How many indicators can we include in an Equity Indicators tool?

To keep the tool manageable, you should choose a limited number of indicators to include. What this means is that rather than including all of the important things to measure, you select a certain number of indicators to serve as proxies for conditions in each particular area.

For the purposes of sustainability, it is important to ensure that the data can be collected and reported regularly. The more indicators that are included, the harder it is to do so. Additionally, the more indicators that are included, the more difficult it is for users to get a sense of the data and the indicators themselves.

Can we add more indicators to an existing Equity Indicators tool?

It is not recommended, largely due to concerns about manageability and usability. Adding an indicator to any given topic will exponentially increase the number of indicators in the tool. This is because the methodology requires that each topic include the same number of indicators; adding an indicator to one topic means that an indicator must also be added to every other topic.

Can we replace indicators in future years?

Yes. There are several reasons that an indicator might be replaced, including discontinuation of data collection, determination that the indicator was not accurately capturing the disparity it was intended to capture, or simply identification of an indicator that serves as a better proxy for the issue being examined.

In order to replace an indicator in a given year, it must also be replaced in the prior year(s) to enable tracking over time. These types of changes may result in updated static scores for previous years of data collection with each successive round of data collection.

Why do we need to have an equal number of indicators in each topic?

So that all indicators have an equal weight. If you have more indicators in a given topic, each individual indicator ends up counting less than indicators in a topic with fewer indicators. Given that selecting measures is in itself a subjective process, we wanted to avoid adding additional subjectivity by giving some indicators more weight than others, or making decisions about which indicators were more important than others.

Topic scores are created by averaging the scores of individual indicators contained in the topic, while theme scores are created by averaging the scores of the topics contained in the theme; an uneven number of indicators means that indicators in topics with fewer indicators individually contribute more to the topic score, and thus to the theme score (and ultimately to the city score).

How are the indicators weighted?

Briefly, they aren’t. In creating the methodology, CUNY ISLG made a deliberate decision not to weight indicators because it would add an additional amount of subjectivity to the tool.

Why score indicators?

Scoring has two important and related benefits. It enables the standardization of data produced in different formats (i.e., ratios, percentages, and rates) and from different modes of data collection (i.e., administrative data and survey data). In turn, that makes it possible to synthesize findings across indicators, topics, and themes to produce higher-level findings, an important feature of our indicators.

Without scoring, the only take-aways from this process would be individual results for the indicators.

How were the ratio-to-score conversions determined?

A ratio of 1 receives the top score of 100 because it mathematically corresponds to equal outcomes. A ratio of 10 or higher was selected to receive the lowest score because it is a large disparity (i.e., one group 10 times more likely to have a particular outcome), but still one that we have found. As indicators move closer to equality (a ratio of 1), it is more difficult to achieve change. For this reason, smaller changes in ratio are required to move from one score to another as ratios move closer to one.

How do we pick the groups to look at?

You can choose which groups to look at based on an examination of local inequities and government priorities. In cases where there are more than two groups to choose from (e.g., racial and ethnic groups), we recommend comparing the two groups with the best and worst outcomes. That said, you are limited by the data that are available; in order for a group to be included, there has to be a local, regularly-collected data source that allows you to break data down by the group. For many groups, there are no such data sources, which means that you will not be able to include them. In those cases, you have to look at a different group—one for which data are collected—in order to examine the disparity you wish to track.

What time period do the indicators cover?

The indicators are scored using the most recent data available at the time of data collection. Because the most recent data available vary depending on the source, different indicators may use data from different years. For example, the Census does not release American Community Survey data for a given year until October of the following year, whereas individual city agencies may be able to provide data for the current year. Generally, we recommend having no more than a two-year lag for any individual indicator. 

Can we use the indicators to learn about problems in real time?

Unfortunately, no. Equity Indicators tools cannot track change in real time because they use annual or point-in-time data to compare outcomes for groups. Furthermore, there may be time lags of up to two years in the data available for specific indicators.

How does our city compare to other cities?

While we understand the desire to compare different cities, overall scores for different cities cannot be compared to each other, because cities’ individual tools have different structures and are made up of different indicators, topics, and themes. The only comparison that is possible across cities is at the individual indicator level. If two cities use the same indicator and data source, and compare the exact same two groups (e.g., women and men), the indicator scores can be compared.