Problems with data: measuring diversity

Diversity – in audiences, employees, and boards of trustees – matters to arts administrators, and to researchers who study organizations and societies. But how do we measure diversity?

The diversity index is constructed like this: suppose everybody comes from one of four different population groups: Blue, Green, Purple and Red. Let the fraction of the population that is Blue be given by B, the fraction that is Green be given by G, and so on, where B, G, P and R are all numbers between zero (if nobody comes from that group) and one (if everybody in the population comes from that group). The diversity index D is found by taking

D = 1.0 – B² – G² – P² – R² 

If B = 0.4, G = 0.3, P = 0.2, and R = 0.1, then the diversity index D = 1.0 – 0.16 – 0.09 – 0.04 – 0.01 = 0.7. A lower value for D means less diversity. The lowest number D could be is 0 – that would be the case if everyone came from one group and nobody came from any of the others. For a population with 4 groups, the highest D could be is 0.75 (that’s what you get when B = G = P = R = 0.25).

The diversity index has a lot of use across the sciences. Biologists can use it to compare the diversity of plant and animal species in an eco-system. Economists studying industrial organization can use it to measure whether a sector is highly concentrated amongst only a few firms (i.e., if most sales come from just one or two firms, rather than being spread equally over many firms).

And of course we can use it to measure diversity in populations, either in general, or specifically regarding audiences or employees, and where the groups can be defined by race and ethnicity, or by religion, for example.

This all sounds straightforward, but in fact there is a big problem underlying it. To illustrate, let’s take a recent post from Richard Florida at City Lab: ‘The Link Between Religious Diversity and Economic Development.’:

How does religion, and especially religious diversity, affect our economies?

I decided to take a look using data from the Pew Research Center’s Religion & Public Life Project on “global religious diversity.” The Pew data track the level of religious diversity, measured as the percentage of the population that belongs to eight major religious groups in countries around the world. These include the five major world religions—Buddhism, Christianity, Hinduism, Islam, and Judaism—which account for about three-quarters of the world’s population, as well folk or traditional religions, the religiously unaffiliated (atheists, agnostics, etc.), and other religious groups (Baha’i, Sikhism, Taoism, etc.). …

Five key findings jump out from this analysis.

• First and foremost, religious diversity is associated with the overall productivity and economic competitiveness of nations. Religious diversity is positively associated with total factor productivity (.32) based on World Bank measures and overall economic competitiveness (.56) as ranked by the World Economic Forum. It is also associated with higher levels of entrepreneurship (.37) based on the Global Entrepreneurship Monitor rankings.

• Pluralism is also associated with more urbanized societies, with a positive correlation to national levels of urbanization (.31).

• Religious diversity tracks key markers of social tolerance. It is positively correlated to the Gallup survey responses on the acceptance of racial and ethnic minorities (.45) and the acceptance of gays and lesbians (.31). There is an even stronger relationship between religious diversity and the treatment of women based on the UN’s Gender Inequality Index, where more gender equal nations also have more religious diversity (.55).

• Pluralism is tied up with the broad shift to a post-industrial knowledge-based economy, as it is positively correlated with share of workers in the creative class (.42).

• Finally, pluralism is associated with the overall the happiness and well-being of nations. It is positively correlated with Gallup’s overall measure of life satisfaction (with a correlation of.32) and even more so with the United Nations’ assessment of overall human development (.38).

The Pew study can be found here, its methodology here, and its results for all the countries here. Note that the Pew study calculates diversity just as I did above, except (1) it has 8 types of religious affiliation (as opposed to my 4 groups), and (2) the index is scaled such that the number 10 represents maximum diversity.

So what’s the problem? Look at the data. Tied in diversity (at a rank considered ‘high’, though not ‘very high’) at a figure of 5.3 comes Canada, and … North Korea. Now I haven’t been to North Korea, but I did spend the first 3/4 of my life in Canada, and based on what I have read of the DPRK, religious life is not very similar in the two places. In the ‘very high’ diversity countries, we have a near tie between South Korea at 7.4 and China at 7.3, and again, I think most observers would say there is a pretty significant difference in religious life between the two.

What’s going on? The problem isn’t in the data collection; I fully trust that Pew got the numbers right. The problem is in the index. To go back to my original example, the diversity index looks at diversity across Blue, Green, Purple and Red without considering which groups are dominant. And a country that is 90% Blue and 10% Green might be very different from a country that is 90% Purple and 10% Red, even though the diversity index will be identical in the two places.

If you were wanting to use diversity as an explanatory variable about some aspects of policy or society in the US, would you treat a zip code that was 80% African-American and 20% white the same as one that was 80% white and 20% Asian-American? The diversity index for the two zip codes would be the same (0.2). Only if you thought that relative shares mattered, but that it did not matter who was in the majority, would the index be useful.

The diversity index is only useful if all that matters is the relative shares in the mix regardless of which shares are which. Sometimes that can be useful, if, say, I am measuring species of butterflies in a field, or industrial concentration in energy production. But religion and race are trickier than that. And in those cases, skipping the index, and instead using actual shares by the different groups, is advisable.

And so I don’t think Professor Florida has given us much to work with here.

Academic footnote: I actually wrote a paper on this once, ‘A Note on the Use and Misuse of the Racial Diversity Index,’ Policy Studies Journal, 36(3) (August 2008): 445-59.