How should we rank the employment prospects of cities?

News sites on the web demand a stream of content, and a sure way to produce something is to make a list. My previous post knocked a recent list of  ‘top cities for culture’, on the grounds that the way the index was constructed made no sense.

Another day, another list, this one from Atlantic’s CityLab: ‘The best job markets for young college grads now‘. Relevant to me, as I teach students in arts administration and hope to understand as best I can what is happening in the labor market. The piece has some of the usual problems of all indices created from thin air, such as choosing five inputs into the index with no systematic way to determine the weights each factor should have. Although all these rankings and indices we see in our news feed come from real data, the weights are a fiction, and completely discredit any claims to objectivity.

The second problem is the factors that make up the index itself. In this case, what are supposed to be the key factors for college grads seeking a good job?

The rankings are based on five key factors:

• The percent change in jobs requiring post-secondary education from 2010 to 2014.
• The percentage of 25-34 year olds who hold these positions.
• The average wages for these jobs requiring post-secondary education.
• The concentration of these jobs based on their “location quotient,” or LQ for short.
• The share of new jobs requiring post-secondary education that can be attributed to local economic conditions or competitiveness.

In order then:

• The percent change in jobs requiring a degree doesn’t capture the employment base. If the number of jobs requiring a BA went up over five years by 6% in Detroit and just 5% in New York, would that indicate better labor market conditions in Detroit? No.
• This is an odd metric. An absence of younger workers might mean that turnover in jobs is due? It is hard to say how this translates into better employment prospects.
• OK, it’s a gimme.
• I have a real problem with the LQ. In short, it tells us the proportion of total jobs made up by a certain type (e.g., in this case, requiring a college degree) compared to the national average proportion. But what matters to job seekers is the number of opportunities, not the number of opportunities relative to the size of the entire regional labor market. If my arts administration students come looking for advice on what regions have the best prospects for jobs, I will be looking at job activity in the arts in that region, not at jobs in the arts as a proportion of all jobs in the metro region in question. This measure has been used in many studies of the creative economy – I admit to having fallen under its spell in the past – but it is time to ask, in any further application of the LQ, is it capturing what you really want to capture?
• The last measure tries to get at what are the factors driving employment changes, in terms of regional economic make-up or national trends. The implications are not clear.

These types of indices overthink the problem, taking data that isn’t all that informative, and ignoring simpler measures that are (how about, for example, ‘number of new hires of college grads’?).

And the odd rankings that come from them should give the authors pause. In this index, Detroit comes off as a better option for new grads than DC, NYC or Chicago. How much confidence do you have in that result? (If Detroit really is a good place to look, these data do not show it).

There is a larger problem here – a demand for articles with data and maps (sorry, interactive maps) and rankings on a daily basis that haven’t been really thought through, with headlines making much bolder claims than are even close to being warranted. Please let’s not allow research on the arts to go this way.