How good will AER: Insights be?

American Economic Review: Insights is a new journal by the American Economic Association. It’s intended to replace the short paper section of the AER, and the first issue will appear this summer. Naturally, I’ve had quite a few discussion with colleagues about its likely quality: will AER: Insights be a top-tier journal like the AER, somewhere in the middle of the pack, or a flop?

Obviously, many factors affect the success of a journal. But how it starts out surely matters. Publish some amazing articles and become the journal at the top of people’s minds when they think about where to submit, which will in turn make it easier to attract high-quality articles. Publish some questionable research, and risk aversion will kick in, prompting people to submit elsewhere first and leaving you mostly with articles that other journals decided against publishing.

So I again dove into the database of Academic Sequitur. We track forthcoming articles, so even though the first issue of AER: Insights has not been published yet, we have 26 to-be-published articles in our database (the first of which were added in November of 2018, by the way!). The question I asked was simple: what percent of authors whose work is scheduled to appear in AER: Insights have published a top 5 article any time since 2000?

The answer is a whopping 67% (61% if you exclude AER articles). 58% have published in the AER, 23% have published in Econometrica, 38% have published in QJE, and 39% have published in ReStud. The overall percentage is non-trivially higher than that of any other journal except for Journal of Economic Literature.

Perhaps these numbers are not surprising to you. In fact, it may very well be a strategy that AER: Insights is consciously employing to gain early traction. And these statistics could signal that it’s difficult to get published there unless you’re well-known, at least at this point (though we don’t know what the distribution of submissions looks like). But more information is better, and this certainly makes me more likely to try for AER: Insights in the future!

Has it gotten harder to publish in top economics journals?

In a fascinating series of graphs, David Card and Stefano DellaVigna document that submissions to the top 5 economics journals have gone up from about 3,000 per year in the early 1990s to over 7,000 per year in the mid-2010s, while acceptance rates have decreased from 10-15% to 2.4-5.6%. A natural interpretation of these trends is that it has gotten much harder to publish in top journals, all else equal. But could there be more to this story? There are (at least) three reasons things are not as clear as they seem at first glance.

First, submitting a paper to a journal has certainly become easier since the 1990s. It used to be (or so I’m told) that you had to physically print out the paper, put it into an envelope, send it somewhere and wait for a return envelope, which meant that even desk rejections took a lot longer than they do now, on average. The whole process should have been inherently harder, which meant people may have thought twice before trying for a Hail Mary at a top journal. With more and more people having internet access and virtually all top journals accepting electronic submissions, I would predict that a lot more people are willing to shop their papers around for a while in hopes of hitting the best journal possible.

Second, the emphasis on having a top-5 publication seems to have increased steadily over the past 20 years or so. Of course, there could be reverse causality here – if publishing a paper in the top 5 is harder, then success should be valued more, on average. But I think it’s likely that the emphasis on the top 5 journals has caused more people to try to publish papers there that they would otherwise send someone else, driving submissions up and acceptance rates down. For example, authors may now try for three of the five journals instead of just one (I may or may not have done this myself), so a single paper will show up as three submissions in the statistics above.

Third, the graphs show that the number of authors per paper has gone up too, so conditional on the total number of published papers being constant, the total number of people with a top 5 publication should be going up!

But shouldn’t the first two trends mean it’s gotten harder to publish in top journals, as the original statistics imply? This is true only if the marginal papers are about as good as the average papers. And that is very unlikely to be the case. Given the high rates of desk rejections at many journals, is seems more likely that there are many people trying for a very unlikely outcome. In fact, it’s possible that the probability of a good paper getting into a top journal is essentially unchanged because each marginal paper has an incredibly tiny probability of acceptance. Of course, it’s hard to believe that the publishing process is that accurate, so it’s likely to have gotten at least somewhat more competitive at the top journals. But I highly doubt that the level of competition is anywhere close to 2-5 times higher, as the statistics imply.

The real question we should be asking is, are there more high-quality papers per high-quality journal (somehow controlling for general progress in methods, computing power, and data)? Or, if we’re interested in the likelihood of a person having a top-5 publication, are there more high-quality academics per high-quality journal? Both of these questions are really difficult to answer, but I found no statistics suggesting that there have been significant increases in the number of academic economists over this time period. For example, this website from 2006 says that “the number of new PhDs who intend to pursue careers in the US has declined.”. This report indicates that there were 24,886 AEA members and subscribers in 1999 and this one says there were 23,170 in 2017. My overall conclusion is that the decline in acceptance rates should not be interpreted as really bad news for good papers trying to get into good journals.