Today, I again used data from the literature tracking tool Academic Sequitur, this time to examine some gender patterns in publishing across fields. I took article data from 2018-2020 and estimated the share of female authorships for 38 different research fields, as determined by the field of each journal.* I excluded names that could not be classified as female or male; thus, the share female and share male add up to 1 in each case.
What are the most male-dominated fields? Mathematics barely clears 20 percent female authors, with computer science and finance close behind (or ahead?). Economics just makes it over the 25 percent hurdle and has fewer female authors than engineering. Business does slightly better, with 32 percent female authors. Archeology rounds out this group with just under 40 percent women.
The bottom half of the male-dominated scale has many fields with that are right around 40 percent female, including urban studies, neuroscience, epidemiology, health policy and pharmacology. Finally, three fields have a greater than 50-50 female representation: demography (60.0 percent female), social work (65.7 percent women), and gender studies (66.0 percent female).
Although a few research fields were excluded from this analysis for conciseness, it’s pretty clear that gender parity has a long way to go in academia in the vast majority of fields, even if we look at the most recent data.
* A journal may belong to more than one field. Highly multidisciplinary journals, such as Nature, Science, and PNAS, were excluded from the sample.
Over a year ago, I wrote a post tabulating the share of AER: Insights authors who have also published in a top-5 journal*. (The answer was 67%, significantly higher than most other journals, except those that generally solicit papers, like the Journal of Economic Literature.)
Now that AER: Insights is in its second year of publishing and has 60 forthcoming/published articles, I decided to revisit this question, again using Academic Sequitur data. The graph below shows the percent of authors that (a) have published/are forthcoming in a given journal in 2018-2020 and (b) have had at least one top-5 article published since 2000. The journals below are the top ten journals based on that metric.
With a score of 66%, AER: Insights still has the highest share of top-5 authors among journals where submissions are not generally solicited.** The next-highest journal, Theoretical Economics, is five percentage points behind. (There is some indication that the share for AER: Insights is coming down: for articles accepted in 2020, the top-5 share was “only” 60%.)
What if we condition on having two or more top-5 publications? That actually causes AER: Insights to move up in the ranking, overtaking Brookings Papers on Economic Activity.
Whether this pattern exists because AER: Insights is extremely selective or because less-established scholars are reluctant to submit their work to a new-ish journal or for some other reason is impossible to know without submission data. But no matter how you look at it, the group currently publishing in AER: Insights is quite elite.
*Top 5 is defined as American Economic Review, Econometrica, Journal of Political Economy, Quarterly Journal of Economics, and Review of Economic Studies.
**AER: Insights would be even higher-ranked by this metric (#3) if we ignored top-5 publications in American Economic Review. Therefore, this pattern is not driven by the fact that both journals are published by the AEA.
We all know that economics is a male-dominated discipline on average. But how does the representation of women look across different journals? Armed with Academic Sequitur* article metadata (going back to around 2000), I determined the genders of 82% of all authors in the data and calculated the prevalence of male authors by journal for 50 top-ranked journals in economics.** To see how things have changed over time, I also repeated this exercise with articles that were published in 2018-2019.
Just to set some expectations: in the gender-matched dataset, 82% of author-article observations are male (80% when restricted to 2018-2019). So if a journal has, say, 75% male authors, it’s doing better than average. With that, here are the top 10 male-dominated journals, ranked by share of male authors over the entire data period.*** To be super-duper scientific, 95 percent confidence intervals are also shown, and I added a vertical line at 82.1% for easy benchmarking to the average.
So three of the top five journals (Econometrica, QJE, and ReStud) have also been the three most male-dominated journals, at least historically, with 90%, 89%, and 88% male authors, respectively. A fourth (Journal of Political Economy) also barely made the top ten, with 87% male authors. These numbers also illustrate that there’s not much difference between the #1 and #10 male-dominated journal.
Encouragingly, there are some improvements as well. The share of male authors in QJE was almost 9 percentage points lower in 2018-2019 compared to the whole sample period. JPE‘s share decreased by 7 percentage points, putting these journals in the top 5 most improved. If ranked based on 2018-2019 shares, Econometrica would be #6, ReStud would be #11, QJE would be #24, and JPE would be #28, just barely in the bottom half.
TheJournal of Finance, by contrast, has taken a small but statistically significant step backwards, with a 3 percentage point increase in the share of male authors. If ranked by the 2018-2019 male ratio, it would be number 1.
Here are the least male-dominated journals (rank 41-50). Economics of EducationReview and JHR are both about 66% male. Surprisingly, both applied AEJs are in the least male-dominated group (AEJ: Applied is 71% male; AEJ: Policy is 74%). This may be because they are newer, though it is worth noting that their overall average is below the 2018-2019 average of 80%.
Here’s the rest of the pack. First, here are journals ranked 31-40 on the male-dominated scale (i.e., next 10 least male-dominated), ordered by share male in the overall sample. AER and ReStat are in this group, with 80% and 81% male, respectively. Thus, AER has historically been an outlier among the top five on this dimension (using 2018-2019 shares, it would rank #19, right in the middle of the other top five journals).
Here’s rank 21-30, all in the low-to-mid 80’s.
And here’s rank 11-20. AER: Insights is 84% male. The other two AEJs are in this group, with males representing about 85% of all author-article observations.
These patterns do not necessarily reflect discrimination: the representation of women in a particular field will obviously make a difference here (as evidenced by the positions of macro and theory journals). I leave it up to you, the reader, to interpret the numbers.****
* Academic Sequitur is a tool I developed to help researchers keep up with new literature. You tell us what you want to follow, we send you weekly (or daily!) emails with article abstracts matching your criteria.
** Close to 1.5 percent of the initial observations are dropped because only the initials of the author are available. About 16.5 percent of the observations cannot be mapped to a name for which the gender is known. This includes a lot of Chinese names, for which it is very difficult to determine gender, according to my brief internet research. Names which can be both male and female are assigned a gender based on the relative probability of the name being male.
*** Each observation in the sample is an article-author, so those who publish in a journal multiple times will contribute relatively more to its average. Each coefficient is from a journal-specific regression. Confidence intervals are based on heteroskedasticity-robust standard errors.
**** If you want the numbers underlying these graphs, you can download the csv file here.
Given the centrality of peer review in academic publishing, it might astonish some to learn that peer review training is not a formal component of any PhD program. Academics largely learn how to do peer review by osmosis: through seeing reports written by their advisors and colleagues, through being on the receiving end of them, and through experience. The result is perhaps predictable: lots of disgruntled researchers and the formation of such groups as “Reviewer 2 must be stopped” on Facebook.
This post is my attempt to make the world a better place by giving some advice on peer review. I have written over 100 reports, and I would like to think I do a good and efficient job (then again, I also mostly learned through osmosis, so you be the judge). Some of my advice is based on a great paper by Berk, Harvey, and Hirshleifer: “How to Write an Effective Referee Report and Improve the Scientific Review Process” (Journal of Economic Perspectives, 2017).
As a reviewer, your job is to decide whether the paper is publishable in its current form and what would make it publishable if it is not. This is a distinct role from that of a copyeditor, whose job is to scrutinize every word and sentence, or a coauthor, whose job is to improve the contribution and substance of the paper. A reviewer’s goal is not to improve the paper, but to evaluate it, even though in the process of evaluating it, he may make suggestions that improve it. Of course, it is difficult for people to completely separate their own opinions from objective facts, but the harder we strive to play the right role, the fairer and smoother the review process will be.
Your explanation of the paper’s strengths and weaknesses is more important than your recommendation. Many of us agonize over whether to recommend rejection or revise-and-resubmit. But reviewers do not know how many other submissions the journal receives or what their quality is. Even if you think the paper is great, it may be rejected because there are many papers that are even better. And a mediocre paper may make the cut if the other submissions are inferior to it. So the biggest service you can do for the editor is to help her rank the paper against the other submissions she is handling. Thus, you should aim to explain to the editor of what’s most impressive about the paper and what is lacking. The recommendation itself is secondary. When I recommend a rejection, I use the letter to the editor to outline the issues that make the paper unpublishable (there are usually 1-3), and why I don’t think they can be fixed by the authors.
In case of rejection, make it clear to the authors what the deal-breakers are.The most frustrating and confusing reports to get are ones that raise seemingly addressable issues but are accompanied by a rejection recommendation. It may seem easier to save the “worst” for the letter to the editor, but it will leave the authors trying to guess why exactly the paper was rejected. Anecdotally, the most likely conclusion they will come to is “The reviewer just didn’t like the paper and then looked for reasons to reject it”, which is how Reviewer 2 groups get formed. Of course, you should use professional and courteous language in your reports. But don’t hide your ultimate opinion about the paper from the authors.
In case of a revise-and-resubmit, make it clear to the authors what the must-dos and nice-to-dos are. Point 1 does not mean you should avoid suggestions that wouldn’t make or break publication. Many of my papers were improved by suggestions that weren’t central to the revision (for example, a reviewer suggested a great title change once). So if you have a good idea for improving the paper, by all means share it with the authors. But keep in mind that they will have at least one or maybe two-three other reviewers to satisfy, and the “to do” list can quickly spiral out of control. Sometimes the editor will tell the authors which reviewer comments to address and which to ignore. But sometimes the editor will pass on the comments to the authors as is. By separating your comments into those you think are indispensable and those that are optional, you’ll be doing the authors a big favor.
Don’t spend a lot of time on a paper that you’re sure you’re going to reject. This is perhaps the most controversial piece of advice (see this Tweet & subsequent discussions) because some authors view the review process as a “peer feedback” system. But it is not (see point 1). And, at least in economics, many of us are overwhelmed with review requests and editors sometimes have a hard time finding available reviewers. Treating the review process as “peer feedback” exacerbates this problem. If you think the authors’ basic premise is fundamentally flawed or the data are so problematic that no answer obtained from them would be credible, you should not feel obligated to give comments on other parts of the paper.This does not mean that you should not be thorough – there are few things more frustrating than a reviewer complaining about something that was explicitly addressed by the authors. But in such cases you do not need to give feedback on parts of the paper that did not affect your decision.
Finally, I’d like to wrap up with an outline of how I actually do the review. First, I print out a physical copy of the paper and read it, highlighting/underlining and making notes in the margins or on a piece of paper. Second, I write a summary of the paper in my own words (it is useful for the editor to get an objective summary of the paper, and the authors can make sure I was on the same page as them). Third, I go through my handwritten comments and type the most relevant ones up, elaborating as needed. Fourth, I number my comments (helpful for referencing them in later stages, if applicable), order them from most to least important, and separate the deal-breakers or must-dos from the nice-to-dos. Fifth, I highlight the deal breakers (if rejecting) or must-dos (if suggesting revisions) in the letter to the editor. Finally, regardless of my recommendation, I try to say something nice about the paper both in the editor letter and in the report. Regardless of its quality, most papers have something good about them, and authors might be just a tad happier if their hard work was acknowledged more often.
