Publication in our best journals is based on factors other than merit, "author
prestige also comes into play":
Everyone has the same chance at the AER, right?, by John Whitehead: Wrong:
We spoke with Virginia economics professor William R. Johnson, who edited
the edition of the Review in which the [Reinhart and Rogoff] paper first
appeared.
This annual
edition, "Papers and Proceedings," differs from all others in that the
papers come out of presentations made at the yearly meeting of the American
Economic Association, he said.
The papers are
personally selected by the AEA's president-elect, in consultation with a
committee.
As a result of
these unusual circumstances, Johnson said, the editing of "Proceeding"
papers is less rigorous.
"Normal peer
review doesn't happen for these papers in the way of other issues of the
AER." ...
But author
prestige also comes into play, Johnson said, adding that that was true for
all AER papers, not just the ones that appear in "Proceedings."
We all knew this but it is a little surprisingly to see its admission. [via www.businessinsider.com]
It was awhile ago, but I once discovered an error in a paper in the
AER (the
author used the level of the price level rather than the log). When I
pointed
out the error, the author -- who is very well known (and now the Fed
chair) --
wrote a letter to the editor of the AER arguing that it didn't
materially affect
the results, and subsequently our note pointing out the error was
rejected (much like Reinhart and Rogoff argue that their results are not
materially affected by their error, but I think the error mattered as
it
weakens the case in the Bernanke and Blinder paper, but even if it
doesn't wrong results should be corrected --
the results cannot be replicated based on the information in the paper).
The
prestigious author won out over lowly me, and to
this day there is a wrong result in a table in a influential paper in
the debate
over how to conduct monetary policy. The only place I know of where the
error is
noted is in a footnote to a (relatively obscure)
paper of mine (along with Jo Anna Gray). The footnote says:
Table 1 of Bernanke and Blinder [1992 ] reports marginal significance levels
for the federal funds rate that are dramatically higher than those for M2.
There are numerous differences between our studies, most of which are
inconsequential. However, this discrepancy in results is due to a
computational error in the Bernanke and Blinder study. While the error
significantly affects some of the F-statistics reported by Bernanke and
Blinder, it has little effect on the corresponding variance decompositions
reported in the paper. We thank Ben Bernanke for providing assistance that
allowed us to confirm and correct the error
Again, whether or not is materially affects the results is debatable.
The F-statistics changed quite a bit, the IRFs less so, but in any case
results containing computational errors should be corrected, especially
in papers as influential as this one turned out to be. (Update: This
also shows that the profession cares very little about making it easy to
replicate results.)
We will never be a science so long as this crap persists.