Fraud at Georgetown Finance Shakes the Foundation of the Corporate Bond Literature

"This is probably one of the biggest cases of fraud or incompetence in academic finance's history."

Today’s story is about the paper Common risk factors in the cross-section of corporate bond returns published in the Journal of Financial Economics, one of the 3 best finance journals in the world.

To put this in perspective, if this were a medical paper, it would be as if a massive case of corruption happened at Lancet, New England Journal of Medicine, or Cell…. one the biggest cases of fraud or incompetence in science history… and nobody cared.

The reason nobody cares is that the corporate bond literature, is, well, boring and arcane. My job is to make you care, by explaining this scandal in a digestible way.

The authors of the paper, Jennie Bai, Turan Bali, and Quan Wen, are all professors at Georgetown Finance. Henceforth, I will refer to their paper simply as BBW.

Note that Bai is an associate editor at JFE, where this paper is published.

Bai, Bali, and Wen

During my research for this article, I spoke on the phone with an expert in the field of corporate bond literature—a finance professor who requested anonymity. He advised against including a photograph of the three researchers involved. He expressed his conviction, albeit unprovable, that not all the authors share equal culpability in this scandal. In his words:

This is probably one of the biggest cases of fraud or incompetence in academic finance's history. But not all of the coauthors might have been aware that it was happening.

— Anonymous finance professor

While on the surface, all three researchers appear equally accountable, the actual distribution of blame might be uneven. It's challenging to ascertain individual responsibility without definitive evidence or perhaps a confession— a remote but potential scenario, as it is actually relatively common for 2 authors to throw the 3rd author under the bus in these types of scandals.

Bear these nuances in mind as you read on.

This scandal first broke on EJMR 5 months ago, in a 50-page thread titled Finally some one called out JB, a great day for corporate bond literature.

As the title of the EJMR thread implies, another group of heroic researchers “called out” the fraud — this is the tale of 2 warring papers, one fraudulent one (BBW) and one by Dickerson, Mueller and Robotti that I will refer to henceforth as DMR.

BBW vs. DMR.

These dueling papers concern a model called the Capital Asset Pricing Model (CAPM), which is a financial model that calculates the expected rate of return for an asset or investment.

For the layman reading this article, it’s not important to understand exactly how CAPM works. Just know that it is the dominant factor model used across the entire bond pricing literature — used by practioners and academics alike; which is important because the overall size of the global bond markets in terms of USD equivalent notional outstanding is approximately $129.8 trillion.

This CAPM model—theoretically—determines how those bonds are priced.

The Primary Error: Lead-Lag

The first major error in BBW is a “lead/lag error”. At first, I thought this lead/lag error would be hard to explain to a layman, but it’s actually rather straightforward. Here is an excerpt from DMR:

The publicly available [factors] provided by BBW are not properly constructed and suffer from lead/lag errors over extended portions of their sample period … First, there is a lead error (look-ahead bias) in the original DRF and CRF factors for the majority of the sample. That is, the return value for month t is in fact the value for month t + 1. This error spans the sample period from 2004-08-31 to 2014-12-31 (the entire sample except for 2015 and 2016). For LRF, there is a lag error which spans the sample period 2015-01-31 to 2016-12-31 (the last two-years of the sample), i.e., the returns for month t are the returns for month t−1.

I bolded the t+1 and t-1 in this excerpt because that is what a “lead/lag error” is: the BBW authors took most of their data points which are supposed to be calculated at time “t”, and added or subtracted a month to them when it was convenient, to get the results they wanted. When those errors are corrected, BBW’s results disappear. As such, BBW’s paper is totally broken and wrong beyond any reasonable doubt.

It’s really that simple.

Different factors are constructed from the same merged data set. If there was a leading error in the merged data, you should ONLY see the same leading error in the constructed factors. It is not possible to see either leading errors from one factor or lag errors from another factor. Factors are from the same merged data. Why sometimes lead and sometimes lag errors depending on factors?

Do you understand what it means? This cannot be a mistake.

— EJMR comment

The Secondary Error: Surreptitious Winsorization

In addition to the lead/lag error, DMR also point out a second error:

In contrast to BBW, we do not rely on winsorization of bond excess returns in factor construction … we show that truncating both tails of MKT B as in BBW, reduces its risk premium and favors alternative factors in multivariate tests.

Winsorization refers to the practice of modifying extreme data values to reduce the impact of outliers on statistical results. In simpler terms, BBW truncated some of their data. Yet, they did not disclose this modification, nor have they shared their winsorization/truncation code. Furthermore, the BBW paper does not mention the word “winsor” or “truncate” even once — so what they did is hide it in their private code and hoped nobody would find it!

As of May 2023, after their mistake was highlighted on EJMR, BBW, instead of sharing their original data/code or clarifying why the errors arose, published "corrected" factors that admitted no mistake and further exacerbated the issue.

As is often the case, the attempted coverup is worse than the crime.

https://drive.google.com/file/d/1UpnZTXmEaGaati07KJcSBFSYkN0k2Wau/view

DMR points out that this BBW response is worthless since it does not correct for bond return winsorization.

As of May 2023, shortly after the public release of our working paper, the original factors were updated on Turan Bali’s webpage, which correct for the lead-lag errors but not for the bond return winsorization. We provide the original BBW factors, which contain the lead-lag errors, and the correctly constructed replicated factors for download on the companion website openbondassetpricing.com

I tried to find the winsorization in BBW’s response code, but was unable. I would think it is unlikely they would let bond winsorization be part of the code. There are 3 possible reasons why I can’t find the winsorization in the code:

1. I am bad at reading SAS code

2. BBW selected bonds in a non-obvious or hidden way that dropped extreme negative returns

3. DMR may be mistaken

I would put the odds of these 3 scenarios at roughly 15%, 80%, 5%.

I think it is highly likely they winsorized without telling us, but it isn't possible to say with 100% certainty. However the onus is on them to prove they didn't and they have said nothing.

In the BBW "response", the authors winsorize the bond returns ex post at the -10% level (i.e., at month t+1). This renders any and all of their "new results" utterly meaningless.

— EJMR comment

Their replication suggests their main results don't hold and they cannot explain what causes lead and lag errors. Then why they don't just publish the original dataset and code? What to hide if it is an honest mistake?

— EJMR comment

If no fraud was committed, I see no reason why BBW would not share the original code and data right away. Not sharing is a major red flag that mounts everybody's suspicion about a fraud BBW are trying to cover up.

— EJMR comment

Truncation at the bottom 10% is very very likely intentional and can be called data manipulation. That also makes the lead lag errors less look like innocent. Intentions tend to be correlated.

— EJMR comment

The Retraction

About a month ago, I barged into the EJMR thread and asked:

Subsequently, I reached out via email to the authors, JFE, Georgetown, AEA, AFA, as well as JFQA—a journal where additional research that relies on BBW is published, and one of the BBW authors and also the chair of finance at Georgetown are editorial board members — and posed a series of pointed questions:

I received two responses.

From JFQA:

We at JFQA are aware of the concerns regarding the bond market factors employed in BBW (2019), and we are in the process of assessing the potential relevance to papers published or accepted for publication in the JFQA. While we take the matter seriously, it would be inappropriate for us to comment further before this process is complete.

From JFE:

This entire matter has not been resolved. We are aware of the issues, as we are handling the DMR paper. I cannot comment until the process is complete. 

Okay, so the journals where the fraudulent work is published seem to be handling this appropriately, at least. Indeed, a couple of weeks after I sent this inquiry, this happened:

And in BBW’s stead, the JFE published DMR’s paper:

This is huge!

First, the correct bond factors are now published.

Second, this is the first ever retraction in the JFE (!!), which has a history of being corrupt and covering up fraud under the last administration. It seems the new JFE administration is turning a leaf.

Unfortunately, though, this is only where the problems begin.

First, the JFE chalks this up to an innocent “error” with no mention of fraud.

Subsequent research conducted by Dickerson, Mueller and Robotti (2023) using a similar dataset of corporate bond returns reveals an error present in the data used by Bai et al. (2019) that consists of temporal misalignment of different data series. Dickerson et al. (2023) find that in data that do not exhibit this error, the factors identified in Bai et al. (2019) do not have incremental explanatory power over the aggregate corporate bond market return, with the marginal exception being the factor that captures liquidity risk.

The authors of Bai et al. (2019) confirm that the original results are based on the data with the above error. The main result in the paper regarding the explanatory power of the bond factors does not survive the correction of this data error. The authors deeply regret the damage this caused to the journal and the scholarly community.

This is so ugly. The retraction uses a data error as an excuse…

— EJMR comment

Second, the JFE is simply publishing the paper showing that 3-4 additional ones they had published earlier were wrong due to manipulated data. So, really, the JFE should retract several more papers

But it goes much deeper than just a few papers at the JFE…

The Wider Impact

Countless published and working papers have utilized these factors, and their validity is now in question. Although BBW has garnered a commendable 334 citations, many of those 334 papers depend fundamentally on the fraudulent BBW factors, and have in turn been cited collectively thousands of times.

This creates a domino effect where the integrity and reliability of a vast body of academic work becomes dubious, if not outright wrong. The academic finance community must now grapple with the challenge of re-evaluating these works and determining the extent of potential misinformation. Will dozens, or hundreds, of published papers eventually be retracted and/or corrected? In a perfect world, that should be the outcome, but it seems impossible. The scale of this is staggering.

An even bigger question is what happens now to the dozens of papers by the same authors and others, whose results rely on the BBW factors?

— EJMR comment

Who would’ve thought this paper would become the standard bond factor paper with thousands of citations? When they were forging the data work, they wouldn’t have even known this would happen.

— EJMR comment

My Job Market Paper was largely fucked up up by DMR. Now my JMP is completely destroyed. I cannot blame DMR for doing the right thing. I cannot blame myself for following the literature like BBW... Then whom or what should I blame for? FML...

— EJMR comment

The Coverup

Where is Georgetown in all this?

I emailed them twice, with no response.

The lack of response on Georgetown’s side is very scary. Trying to cover up and not being transparent about research ethics issues is the worst a research-oriented university can do. Many would claim that they will investigate and then do nothing but at least they would react and show to the academic community that they were willing to take the matter as seriously as it deserves. Staying silent is the worst.

— EJMR comment

If Georgetown refuses to act then the AFA as the association of the finance profession should be involved even though I ignore if they have any protocols for these cases. By any means I would be astonished if their school did not want to set up an investigation to determine if their researchers committed academic fraud.

— EJMR comment

So is Georgetown running an independent investigation about this potential academic fraud by three of their tenured faculty? If not, how serious and credible can they be as a research institution?In other fields, schools routinely run investigations based on allegations of research fraud by their faculty and they take action if the allegations are proven valid. The reason is obvious. Keeping fraudulent researchers on one’s roster is so much more damaging to any research institution than taking action against those who commit fraud.I don't think we can officially claim this is a "fraud" yet. But it raises enough concerns to launch a formal investigation on this.

— EJMR comment

Quite a bad hit for Georgetown’s reputation mostly because these are three researchers all at that same institution. The only similar case I can think of is CMU which effectively ended up firing all three of AB, SK and TR and the latter was not even involved. Easier for them because none was tenured. They put in writing that no evidence of fraud was detected and yet told them all to leave, which restored their reputation. When you have three tenured faculty involved in fraud it’s much harder to restore your reputation. A serious investigation seems like the minimum you can do

— EJMR comment

What shocks me about this whole story is not the fact that a major error was detected, which is most likely fraud, and that this will impact many papers written after BBW 2019, but that nobody seems to be acting to punish the malevolent authors. Retraction is only the starting point that certifies that the paper was wrong. It’s not. A punishment for wrongdoing. This behavior cannot be unpunished or else everybody will start to do the same knowing that they would pay no consequences if caught with the hands in the cookie jar.

— EJMR comment

At a minimum Karl’s email made the university formally aware of the situation and if they do not act they cannot say that they were not aware or that nobody brought up this issue to them.

— EJMR comment

If Georgetown doesn’t do a formal investigation to understand what happened and if it was fraud or a major error without fraud intentions they will lose all credibility researchwise.

— EJMR comment

The retraction will hopefully be the base for an investigation by Georgetown or the AFA. The JFE is not required to go into any details about what caused the mistake and if it is fraud or not. The JFE states that the paper is retracted because the results are false, that's all they have to do. Now it's on others to investigate if the researchers committed fraud … Also if Georgetown doesn’t care about research and reputation it’s their problem but what about the RFS and JFE? Are they keeping those people as AEs on their boards?

— EJMR comment

BBW retraction is … just the scratching of the surface. Is it too much to ask that somebody (AFA, Georgetown, JFE, other journals, ideally all of the above) investigates these issues and gives us answers?

— EJMR comment

I do not see how Georgetown could not act. An investigation is unrelated to any potential retraction. There is now published evidence (DMR in JFE) that shows major mistakes in their researchers’ published results and these mistakes could be due to fraud or to a major error. How could the institution not investigate to protect their own reputation of research ethics? … How come that Georgetown hasn’t acted for months? They might have an investigation ongoing, who knows. For the sake of their reputation I hope that they do.

— EJMR comment

Anybody knows if Georgetown or AFA are investigating or at least releasing any official statement about this situation? Here we are talking a whole line of research and hundreds of papers being nullified by the both outright frauds and severe ineptitude of some researchers who try to deny the evidence in every possible way and to cover up their wrongdoing. If I was responsible for those institutions and their academic integrity this problem would not make me sleep at night. Here it seems that either they are not even informed about these developments or they just stay silent and hope that people stop talking about it and pretend nothing has ever happened.

— EJMR comment

Guys, if we don’t eradicate this way of reasoning from finance academia we will never be a serious and scientific discipline. We will always be a mob. The mob is the claque who, despite obvious evidence of major mistakes and very suspicious behavior, such as not sharing data and code, is willing to show their loyalty to the offenders by attacking those who want justice. This is the mob, my friend. It’s like protesting that judges do their jobs by evaluating convicts in courts because they are unjustly trying to punish a poor innocent. This is a mob and like the mob operates all over the world.

— EJMR comment

This story is also a warning about perverse tenure incentives. If a school only counts numbers to grant tenure they create strong incentives for juniors to do ugly stuff with the data in order to obtain publishable results especially when the data are not public and the chance of being caught are low. Except when similar data later become available to all and the original results can be finally replicated and shown wrong. Seniors should also be more careful when coauthoring with juniors who have so strong incentives to commit fraud.

Otherwise why wouldn’t all untenured faculty manipulate data, get tenure and the wait for somebody who detects the fraud years later without any consequences for them?

— EJMR comment

This whole fiasco is an embarrassment to financial economics.

— EJMR comment

The moral of the story: Let us all fake some results to get publications and then tenure. Let's worry about being caught later.

— EJMR comment