Benford's Law

<body bgcolor="#E8FFF0"> <div align="left"> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table17"> <tr> <td width="740"> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" height="37" id="table18"> <tr> <td height="35" width="100%"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" id="table19"> <tr> <td> <p align="center" class="header_green"><map name="FPMap4"> <area target="5" href="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog_071210_post.htm" shape="rect" coords="0, 0, 149, 29"> </map>Benford's Law</td> </tr> </table> </td> </tr> <tr> <td width="100%" bgcolor="#CC9966" height="2"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td width="0%" valign="top" bgcolor="#FFFFCC"> <table border="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" id="table20"> <tr> <td align="left" valign="top" bgcolor="#E8FFF0"> <p class="blog_body" align="center"> December 10, 2007  <font size="2"> <a target="_top" href="http://www.glynholton.com/link/blog_071210.htm">permalink</a></font>  <a target="5" href="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog_071210_post.htm"> comment</a>  <a href="mailto:glyn@contingencyanalysis.com?subject=Blog Feedback"> e-mail</a></td> </tr> </table> </td> </tr> </table> <p class="blog_body">Benford's Law is a fascinating theorem from statistics that states, for most forms of data, the leading digits of numbers are not uniformly distributed among 1 through 9. Instead, any given data point has a 30.1% probability of having a 1 as its leading digit. There is a 17.6% probability of the leading digit being a 2. The probabilities continue to decrease with successive digits. There is just a 4.6% probability of a data point's leading digit being 9.</p> <p class="blog_body">Benford's Law is widely used by auditors and tax authorities to detect fraud. They apply Benford's Law to the numbers reported in a financial statement or tax return to see if the leading digits are distributed according to Benford's Law. If they aren't, there is a high likelihood someone fabricated the numbers.</p> <p class="blog_body">Hedge fund returns pose a similar challenge. Reported numbers are routinely manipulated, inflated and smoothed to make the returns appear higher and less volatile than they actually are. But how might we prove this when the hedge funds are unregulated and don't have to show their books to anyone? Recently, researchers have been getting around this problem by applying statistical tests to reported hedge fund returns. Their techniques aren't as sophisticated as Benford's Law. They don't have to be. Hedge fund abuses are so blatant and widespread, even simplistic statistical tests make them stand out. </p> <p class="blog_body">One technique is to look at the distribution of returns that happen to fall close to zero. We would expect about half of these to be very slightly positive and about half to be very slightly negative. For hedge funds, this is not the case. The vast majority are slightly positive.</p> <p class="blog_body">Researchers Bollen and Pool report this result in a working paper released last month. They analyzed individual hedge funds' monthly returns reported in the Center for International Securities and Derivative Markets (CISDM) database from 1994 to 2005. They found "a significant discontinuity in the pooled distribution of reported hedge fund returns." The number of small gains far exceeded the number of small losses. This was true for active funds, defunct funds, and funds of all ages. Interestingly, the discontinuity disappeared during the three months leading up to a hedge fund being audited.</p> <table border="0" cellpadding="0" style="border-collapse: collapse" bordercolor="#111111" width="345" align="right" id="table21" bgcolor="#FFFFFF"> <tr> <td align="right" width="15"> </td> <td> <p class="blog_body" style="margin-bottom: 12px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/_blog_071210/exhibit_1.gif" width="331" height="271"><br> <font style="font-size: 9pt"><b>Exhibit 1: </b>Histogram of reported monthly hedge fund returns that happen to fall close to zero. Each bucket spans 19 basis points. Source: Detail from an exhibit in the Bollen and Pool working paper.</font></td> </tr> </table> <p class="blog_body">The exhibit on the right is reproduced from that paper. It indicates the distribution of hedge funds' returns that happen to fall near zero. The discontinuity Bollen and Pool found is pronounced, and it falls precisely at 0. Wow, isn't that interesting! Anyone who analyzes data for a living knows there is something profoundly wrong with this distribution. The conclusion is inescapable. Hedge fund managers are inflating their returns to avoid reporting negative returns. The fact that they don't do so in the months prior to an audit suggests they know what they are doing is wrong.</p> <p class="blog_body">There are various ways hedge funds can inflate their returns. One approach is to outright lie. this is fraud, and a number of hedge fund managers have been prosecuted for it. Because hedge funds are unregulated, fraud is easy to hide. Unless you were born yesterday, you know there are more hedge funds engaged in fraud than the few that have made headlines.</p> <p class="blog_body">Hedge funds that trade illiquid instruments can easily inflate returns without resorting to actual fraud. There is a lot of subjectivity in how illiquid assets are marked to market. In the present credit crunch, the market for CDO's has pretty much dried up. Without market quotes to base their valuations on, hedge funds are free to value them pretty much any way they want. Bollen and Pool found that their discontinuity was most pronounced for hedge funds that trade illiquid instruments.</p> <p class="blog_body">Even if broker quotes are available for instruments, a hedge fund can cherry pick those broker quotes to bias their returns upward, so hedge fund managers can manipulate their returns even in modestly liquid markets.</p> <p class="blog_body">Bollen and Pool estimate that 10% of the individual monthly hedge fund returns they looked at were manipulated. They emphasize they were looking at one specific form of manipulation—small negative returns being inflated to make them positive. Other forms of manipulation—such as inflating a return to hit a hurdle rate or to outperform some benchmark, or inflating a return simply to make it bigger—may be just as common. Clearly, there is widespread deception in reported hedge fund returns.</p> <p class="blog_body">Another working paper released earlier this year took a different look at hedge fund data and came up with another disturbing result. Hedge fund monthly returns, it seems, spike each December. Authors Agarwal, Daniel and Naik analyzed monthly return data from four of the more prominent hedge fund data/propaganda disseminators: the Center for International Securities and Derivative Markets (CISDM), Hedge Fund Research (HFR), Morgan Stanley Capital International (MSCI), and Tremont Advisory Shareholder Services (TASS). They found that hedge funds report average monthly returns of 0.9% from January to November, but average returns spike to 2.5% in December. Reasons may include:</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">Hedge fund managers may inflate returns to boost their annual return for the year. This would enhance their typically 20% incentive fee. Think of it as writing themselves a Christmas bonus.</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">They may also want to boost their annual return because investors primarily look at annual returns when deciding whether to invest in a fund.</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">The December spike may also arise due to return smoothing. Hedge funds with less volatile returns have higher Sharpe ratios, so there is incentive to smooth returns. The authors found evidence that hedge funds were under-reporting returns early in the year and holding back the excess as a reserve to smooth out any poor returns later in the year. If they reached December without having to call on that reserve, the managers might recognize it then, thereby contributing to the December spike.</p> <p class="blog_body">December is always a peculiar month for equities. Taxable investors tend to sell losing stocks to realize the capital losses, which tends to drive those losing stocks still lower. Mutual fund managers often window dress their portfolios, buying up winning stocks so they can include them in the year-end portfolio they report to investors. It is easy to overstate the impact of such non-economic trading, since arbitrageurs intervene with offsetting trades. Also, the effects are mostly limited to the stock markets. The December spike Agarwal, Daniel and Naik discovered is more pronounced for hedge funds trading illiquid instruments, such as CDOs or emerging market debt.</p> <p class="blog_body">What is alarming about the December spike is its size. A 1.6% surge in monthly returns every December may not seem like much, but keep in mind it is an average across all hedge funds. Certainly not every hedge fund is inflating its December returns. If we assume one hedge fund in four is doing so, this would mean that, among hedge funds that do inflate their December returns, they do so on average by a whopping 6.4%. This is worth emphasizing. The authors' results are consistent with a situation where one hedge fund in four is inflating its December return by an average 6.4%, and this recurs every year. This is further compelling evidence of blatant and widespread manipulation of hedge funds' reported returns. Does anyone doubt that, with more time and more data, researchers would be able to uncover still more ways hedge funds are manipulating their reported returns?</p> <p class="blog_body">Now for even more bad news. The two papers I have been discussing describe ways hedge funds manipulate their own reported returns. Those reported returns are further manipulated by the data/propaganda disseminators who aggregate the data. Earlier researchers have documented a number of ways they can do so:</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10"><b>backfill bias:</b> When a hedge fund is added to an index, the fund's past performance may be "backfilled" into the index. For example, if the fund has been in business for two years at the time it is added to the index, past index values are adjusted for those two years to reflect the fund's performance during that period. Indexes generally require that hedge funds have achieved a certain size before they can be added. This pretty much assures only hedge funds with successful track records are added to—and hence backfilled into—an index.</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10"><b>survivorship bias:</b> When a fund is dropped from an index, past values of the index may be adjusted to remove that dropped fund's past data. Inevitably, a fund will be dropped from an index if it stops providing its performance data to the index provider, and a fund will be more likely to do so following poor performance than good. Also, providers may have criteria for dropping a fund, and this may naturally cause poor performers to be dropped more often than good performers.</p> <p class="blog_body"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10"><b>liquidation bias:</b> Due to their considerable leverage, hedge funds can fail suddenly. In the midst of such a calamity, the managers are going to have more important things on their minds than reporting their mounting losses to index providers. An index provider has little choice but to drop the fund from the index. They may go back and purge the index of that fund's past performance or they may not. Either way, the index will not reflect the fund's staggering losses.</p> <p class="blog_body">There was considerable controversy about these biases a few years ago, as unsavory practices were found to be widespread among the data/propaganda disseminators. Those disseminators complained that their hands were tied, that there was little they could do to eliminate the biases. They have since gone quiet on the issue. If you visit almost any data/propaganda disseminator's website today, you are likely to find little if any technical information clarifying how their indexes are compiled. The message is that the data/propaganda disseminators will compile the indexes as they choose, and prying researchers can't complain if they don't know the details.</p> <p class="blog_body">Three weeks ago, I took the <i>Wall Street Journal</i> and their reporter Gregory Zuckerman to task for reporting inflated, manipulated and smoothed hedge fund statistics as if they were factual. I explicitly challenged their claim that hedge funds have, overall, returned an average 10.5% so far this year. That ludicrous number came from one of the hedge fund data/propaganda disseminators, which the article cited. I contacted the <i>Journal</i> to make them aware that the numbers Zuckerman was reporting could not be trusted, and I provided a link to my blog posting. Last Friday, Zuckerman contributed to another <i>Journal</i> article in which he repeated the same bogus statistic, asserting that hedge funds have achieved a "roughly 10% gain" this year—and not even citing a source this time. I guess he is thumbing his nose at me. I don't care. I am as thick-skinned as a rhinoceros. If hubris is getting in the way of journalistic integrity, that is between Zuckerman and his editors at the <i>Wall Street Journal</i>. Millions of future pensioners and students, not to mention thousands of charities, are going to suffer because the pension plans, endowments and foundations they depend on are being lured into hedge funds by slick marketing lent credibility by journalists who either don't know or don't care. I can educate journalists, but I can't make them care.</p> <p class="blog_body">Returning to Benford's Law, I won't derive the result, since it is technical. It may seem counterintuitive, but most quants should spot why it must hold. If you would like to test your intuition, consider the following five forms of data:</p> <p class="blog_body" style="margin-top: 6px; margin-bottom: 6px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">the populations of towns</p> <p class="blog_body" style="margin-top: 6px; margin-bottom: 6px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">stock quotes</p> <p class="blog_body" style="margin-top: 6px; margin-bottom: 6px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">half-lives of nuclear isotopes</p> <p class="blog_body" style="margin-top: 6px; margin-bottom: 6px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">lottery numbers</p> <p class="blog_body" style="margin-top: 6px; margin-bottom: 6px"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/bullet_blog.gif" width="20" height="10">a tournament's tennis scores</p> <p class="blog_body">Benford's Law applies to four out of the five. Can you identify the one it doesn't apply to? If you understand what makes Benford's Law hold, the answer should be obvious.</p> <p class="blog_body" align="center"> Glyn A. Holton</p> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" height="37" id="table22"> <tr> <td height="35" width="100%"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" id="table23"> <tr> <td width="100%"> <p align="center"><map name="FPMap0"> <area target="5" href="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog_071210_post.htm" shape="rect" coords="0, 0, 149, 29"> </map> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/button_comment.gif" usemap="#FPMap4" width="150" height="30"><p align="center"> </td> </tr> </table> </td> </tr> <tr> <td width="100%" bgcolor="#CC9966" height="2"> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td width="0%" valign="top" bgcolor="#FFFFCC"> <table border="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" cellspacing="1" id="table24"> <tr> <td align="left" valign="top" bgcolor="#E8FFF0">  </td> </tr> </table> </td> </tr> </table> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table25"> <tr> <td bgcolor="#FFFFCC"> <table border="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="6" bgcolor="#E8FFF0" id="table26"> <tr> <td width="100%" bgcolor="#E8FFF0"> <p align="center" style="margin-top: 18"><map name="FPMap2"> <area target="_top" href="http://www.glynholton.com" shape="rect" coords="0, 0, 192, 76"> </map> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/logo_blog_green.gif" width="193" height="77" usemap="#FPMap2"></p> <p align="center"><span class="verdana10"> <a href="http://www.riskglossary.com/main/disclaimer.htm" target="_self"> Disclaimer</a></span></p> <p align="center">website: <a target="_top" href="http://www.contingencyanalysis.com"> http://www.contingencyanalysis.com</a><br> blog direct link: <a target="_top" href="http://www.glynholton.com"> http://www.glynholton.com</a><br> copyright © Contingency Analysis, 2006 - current<p align="center"> <map name="FPMap3"> <area target="_top" href="http://www.contingencyanalysis.com" shape="rect" coords="0, 0, 240, 88"> </map> <img border="0" src="file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/web_ca_blog_071210/blog/images/logo_ca_green.gif" width="241" height="89" usemap="#FPMap3"></td> </tr> </table> </td> </tr> </table> </td> <td>  </td> </tr> </table> </div>  </body>