I recently read two books on data crunching or analytics: “Competing on Analytics” by Thomas Davenport and “Super Crunchers” by Ian Ayres. I started reading these books with a bit of skepticism. This is most probably because it was after I read Nassim Nicolas Taleb’s “Fooled by Randomness” and “The Black Swan”, and Roger Lowenstein’s “When Genius Failed”, among others, in which I was so intrigued by the author’s view of the world that it is full of asymmetries and the highly improbable as well as the spectacular fall/implosion of the quantitative investment strategies taken by LTCM in 1998. Coupled with these are my own experiences in my professional career to date as well as an amateur quasi-quant trader, in both of which I come across many “black swans”/luck/the improbable whatever you call it, including both good and bad.
Professors Davenport and Ayres both addresses basically that there are a number of areas where data crunching can produce lots of insights as well as fairly accurate forecasts against which human brains usually cannot match, and the both books are chock full of real-world examples, with “Competing on Analytics” centering mostly on business with focus on how to build a company with competitive advantage capitalizing on analytics capabilities, while the “Super Crunchers” encompassing broader range examples in legal and governmental areas with focus on statistical methodologies (i.e. randomizations and regressions) used in crunching data. Upon reading these books, I personally found the Super Crunching more interesting due to the width of areas covered and the methodologies explained, but was pleasantly surprised by the fact mentioned in the two books that there are so many real life situations, where data crunching plays a pivotal role in making business decisions and driving institutional policies, and my skepticism about the validity of data crunching based on statistical methodologies are largely gone….but not entirely.
When data is available for crunching and that data has more or less a normal tail (i.e. not too many outliers), relying on the results of crunching should provide reasonably good forecasts/predictions and would be of great use. However, I still tend to believe that investment is a different animal to which super crunching may or may not work effectively. This notion comes from my observations that quant hedge funds collapse here and there when the market acts capriciously or atrociously, as well as from my experience in investment where I have been using extensive statistical modeling but have had quasi-implosions here and there…… The correlation suddenly gets close to almost 1.0 when the market volatility roars. And what’s significant here is that it is not just counts/frequency of outliers that is relevant but it is more the amount involved that should be kept in mind, as the impact is frequency times the amount of money involved. Imagine the consequences when a high degree of leverage is involved. Surprise! …and you are out. One can say that it does not mean the super crunching does not work in investment modeling but indicates that the modeling is deficient or not robust enough to deal with the unexpected market volatility, and/or coupled with inadequate risk management such as too aggressive leverage. Well, that may well be the case…. Just wondering if there are any quant-funds that have sustained, say, for 15 to 20 years through market volatilities and continue to produce stellar returns….?
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment