Analytic systems share system limits with financial markets.
Correspondent Lew G. recently sent me a thought-provoking commentary on the limits of "total information awareness" in terms of any information system's intrinsic rate of generating false positives.
In essence, the rate of false positives limits the effectiveness of any predictive system. The process of attempting to eliminate false positives is inherently one of diminishing return: even with no expense spared, the effort to eliminate false positives runs into boundaries of signal noise and generation of false positives.
To the degree that financial markets are ultimately predictive systems, this suggests a systemic cause of "unexpected" market crashes: signal noise and the intrinsic generation of false positives lead to a false sense of confidence in the system's stability and its ability to predict continued stability. ...