Saturday, November 30, 2013

More Predictioneers


Radinsky, along with her partner Eric Horvitz, co-director at Microsoft Research in Redmond, Washington, developed software that parses the web, seeking patterns — in news and historical archive sites for hints on patterns that have led to outbreaks of disease, deaths, and riots in the past – and comparing those patterns to current conditions. It’s a very sophisticated form of data mining, enabling deep analysis of disparate events and seeing how they repeat themselves time after time.

A paper published by Radinksy and Horvitz provides a good example: In 2012, Cuba suffered a major outbreak of cholera, its first in 130 years. Authorities there were totally unprepared to deal with the situation; according to news reports, doctors had declared states of emergency in numerous areas (although there was little official comment from the Cuban government).

But the software designed by Radinsky and Horovitz, their paper said, specifically pointed to the likelihood of a major cholera outbreak in the country. 2011 was a dry year for Cuba, but by mid-2012, rain returned to the country, with the above-average rainy season culminating with Hurricane Sandy in October of that year. The summer rains, and especially Sandy, caused major flooding in some parts of the country, and as the flooding increased, the cholera infection rates rose, the paper said.

While the events – drought, flooding, and cholera – seemed random, the software determined that it should have been expected. Searching 150 years of news reports and historical archives, the software determined a specific correlation between a drought state followed by major flooding, and a subsequent cholera outbreak, especially prominent in poor countries, where flood control was often substandard or non-existent. Weather researchers had long suspected a correlation between flooding and cholera, but it took the “prophecy software” designed by Radinsky and Horovitz to figure it out.

In another cholera example, the system would have predicted a major outbreak of cholera in Bangladesh in 1991, giving medical officials several days to prepare for it, the paper said. Not that the system is foolproof, the paper noted – but it has shown an accuracy rate of between 70% and 80%. That would be better than the 50/50 rate most of us can boast, and could help determine trends and events in many spheres.

Que padre. Que chido! 

Accuracy rate, especially for this kind of prediction, is very good. Congrads.