The New York Time recently called Big Data an important tool for analyzing data, but no silver bullet.  Here are eight reasons why from their article, “Eight (No, Nine!) Problems With Big Data)”

The first thing to note is that although big data is very good at detecting correlations, especially subtle correlations that an analysis of smaller data sets might miss, it never tells us which correlations are meaningful…

Second, big data can work well as an adjunct to scientific inquiry but rarely succeeds as a wholesale replacement…

Third, many tools that are based on big data can be easily gamed…

Fourth, even when the results of a big data analysis aren’t intentionally gamed, they often turn out to be less robust than they initially seem…

A fifth concern might be called the echo-chamber effect, which also stems from the fact that much of big data comes from the web. Whenever the source of information for a big data analysis is itself a product of big data, opportunities for vicious cycles abound…

A sixth worry is the risk of too many correlations. If you look 100 times for correlations between two variables, you risk finding, purely by chance, about five bogus correlations that appear statistically significant — even though there is no actual meaningful connection between the variables…

Seventh, big data is prone to giving scientific-sounding solutions to hopelessly imprecise questions…

FINALLY, big data is at its best when analyzing things that are extremely common, but often falls short when analyzing things that are less common.

For more information on each point I’d recommend clicking on the link to the article.

Advertisements