I do not typically read Salon, but when they had an article on how Netflix is a pioneer with Big Data and entertainment I could not resist. The article focuses on the new Netflix series, “House of Cards” and how they used big data analytics to know they had a market for the show.

For at least a year, Netflix has been explicit about its plans to exploit its Big Data capabilities to influence its programming choices. “House of Cards” is one of the first major test cases of this Big Data-driven creative strategy. For almost a year, Netflix executives have told us that their detailed knowledge of Netflix subscriber viewing preferences clinched their decision to license a remake of the popular and critically well regarded 1990 BBC miniseries…the company committed $100 million for two 13-episode seasons.

Netflix has used their algorithms based on what people watch, give stars to, parts that are rewound and watched again and many other data “events”. Instead of find a focus group, the at home consumers are their focus groups and every action is recorded to be analyzed. They do not need to spend millions to find out what people want because that is already told to them based on what you search for, add to your queue and how you rate it.

Aside from the sources listed above, there is also data coming from third-party providers such as Nielson and many other sources. The author goes on to reference a zdnet article by Rachel King entitled, “Panel: Netflix, StubHub, IBM execs discuss value of big data.”

The sheer amount of data available to crunch is already phenomenal and is growing at an extraordinary rate. Last summer, at a panel discussion that included several significant players in the emerging Big Data universe, Michael Karasick, a V.P. at IBM Research, estimated that there is “a thousand exabytes of data on the planet anywhere.” An exabyte is one quintillion bytes, or one billion gigabytes. That’s a lot of ones and zeroes all by itself, but the mind-boggling part of the equation is that Karasick predicted that just two years from now there will be 9,000-10,000 exabytes of data on the planet.

The companies that figure out how to generate intelligence from that data will know more about us than we know ourselves, and will be able to craft techniques that push us toward where they want us to go, rather than where we would go by ourselves if left to our own devices.

Ironically, the article by Andrew Leonard is entitled, “How Netflix Is Turning Viewers Into Puppets”, but it almost seems like Netflix is becoming the puppet as viewers pull their strings based on the data that is received. Will the creative process still allow for thinking outside of the box or only produce what Big Data produces will succeed? The truth of the matter is that what is popular is not always good or innovative.

As information managers we face the following challenges:
•How do we ensure that redundant data is not being captured?
•Is data being captured in a timely manner with the proper metadata?
•Is data indexed so that it is searchable and accessible?
•Are proper retention and storage plans in place so Big Data overload does not take place?