By Jeff Weidauer
Feb. 2, 2012
If you’re a marketer and you haven’t heard about big data yet, you will soon. What’s big data? The broadest definition is a dataset that’s too large for typical database software tools to manage, store or analyze.
Most of us are familiar with terms like megabytes and terabytes; big data introduces new terms: petabytes and zettabytes. What’s the difference? The first iPod held about 10 gigabytes of data and was touted as holding “a thousand songs in your pocket,” or roughly 50 hours of music. A terabyte is about 200 hours of music, and a petabyte is about 280,000 hours, or 32 years of music.
In short, the term big data is a simple explanation for quantities of information that are virtually unfathomable for humans. But as storage and processing power get cheaper, we store more and more information knowing we can tap into whatever we need quickly. Today much of that storage is moving out of our offices and homes and into the so-called “cloud”—remote data storage someone hosts on our behalf.
Worldwide data is currently estimated at 1.5 zettabytes (a 1 followed by 21 zeroes), and is projected to reach 35 zettabytes by 2020. The growth of social media is adding to this store of information every minute, as every post and tweet is captured and filed away.
There are two major types of data being stored: one is the social media-driven portion, which is commonly referred to as “unstructured.” The other is what we normally think of when it comes to data: structured forms that are created and managed in relational database tools. Both of these types are growing at unprecedented rates, and the growth will only continue to accelerate.
For marketers, there are both opportunities and challenges in big data. The first challenge is to determine what to do with it; big data can’t be ignored, and some estimates peg margin enhancement at over 50 percent with proper use of big data. This goes beyond the usual data mining techniques though; specialized algorithms and analysis tools need to be built. More importantly, specialized skill sets will be required to build and manage those tools.
This leads to an even greater concern: who will write the programs to mine these massive data sets for actionable insights? One can assume that this is an IT problem, but there is also the question of being able to take those insights and know how to act on them. That is definitely a marketing problem, and those few with the required skill sets are quickly being snapped up by forward-looking companies.