Home » Blog

Big Data, Big Problems, The Next Big Thing?

By on May 4, 2012No Comment

So much data to think about...

Slowly but surely, the corporate world is beginning to realize that the amount of information and data it generates is digging the enterprise into a hole.

It’s no surprise that the buzzword for 2012 is “Big Data,” a nebulous term that attempts to define parameters for mammoth stores of data that have become seemingly unmanageable with traditional tools.

We keep building systems that generate voluminous data. These systems include everything from information posted on social media sites, climatological information, digital images (think of the number of camera-equipped smartphones out there today), sales transactions, the list goes on and on.

Between people and machines, over 2.5 quintillion bytes of data are created daily (a quintillion is a one followed by 18 zeroes, just in case you weren’t sure).

If you check Wikipedia, you’ll learn that Big Data “consists of data sets that grow so large that they become awkward to work with using on-hand database management tools.”

Put more clearly, it’s too much of a good thing and the tools that we have simply aren’t up to the task. However, it seems that potential solutions are cropping up everywhere as is the effort to get the word out on Big Data.

You may not be aware but this is Big Data Week. If you didn’t know, you’re not alone.

Perhaps in anticipation of the festivities, Splunk, a company that offers such solutions and promises to help companies organize, listen to, and make sense of all of the information they have, rose 109% in value on its first day of trading after its IPO last week.

Also last week, perhaps as part of its ramp-up to celebrate Big Data Week, IBM announced plans to acquire Vivisimo, a search company that focuses on the federated discovery of structured and unstructured information.

The funny thing is that Big Data sounds a lot like other ideas and buzzwords we’ve seen come and go over the years. It’s closest to business intelligence and knowledge management but it promises to let you use all of your data, all the time, instead of just a sampling.

Big Data isn’t necessarily a solution, nor is having access to all of your information. Roughly a dozen years ago, the military ran tests of its Force XXI, a tool that would support complete battlefield tracking. What the military didn’t count on was the amount of Information Overload that would result from a system that provides “exquisite situational awareness.” Having more data didn’t do very much for the big brass. It was simply too much to swallow.

Around the same time, I wrote in this space about the problem that knowledge management had (and still has at this time), namely the fact that KM “doesn’t get much play in the general business press, much less attention by CEOs of major multinationals.”

I used Siemens’ CEO Heinrich von Pierer as an example of one of the few who actually did pay attention to KM, something that didn’t surprise me. Indeed, Werner Maly, a member of the Siemens management board, mentioned, in an address at a forum on Idea Management in 1998 in Frankfurt what he explained was not a new saying at Siemens: “If Siemens only knew what Siemens knew.” In a company with 60,000 engineers and scientists, the knowledge and know-how had to be somewhere. Maly asked, point blank: “How do we succeed in connecting that together?”

This is, ladies and gentlemen, what Big Data hopes to accomplish. Of course, it may succeed, but it may also take its place in a long line of attempts to connect knowledge, know-how, and information, an admirable but rather elusive goal.

Plus ça change, plus c’est la même chose.

Jonathan B. Spira is CEO and Chief Analyst at Basex and author of Overload! How Too Much Information Is Hazardous To Your Organization.

Leave a comment!

You must be logged in to post a comment.