My head hurts!

Do you ever get the feeling that you can't take one more email or survey or blog? When you feel like your drowning in a sea of information that you didn't even ask for? I know I have. In the past, I was an IT guy, an instructor and I worked for a department in the corporation where we were tasked with finding out new ways to handle data; from the simple to the complex. There were times when I was overwhelmed by it and had to shut down the computers and hide for awhile. Okay it was just a weekend, but…That was seven years ago, an eternity when it comes to the growth of available data and the programs to handle it. I couldn't possibly jump back into that sea, I would drown in an instant!

Anyway…the current issue of the Economist has a fascinating 14 page article on 'Data' and how it is being managed or not. All very revealing. Today's corporations and companies are going to drown in that same sea if they don't get onboard with data management. Even the smallest of enterprises needs to be fully aware of how this data flood can change their lives. And I haven't even mentioned how it affects our private lives…scary! Unless you are prepared mentally for it by being aware. Some odds and ends from the articles

 

The amount of digital information increases tenfold every five years. Moore's law, which the computer industry now takes for granted, says that the processing power and storage capacity of computer chips double or their prices halve roughly every 18 months. The software programs are getting better too. Edward Felten, a computer scientist at Princeton University, reckons that the improvements in the algorithms driving computer applications have played as important a part as Moore's law for decades.

Researchers at the University of California in San Diego (UCSD) examined the flow of data to American households. They found that in 2008 such households were bombarded with 3.6 zettabytes of information (or 34 gigabytes per person per day). The biggest data hogs were video games and television. In terms of bytes, written words are insignificant, amounting to less than 0.1% of the total. However, the amount of reading people do, previously in decline because of television, has almost tripled since 1980, thanks to all that text on the internet.

Nestlé, for example, sells more than 100,000 products in 200 countries, using 550,000 suppliers, but it was not using its huge buying power effectively because its databases were a mess. On examination, it found that of its 9m records of vendors, customers and materials around half were obsolete or duplicated, and of the remainder about one-third were inaccurate or incomplete.

…the New York Times a few years ago used cloud computing and Hadoop to convert over 400,000 scanned images from its archives, from 1851 to 1922. By harnessing the power of hundreds of computers, it was able to do the job in 36 hours.

Wal-Mart, a retail giant, handles more than 1m customer transactions every hour, feeding databases estimated at more than 2.5 petabytes—the equivalent of 167 times the books in America's Library of Congress