The GigaOm Structure: Big Data 2011 conference on Wednesday in New York brought together top experts in their fields to discuss the opportunities presented by society's growing "data warehouses." As Google's Eric Schmidt has noted in the past, "Every two days now we create as much information as we did from the dawn of civilization up to 2003." What are companies doing with all this data? Some have developed new businesses from analyzing and selling data to third-parties. Others are working on processing and moving big data in real-time.
Jeff Jonas, Distinguished Engineer, IBM, spoke about how large amounts of data can be used to make better predictions, including things like human behavior. His most recent work is a "big data analytic sense-making" engine. He emphasized the use of context to increase understanding of data, which could be leveraged to produce better results than algorithms alone.
NetaApp Founder and EVP Dave Hitz spoke about "big analytics" might be a better term to describe the current state of the large data environment. While his company is about storage, he said that the point of having all the data is discovering meaningful trends in the data, not simply storing it.
The audience for the one-day event filled the room at the Chelsea Piers venue in New York City. GigaOm founder Om Malik and his team interviewed speakers, introduced panelists and read audience Tweets.
Jaidev Shergill, of CEO and Founder of Bundle, a company spun off from Citibank, discussed how anonymized credit card and financial transactions could be mined for trends in how consumers really spend money, such as on restaurants. Self-reported reviews are fraught with bias, he said. Analyzing how much and how often customers spend on dining can lead to new insights and recommendations based on financial data.
More Scenes from GigaOm's Structure: Big Data 2011