Friday 3 June 2011

Move over Big Data

Yeah I have got to say it, I am over 'Big Data' - Over. It.

I've heard the stories of 1 zetabyte of data being created every nanosecond at the intersection of business, social and location-based services. Still, aren't we missing the point?

'Big Data' and the second and third comings of IBM hadooping everything (yes it is now officially a verb) is being hailed as the next opportunity for business advantage but we still need to redux that data to identify the critical insight that will generate business value from it. Most analytic players today are living in a world of Analytics 1.0, its now remiss of us if we don't invent Analytics 2.0

Analytics 2.0 is the ability to spot patterns or outliers in transactional, behavioural and contextual data at lightning speed with high agility to make a change in your business and out-smart the competition.

The critical thing about this, in my mind, is that the window of opportunity to act on this information is short and decreases in value the longer it takes to capitalize on it. If you go to St Louis or Boston you can find High Frequency Trading companies that buy and sell large quantities of stock on the NYSE. The average time they hold that stock is seven seconds! At that frequency it's not about predicting outcomes its about jumping on outliers and gaining advantage from them

The way we are looking at this today is by capturing and analyzing data from potentially an unlimited number of sources. Some of that is highly structured from old-school databases like sales and customer profiles but augmented with live click-streams from websites and also Facebook feeds which are completely unstructured.

The trick and the opportunity is to conduct the analysis iteratively and act on it immediately rather than at predefined points in time. That phenomenon will kill the traditional ways of working and the legions of smart people who work in boutique agency shops unless they get with the 2.0 program

2 comments:

Scott Hoffman said...

Data, Data everywhere - but you can't drink data to survive - you have to use it to find the water! Data + Insights = Revenue - great piece!

everheardofaspacebar said...

I would argue that there seems to be a paradox in your post, in the sense that to be able to act on outliers, etc. the "big data" infrastructure (or something very like it) is a necessary to be able to do the analysis on top of it.

e.g. with hft, these co's aren't using ms excel and an access database.

look at the likes of zynga too ($10bn valuation, $1bn pa rev) - http://venturebeat.com/2010/09/27/zyngas-data-junkie-leaders-aim-to-level-up-game-design-for-hit-social-games/