Big data - requires cultural changes for success

Much has been written lately about ‘big data’ being the next big thing in business. There is no doubt that the explosion in data being collected by organisations opens opportunities for exploitation, especially if you can exploit it quicker than your competitors. However just deciding to trap and analyse 100 terabytes of data does not make an organisation expert at ‘big data’ or guarantee any competitive insight will result. In many companies the only real experience with large quantities of data and its use is through the finance team. The problem with the accountants is that they treat data in a ritualistic way (remember the comedy sketches of accountants making sacrifices to the god of balanced books?) so that vast quantities of the data they tend has little relevance to the organisation. To a lesser extent marketing also use data extensively though often those number are externally summarised so the nitty gritty of data management is avoided. Also most people are very weary of data and statistics, the political misuse of statistics has a lot to answer for.

All of this means that without considerable effort an organisational initiative to hop on the “big data” bandwagon is doomed to failure. Having spent 6 years of my life championing data management and analysis in a large multinational organisation here are a few suggestions to improve the chance of any ‘big data’ project:

1. Reverence for data – everything else in this list really reinforces this point. Data needs to be taken seriously to be effective. Discounting it should not be done lightly and there should be good reasons for ignoring data.
2. Culture of data accuracy – “garbage in - garbage out” is as relevant today as it ever was. Big data will only ever be useful when everyone involved in its collection puts a priority on it and is fastidiousness in ensuring quality, accuracy and timeliness of data inputs.
3. Trust the data – many a plane has crashed because the pilot decided that the instruments were wrong and they knew better. Once the systems have been set up and data validated it should be believed, nothing erodes the value of data quicker than discounting it because it tells us something we don’t want to believe.
4. Don’t shoot the messenger – when profit is sinking do you blame the finance team for recording such poor results? Data is often used as a scapegoat when times are tough, it is nothing more than a tool.

Effective use of data therefore, is dependent on a positive organisational ‘mind set’ towards the value that the data can bring. All the latest analytic tools, consultants and trends will not mean a thing, indeed will be a big waste of money unless the organisation is willing to change its mindset and accept the value of data.

David Gwillim
Exploring the value of IT to organisations
email: david.gwillim@optusnet.com.au
blog: http://www.businessitvalue.blogspot.com/