Big data holds big challenges for 2012
As 2012 rolls in amid continued economic uncertainty, businesses will have to work smarter to achieve sustainability and growth. One way of succeeding is to extract more value from data through business intelligence (BI) and analytics. This poses a huge challenge to the organisation, particularly in face of the growth of ‘big data’.
“Last year saw the beginning of the ‘democratisation’ of data with more tools becoming available to assist companies with the process of data discovery,” says Sean Paine, COO of information solutions specialist, EnterpriseWorx. “The exclusive use of information is being taken away from analysts and being vested in the hands of ordinary people via smart devices. This trend will continue to gain momentum.
“Cloud computing will remain as a dominant theme and this year will see tangible evidence of the move to BI in the cloud. We don’t have consistent high bandwidth in this country, but success will go to those who act now to secure their place in the South African cloud computing environment.
“However, let’s not get confused by all the hype about the cloud and mobile devices. The cloud is an IT construct, a technology for information storage, while the smart phone is just another device for viewing and transmitting information. And of course, cloud computing makes BI accessible to business people via mobile devices. But these are just tools. What’s important is how the business deploys these tools.”
Paine argues that success in maximising the benefits of BI, data discovery and analytics is threatened because data management and data warehousing are lagging behind, despite their key roles as the foundation of the BI pyramid.
“The exponential increase in data means an endless supply of information to analyse,” he says. “However it’s not just about the volume of data, it’s about the complexity and variety – text, voice, image, video, multi-media.
“The technology is available. Multi-core, big-memory servers and cloud-based analytical services make it possible to crunch huge volumes of data. Leading database analytical databases and data warehouse appliances like Kognitio WX2 that can handle terabytes of data in seconds.“
The question is: How do you even begin to get value from the data?
“There’s the issue of complexity – just how many disparate data types do you have? Where are they located in your business, perhaps across multiple systems? And what do you want to get from your data? A marketing professional will want very different information from what the CFO needs. Then you have to think about speed – how quickly do you need to get to your data, and how fast does your business need to get the insight in order to remain competitive?
“We need to determine what data is worth keeping or analysing and what is not that useful. Take video surveillance or voice recordings of interactions with clients – should we be keeping it and for how long? And – finally – how much is the business prepared to invest in gleaning profound insights from its data?
”Paine believes there is a compelling case for management to define and refine its data environment and implement sound data management strategies, so as to be able to handle the challenges of the expanding data spectrum.
The Data Warehouse Institute provides some trends driving the development of comprehensive data strategies:
Advanced analytics are pushing the limits of BI and online analytical processing (OLAP) systems, and data strategies are needed to support these trends.
Data strategies must extend beyond the organisation to include social media. Social media data is being generated by customers in real time and offer a mine of intelligence for marketing and product and service development. Companies need to integrate big data from a variety of sources into a multi-terabyte data warehouse.
Governance and regulatory demands are putting pressure on organisations to know where sensitive data is stored and who is using them. Organisations need to develop governance, data quality and metadata capabilities to address this.
“Organisations need to renew their data strategy to keep up with big data dynamics, changing technology and user requirements, says Paine. “At the same time, the data strategy must be aligned with the strategic objectives of the business.”