Big data is not just about huge volumes; it is about a variety of data delivered at different speeds. As a result, there is a need to consolidate different data sources located all over the organisation.
“This can be a daunting challenge,” says Michael de Andrade, CEO of information solutions specialist, EnterpriseWorx. “To gain a competitive advantage, businesses need to bring together big transactional data and interactive data and analyse them in an integrated fashion. Without a strategy in place, they increase the risk of disruption and unforeseen costs.”
According to the Financial Times, 90% of stored data globally has been created in the past two years. “If you know what you want to do with the data, you won’t be tempted to follow the trend of storing every piece of data for the sake of doing so,” says De Andrade. “It takes time, effort and money to upgrade the data centre with new technology. You need wins out of that.
“Traditional relational databases don’t have the power to address large sets of data. The combination of structured data, such as bank transactions and point-of-sale information with unstructured sources such as e-mail, blogs and electronic sensors makes big data difficult to interpret using traditional databases and data analytics.
“Businesses need to take advantage of the power of new technology and lower costs to improve on the output of their data for better decision making. With the right technology and infrastructure, it’s possible to analyse the entire dataset and not just a portion of it. Unless you analyse the data you are incurring additional costs for no gain. The in-memory capabilities of data warehouse appliances like Kognitio, and BI programmes like QlikView allow train-of-thought analysis.”
According to a recent Data Warehouse Institute survey, 61% of organisations believe implementing big data analytics will allow more targeted social influencer marketing. Some 45% see benefits from more accurate business insights.
“Companies shouldn’t disregard the benefits of structured data analysis and the intellectual property they’ve built over many years,” says De Andrade. There’s a lot of value in structured data. Now it’s necessary to identify the value in unstructured data. Management needs to develop a strategy and consider adopting new techniques, such as Hadoop, or taking a hybrid approach. In that way, it’s possible to integrate traditional transactional data with unstructured data.
“Relevant nuggets of information can be extracted using a mature data warehouse appliance like Kognitio with its in-memory data store running on top of Hadoop’s open source software framework. Both support data-intensive applications and petabytes of data. The data can then be analysed using advanced analytics tools.”
According to Kognitio chief technology officer Roger Gaskell, the market is coming to Kognitio for its in-memory analytics platform, not just for data warehousing. “The traditional method of sucking in data into Excel and building OLAP cubes is highly data-intensive,” he says. “Kognitio is the perfect analytical accelerator to Hadoop or to an enterprise data warehouse. It seems that this is the way the market will move over the next two to five years.