The success from Big Data and data analytics initiatives for a lot of traditional companies has been restricted to only small parts of their business. A huge impact through big data projects is something that not many have been able to achieve. Gartner predicted that only 20% of analytic insights will deliver business outcomes through 2022. A number of factors like strategy, competence, lack of resources and ignoring the details of analytics, among many others, lead to problems in big data implementation and sometimes to its failure.
Big Data and advanced analytics are indispensable to business processes. Companies, both laggards and innovators, are implementing various big data projects for a variety of use cases including marketing, 360-degree view of the customer, sales, price optimization, security intelligence, and more. The need to integrate and analyze huge amounts of data from multiple sources and drive business decision-making is becoming key to staying competitive, improving customer experience, increasing marketing ROI and reducing operational costs.
Leaders of the organisation should envision a powerful roadmap to implement big data, clearly lay down its purpose and take decisive actions to lead the entire organisation in successfully incorporating big data. In this article, we outline the 9 common pitfalls that affect the implementation of big data projects and tell you how you could avoid them.
The 9 Big Data Mistakes And How To Avoid Them
1. Not Identifying The Right Business Use Case
Not identifying the right business use case and establishing clear success criteria and KPIs is one of the most common reasons most big data implementations fail. In order to capture a viable use case, Chief Analytics Officers can consider the following actions:
- Work with different business and functional leaders to identify their challenges and goals
- After the initial brainstorming develop a list of 30-40 use cases with relevant KPIs
- Select the most relevant use case based on
- The use cases’ viability for an analytics project
- The complexity of analytics and data required
- Ease of implementation
- Potential ROI that can be achieved
2. Not Taking A Thorough Technology And Business Readiness Assessment
Before starting a big data project, organizations often don’t look at answering critical prerequisite questions like:
- How will the big data align with our current technology environment?
- Where and how to re-tool?
- Do we have the sufficient technology infrastructure?
- What are the training and staffing requirements needed?
- Are our existing business intelligence processes flexible enough to support the project?
In addition to these, organizations should also look into their current data quality. The reliability of data analytics output begins with data quality. Ramifications of poor data quality can be detrimental to organisation. A Gartner’s study reveals that poor data quality costs companies $15 million annually. Advanced data integration tools that are beneficial for big data projects need structured and cleaned data. With poor data quality these systems will prove ineffective leading to manually cleaning data which will be time intensive, cumbersome and unreliable.
Organizations embarking on the big data journey for the first time can consider partnering with a big data consulting company for a thorough assessment of their current technology, processes and people.
3. Big Bang Or Very Conservative Implementations
Business leaders many-a-times make the leap to implement big data for a number of use cases at the same time as a part of their expansion plan without fully grasping the potential benefits. Though some initiatives might be beneficial, others however fall short. This leads to an incomplete understanding about the benefits that can be yielded from Big Data. Such an approach to implement Big Data reflects that organization might not have evaluated the ramifications, especially when procuring infrastructure.
There are however, some companies that take a low-risk, low-return approach by implementing very few initiatives in their pilot project to test the viability for bigger investment. Such a conservative approach to execute a pilot in isolation means that the organization is unsure about potential benefits of Big Data. These approaches represent two ends of the spectrum of ineffective approaches to implement big data without evaluating the actual benefits.
It’s recommended that organizations start their big data journey with a minimum viable program (MVP). Here’s why:
- With an MVP, stakeholders can quickly assess the viability of a use case, understand the potential challenges and risks with the project and get a better estimate of the potential ROI
- Stakeholders can perform A/B testing and evaluate different analytics models in parallel
- Once the MVP is deemed fit, improvements can be made to the program and the model’s usage can be expanded
4. Inaccurate estimation of time and costs
Accurately estimating the time and cost of implementing big data is crucial to businesses. Many big data projects model their time and cost based on the initial use case, however, depending on the methodology used during implementation, use cases can have a different cost model. Organizations also overlook the fact that the initial use case is usually a low-hanging fruit.
Organisation should therefore estimate their cost based on the architecture of their plan as it provides a better visibility of costs that can be incurred during implementation.
5. Compromising Data Security For Innovation
Mitigating risks is an essential aspect while bringing in innovative big data solutions. As data is invaluable to enterprises, protecting it becomes imperative and non-negotiable.
Organisations should have a deep understanding of their data and audit them regularly to check for data consistency. Providing access to users should be done with great scrutiny and data security needs to be devised by a unified system of controls.
6. Ignoring Data Governance
Data governance lays the foundation to manage data and brings in reliability to organisations’ data. Data governance establishes responsibilities, develops robust processes to ensure quality in data and lays down guidelines to build data according to the organisation standards.
Without an effective data governance, enterprises not only fail in their new initiatives, but also pose a threat to the existing analytical programs. Absence of data governance weakens the integrity of data that the organisation holds and also hampers the results of data analytics.
7. Improving Stakeholder awareness
The success and expansion of big data analytics usage within your organization depends on engaging and getting the buy-in from cross-functional stakeholders. If you’re an analytics executive, you must strive to educate your leadership team and functional stakeholders about the benefits being generated by the big data initiative. Regular reporting about the project is key to ensure transparency and expanding the business use cases.
8. Data Silos and Ineffective Use Of Data
Access to data has become easier than ever and companies hoard expansive amounts but seldom is this data used to extract insights to aid business goals.
Organisations should resolve obstacles that allow them to judiciously use data than have it stored in a silo.
9. Emphasis on Technology rather than the Business Requirement
Although infrastructure is important for big data analytics, what actually drives the infrastructure is the business requirement. Accumulating technology that is not centered around the business objectives can be another pitfall.
Achieving business outcomes should be the prime focus of IT leaders as this drives the business context for the data acquired and also ensures that IT delivers what business really needs. Organizations need to consequently build the technology around these business outcomes in order to optimize the spend on technology needed to support the Big Data initiative.
Conclusion
The potential in Big Data and its benefits have been accepted, yet not many achieve it. Organisations need to have an assessment of how the big data initiatives might fail and need to approach the implementation with robust strategies that mitigate the risks involved.
Although it might be common for organisations to stumble upon the pitfalls of big data initiatives as a natural course of their implementation, devising ways to overcome these challenges is what differentiates a successful implementation from the ones that don’t make it.
