In my last post on big data advantages, I wrote about the potential impact of big data and the types of things companies are looking to get out of their information. However, only four percent of companies extract the full value of their information assets, while 43 percent “obtain little tangible benefit from their information,” according to PwC and Iron Mountain in their report “Seizing the information advantage.”
Four percent — ouch! You see, although the internet of things and big data analytics have paved the way to previously unimaginable possibilities, they have also opened a Pandora’s box of complexity. And that leads me to another famous quote from “The Big Lebowski.” “This is a very complicated case . . . You know, a lotta ins, a lotta outs, lotta what-have-yous.”
Why are so many enterprises struggling with finding their big data advantage? Well, there’s a lot of moving targets and dynamic plot lines. But a few things are clearly slowing companies down:
- Complexity is growing on all fronts. There are varied and increasingly difficult analytics needs: from batch and streaming ingest to machine learning and graph analytics workloads. Plus, there is an ever-increasing volume, variety and velocity of data to adapt to.
- The analytics tools landscape is overpopulated and rapidly changing. Many of these tools are open source. (How many of you had heard of Spark™ or Kafka two years ago?) The fungal sprawl in analytics tools and environments is significant.
- It’s extremely difficult to keep the right skill sets and infrastructure to transform these massive data collections into insights.
- Time to insight and decisions aren’t in pace with business needs. Workloads might be running too slow, you might have to wait for resources or perhaps it’s excessive data movement. There are many phases of and causes for slowdown, and they can be anywhere in the analytics pipeline. IT has tried to solve this challenge with proprietary and expensive tools, but had limited success, and with added silos and complexity.
Partly responsible for these struggles are the inadequacies of common approaches to big data platforms:
- Homegrown solutions that provide complete control and customization but can take up to a year to fully implement and invariably lead to cluster sprawl and management headaches;
- Big data appliances with fast time to value but with single-use focus and little flexibility for emergent technology; and
- Cloud solutions that eliminate upfront capital costs but are often more expensive after a few years – not to mention suffering from loss of data control.
But these hurdles can be overcome, and we believe an agile analytics environment is the best possible outcome and approach. I’ll dive into that a little more next time and talk about key factors needed for successfully seizing your big data advantage.
The post Big Data Advantage, part 2: “This is a very complicated case . . . You know, a lotta ins, a lotta outs, lotta what-have-yous.” appeared first on Cray Blog.