Not enough skilled Big Data practitioners
"Raw" and relatively immature technology
A lack of compelling business use case
No-one knows how to do it
We don't have good tools to do it
There's no good reason to do it
What do you expect from big data mining: easy-to-find gold nuggets of information from the dark pits of the data dumps? According to a recent report from the Wikibon consulting group, almost half of big data projects fail. It also found that the payoffs – using current technology – are not yet worth the costs of running the …
Exactly: a castle built on hot air. One wonders if the driving force behind the drive to do it in the first place wasn't some politician. They are experts at starting projects which require non existent technology to provide deliver star trek sourced results to nobody in particular.
See: Airport body scanners, biometric passports, SDI etc etc etc.
There still seems to be many cases of "if we build it, they will come" - mining any data that comes to hand in the hope that some third party will see value in it.
The bottom line has to be that a company has to be willing and able to take some clear action in response to big data insights. Knowing more about your business/customers does not in itself unlock value.
On the other hand, I've seen clients add 8 figure sums to their annual turnover by incorporating simple insights to their interactions. If a company can make that connection, from insight to action then it can see good returns on big data.
That's a failing of many businesses. They want to build something they can ride into the future, and they want to build it once and be done with it. They really believe they can build the ultimate tool and never have to work on it again. Business doesn't work like that or everyone would be doing it (successfully).
They're building something that absolutely will not be used in the way they predict. They use all their resources trying and are then unable to respond because they've built something so massive. By the time they've made changes the problem has evolved and they're still behind. Any business that tells you (x) is definitely going to happen because of something they've done/doing is full of shit.
Build small, adaptable and fast, otherwise you're going to be left sitting when the rules (which you don't control) change. Accept the fact you're always going to have to rework your systems and the IT department is there to stay. IT departments also have to realize that they're never going to be 'done' building. There will be no point where everyone sits around with nothing to do because everything is running so well. If that is the case then you should be spending that time polishing your resume. You're going to be job hunting when the business collapses.
Because of the 4th reason Wikibon did not list : managers jumping willy-nilly on the bandwagon to not look like they're behind the times.
Now they're just going to look stupid. Especially those who played the "I'm smarter than you all because I'm managing a Big Data project" card.
I have no pity for them either.
" In order to "overcome these obstacles", and derive "significant value" from big data, enterprises should consider engaging professional services organizations and/or cloud services. "
In actual fact, the ONLY way to derive significant value from *cough* Big Data is to have a compelling business case constructed from actual business scenarios derived from key business users on the ground who actually understand the business data itself. Far too many projects do not realise their benefits because none of this "difficult" but fundamental work is done; rather it's CIOs being sold "visions" of future business from the snake oil merchants at IBM, Oracle, SAP, Accenture et al - who are generally the root cause of the problem in the first place, and certainly won't understand the intricate nuances of the business data or operations to derive these insights during project ramp-up or design.
If a business sets off down this path with anything other than a compelling business case and realisation plan then it only has it's self to blame.
Isn't the operative phrase on the ROI "to date"?
How many of these projects have been out ther for a year? Let alone 3 years... Most cases don't make thier big paybacks in the first year or 2, normally year 3 will provide the bulk of the savings as the project is up and running and the user community mature.