Big Time for Big Data

Data overload is not a new problem. The prophet Sting was singing about it years ago: "Too much information, running through my brain... driving me insane." Duran Duran also lamented this condition.

What is new is the massive tons and tons of unstructured data, from all sorts of digital sources – or what people are now calling Big Data – and the belief that all this information is like gold. More important are tools arriving on the scene to help process it and analyze it and, in the hands of someone smart and lucky, monetize it. These challenges present new opportunities for IT service providers and BPO companies that invest in learning how to use these new tools.

As Joe Cordo, CMO of Extraprise, sums it up, there are three big drivers right now:

• Data explosion, online and offline, at a nearly 1,000% increase in the last five years, according to an IDC report
• Social media and mobile communications
• Cloud computing and associated technologies, like sales force automation and "listening platforms"

Visualization software has been one way to get a grip on gigantic data sets and understand connections, but even more powerful systems are available now. Many of them are based on the open source Apache Hadoop framework. Then there are tools that integrate with Hadoop but go beyond it, such as Splunk – described here in a great conversation about Big Data between CIO Doug Harr and Harvey Nash analyst Anna Frazzetto.

And just this week, in fact, EMC officially announced the Greenplum Unified Analytics Platform. As the company describes it, UAP combines "co-processing of structured and unstructured data with a productivity engine and social network." EMC says the big deal here is that it enables businesses – or the "data science teams" they've outsourced the work to – to manage both structured and unstructured information.

One early customer sums it up like this: "It is the fusing of these multiple types of data at large scale that is where the magic lies to drive innovation and agility in business today."

There are other tools out there as well, and no doubt there will be lots more as the Big Data train picks up even more steam. The important thing for outsourcing companies is to put the time and money necessary into these tools, and hiring people who know how to use them. We often hear sourcing experts talk about nearshore IT companies focusing on higher-value services and innovative solutions rather than doing the dull, process-oriented tasks that lower-cost locations dominate – and becoming masters of these tools is a good example.

As Dan Berthiaume of BPO Outcomes points out, "while some companies may possess these tools and the expertise to use them in-house, most probably do not." You don't have to sort through a mountain of data to see the opportunity there.