Big data’s dark side
The big data backlash started brewing a long time ago – it’s inevitable (and often justifiable) for any big buzzword – and while numerous analysts pointed out the potential for big data to be misused in certain contexts, few predicted it would so swiftly be caught in the crossfire of major international news about how corporations collect data and how governments harvest that data.
Right before the PRISM story broke, I happened to be finishing Viktor Mayer-Schönberger and Kenneth Cukier’s new book, Big Data: A Revolution That Will Transform How We Live, Work, and Think. After taking readers through a tour of how big data is leveraged by powerful brands like Google and Amazon on a daily basis, as well as creative ways that big data is already quietly improving our daily lives (like predicting and preventing major manhole disasters in Manhattan), the authors take a turn at the finish line and talk about the potential dark side of big data, focusing on three major areas.
The notion of opting out of sharing your data has become almost fanciful. Your data is collected, packaged, repackaged, sold and scrutinized – and often probably neglected by entities that may be unaware they possess it or just don’t know what to do with it. In a previous post on big data serving as a bridge between CMOs and CIOs, we talked about the potential for brands to harness big data to serve customers in new ways, rather than just shove ads in faces in new ways.
Here’s where we take a shadowy turn. “Penalties based on propensities” is basically the idea behind Minority Report, except instead of mutants or psychics, it’s data that’s seeing into the future. The Big Data authors note the possibility of “using big-data predictions about people to judge and punish them even before they’ve acted.”
Propensity doesn’t have to pertain to something dramatic like arresting people before they commit murders they haven’t yet planned – and it’s also just as applicable to scenarios in which you willingly share your data. If you’re in a hyper-transparent company that shares salary information and sleep habits, for instance, big data will eventually shine a light on propensity. Employees who get eight hours of sleep may be more likely to be productive than those who get four hours of sleep (hopefully the data would lead to some less obvious findings). What big data doesn’t do particularly well is predict exceptions and outliers.
For more – a lot more – on the limitations of big data’s prophetic capabilities, another worthy addition to your bookshelf is Nate Silver’s The Signal and the Noise.
Big data dictatorships
“In addition to privacy and propensity, there is a third danger,” the authors warn. “We risk falling victim to a dictatorship of data, whereby we fetishize the information, the output of our analyses, and end up misusing it. Handled responsibly, big data is a useful tool of rational decision-making.”
A useful tool – precisely. Big data is shiny and new, and can be a powerful tool when wielded correctly, but that doesn’t mean you burn down the rest of the tool shed. As Steve Hall points out, predictive analytics shouldn’t be a substitute for risk or old-fashioned gut instincts.
“The real revolution is not in the machines that calculate data but in data itself and how we use it,” Big Data‘s authors write. “A gold mine isn’t worth anything if you can’t extract the gold.” The miners in your company are poised to have real power, and deservedly so. Don’t let them become data dictators.
A warning to mid-sized companies
Mayer-Schönberger and Cukier predict that big data will wind up being bad news in many sectors for mid-sized companies hoping to remain mid-sized. “Big data squeezes the middle of an industry, pushing firms to be very large, or small and quick, or dead. Many traditional sectors will eventually be recast as big-data ones, from financial services to pharmaceuticals to manufacturing.”