BLOG

The Human Element

Big data is nothing without the right people and culture

 

It seems everywhere you look these days, you can feel the effect of Big Data analytics on our culture. From retail operations that offer customers curiously accurate coupons immediately after a purchase; to police departments that use crime statistics to place officers in riskier areas; Big Data Analytics has gone past early adoption and is now the mainstream. And it’s not surprising considering the velocity, variety and volume of data available. The vast collection of data floating around has only recently started to be tapped into, for the benefit of the company and consumer.

As the data explosion continues, companies are employing teams of data wizards to collect this data, write impressive algorithms to manage it, and create dynamic visualizations to interpret it. And the results have been amazing. Yet in the midst of this exhaustive data boom, there is a temptation to feel that the machines are doing all the work now and that humans are becoming obsolete.

But the truth is, without the human element in data analytics, all that data and algorithms are useless. We are taking a look at some of the crucial ways in which the human mind is the basis for all Big Data insight.

Dirty Data

Big Data analytics promises a utopia of insights and solutions to some of the world’s greatest problems. Whether it’s speeding up a manufacturing process to lower a company’s bottom line, exponentially increasing customers, or taking leaps in the fight against cancer, Big Data is often seen as the answer to everything. And yet the reality is so much different than that. Most data reserves are dirty; that is to say that they contain obsolete, inaccurate or missing information. Data that was collected before these analytics programs and machine-generated data were created therefore take significant investment by people to create dedicated mechanisms to clean, standardize and make worthy for analytics.

Steering the Truth

When uber-retailer Target famously started using data analytics to focus on the maternity market, many observers were stunned. It was brilliant, algorithms were established to single out certain purchases like unscented lotions and soap, supplements like calcium, magnesium and zinc, and hand sanitizers and washcloths. It is these products that are sought after when a woman typically is in her third trimester. Once they had the baby, it would be too late. This was seen as very interesting, even a little frightening. But once a father stormed into a store, complaining about why his teenage daughter was receiving coupons about pregnancy, Target had to entirely rethink their marketing approach, to avoid possible future lawsuits. So it’s up to humans to decipher what information should and should not be used, along with how to use it. The era of Big Data also happens to occur at the same time as the era of complete transparency; and if we’re not careful, privacy will be a thing of the past.

The Human Brain, an Insight Machine

Data analytics is an impressive science. But the interpretation of Big Data is an art. Giant sources of data are collected, cleaned and analyzed, usually put into some sort of vibrant visualization; a huge undertaking, and when finished, a sight to behold. But without interpretation, utterly useless. It is up to humans to provide insights into what the data actually means. When a platform processes a huge data set, and is left with a clear picture of this multitude of information, it is up to the people receiving it to make important inferences and discover why. When the why has been answered, the great thinkers in the company can take massive steps to shift its direction. In basic terms, technology sees the trees, humans see the forest.

 

New technology always instills fear into the old-fashioned. The sheer speed of change in the modern world is daunting. But the important thing to remember is that these technologies challenge us to find new ways of working and living. Technology doesn’t have to eliminate old roles and positions, it’s that these roles and positions must adapt to form a new value in the company. And it’s comforting to know that the great advances in Big Data could not exist without the one most important factor in tech, the human mind.