Buzzwords Defined: Big Data

At the dawn of the information age, the term “big data” is bandied about by everyone from media luminaries and consultants to tech CEOs and government officials. Following earlier Buzzwords Defined posts on prototyping and human-centered design, this post demystifies big data and highlights the opportunities that big data presents.

Contrary to popular belief, the biggest challenge with big data is not technological, but social. Relying on technical solutions like machine learning, natural language processing, and artificial intelligence will only go so far in the information age.

With increasing amounts of data and processing power, the greatest advances in managing big data will be in creating human organizations that can think critically and creatively at every level, from entry-level analyst to CEO.

Processing massive amounts of information in clever ways is not enough to ensure success; human systems need to modernize just as much as machine systems do.

Definition

Big data isn’t “big” just because there’s a lot of it. Big data is defined by three factors (the three “Vs”): volume, variety, and velocity.

• Volume: Yes, big data involves large amounts of data, usually at a volume that is effectively impossible to store on a local computer. How data is stored also matters, though, whether it’s on the cloud or on the premises (“on-prem”), since this impacts how data scientists and other users can access and analyze it.
• Variety: Big data also involves a variety of structured data (in traditional spreadsheet form, such as sales figures or case processing data) and unstructured data (such as social media posts, video files, or news articles). These datasets might also be related to one another in a variety of ways; data scientists often find themselves developing creative means to clean and combine datasets to produce useful insights.
• Velocity: Big data relies on combining and analyzing data at speeds that allow it to influence decision-making or streamline business processes, especially in real-time. In order to be useful, there must be ways to make big data accessible in a way that matters—the right data, to the right people, at the right time.

Opportunities

We operate in a data-rich world, but this doesn’t mean that we are informed. To turn data (a collection of facts) into information requires the coordination of human and machine systems. This is the key to structuring and organizing data, and it is an opportunity for organizations to stay on the leading edge of the information age. Organizations that store and process the right information efficiently hold a competitive advantage over organizations that don’t.

To succeed in the information age, leaders need to go beyond outsourcing data-driven thinking to the “data folks.” Leaders need to be involved because the structure and culture of their organizations determine the extent to which they can exploit big data for their advantage.

Consider some ways that leaders can influence the human organizations they lead to exploit big data:

• Data Standards and Governance: It is not enough to declare how data should be managed. It takes leadership at the everyday level to create the human behavior necessary to maintain these standards. This is the only way for leaders to have confidence that their data is telling them what they think it’s telling them.
• Data Cataloging: At least as important as the standards and governance that produce data quality is knowing what data is on-hand. This is especially important when dealing with a wide variety of data, from structured to unstructured. Data cataloging is also important for assessing different levels of quality in data, and maintaining a data catalog is a human leadership challenge.
• Aligning Structured and Unstructured Data: Even the most clever computer algorithms can’t align structured and unstructured data without some level of human intervention (either at the coding or the cleaning and merging stage). Leaders can provide the human resources in order to make these important links a reality.
• Incubating Innovation: Setting the conditions for innovation is an inherently human activity, and since human systems are complex leaders can’t rely on the newest software or systems to make innovation a reality. Innovation requires leaders to provide clear incentives for creativity and to underwrite failure, because the key to innovation is being able to fail.

Most organizations sit on a veritable gold mine of data, but turning data into a competitive advantage often requires reorganizing both human and technological systems. The Clearing specializes in human-centered design, organizational transformation, and building cultures that engender success. If that success is to exploit the opportunities inherent in big data, then it will have develop more modern human systems.