In their quest to extract insights from the massive amounts of data now available from internal and external sources, many companies are spending heavily on IT tools and hiring data scientists. Yet most are struggling to achieve a worthwhile return. That’s because they treat their big data and analytics projects the same way they treat all IT projects, not realizing that the two are completely different animals.
The conventional approach to an IT project, such as the installation of an ERP or a CRM system, focuses on building and deploying the technology on time, to plan, and within budget. The information requirements and technology specifications are established up front, at the design stage, when processes are being reengineered. Despite the horror stories we’ve all heard, this approach works fine if the goal is to improve business processes and if companies manage the resulting organizational change effectively.
But we have seen time and again that even when such projects improve efficiency, lower costs, and increase productivity, executives are still dissatisfied. The reason: Once the system goes live, no one pays any attention to figuring out how to use the information it generates to make better decisions or gain deeper—and perhaps unanticipated—insights into key aspects of the business.
For example, a system that an insurance company installs to automate its claims-handling process might greatly improve efficiency, but it will also yield information for purposes nobody articulated or anticipated. Using the new data, the company can build models to estimate the likelihood that a claim is fraudulent. And it can use data on drivers’ speed, cornering, braking, and acceleration—gathered in real time from sensors installed in cars—to distinguish between responsible and less responsible drivers, assess the likelihood of accidents, and adjust premiums accordingly. Yet simply putting the system in place won’t automatically help the company gain this knowledge.
Our research, which has involved studying more than 50 international organizations in a variety of industries, has identified an alternative approach to big data and analytics projects that allows companies to continually exploit data in new ways. Instead of the deployment of technology, it focuses on the exploration of information. And rather than viewing information as a resource that resides in databases—which works well for designing and implementing conventional IT systems—it sees information as something that people themselves make valuable.
Accordingly, it’s crucial to understand how people create and use information. This means that project teams need members well versed in the cognitive and behavioral sciences, not just in engineering, computer science, and math. It also means that projects cannot be mapped out in a neat fashion. Deploying analytical IT tools is relatively easy. Understanding how they might be used is much less clear. At the outset, no one knows the decisions the tools will be asked to support and the questions they will be expected to help answer.
Therefore, a big data or analytics project can’t be treated like a conventional, large IT project, with its defined outcomes, required tasks, and detailed plans for carrying them out. The former is likely to be a much smaller, shorter initiative. Commissioned to address a problem or opportunity that someone has sensed, such a project frames questions to which the data might provide answers, develops hypotheses, and then iteratively experiments to gain knowledge and understanding. We have identified five guidelines for taking this voyage of discovery.
The logic behind many investments in IT tools and big data initiatives is that giving managers more high-quality information more rapidly will improve their decisions and help them solve problems and gain valuable insights. That is a fallacy. It ignores the fact that managers might discard information no matter how good it is, that they have various biases, and that they might not have the cognitive ability to use information effectively.
Click here to view original web page at hbr.org