Credit: Jennifer Tapias Derch
A successful data strategy turns a company’s data into important insights and financial gains — but it shouldn’t stop with information from inside the firm. From historic weather information to customer preferences to shopping trends, organizations have access to a vast number of data sets outside their walls, some for a price and others for free, that can sharpen analytics and ultimately boost the bottom line.
A 2018 MIT Sloan Management Review data and analytics report found that the most analytically mature organizations use more data sources, including data from customers, vendors, regulators, and competitors. “Analytical innovators,” or companies that incorporate analytics into most aspects of decision-making, are four times more likely than less mature organizations to use all four data sources, and are more likely to use a variety of data types, including mobile, social, and public data.
And organizations that share their own data with customers, vendors, government agencies, and even competitors report increased influence in their business ecosystem, the survey found.
“In all sorts of areas, people are using third-party data to augment the data they already have,” said Asif Mahammad Syed, the vice president of data strategy at the Hartford Steam Boiler Inspection and Insurance Co. “In most cases, you can’t build high-quality predictive models with just internal data.”
Companies use this external data to augment their decision-making, better meet customer needs, predict supply and demand, and more.
Experts from MIT Sloan and elsewhere shared successful strategies for making the most of external data.
Know the external data landscape
The practice of bringing in external data isn’t new, but many companies have focused their data strategies thus far on making use of structured and unstructured data within their organizations. In a 2019 Deloitte survey, 92% of data analytics professionals said their companies needed to increase use of external data sources.
“Digital is giving you many more sources of data,” said Stephanie Woerner, a research scientist with the MIT Center for Information Systems Research. New sources include social media data, information from Internet of Things-enabled sensors, and even fingerprint data.
Some companies have business models that rely on untraditional external data sources, Woerner said, citing the fintech company Kabbage, which uses data from social media, sales, shipping records, and more to help determine the creditworthiness of small businesses.
The insurance industry has always been data-driven, Syed pointed out — insurance companies have a strong interest in finding out information about risks they are taking and pull in data like credit scores and historic weather information. “These things were happening, but it sped up in the last decade or so,” he said.
COVID-19 has likely accelerated that trend as firms try to anticipate consumer demand in a changing, unstable landscape. As internal data became less of a reliable indicator, companies turned to external factors to predict consumer demand, supply availability, and the future course of the pandemic.
Know your options
Types of external data vary. Syed noted that high-level data is available for free at data.gov, which has more than 200,000 data sets from a variety of government agencies. “That's really an exciting place to start, because the government gives away so much free data, and it can actually help you to predict a lot of things,” he said.
His company, Hartford Steam Boiler, is part of the global company Munich Re, which has a “data-hunting team” that looks for data sources outside the company for a given use case, and then helps acquire, clean, and incorporate the data within the company.
There is a host of other information available, including regulatory data, property data, weather data, telematics data, which includes information such as how fast a vehicle is being driven, and even information about people’s emotional states. Image-based data is also available — satellite or drone images after a catastrophic event can help an insurance company direct resources before people can visit in person, Syed said.
“The benefit of using external data is so great that there are businesses built around gathering this data, consolidating it, cleaning it, and packaging it up for use by other companies,” said Angie King, PhD ‘15, an analytics innovation senior principal with End-to-End Analytics, which is part of Accenture. Accenture engages with more 300 external data providers and uses relevant external data to enhance clients' own internal data when building analytical models for them, she said.
Depending on need and their own tech savvy, companies may opt to gather the data themselves, King said, or they can reach out directly to data providers, or engage with a firm that can help them through the data selection and model building process.
Use external data to know customers better
Related Articles
With third-party data and information, companies can avoid asking customers to provide too much information or fill out multiple forms, Syed said. “A customer’s attention span is very limited,” Syed said. “If you ask too many questions, they say, ‘Oh, I'll go to someone else.’” For example, a customer looking for an insurance quote can provide their name and address, and companies can use external sources for further information about the customer, their home, or their car, equipment, or machines.
Dwane Morgan, MBA ‘08, the director of global consumer insights at Under Armour, calls this the “Amazon effect.”
Amazon customers are used to the company tracking certain information — for example, when Amazon sends a reminder that it’s time to purchase something, most customers are used to the company’s practices and don’t stop to wonder how it knows that they’re running low.
“The challenge is getting to the point that you're being useful enough with the recommendation based on the data,” Morgan said. “If you run into a situation where you cause the consumer to pause and think, ‘Well, how do they know this?’ and it feels like something that's not useful, they might delete the app or whatever they think is allowing you to track that data.
“There's a balance between making sure you're being prescriptive, so you can be helpful, but not overstepping in a way that feels intrusive.”
Use external data to add real-world context to internal decision-making
Businesses often need to know what’s going on with other companies and predict the impact of outside factors, from industrywide purchasing trends to pandemics and other catastrophic events.
“One of the most classic use cases for external data is to understand the major impact that external events can have on demand for a company's products,” King said. For example, a company’s sales could dip if a competitor’s product goes on sale, or product sales could go up because of social or traditional media, like a celebrity mention on Twitter, a viral video, or a shoutout from a talk show host.
King said using external data can improve a company’s predictive analytics and machine learning models. “Without having external data capturing these events, the predictive models wouldn't be able to infer the reason for the resulting spikes or drops in sales,” she said.
Beyond predicting their own sales, companies use social media and survey data to assess consumer sentiment and guide product decisions, King said. Companies also scrape competitors’ websites to track inventory for products with limited availability (such as event tickets or airplane seats) and to inform their pricing.
Use external data with care
Using third-party data sources can raise concerns about protecting privacy, avoiding biased or inaccurate data, and using data for the right purposes, Syed said.
In a survey, 92% of data analytics professionals said their companies needed to increase use of external data sources.
“Data has an incredible power to change the outcome of whatever you're trying to do. If your data is not trustworthy, then the business decisions that you make from that data are going to be wrong as well,” Syed said. Vetting any data coming from outside the company is important, he said. Companies should know how the data is collected and how ethically the company gathering data manages its business.
Morgan agreed. “It's not all about price. I think a lot of it comes down to the comfort level you have with that team and their ability to pull in the data accurately,” he said.
Establishing a lasting relationship with data companies is also key. “Just like any type of data, the more you have, the more you can learn from it,” Morgan said. “The longer you're with that partner, the better it is for you.”
For example, data from 2020 alone would show impacts that were unique to that unusual year. “Unless you have multiple years of data to look at and compare, you don't really know what's a seasonal impact, what was just the COVID effect, and what's normal.”
Changing vendors could mean slight changes in methodology, he added, which makes it hard to compare and contrast data over time.
Above all, being able to trust the data is vital. “I always say, scrutinize the data first,” Morgan said. “The first thing you want to do is make sure you scrub the data and that you're fully confident in what the data is saying.”
Remember that external data is only successful as part of a centralized digital strategy
Anyone can buy third-party data, Syed pointed out — if a company’s strategy relies on data gathered outside the firm, other companies could have the same insights. “You really need to have your internal data, or have some special process where you're synthesizing the external data and making some intelligence that is specific to you,” he said. “Data by itself doesn't produce that intelligence. Data has to be put in the context of the business outcome to give you a valuable insight.”
Most often, internal data is what ends up differentiating a company from competitors.
“Buying third-party data has to be seen in a broader context of what the business is trying to achieve with internal data,” Syed said. “If you don't have a solid understanding of all the internal data that you have in your entire company, then buying third-party data probably wouldn't help you that much. You'd be spending a lot of money, and third-party data doesn't come cheap.”
Companies should also coordinate data acquisition so different departments don’t purchase the same data, he said.
“You have to have a reason to bring [external data] in,” Woerner said. “You have to have some way of connecting it to what you do to have it be a useful type of exercise.
“If you can't get the small data done right, then you're not going to get the big data. You don't want to go willy-nilly collecting sources of data, because then you're going to have this tangle,” Woerner said. “It’s really a matter of, can you ask the right question and then find the right data source?”
