How auto insurer Progressive collected 10 billion miles of driving data from its customers

Progressive has quietly amassed a trove of driver data with Snapshot, its opt-in driving tracker device.
Written by Natalie Gagliordi, Contributor
Flo and Snapshot. Via Progressive.

The auto insurance company Progressive is known for its quirky commercials helmed by the caricatured sales agent Flo.

But buried beneath the cheap commercial laughs and the drab exterior of insurance policies, Progressive is quietly amassing a trove of driver data — 10 billion miles of driving data, to be exact — with Snapshot, its opt-in driving tracker device.

Progressive began to offer Snapshot to drivers in 2008, right at the birth of the Big Data boom, marketing the device to customers as a means to lower their insurance premiums. Since then, use of the device has been approved in 45 states.

The device plugs into a vehicle's onboard diagnostic port and collects speed at one-second intervals. At the end of every trip, the collected data is sent wirelessly back to Progressive via AT&T's network. Progressive says it plugs the speed readings into algorithms to calculate mileage and hard braking events, with the theory that drivers who exhibit safe driving habits can lower the price of their premiums.

But back to Big Data...

Progressive's usage-based insurance leadership team, which included Dave Pratt, Bill Everett and James Jiang, held an interactive chat session recently to give an inside look at the Snapshot program, how it works, and how they’ve leveraged Big Data as a result.

It turns out Progressive's tools for harnessing and analyzing Big Data run the gamut from refined (version 2.4.1 of Hadoop, R and the full spectrum of SAS software) to bare bones (Excel). The team said Hadoop and SAS is more widely used by Progressive, while the more complex analytical language R is used by only a small number of its internal analysts.

"Hadoop works well for massive amounts of data, and it allows us to process the data via multiple processors at the same time," Everett said.

Currently, Progressive collects only two data elements with Snapshot: time and speed. According to Everett, Progressive is beginning efforts to collect GPS data as well.

"Our current product does not include location data. We're currently collecting GPS data from a sample of customers for research purposes only," he said.

And while Progressive uses AT&T's wireless network to transmit data from the Snapshot device back to Progressive's data warehouse, the process is not done in real time. But at some point, mobile phones could be used for that purpose, Pratt said. 

"We're currently testing mobile phones as an alternative data gathering option," he said. "If mobile proves to be accurate, we might be able to offer new, real-time services."

Snapshot collects up to six months of data from drivers for discount purposes, and the data is archived for years as per the insurer's regulatory protocol. The drivers who opt in for the Snapshot program have online access to the data they generate at the end of each trip, using their Progressive policy account to get visualized reports.

No intention of sharing its data 

So, with all of that data building up, one could wonder if Progressive has ever offered — or considered offering — analysis of the data to external entities. 

After all, there are a number of ways that data could be used beyond lowering insurance rates. For example, planners of urban infrastructure could use the data to determine which roads are the most problematic for drivers and in need of work or repair.

But Progressive has no intention of sharing its data trove, according to Pratt.

"It's very important that we keep individual customers' data secure," he said. "Even if it doesn't include name or address, we need to avoid any disclosure of individual information."

Looking ahead to the maturation of smart automobiles, Progressive is hopeful that its data collection will not only continue to remain relevant, but also become easier and more efficient.

"We hope that new technology (whether OEM telematics or mobile phones) will make Snapshot even easier for our customers," Pratt said. "We can use the same analytic techniques on data from new sources."

Editorial standards