Could thermal sensors have prevented Uber's fatal crash?
Lasers and ultrasound? Radar and cameras?
Those questions have dominated the great sensor debate in self-driving technology. But there's been less attention paid to the promise of thermal sensing.
Following the highly publicized fatality in Arizona involving one of Uber's self-driving vehicles, the case for thermal has gotten a lot stronger.
Now companies like FLIR Systems and AdaSky, which make competing thermal cameras, are hitting the publicity trail to press their case to a public that's newly wary of the blistering pace of development of self-driving vehicles.
At issue are the compromises inherent in any sensor technology.
Companies like Waymo and Uber have relied primarily on spinning LiDAR sensors to help accurately resolve the world around their vehicles.
But top-of-the-line LiDAR, which has come down in price substantially over the last couple years, is still prohibitively expensive for most market applications. Velodyne's 64-laser LiDAR system costs about $80,000, for example.
Market-minded developers have made do with cheaper LiDAR technology. A new generation of so-called solid-state LiDAR, which is much more compact than the spinning laser systems you're used to seeing on Waymo's cars, will likely cost less than a hundred bucks in a few years.
That cost reduction, it should come as no surprise, comes with significant tradeoffs. Lower-resolution LiDAR sensors have trouble accurately resolving objects at distance compared to their high res siblings, which means autonomous vehicles relying on those systems will have less time to react.
More troublesome, LiDAR is a visible spectrum technology. Fog, rain, and snow can severely impact accuracy.
Tesla has opted for a starkly different sensing strategy, getting rid of LiDAR in favor of radar and optical cameras.
But those technologies, too, have limitations. While radar can sense objects at a great distance and performs well in adverse weather, it has difficulty identifying objects.
In Tesla's system, cameras work in conjunction with radar to sort out what's a plastic bag and what's a small dog. But cameras don't work well in low light, which is a huge Achilles heel of the system.
Pedestrian fatalities, it's worth noting, are rising fastest after sundown, according to new research from the Insurance Institute for Highway Safety. In fact, the vast majority (three-quarters) of pedestrian deaths happen at night.
Watching the horrifying incident video released by police following the Uber-involved fatality, it's plain that light conditions played a leading role.
Thermal could be the needed stopgap in either LiDAR-based or radar-based systems. The technology is already in use on some high-end cars from the likes of BMW and Porsche, but it hasn't yet become part of most self-driving sensor payloads.
"A new type of sensor that employs far infrared (FIR) technology can fill the reliability gaps left by other [autonomous vehicle] sensors," according to a recent white paper by AdaSky, one of the companies that's commercialized infrared for autonomous driving. "FIR has been used for decades in defense, security, firefighting, and construction, making it a mature and proven technology."
FIR-based cameras detect differences in heat naturally emitted by objects using infrared light waves. Because FIR cameras sense the infrared spectrum, they can pick up objects that visible light cameras, radar, and LiDAR systems can't.
Crucially, they work phenomenally well in low-light and poor weather situations.
In light of public reaction to recent accidents involving self-driving vehicles, developers will be taking a hard look at ways to recapture public confidence and consumer enthusiasm.
We'll be seeing more talk of improved sensor packages, and it's a safe bet thermal cameras will be part of that conversation.