How cars make sense of sensor info

Automakers are increasingly turning to cameras, radar, lidar and other sensor technologies to make vehicles safer and more autonomous.

Each technology serves a purpose of its own, often to fill in the information gaps that the other technologies leave, said Jeremy Carlson, automotive technology analyst at IHS Markit.

But to understand how various vehicle technologies work together, it is important to understand how they work individually -- and what their advantages and disadvantages are.

Cameras serve a central purpose in vehicles, Carlson said: "They replace the human eye."

He said cameras, whether mounted to the windshield or elsewhere on a vehicle, can recognize what objects are to a degree that radar, for instance, is not able to. 

"There's a difference between detection and recognition," Carlson said. "The one thing that a camera is good at is recognizing and interpreting that image. It can say if an object is a tree, car, stop sign, pedestrian, whatever." 

That information can be used to, for instance, avoid collisions. But cameras have a significant drawback that other technologies must supplement: They aren't great at being able to measure distance.

To address the shortcomings of cameras, vehicles utilize radar and lidar, Carlson said.

Radar "serves a pretty simple purpose," he said: to determine an object's positioning relative to the vehicle via radio waves. 

Radar has proved to be useful for systems such as blind-spot detection, which rely on accurate measurements of distance.

Radar's technological cousin, lidar, works in a similar manner. Instead of radio waves, lidar sends out pulses of light to determine an object's distance.

Lidar is typically costlier than radar. But, as Carlson notes, lidar also can help determine what an object is, not unlike a camera. He said lidar helps with "building a picture" of the road ahead by supplementing information that can be gleaned from cameras and radar.

Carlson said other technologies, such as ultrasound and vehicle-to-vehicle communications, work hand-in-hand

with cameras, radar and lidar to build that picture.

He said the concept of mapping "localization" has gained momentum in recent years thanks to suppliers such as Mobileye. 

Map crowdsourcing, as Carlson called it, utilizes information gleaned from cameras, radar, lidar and other sensors on multiple vehicles to paint a detailed picture of the roadway. For instance, a crowdsourced map could make note of whether an object a camera detects, such as a tree or pedestrian, is typically on the roadway and adjust the vehicle's path. 

"Being able to localize yourself with these highly detailed maps, it provides that extra piece of information," Carlson said.