Waymo reconstructs real-life crashes, eliminates most of them in simulation

Tech company Waymo has provided a glimpse of how self-driving systems could one day dramatically reduce road carnage.

In a first-of-its-kind analysis, the company reconstructed real-life fatal crashes that occurred over a decadelong period in and around its Chandler, Ariz., operating area.

When Waymo replaced human drivers with autonomous systems and simulated the crashes, it found its system avoided collisions in 84 of 91 scenarios studied. Further, Waymo's system mitigated the severity of crashes in four of the remaining incidents.

Of the three incidents in which no change occurred, all were instances in which the Waymo vehicle was struck from behind.

"This is an important step forward, because they've been able to do a very careful comparison on specific fatal crashes that occurred within the same area where they're doing vehicle operations," said Steve Shladover, research engineer at the California Partners for Advanced Transportation Technology, an R&D program at the University of California-Berkeley. "That makes for a nice, direct comparison with their automated system."

Self-driving vehicles hold the promise of delivering a safer future; findings from the Google offshoot underscore that potential with newfound specificity.

They arrive one week after the National Safety Council warned that its preliminary estimates showed as many as 42,060 people died on American roads last year, an 8 percent total increase over 2019 despite an overall reduction in travel stemming from the pandemic.

Though the number is only an estimate, the nonprofit safety organization said the accuracy of its estimates has historically been within 1 percent of the eventual final figures compiled by the National Center for Health Statistics.

In 2019, 36,096 people were killed in traffic crashes, according to NHTSA.

Self-driving systems are hardly the only means to reducing that toll. Some states are looking to combat drunken driving with new laws that would lower permitted blood-alcohol content to 0.05 percent. Driver-assist technologies that enhance the performance of human motorists rather than outright replace them can lower crash rates by as much as 56 percent in certain scenarios, according to a December 2020 report from the Insurance Institute for Highway Safety.

Waymo's results highlight the potential for automated drivers to further decrease the number of crashes and fatalities.

Working with an independent third party which helped re-create the crash scenarios from Arizona Department of Transportation data, Waymo's researchers replicated scenarios involving 72 fatal crashes. Next, they ran simulations in which the Waymo Driver played individual roles — first replacing the vehicle that initiated the crash, then separate simulations in which Waymo's Driver adopted the role of the vehicle responding to an instigator. (Waymo could not swap its system in place of pedestrians or bicyclists.)

In 52 scenarios where Waymo's system replaced drivers who instigated crashes — by running red lights, speeding and crossing the median and more — the cars responded as anyone might expect. Because the systems are designed to stay within the speed limit and not run red lights or violate other rules of the road, the systems avoided crashes in 100 percent of those scenarios.

Where the study breaks new ground is in crystallizing how autonomous vehicles could avoid crashes in which they're responding to other road actors who have broken road rules. In 39 scenarios in which Waymo's Driver played the "responder" role, the system avoided crashes in 32 scenarios and mitigated four more.

"These are incredibly important results," said Trent Victor, director of safety research at Waymo. "It shows that the Waymo Driver is not only capable of driving safely on its own, but is able to respond effectively when the opposing party makes a mistake."

One caveat in the company's methodology: While the study analyzed the way automation could eliminate human error, it did not account for the rare possibility of automation error.

In the four crashes in which the severity was diminished, Waymo says the driver was 1.3 to 15 times less likely to sustain a serious injury.

Intersection crashes were the most common collision scenario in the data set, accounting for more than 35 of the crashes. Three of the most common intersection-crash scenarios account for 93 percent of the fatal intersection crashes annually recorded in the U.S., according to Waymo researcher John Scanlon.

Preventing intersection crashes could yield significant results in efforts to reduce deaths, but they occur in areas that may prove vexing. All four of the crashes in which Waymo's system mitigated a crash occurred in intersections.

One instance "illustrates how incredibly fast things can change at intersections and how the opportunity for avoidance is very small," Victor said.

Shladover said it's understandable that there would be some skepticism over the internal study, which could be construed as Waymo "tooting its own horn." But he and other independent academics who consulted on the study felt the results are revelatory in an area that's been otherwise difficult to assess.

"Fatal crashes are so rare, and it took a lot of work for them to look at these individual scenarios," he said. "I liked the separation between the 'initiator' and 'responder' and I thought that was a very good way of doing the analysis and showing the differences. This is a very direct comparison to where they're operating in Chandler. I hope they can do this in other locations in the future and extend it."