Disabled fail-safe at issue in Uber crash

Fresh questions have emerged about Uber's self-driving technology and safety practices on the eve of a federal meeting at which the probable cause of a landmark crash likely will be determined.

Two people with firsthand knowledge of the company's self-driving system say company officials removed a key fail-safe from test vehicles months before one of them struck and killed a pedestrian in Tempe, Ariz.

The fail-safe, an internally developed feature called Reflex, was shelved because some members of Uber's Advanced Technologies Group, or ATG, thought it triggered too many braking events at a time when the program was under pressure to show progress in developing software, said the sources, who asked not to be identified because they are not authorized to speak on the matter.

Reflex provided an independent cross-check on the main self-driving systems by using short-range radar signals to sense imminent hazards directly in the path of vehicles. The sources, who had roles within ATG, believe that had it been enabled in Uber's Volvo XC90 fleet, Reflex would have mitigated the collision that killed Elaine Herzberg — and possibly prevented it.

"It was designed to prevent exactly this sort of crash," one source said.

The existence of Reflex has not been disclosed publicly, nor is it referenced in any of the 43 documents and 439 pages of information the National Transportation Safety Board has released in its public docket on the March 18, 2018, incident, the first fatal crash in history involving a self-driving vehicle.

Asked about Reflex, a spokesperson for Uber said Friday, Nov. 15, the company could not comment pending the ongoing NTSB investigation.

With NTSB investigators set to meet Tuesday, the two sources, plus a third person who also held a role within ATG, spoke to Automotive News because they were concerned that investigators did not yet have a comprehensive picture of Uber's safety capabilities and culture. Further, they worried that lessons learned from the crash have gone unheeded.

The three people described a culture intent on keeping pace with rival Waymo, which regularly makes public pronouncements about the number of cars in its fleet and the number of miles it drives. Even though many in ATG found little value in pursuing a similar course, leaders deemed it necessary to put a large number of cars on the road because those were metrics they believed investors valued.

When Uber named Dara Khosrowshahi CEO in August 2017, pressure to show progress intensified — not because pressure was directly applied by the CEO. In meetings before and after the crash, two of the sources say, Khosrowshahi showed tepid enthusiasm for Uber's self-driving ambitions. In turn, ATG leaders ramped up efforts to show progress.

That included eliminating Reflex, because it caused frequent braking events that could make rides uncomfortable or jarring.

Smooth rides weren't yet an objective of engineers. But at Uber and elsewhere, smooth rides give the appearance of progress to those unfamiliar with self-driving development, and it became important to demonstrate for Khosrowshahi and potential investors the ability to provide smooth rides. In every sense, the three people said, this was a matter of Uber choosing comfort over safety.

Earlier in development of the system, there was concern that harsh braking events, a frequent occurrence when AVs are being tested, could lead to human drivers rear-ending test vehicles. To reduce that likelihood, Uber installed a function called action suppression, sometimes internally called plan suppression, that would delay hard braking by one second to account for the possibility these frequent braking events were false positives — braking for hazards that didn't exist.

This worked but at the cost of delaying reaction to real problems — a scenario that would play out in the Tempe crash. Recognizing that potential pitfall, Uber engineers designed Reflex as a rudimentary backstop to the main self-driving system and as a counterbalance to action suppression. But it was removed in the months before the crash. The three sources could not provide a specific date that Reflex was removed.

The action-suppression efforts are described in the NTSB docket, and the suppression system delayed a braking response when a collision with Herzberg was imminent. Since then, Uber has changed its system so hard braking is initiated immediately when a collision is imminent.

There was another system available that held the potential to apply emergency braking. Volvo's City Safety driver-assist system came factory-installed on the XC90s that Uber used for testing. But it was deactivated because Volvo and Uber engineers said radar signals from the systems interfered with each other. The three sources dispute that.

All three sources viewed the deactivations of Reflex and City Safety as links in the broader set of circumstances that contributed to Herzberg's death. Other circumstances include a human safety driver who watched a TV show on her phone instead of the road, software that could not properly classify Herzberg as a pedestrian for 5.6 seconds before impact, the culture of Uber's self-driving program and even the roadway infrastructure.

As the NTSB prepares to meet, the three people all caution that, while Uber deserves its share of blame, recommendations for improved safety and revised public-testing procedures should be given to the entire industry.

"We had unrealistic timelines and internally driven pressure," one of the sources said. "That's not unique to Uber. The way we develop and test as an industry is problematic."