Cruise faces a safety reckoning that will determine its self-driving future

Long before the California Department of Motor Vehicles deemed Cruise's robotaxis "unsafe" and took the extraordinary step of suspending its driverless permits Oct. 24, concerns surfaced about the company's safety culture.

The General Motors-backed company's bumpy history and current crisis have brought questions about its future to the fore.

Problems arose as early as April 2022, when a Cruise robotaxi drove at night without its headlights on. A San Francisco police officer pulled over the car. The incident prompted Carnegie Mellon University professor and embedded systems safety expert Phil Koopman to write that Cruise should consider the event "a wake-up call to get their safety house in order."

A month later, a person claiming to be a Cruise employee warned California regulators the company's development approach was not "consistent with a safety-first culture." Cruise leadership hid information on crashes from a team that worked on critical safety systems, the whistleblower said. Pressure from executives and investors, they said, pushed the company toward a launch when employees did not feel ready.

Cruise rebuffed concerns and started driverless commercial service in June 2022.

A succession of collisions, incidents and gaffes have followed these past 17 months. They reached a crescendo the night of Oct. 2, when a Cruise robotaxi ran over a pedestrian who fell into its path after being struck by a human-driven car in the intersection of 5th and Market in San Francisco.

Cruise executives omitted a troubling detail about the crash in a meeting the next day with regulators, California DMV officials allege. After its initial stop, the vehicle resumed driving, dragging the victim underneath and potentially causing more injuries.

The DMV said it only learned about this aspect of the crash from another regulator conducting a separate Cruise investigation. When asked, Cruise provided the DMV with complete footage on Oct. 13.

Concerns about technical shortcomings and Cruise's ethical backbone now imperil the robotaxi company.

"If we can't trust Cruise about this crash, then we shouldn't trust Cruise about anything," said Bryant Walker Smith, a University of South Carolina law professor who co-authored the SAE Levels of Driving Automation. "And an AV company that isn't trustworthy has no business being on our roads."

Cruise did not respond to a request for a comment regarding its trustworthiness. In a written statement issued Oct. 26, the company said its self-imposed "pause" in operations was "the right thing to do during a period when we need to be extra vigilant when it comes to risk, relentlessly focused on safety, and taking steps to rebuild public trust."

With its driverless deployment and testing permits suspended in its home city and prime market, Cruise has indefinitely idled its 400-vehicle nationwide fleet. The company further said the pause would give it “time to examine our processes, systems and tools, and reflect on how we can better operate in a way that will earn public trust.”

But the company declined to answer key questions about how its self-driving technology and remote operations center acted during the crash sequence. Separately, Cruise disputed the DMV's determination that company executives misrepresented the crash's circumstances. "DMV was shown the complete video multiple times," spokesperson Hannah Lindow said.

That position marked the latest contentious dealing with California officials — the San Francisco Municipal Transportation Agency has longed complained that Cruise brushed aside efforts to forge a collaborative dialogue — and it left one national safety expert unimpressed.

"Companies with safety culture problems blame everyone but themselves," said Missy Cummings, director of George Mason University's Autonomy and Robotics Center and a former safety adviser at NHTSA.

"Cruise should wipe out management, clean house and bring in some people who know what it means to have a safety culture," she said. "I really think that's their only choice going forward."

At first, Cruise used the Oct. 2 incident to reinforce its notion that "Humans are terrible drivers," the headline of a full-page advertisement the company ran in The New York Times on July 13.

A hit-and-run human driver first struck the woman, who toppled from that car's hood directly into the path of the Cruise vehicle. Cruise vowed to work with police to "find the responsible driver."

But the collision's circumstances proved more nuanced and problematic.

With the robotaxi stopped at a traffic light prior to the crash, Cruise's self-driving system detected the pedestrian, the company said. Cruise declined to say whether the system continued to classify her as a pedestrian once she was struck by the hit-and-run vehicle and lodged on its hood.

This remains an important question, Cummings said.

If the pedestrian had been detected in the intersection and then suddenly was no longer detected, it would be reasonable to believe the self-driving system would infer the object on the hood might be her — or at least question her whereabouts. But artificial intelligence is too simplistic to reason in that fashion, she said.

"AI is not particularly intelligent," Cummings said. "That's the problem. There's no real knowledge. The system didn't know anything. It's 'thing detected, thing not detected.' It's the problem with all large-language models."

Moments later, the pedestrian fell into the path of the oncoming Cruise vehicle. The robotaxi aggressively braked in its lane. The company said it found its AV responded within 460 milliseconds, faster than most human drivers could react.Nonetheless, it ran over the victim and came to a stop on top of her.

It remains unclear how much time elapsed, but the robotaxi soon accelerated to 7 mph and traveled another 20 feet while dragging the pedestrian underneath, according to the DMV's suspension order. She sustained "multiple traumatic injuries," according to the San Francisco Fire Department. This compelled the DMV to determine Cruise robotaxis are "not safe for the public's operation."

It was not the only reason for the suspension. DMV officials wrote that Cruise "failed to disclose" the pullover maneuver, and the "omission" amounted to a misrepresentation of the crash circumstances.

Cruise owes regulators and the public "the whole truth and nothing but the truth," said Smith, the University of South Carolina law professor. "When we're talking about the trustworthiness of companies, that's what they owe us and that's what we deserve."

Experts said this additional movement likely represented an attempt to place the car in a "minimal risk condition," in which it moves either to a curb or shoulder of a road in the event of a problem, ostensibly to stay out of harm's way.

Cruise declined to say whether the car initiated this maneuver or whether a human in its remote operations center issued such directions.

Either way, "it seemed like exactly the wrong thing to do," said Mike Wagner, CEO of Edge Case Research, a Pittsburgh company that conducts safety analysis and defines metrics for automated vehicle companies.

Cruise should have foreseen the detected pedestrian may still be in traffic at the outset of the crash sequence and, after the strike, accounted for the possibility she remained underneath the car, he said.

"This thing is not an edge case," Wagner said. "Almost by definition, if it's happening right off the bat in the first year of operation, these are things you could have anticipated with a little bit of critical thinking. So that's a potentially big gap."

Cruise is no longer supposed to have technical gaps.

Kyle Vogt, the company's CEO, said in April that Cruise had shifted its focus from technology development to widespread commercial deployment.

"The business has grown up in many ways and reached this rapid-scaling phase where we're no longer trying to prove that this technology works or even that we can do it inexpensively," Vogt said. "It's about reaching scale and driving that top-line revenue and path to profitability."

Cruise was on track to surpass its goal of generating $1 billion in revenue by 2025, he said. Beyond San Francisco, the company operates commercially on a smaller scale in Austin, Texas, and in Phoenix, where it has encountered some familiar snags. Plans to scale in those two cities and add Miami; Nashville; Los Angeles; Atlanta; San Diego; Charlotte, N.C.; Raleigh, N.C.; Washington; Seattle; Las Vegas and Tokyo quickly took shape.

Behind those ambitions lies a business reality. Cruise lost $263 million in the first quarter of 2023. Scaling becomes a commercial imperative, which Bryan Reimer, a Massachusetts Institute of Technology research scientist and co-founder of the Advanced Vehicle Technology Consortium, worries pushed Cruise toward deployments before the technology was ready.

"Cruise's funding stream and runway forward is somewhat limited," he said. "Their back is close to being against the wall, so they're moving faster and not necessarily respecting the full implications of that."

Cruise did not return a request for comment on how the suspension might affect the company's long-term plans in San Francisco and elsewhere.

The backdrop of Cruise's latest pedestrian crash is reminiscent of the one behind a March 2018 incident in Tempe, Ariz., in which an Uber self-driving test vehicle struck and killed a pedestrian at a time the company held expansive ambitions, he said.

A National Transportation Safety Board investigation found an "inadequate safety culture" at Uber created conditions that contributed to that incident, including a lack of safety risk assessment procedures.

"This has all the telltale signs of the Uber incident," Reimer said.

Cruise pressed ahead with its expansion plans despite problems in San Francisco. Those include an injury crash days after commercial service began, a crash into the rear of a municipal bus in March that sparked a safety recall and a crash involving a fire truck in August.

There have also been instances where robotaxis clustered at intersections and stopped for hours, repeated occasions in which city officials said Cruise robotaxis interfered with emergency responders, and one incident in which a car drove into wet cement.

Uber's self-driving program never recovered from the fatal crash. The company sold its Advanced Technology Group to Aurora in early 2021. Whether Cruise can reestablish itself — its California permits are suspended indefinitely — remains unknown. 

"This should be an existential question for a company that betrays the public trust," Smith said.

Cruise retains one key strength that could sustain it through a tumultuous period — the backing of General Motors. That gives the company a path to viability, Cummings said.

"Uber was not quite as well resourced," she said. "So this is a watershed moment for Cruise, but not a do-or-die moment. At least in the short term."

Over the long term, she said, Cruise's future hinges on whether it can get serious about a safety culture.

The first step is straightforward. Cruise must mend relationships.

"I'm struck by how Cruise is so antagonistic in almost every interaction they have with … anyone who seems to be remotely critical," Cummings said.

From a technical and operational perspective, Cruise said its team is conducting an analysis to “identify potential enhancements” and that the pedestrian incident will be included in future simulation testing. Further, the California DMV has provided Cruise with a list of requirements it must complete before applying for reinstatement of its suspended permits. 

Beyond those steps, Cruise must evolve its thinking to recognize the broader impact robotaxis have on the entire transportation system, Reimer said. Robotaxis can function as designed but still "increase risks in other elements of the transportation system," he said.

Those may include assessing robotaxis' interactions with emergency responders and movements around pedestrians, the latter of which is the subject of an ongoing NHTSA investigation.

It may also include examining aggressive braking when no obstacles are present, leading human drivers to rear-end Cruise vehicles. NHTSA is examining five such incidents involving Cruise vehicles in a separate probe.

If self-driving technology is to fulfill its potential of delivering an orders-of-magnitude reduction in the 40,000 traffic deaths that occur every year on American roads, its developers must implement a holistic approach to solving problems, Reimer said.

That will take a long time.

"If we want to look long term — decades out — at seeing the benefits of automation on how we live and move, we need to think about how we evolve the system," he said. "It's not going to be created overnight and just work."