Interior sensing emerges as a new frontier in vehicle safety

Every year, 1.3 million people die from road traffic injuries. To combat this loss of human life, automakers have implemented driver monitoring systems that improve road safety by understanding signs of driver impairment, such as drowsiness or distraction. Driver monitoring system adoption also is driven by regulatory frameworks such as the European New Car Assessment Program, or Euro NCAP, which require the systems for a five-star safety rating.

However, forthcoming regulations from Euro NCAP, the U.S. Hot Cars Act and the U.S. infrastructure bill will soon require systems that measure not only driver state but the cabin as a whole. This requires a deeper level of insight than what driver monitoring systems can provide.

As a result, we're witnessing the evolution of driver monitoring systems to "interior sensing," a rapidly emerging market in which artificial intelligence and several modalities such as computer vision are applied to measure the state of the driver, the cabin and occupants. In addition to meeting regulatory requirements, these systems present compelling opportunities for automakers to differentiate on both safety and experience.

Interior sensing systems encapsulate the key features of driver monitoring systems. They monitor:

• Driver state, analyzing eye gaze to detect distraction
• Blink rate and eye closure to assess drowsiness
• Face ID to inform benchmarks for a driver's normative behavior, helping recognize deviations.

But understanding the full context of dangerous driving behavior requires an awareness of what's happening elsewhere in the cabin. A driver may be distracted by a number of things, be it their cellphone or a crying baby in the back seat. Interior sensing provides that context through occupancy detection (how many people are present, where they're sitting and seat position); and activity detection, identifying cellphone usage, eating, drinking, smoking or other activities that introduce distraction.

Another key safety feature is the ability to detect children or pets left behind in a vehicle — another cause of death in vehicles. Interior sensing systems address this using occupant detection to recognize the child, or object detection to identify a child seat.

Object detection also has interesting applications for ride-sharing or other shared mobility environments. For example, the system could classify and detect when objects such as bags or cellphones are left behind, improving the odds that they're returned to their rightful owner.

Interior sensing provides opportunities to improve the mobility experience, too. Insights on occupancy coupled with data gathered from analyzing gaze, reactions, expressions, emotions, activities and the objects people use will inform the state of the cabin, driver and passengers. This human-centric approach is powerful. It will enable automakers to deliver mobility experiences that enhance comfort, wellness and entertainment in addition to unparalleled safety for all vehicle occupants.

For example, if a driver expressed frustration sitting in traffic, an interior sensing solution could prompt the in-vehicle system to play soothing music, or suggest a pit stop to take a break. Or, if the system recognized antsy children in the back seat, it could recommend playing a movie to provide some much-needed entertainment.

These applications will only become more enticing as we reach increasing levels of autonomy. When the mobility experience is no longer about the act of driving, consumers won't choose their vehicle based on vehicle operation alone — that's a big selling point for automakers considering how to differentiate and gain a competitive edge.

Understanding the state of the entire cabin is complex. It requires a multisensory approach — similar to what we've seen with exterior sensing systems, which have expanded from one or two sensors to include numerous cameras, radar, lidar and more. Interior sensing will follow suit, leveraging existing cameras used for driver monitoring systems, alongside additional RBG and infrared cameras positioned throughout the cabin to provide a holistic view in a variety of lighting conditions.

Interior sensing systems also require advanced AI software, built with deep learning and massive amounts of data. Deep-learning models are data-hungry and must be developed and trained using massive amounts of real-world data. Once trained, these systems can then learn to identify and track the state of the cabin, as well as individuals inside the vehicle. From there, additional layers of analyses can determine if occupants are enjoying their experience, interacting with an object, are seated in the right location, wearing a seat belt and more. Only then will the software be able to understand what's happening in a vehicle and prompt relevant, safe vehicle adaptations that work for all people without bias.

Nevertheless, with technical advancements in both automotive-grade hardware and software, the time for automakers to evolve is now — before regulatory requirements, competitors and consumer expectations pull ahead.