Cars hear you, but when will they get you?

Almost all vehicles sold in America today are programmed to respond to a driver's voice commands. But still on the frontiers of automotive research are vehicles that can determine what a driver really wants -- and respond intelligently.

Getting to that point is a profound engineering challenge that is leading researchers beyond voice recognition and into the realm of artificial intelligence. It will require auto companies, suppliers and their technology partners to delve into the building blocks of language and to teach computers to understand humans as well as humans understand one another.

It's one of the "hardest, long-standing challenges in artificial intelligence," says Tim Tuttle, CEO of Expect Labs, a San Francisco company that works on refining voice-recognition technology.

For Expect Labs, context is everything. Its MindMeld platform helps app developers and device makers integrate voice commands with other inputs to make more sense of the directions. For instance, if a driver is in the car at 8 a.m. and asks the navigation system to find a coffee shop, the car can surmise that the driver is likely on the way to work and can identify a coffee shop along the usual route. The idea is that these voice-powered interfaces should be able to process more than the simple command and respond the way a human would.

A competing platform from a company called api.ai would allow a car's system to respond to less specific commands: for example, navigating to the nearest grocery store, even if all the driver says is, "I'd like to do some shopping and buy eggs and carrots."

One key objective for technology companies is to simplify the language itself, allowing humans to interact with a car's computers in a natural, more conversational way, rather than through prescribed commands or a stilted syntax.

Gint Puskorius, manager of voice signal processing at Ford's Research and Advanced Engineering division, explains that if a driver issues a command to set the temperature at 70 degrees and then says, "Make it a little cooler," the car's system should remember the temperature was at 70 and set it to 69.

"The memory of what we were in a dialogue about" needs to be part of "future-generation systems that are more natural and more interactive," he said.

That level of memory and comprehension requires huge leaps in the onboard computing power of a car.

Most cars today have a limited vocabulary, which can be matched to a limited set of words and doesn't require much computing horsepower, says Danny Shapiro, senior director of automotive at chipmaker Nvidia, which has mobile processors in 7.5 million cars, including Tesla's Model S.

"But if you want to be able to understand anything the driver says, you have to match what's coming out of their mouth to the entire dictionary or beyond," Shapiro says. That requires a system that operates like a human brain, making connections to different sounds and forming the words and then sentences or phrases.

Typically to access this type of information, the onboard computer would need to connect with a data center in the cloud to process the request and return it to the car. Nvidia, instead, has developed an energy-efficient chip for the car that's powerful enough to perform the natural-language processing on board.

Puskorius says such embedded systems are becoming more powerful, allowing for use of a large vocabulary and continuous speech-recognition systems without the driver having to experience any delay.

Nuance, a voice-recognition technology company whose automotive clients include Ford, Hyundai and Fiat Chrysler, is working on a hybrid system in which some of the processing is offloaded to the cloud. The car can analyze a driver's utterance and query, send it to the cloud, compare the results from the cloud with those of the embedded processor and use the better answer. The answers may be a bit slower to arrive but will be more accurate than those from the embedded system alone, says Arnd Weil, Naunce's vice president of automotive.

Even so, "The main challenge is that all people speak differently, not just in terms of accents, but they also formulate their thoughts in different ways," said Ilya Gelfenbeyn, CEO of api.ai. "We have to teach the system to understand different ways of saying the same thing."

Beyond that, just being in a car makes things more complicated. Puskorius of Ford says voice-recognition experiences "have been substantially less pleasing for our customers than we would like them to be," largely due to the vehicle's "harsh acoustic environment."

If the car's fan is on high, for example, it impinges on the microphone.

David Pop, engineering manager of core technologies for General Motors, says people move around in the vehicle, and they're often not near microphones. "We're trying to get the voice captured at the highest level of quality so the speech-recognition engine can determine what it needs to do."

At Baidu, a Chinese tech company that's working on teaching human speech to machines, researchers are factoring in the background noise. They're feeding 100,000 hours of spoken audio, and the corresponding transcripts, into supercomputers that can compile the two inputs into a robust voice dictionary.

Andrew Ng, Baidu's chief scientist, who formerly led the Google Brain project, says his team spent time driving with the windows rolled down, playing loud music and having passengers speak loudly in the back seat, then fed data to the supercomputer.

Amer Aijaz, director of strategy and concepts for Volvo's electrical r&d, says autonomous driving combined with more natural voice recognition creates an environment for more interaction between car and driver -- if that's what people want.

For example, Aijaz said, if an email arrives with a calendar appointment, the car could ask, "Shall I route the car to your destination?" or "It's your anniversary today. Would you like to pick up some flowers?"

He added: "We don't want to be too intrusive" if a driver or passenger would prefer to just relax. "So the future is not just voice command. It's a combination of modalities in human machine interface with the car."

For starters, Volvo has worked on having a more human-sounding voice. The voice should be friendly rather than instructive and cold, Aijaz said.

Tuttle expects recent technological breakthroughs will solve most of the problems people experience with speech recognition today. "With the best systems," he said, "you'll be more likely to have your device be able to understand what somebody said better than you can understand it."