Highway Stars: How Autonomous Trucking Became The Unlikely Hero Of Autonomous-Vehicle Development

Freight movement in the U.S. is expected to rise from 17.4 billion tons in 2015 to 25.5 billion tons in 2045, according to a report by the consulting firm Deloitte. Some AV developers are already taking advantage of that growth. 

“We launched our freight network last July,” says Jason Wallace, marketing director for AV trucking startup TuSimple. “Since then, we’ve opened a wholly owned freight terminal in Alliance, Texas, and mapped out roads all the way from Phoenix to Orlando.” 

TuSimple’s version of the next-generation truck will be built by partner Navistar. Its fleet of 52 trucks operate as Level 4 vehicles with drivers, but the freight network, Wallace says, helps demonstrate the technology to customers while earning revenue for the company.

“Most recently, the hot topic among new customers has been food distribution,” he says. “Through the pandemic, there was a lot of demand for food banks and a real need to distribute food more efficiently. We did a run from Nogales, Arizona, to Oklahoma City using one of our trucks with a team of two drivers and shaved 10 hours off of the trip.”

To pave the way for its trucks, TuSimple uses a fleet of SUVs equipped with cameras and sensors to map out the highways within the network, then creates a digital model of the environment for the trucks. As trucks are deployed, Wallace says, the data is harvested   via AWS devices such as Snowball Edge, a storage device used to transfer the data to the cloud. In the physical world, the trucks keep the maps updated and fresh, scanning, rescanning, and updating changes.

TuSimple has large servers of its own. But as with Plus, its large data needs eventually called for the scale of AWS’ services. The accessibility of data also makes it possible to review recorded drives and differentiate between human and machine driving.

“What you’re looking for is harsh braking or cornering,” Wallace says, something that is much more common with a human. The autonomous system will keep the vehicle centered to within about 5 centimeters of the middle of the lane and manages the throttle much more precisely than a human. It can yield up to 10 percent better fuel economy, Wallace says.

The data management of trucks in the field also allows operators to better track their fleets. 

For now, TuSimple is still at Level 4, but Wallace says “long term, the goal is to remove the driver, and we’ll be demonstrating operations without a driver next year.” 

“Trucking is the tip of the spear,” a spokesperson for autonomous-truck developer Aurora told Automotive News. The company’s Aurora Driver system promises to offer efficiencies similar to other AV developers’ products, with increased operational hours, fuel and maintenance efficiencies and improved safety. The Driver, however, is designed for wider application.

“Trucking enables us to rapidly and efficiently move into adjacent verticals, like ride-hailing,” the Aurora official said. “Trucking is well over a $700 billion business in the U.S. By going first to market with an autonomous truck, we can build a strong, scalable product and revenue base. That experience and scale will be inherited by our ride-hailing product.”

Aurora has a pair of manufacturing partners, Paccar and Volvo, which combined account for more than 40% of all Class 8 trucks sold in the United States. That makes a path to revenue from implementing Aurora Driver clear. 

Unlike other AV developers, Aurora is leveraging what it learns from trucking and applying it to other use cases and vehicle types—including ride-hailing—by developing a “common core” of software, hardware, infrastructure and development tools.

Partnering with manufacturers allows AV developers to leverage the manufacturing and hardware experience of those manufacturers, and it puts the existing relationships of those manufacturers at Aurora’s fingertips.

“OEMs have relationships with large shippers and carriers because they’re serious players that need to operate with such efficiency and speed, so the value of trust between the two is high,” the company says.

“We recognized early on that if we didn’t have the endorsement of a manufacturer, that would impact our ability to launch commercial pilots with big networks this year and ultimately launch a self-driving truck without a safety driver by late 2023.”

This is not the only symbiotic relationship AV developers have, however. 

Handling the computing scale of Aurora’s needs requires computing power that can scale and adapt to geographically dispersed teams handling huge amounts of data. “In June 2021, we exceeded 5 billion virtual miles as our enhanced virtual development tooling and an expanded team allowed our engineers to chew through an average of over 22 million miles each day in our virtual testing suite,” the company says.

To handle that data, many AV developers look to AWS. 

“During development, AV companies have geographically distributed test fleets collecting tens of terabytes per vehicle every day,” says Vijitha Chekuri, global business development leader for autonomous vehicles at AWS. “That data quickly scales up to petabytes for companies with fleets with dozens or even hundreds of test vehicles.”

“The processing and usage of the vehicle data requires pipelines that are agile enough to enable unpredictable iterations upon driving models and algorithms for faster time to market,” says James Barr, an AV business development specialist at AWS. “Autonomous-vehicle development teams also require intelligent data management, training, simulation, verification and validation. Implementing these functions requires cost-effective mechanisms to leverage thousands of standard and specialized compute instances  to develop and deploy self-driving functionality. 

“If you buy hardware and use on-premises storage and computing, you’ve paid for it whether or not you’re using it for a highly unpredictable set of workloads. One of the key value propositions of the cloud is spinning up the latest in computing power when you need it and switching it off and not paying for it when you don’t. This concept is key when you consider the iterative nature of AV development.” 

The flexibility, security, privacy and complexity of the data needs, both in terms of scale and management, also discourage home-grown hardware, says Chekuri. AWS offers online and offline data transport solutions that can quickly collect data from AV vehicles and upload it to “data lakes” in the cloud, where it can be stored and managed long term. The use of autonomous data lakes on AWS provides AV customers the ability to cost-effectively store the data, search, analyze, visualize and be leveraged by diverse development teams for downstream workloads such as simulation and training.

“Storage is one of the biggest cost drivers for AV development,” Chekuri says. AWS offers six classes of storage for cost-effective archiving and accessibility. AWS Intelligent tiering features allows data to be moved automatically between classes based on usage, allowing information to be quickly accessed at one tier of pricing or be archived long term at a much lower cost. 

“It may be that regulators require 10 to 15 years of data retention,” Chekuri says, “so low-cost, archival solutions like Amazon S3 Glacier and S3 Glacier Deep Archive are essential for AV companies.

“Once data is uploaded, products like AWS Glue can be used to crawl, discover and catalog data for analysis.” That helps identify unusual scenarios that may be hard to diagnose, such as a repeated sensor failure in a specific locale or repetitious errors in a certain driving situation.

According to estimates from Synergy Research, AWS accounted for 32% of the cloud infrastructure market in the first quarter of 2021. That scale means it can offer solutions that other providers cannot, something that many AV developers are aware of. 

“AV companies Toyota, Mobileye, Aurora, Torc Robotics and Lyft are all building autonomous-driving models and HD maps on top of AWS,” Barr says. The company’s long history of assisting such developers with a constellation of needs makes it a highly capable partner. 

“We have reference architectures, open source code, partners and best practices for the various phases of AV development workflow ,” Barr says.

Because of the frequency with which developers have asked AWS for niche solutions, it also makes a variety of highly specialized tools available. AWS SageMaker Ground Truth provides 2D and 3D first-party managed labeling services. SageMaker, for instance, allows developers and data scientists to quickly build, train and deploy machine-learning models at scale, simplifying workflow and allowing faster iteration.

 “Our customers need hyper-scale infrastructure that is secure, global and compliant with their specific needs,” says Jon Jones, director for AWS Compute and AI/ML services, including autonomous vehicles.

“We’re helping the leaders in long-haul autonomous-trucking technology with their infrastructure, AI and machine-learning needs just as they are helping their manufacturer partners build the trucks of the future.”

In the first half of 2021, investors showered autonomous-trucking developers such as Plus and TuSimple with a record $5.6 billion. That’s up from $4.2 billion in all of 2020, according to research firm Pitchbook. 

Class 8 commercial trucks, the tractor trailers you pass every day on the highway, typically log astronomical mileages on long routes with relatively predictable conditions. That’s an ideal environment for developing and proving self-driving technology. 

“The type of more structured driving that trucks do on the highway is really where autonomous technology shines,” says Plus co-founder and COO Shawn Kerrigan. The autonomous-vehicle developer currently deploys Level 4 autonomous trucks in the U.S. and China.

“The business cases are clear in terms of cost reductions, asset utilization and especially safety,” Kerrigan says. 

After years of waiting on robotaxis and autonomous passenger cars, many investors see that advancements in autonomous trucking have real potential to alleviate labor shortages, build operational efficiencies and, most of all, improve safety in a shorter time frame than cars. 

But while controlled highways are a good petri dish for developing algorithms for artificial intelligence, the challenge of full autonomy is still a big lift for cars or trucks.

“There are two problems every autonomous-vehicle developer has to solve,” says Kerrigan. “First, you have to develop technology that's much safer than a human driver. Second, you have to demonstrate that safety to regulators and the public before you can deploy at scale.”

“Our view has been that the only way to really address both of those problems is to collect massive amounts of data. Autonomous driving is really a big data problem.”

Safety is the foremost challenge for every AV developer, Kerrigan says, but it’s also one of trucking’s biggest hurdles generally. According to the National Highway Traffic Safety Administration, more than 500,000 truck accidents occur in the U.S. every year. In 2017, just under 5,000 of them involved fatalities. Autonomous vehicles, Kerrigan says, can truly redefine safety. 

“This is technology that doesn't get distracted, doesn't get drowsy and always follows the rules,” he says.

Like most AV developers, Plus tests its algorithms in computer simulations and the real world—which, due to the scale of the processing and data required, means a big appetite for cloud computing and storage   as well as logistical challenges in harvesting and redeploying data to test vehicles. 

“We have a heavy need for cloud computing capability, both for simulation and data processing,” Kerrigan says. “Simulation is critically important, as it enables you to test against situations you have seen in the real world or have conceived of as something that could happen out there. 

“But in addition to simulation, you need to have extensive real-world data collection to ensure you have enough long-tail scenarios captured in your simulations.” 

Furthermore, Kerrigan says, real-world experience is the best way to prove the safety of your system in the real world.
A big hurdle, however, is just accumulating all the miles needed to identify all the long-tail situations and prove the efficacy of the system. 

“We are applying our Level 4 technology into trucks with drivers that supervise the system, like a really high-functioning driver assistance system with limits on when and where it can operate,” Kerrigan says. Long-tail scenarios and data gathered in these driver-in trucks is then fed back into the driving AI to improve it.

To handle the amount of data harvested from real-world operation, Plus relies on Amazon Web Services (AWS) for its cloud computing. “The AWS team rose to the challenge with a solution that matches the scale we need for our driving system,” Kerrigan says.

“We use AWS’ petabyte-scale object storage service, Simple Storage Service (S3), and their managed compute service, Elastic Cloud Compute (EC2). We also employ some higher-level services including FSx for Lustre, a high-performance file system to more efficiently feed our data into our EC2 infrastructure for processing, and Elastic Kubernetes Service (EKS) for orchestrating and managing our EC2 instance fleets. These services allow us to more easily process all of our data and ultimately reduce our time to results, speeding up the development of our driving system.”

Those computations have already yielded a production-ready driving system, Kerrigan says. And even with the driver-in solution, there are real safety gains.

 “You get much happier drivers because this makes their driving experience much less taxing,” he says. “If you're driving 11 hours a day with this technology to handle the traffic and the wind and all these other conditions, it makes it much more manageable.”

Safety gains are the most valuable benefit of all autonomous-technology applications, but the commercial case is stronger for self-driving trucks than for other sectors. 

“Trucking is the backbone of the whole economy, and there’s a shortage of truck drivers just at a time when e-commerce has taken off,” says Michael Fleming, co-founder and CEO of Torc Robotics, which became an independent subsidiary of Daimler Trucks in 2019.  

Driver shortages in trucking have been increasing since the 1980s because of an aging workforce, the damaging health effects of long-haul driving and historically high industry turnover. Autonomous trucks can do far more than alleviate that labor shortage.

“Truck drivers are limited by the number of hours that they can drive per day,” Fleming says, “so now we have trucks sitting around unused and a lot of trailer loads waiting to be picked up. As we shift to self-driving trucks, the greatest optimization and efficiency gains that we see is being able to run trucks 20 or 22 hours a day as opposed to 11 or 12.”  

One of the most important lessons in Torc’s 16-year history, Fleming says, “is that the only way to safely commercialize self-driving technology is by partnering with a manufacturer. They have to be committed to reinventing the chassis for self-driving applications.”

 In seeking a best-fit partner, Torc allied with Daimler, whose products make up 40% of the new Class 8 trucks sold in the U.S. every year. 
To build the next generation of Daimler trucks, Torc’s sensor array needs to be integrated from the start into a chassis that also integrates redundant systems such as power braking and steering. But as the number of sensors and systems grows, so, too, does the data and complexity. For awareness of what’s happening on the road, Torc’s virtual driver relies on a sensor suite made up of cameras, lidar and radar sensors that provide redundancies. 

The virtual driver combines the inputs from the three types of main sensors in real time. 

“Imagine coming up on a dark vehicle at night when it’s raining,” Fleming says. “Detecting what’s being seen with the cameras is difficult because of the lighting conditions. Lidar sensors perform better in low light but might have their range reduced by the rain. Radar, however, is a stronger performer in heavy weather conditions.” 

The complexity of solving for all situations, including that one, is the driver of the computational needs for driving algorithms. This is an example of using all the strengths of the sensors.

 “Each time we test Torc Robotics’ vehicles on the road, we generate significant and valuable data,” Fleming says. “This data must be transferred, stored and analyzed. Testing scenarios in simulation also create a large computational load.” 

This is no mean feat, Fleming notes. “The problem we’re trying to solve is one of the most difficult of our generation. In order to solve that, you need to partner with the best.” 

The large data volumes generated by Torc’s test trucks are ingested to Amazon S3 using AWS Direct Connect. Proprietary Torc services analyze the data and make it available to any location on demand. Monitoring and other data from autonomous-driving vehicles is securely transmitted in real time using AWS IoT Core. 

“We’re pushing the envelope on hardware capacity to collect data in real time,” Fleming says. “We’re transmitting data 500 times faster than a solid-state USB device in every truck, and we’re running a lot of trucks.” 
The data used to improve driving algorithms also informs hardware and maintenance changes that will be needed in future generations of vehicles. 

“Experienced truck drivers have tuned their brains to serve as accurate sensors,” Fleming says. “If a tire blows out or a wheel bearing fails, we can feel that and quickly pinpoint the problem. A machine can’t. 
“We could develop deep-learning algorithms to diagnose those failures, but we're solving for a product, not a research project. Sometimes we look at our data and find that it’s easier to just integrate a sensor somewhere.” 

When commercial self-driving trucks begin rolling off the line, Fleming says: “We’ll have to rethink preventative maintenance on the truck because now we’ll have a lot more data than we do today. One of the beauties about truly innovative technology is it starts creating new markets that don't exist today.”