Virtual companion increases safety for trucks on the road

Stuttgart, Germany – Trucks are the mainstay of logistics. In Germany, three-quarters of all goods are transported by road. Time pressure, long working hours, and monotonous convoys are part and parcel of the truck driver’s everyday routine. If the driver is distracted, overtired, or reacts too slowly, the consequences can be disastrous. Automated driving functions are key to avoiding accidents and dangerous situations. They alert drivers and support them in critical situations, and are increasingly in a position to assume monotonous and fatiguing tasks. At the same time, people remain irreplaceable even at subsequent levels of automation: they must be able to take control if necessary. What is the ideal way to coordinate the interaction between vehicle and driver? This was the focus of the TANGO project over the past three and a half years. TANGO stands for “Technologie für automatisiertes Fahren, die nutzergerecht optimiert wird” in German. Its English equivalent is “Technology for automated driving, optimized to the benefit of the user.”

The project focused on partially and conditionally automated driving (SAE Levels 2 and 3). It produced a prototype for an “attention and activity assistant” – a virtual companion that keeps the driver alert, counteracts fatigue, and makes driving as pleasant as possible. In addition to Bosch, which acted as consortium leader, the project’s other participants were Volkswagen, MAN Truck & Bus, University of Stuttgart, Hochschule der Medien, Spiegel Institut, and CanControls. TANGO was funded with a grant of some five million euros from the German Federal Ministry for Economic Affairs and Energy (BMWi).

Keeping the assistance level, driving situation, and driver’s condition in view

“In the future, manual and automated driving phases will alternate: as in a relay race, responsibility for steering will pass from vehicle to driver and back,” explains Bosch project manager Michael Schulz. “In partially automated driving as per SAE Level 2, the driver must be prepared to intervene at any time; in conditionally automated driving as per SAE Level 3, only when necessary. The objective is to keep the demands placed on the driver at the optimum level so that they are always capable of taking control.”

Here is where the attention and activity assistant comes into play. Depending on assistance level, the driver’s condition, and what the actual driving situation will permit, the assistant can provide the driver with various secondary activities: listening to music or audiobooks, reading emails, writing text messages, watching movies, scheduling the day’s appointments, planning the route, and doing fitness exercises. The system has to not only reliably determine the specific driving situation, but also detect and properly interpret the driver’s condition at any given moment – for example, if the driver is tired or distracted.

In addition, the research project installed sensors to monitor the interior and combined them with artificial intelligence methods. Cameras detect if the driver’s eyes are closing, if they are blinking frequently, are losing sight of the road, or even if their head is drooping to the side with fatigue. Intelligent algorithms evaluate the images, interpret them, and launch countermeasures. These might be a warning, an offer to perform a secondary activity, or an active intervention such as braking.

Development focuses on the needs of the user

To get to the root causes of fatigue and distraction and to precisely determine the requirements for the attention and activity assistant, the researchers accompanied truck drivers on their routes, interviewed them about their experiences, assessed online daily logs, and repeatedly compared the intermediate development stages against test subjects so as to make revisions. This process involved driving simulators, drives in test vehicles, and even a little magic: in a right-hand-drive truck, the passenger side of the cab was also equipped with a steering wheel, brakes, accelerator, and all the necessary control and display elements, with the two halves separated by a privacy screen. This approach, dubbed the “Wizard of Oz,” enabled the researchers to simulate the automated function and observe user behavior under realistic conditions.

“We deliberately focused our work on the user and user requirements,” Schulz says. The researchers started from the premise that truck drivers are often wary of automated driving because they associate the technology with being stripped of responsibility and worry that their job will be eliminated. As a result, they don’t always use even today’s assistance systems, sometimes deliberately turning them off altogether. The project determined that the strongest argument in favor of more automation was the increase in safety, though comfort as well as more relaxed driving and arrivals play a role as well.

Applying the findings for future development – including in cars

Against this backdrop, there was particular emphasis on the human-machine interface (HMI). “In the future, a vehicle will have to interact with the driver and serve as a partner,” Schulz says. “Perfect interaction between vehicle and driver requires operation to be simple, intuitive, and inviting.” The prototype contains a control panel consisting of several displays, which combines visual, acoustic, and haptic elements and depicts the attention and activity assistant as an avatar. Findings from the research project will flow into further development in areas such as interior monitoring, automated driving, and entertainment systems. These findings can apply not only to trucks but also to cars, where they also stand to increase safety in road traffic.