How cars and infrastructure work together in urban automated driving

Stuttgart, Germany – Pedestrians obscured from view by vehicles, cyclists passing in front of the car, buses that suddenly approach: negotiating city traffic can quickly become a difficult task. Of all things, it could be streetlights that make urban traffic safer and provide automated vehicles with an overview of the traffic situation. This was the subject of the MEC-View research project. The project required streetlights to be fitted with video and lidar sensors, which then used advanced cellular technology to provide vehicles with critical information in real time, allowing them to detect obstacles – whether other cars, bicycles, or pedestrians – quickly and reliably. After more than three years of development, the project is now ready to present its findings. Partners in the project, which received 5.5 million euros in funding from the German Federal Ministry for Economic Affairs and Energy (BMWi), were Bosch, the consortium leader, together with Mercedes-Benz, Nokia, Osram, TomTom, IT Designers, and the universities of Duisburg-Essen and Ulm. The project’s associated partner was the city of Ulm, which for the past three years has been the testing ground for the sensors on the streetlights and the connectivity technology. The insights gained in the project will now be used to further develop automotive technology, automated driving, and cellular technology. In addition, the infrastructure the project has built up will now be available for further research projects to use.

Bird’s-eye beats worm’s eye

Reaching up to six meters in height, streetlights tower above road traffic. They have a precise bird’s eye view of developments at busy intersections, say – and it is knowledge like this that automated vehicles will need in the future. While a vehicle’s sensor systems – cameras, radar, and lidar sensors – give it precise 360-degree vision, the view from the ground – from the vehicle alone – is not always sufficient for it to see a pedestrian currently obscured by a truck, a vehicle emerging from a concealed entrance, or a cyclist approaching from behind and changing lanes quickly. “Because the vehicle itself cannot see around corners or through walls, we use the streetlight sensors to extend the vehicle sensors’ field of view,” says Dr. Rüdiger Walter Henn, who heads the MEC-View project at the consortium leader Bosch. The project partners have developed the corresponding hardware and software for this purpose; the system processes the images and signals from the infrastructure sensors, combines them with high-resolution digital maps (HD maps), and transmits them to the vehicle over the air. There, the data merges with the vehicle’s own sensor information to create an accurate picture of the situation, including all relevant road users.

Wireless data transmission

Advanced cellular technology makes extremely low-latency transmission of sensor information possible. While the MEC-View project used LTE mobile communications technology with an optimized configuration for this purpose, in the new 5G communications standard, real-time data transmission is a basic function. The core task of latency-optimized mobile communications is not only the virtually instantaneous wireless transmission of data, but also the processing of that data as close to the source as possible. This task is performed by special computers, known as mobile edge computing servers (or MEC servers for short), which are integrated directly into the cellular network. They combine the streetlight sensor data with data from a vehicle’s surround sensors and highly accurate digital maps. From this, they generate a model of the surroundings that includes all available information about the current traffic situation, and make this model available to vehicles over the air. In the future, facilities such as the city traffic control centers could be equipped with such servers, so that they can share the data with all vehicles, regardless of manufacturer, and other road users.

Seamlessly merging with traffic

In Ulm, the project partners have been testing the interaction of automated vehicle prototypes and infrastructure sensors in real traffic conditions since 2018. One intersection in the Lehr district of Ulm is notorious for its lack of good all-round visibility. The streetlights there were equipped with sensors to help automated vehicles negotiate the intersection. Vehicles approaching the difficult intersection from a side road have to merge onto the main road. Thanks to the newly developed technology, the automated prototype now recognizes road users early on and can adapt its driving strategy accordingly. As a result, the vehicle targets gaps in the traffic on the main road and merges seamlessly, without stopping. Such a development will make urban traffic not only safer, but also more fluid. The infrastructure built up during the project will remain in Ulm, where it will be available for use by subsequent research projects.