New and efficient AI training methods

In a constantly evolving mobility world, highly and fully automated vehicles are confronted with a wide variety of complex situations. So far, however, its core component, artificial intelligence (AI), has not been able to cope well with changing traffic situations. In a final presentation on March 9, 2023, the "AI Delta Learning" project team (a consortium lead by Mercedes-Benz AG) presented their latest research results on the scalability of AI in the field of autonomous driving. The University of Stuttgart's Institute of Signal Processing and System Theory is one of the 17 project partners.

Perception of the environment is a major research focus in the field of highly and fully automated driving: AI is a key technology in this area and has made great strides in recent years. But so far, AI for autonomous driving applications has only been trained to deal with specific traffic scenarios, such as "highway driving in good weather." In this scenario (domain), AI works in a reliable manner. To make it work in other environments, such as "driving on the highway in the rain," AI algorithms need to be retrained for this new domain, which incurs extremely high development costs Researchers in the AI Delta Learning project have therefore been investigating new machine learning approaches, which enable AI modules to be trained in a more efficient manner. 

The goal was to use the transfer of existing knowledge to learn so-called deltas, i.e., the different requirements between a familiar domain and a new target domain. If the domain changes, then previously learned information as well as tested and validated development stages are retained. This approach is proving to be efficient against the backdrop of ever shorter innovation cycles and in view of constantly changing mobility requirements. 

The deltas addressed in the project include different sensors, different traffic scenarios including everything from rural roads to complex urban traffic, different countries, different times of day, seasons and weather conditions, long-term traffic system changes resulting from new mobility concepts, new types of road vehicles and devices such as e-scooters and cargo bikes, and last but not least, the ongoing development of AI technologies such as improved training strategies and more efficient neural networks.

Focus areas: data, transfer learning, didactics, the suitability of cars

A specific data set tailored to the project objectives was generated and annotated to provide a basis for development in these fields of study. This is necessary to ensure that the required deltas are covered whilst taking data protection into account, which is why both real driving data was recorded and synthetic data was generated. 

Transfer learning is a machine learning approach that can be used to transfer information about specific domains. Its importance in the automotive sector is increasing because it improves the perceptual accuracy of autonomous vehicle systems. The method was applied during the project to improve the overall performance of vehicles.

As is the case in schools, didactics facilitates learning by structuring the learning process. This requires the annotation of data, which has previously been done by humans. Various approaches to learning not only with less data, but also with little to no annotated data were developed in the course of the project, in addition to other approaches for expediting the learning process through optimized network architectures as well as improved learning processes.

It has become generally accepted that in order to develop AI systems for automotive applications, AI functions first have to be developed, trained, and verified in the laboratory, using previously recorded driving data. But there are two problems with this: on the one hand, there are significant differences between the high-performance computer hardware in the laboratory and the hardware embedded in the vehicle, and on the other, the traffic situations that the vehicle encounters in reality may well differ from the recorded training and test data. Both problems were addressed in the AI Delta Learning project in order both to ensure the reliability of AI systems for autonomous driving functions and to increase the robustness of AI systems in the face of unexpected and unknown scenarios.   

Several methods developed at the University of Stuttgart

Several new methods for the use of AI for autonomous driving have been developed by the University of Stuttgart's Institute of Signal Processing and System Theory (ISS, Prof. Bin Yang) in the context of AI Delta Learning including: 

1. the off- and online adaptation of trained AI models to new test conditions, in particular during the ongoing use of the model in real time (test-time adaptation); 
2. the semi-supervised and unsupervised generation of photo-realistic images of road scenes (unsupervised semantic image synthesis) to be used as synthetic annotated training data, with the aim of reducing the effort required to collect and annotate extensive AI training data, and for the targeted simulation of rare case road traffic scenarios; 
3. generative models for artificially improving the resolution of low-cost lidar sensors (lidar super-resolution), and
4. multimodal models for the detection of objects using lidar and cameras and their adaptation to new test conditions.

The research resulted in 90 papers, which were published in prestigious journals and presented at conferences, six of which were presented at the ISS. A further eight publications are still being prepared at the ISS. The research is embedded in the potential area of autonomous systems at the University of Stuttgart.

The AI Delta Learning project is made up of 17 consortium partners from prestigious universities and research institutions, as well as automobile manufacturers, suppliers, and technology providers. The “Autonomous and Networked Driving” flagship initiative initiated and developed by the VDA as part of the AI family has been awarded funding by the German Federal Ministry for Economic Affairs and Energy for a period of three years.