Authors: Anne Stockem Novo, Christian Hürten, Robin Baumann & Philipp Sieberg

Even after several announcements of deadlines for bringing fully autonomous vehicles to the market, it still has not happened today and seems to be a bigger challenge than expected. The main reason for this are the complex vehicle architectures of traditional car manufacturers which do not scale for the transition from SAE level 2+ (assisted driving or ADAS) to level 4 or higher (highly automated or autonomous driving). Tech companies like Google, Tesla or Uber have shown impressive results with a completely different – an often-called revolutionary – approach, comprising of major Deep Learning blocks. However, a hindering problem still remains concerning the safety strategy of such vehicle architectures. As long as the human driver is still responsible for the system, acting as a supervisor at any time (SAE level 2), assisted functions serve as a comfort enhancement but do not need to guarantee safety. It is well known that interpreting and estimating uncertainty in Deep Learning models, especially for large and complex models, is challenging and an issue of heightened importance in discussions of safety considerations.

The question that we addressed as authors of this work is: How can we know at any time if the automated vehicle (AV) operates in a safe state? And if so, is it possible to forecast the safety state for several seconds ahead? There is the need for a self-assessment of the automated system, such that expected problems can be communicated and measures can be taken before the actual problem occurs.

Additionally, we included one more comfort aspect: Even if we could drive safely, it does not feel good if there are frequent decelerations or accelerations, or if a lane change manoeuvre has been started but cannot be terminated. Thus, we addressed a second question, going beyond safety: How can we also guarantee a comfortable driving experience?

We joined forces of Deep Learning experts from the computer science department of the Ruhr West University of Applied Sciences (HRW), simulation experts from the chair of mechatronics from University Duisburg-Essen and the SME Schotte Automotive GmbH & Co. KG in order to design such a safety assessment system. The core principles of the concept are the following (see Figure 1, 2): The vehicle state is evaluated in real-time. A Deep Learning forecasting model predicts the future traffic situation. The reliability of this prediction is evaluated on a huge dataset of training and validation data, resulting in a quantification of a safe prediction horizon. This tells us how far into the future we can reliably predict.

The assessment of the distance of surrounding vehicles is used to compute the maximum braking time in case of an emergency brake to stand-still. This calculation is based on pure physics.

As for the comfort requirement, we faced the difficulty that comfort is a subjective parameter. As a first approach, we chose manoeuvre completion as an indicator of positive user experience. This way, the subjectivity of the comfort parameter could be made quantitatively assessable. In the future, we plan to integrate further aspects. Simulations were used in order to generate driving scenarios and extract the typical times of such manoeuvres.

If the assessment of the operation state shows a safe and comfortable driving situation, the driver can relax and enjoy the ride. Problems can occur due to faulty or dirty sensors, limited perception ranges due to fog or heavy rain, direct sunlight, unknown vehicle type ahead (heavy transports which seldomly happen), drunk drivers showing untypical driving behaviour, or many more. If in such cases the assessment system observes a problem, the issue is reported to the driver via a Human Machine Interface (HMI).

Now the different levels of automated driving need to be looked at separately:

For SAE level 3 AVs, the driver is still in the loop and can be integrated in the driving process in case of a problem. The driver will be asked to take-over control. In case of a SAE level 4 or 5 AV, the vehicle must operate without the intervention of the driver. If a problem is expected, it needs to go to a safe state, which can be for example, driving with degraded functions (drastically reducing speed, prohibiting lane changes and overtaking) or even bringing the vehicle to a full stop on the hard shoulder.

The presented assessment system focuses on driving with at least moderate speeds as well as roads where traffic lanes are present. For urban environments with speeds below 2.5 m/s, where the main interactions will be between the AV and pedestrians or cyclists, our general concept can still be applied. However, predicting the motion patterns of pedestrians is much more challenging than predicting vehicles. Therefore, the reliable prediction horizon is lowered, but at the same time the AV speed is drastically reduced as well. We plan to address such urban scenarios in more detail in the future.