Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS

Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS
Like

The development of high-level automated vehicles faces considerable safety challenges nowadays. The major obstacle preventing the mass production of high-level automated vehicles is the safety of the intended functionality accidents caused by functional deficiencies under a triggering condition, especially those related to perception algorithms. Humans are essentially a special kind of sensor, and it is a meaningful thing to study human-in-loop decision-making based on humans’ states for over coming perception algorithm function deficiencies and improving safety of the intended functionality. For high-level automated vehicles, human will serve as the role of passenger, and the mental activities of passenger may be different from diver. By analyse the influence on passengers’ mental activities caused by driving scenario risk via fNIRS, we provides a possible solution to design a human-centred intelligent system to promise safe driving for high-level automated vehicles using passengers’ driving risk cognition.

Experimental Paradigm

This experiment was performed in driving simulator. The participants were requested to sit on a driving simulator and to look at the front scenario. Beside, the participants were required to press the keyboard when they hear a stimulating sound or they feel dangerous, they were not required to do other things. Four kinds of challenging scenarios were used in our experiment: lead vehicle autonomous emergency braking in a short distance, lead vehicle cut-in from left lane in a short distance, lead vehicle cut-in from right lane in a short distance and pedestrian crossing road from right scenarios.

Data Analysis Process

Risk field is a safety indicator that indicates a dangerous degree of driving scenario. The risk field indicator at a moment involves relative longitudinal distance between target vehicle and ego vehicle, relative speed, and those information is related to time to collision (Fig. 1). Firstly, each driving scenario were divided into a low-risk episode and a high-risk episode based on the objective risk field indicator. Specifically, the episode in which the indicator is larger than 0.05 is considered as a high-risk episode, and otherwise it is considered as a low-risk episode.  Secondly, low-risk features and high-risk features of concentration of oxygenated hemoglobinwere extracted by K-SVD dictionary learning and mean values from low-risk and high-risk episodes, respectively; Finally, the T-value analysis method was used to assess the area sensitive to driving scenario risk in the prefrontal contex of passengers based on fNIRS for high-level automated vehicles.

Figure 1. The sketch maps and risk fields at a moment of autonomous emergency braking in a short distance, vehicle cut in from the left lane, vehicle cut in from the right lane and pedestrian crossing road from the right scenarios.

The differences in the T-value based on mean value features of   between low-risk and high-risk episodes are shown in Fig 2, From the T-value result based on mean value features, it is derived that the T-values of lead vehicle autonomous emergency braking in a short distance, lead vehicle cut-in from left lane in a short distance, lead vehicle cut-in from right lane in a short distance and pedestrian crossing road from right in the channel 8 are maximum, compared with other channels.

Figure 3. The T-values of  between low-risk episode and high-risk episode in driving scenarios of lead vehicle autonomous emergency braking in a short distance, lead vehicle cut-in from left lane in a short distance, lead vehicle cut-in from right lane in a short distance and pedestrian crossing road from right. are shown in (a–d).

Besides, we also analyzed the influence on mental activity caused by sex and driving experience,  using boxplots of  in channel 8 about driving experience and sex. The boxplots of  is shown in Fig 3. LN stands for the low-risk episode data about those participants who do not have valid driving experience; HN stands for the high-risk episode data about those participants who do not have valid driving experience; LY stands for the low-risk episode data about those participants who have valid driving experience; HY stands for the high-risk episode data about those participants who have valid driving experience; LM stands for the low-risk episode data about those male participants; HM stands for the high-risk episode data about male participants; LF stands for the low-risk episode data about female participants; HF stands for the high-risk episode data about female participants.  For lead vehicle autonomous emergency braking in a short-distance scenario, the difference of  between low-risk and high-risk episodes for those participants who have valid driving experience are more obvious the difference of those participants who do have valid driving experience; similarly, the difference  of   between low-risk and high-risk episodes caused by male participants that female participants.

Figure 3. The boxplots of cerebral oxygen exchange in channel 8 about the driving experience and sex. The boxplots of autonomous emergency braking in a short distance, lead vehicle cut-in from the lef lane in a short distance, lead vehicle cut-in from the right lane in a short distance, and pedestrian crossing road from right scenarios are shown in (a–d).

Conclusion

In this study, We compared passengers' mental activities in low-risk and high-risk episodes and explored the impact of driving scenario risk on their cognitive processes using fNIRS technology. By compare   between low-risk and high-risk episodes  using the T-value and boxplots, three conclusions were found: 1) the mental activities of passengers caused by driving scenario risk in the Brodmann area 10 are very active; 2) driving experience makes people more sensitive to longitudinal risks; 3) Male participants are more sensitive to longitudinal risks than female participants.

Implications for Safer Driving

The research has important implications for improving safety of the intended functionality. By recognizing and quantifying passengers' mental activities related to driving risk, it becomes possible to implement appropriate measures to enhance overall safety. Additionally, passengers’ mental states can be incorporated as an essential variable in driving safety systems. By continually monitoring passengers’ mental states, systems can provide timely alerts or interventions to mitigate risks and sustain safe driving conditions. Besides, the  of passengers on prefrontal cortex could be combined with reinforcement learning to realize passenger-in-loop decision-making for high-level automated vehicles.

Support

The more details content can be find in the paper of “Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS”,(Wang, H., Zhang, X., Li, J. et al. Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS. Sci Rep 13, 15839 (2023). https://doi.org/10.1038/s41598-023-41549-9)

The code and supplementary material available at https://github.com/SOTIF-AVLab/fNIRS

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Subscribe to the Topic

Electrical and Electronic Engineering
Technology and Engineering > Electrical and Electronic Engineering

Related Collections

With collections, you can get published faster and increase your visibility.

Retinal imaging and diagnostics

This Collection invites works providing insight into using novel or existing retinal imaging technologies in clinical applications or presents new or adaptive forms of these techniques.

Publishing Model: Open Access

Deadline: Jan 31, 2024