Reinforcement learning paves the way for safer and smarter highway autonomous vehicles

Autonomous vehicles (AVs) have the potential to transform transportation systems by improving safety, efficiency, accessibility, and comfort. However, developing reliable control policies for AVs to handle the complexity of real-world driving remains an immense challenge. The AVs technology can save lives, reduce crashes, congestion, fuel consumption, pollution, and parking space; but it can also increase traffic, disrupt transit and insurance sectors, and pose ethical and legal issues. A recent breakthrough study presented by researchers from the Oakland University and Guizhou China reviews recent advancement on the application of DRL to highway lane change, ramp merge, and platoon coordination.

AV control methods are very sensitive to application scenarios. Three major scenarios for RL-based control, namely highway lane change, highway ramp merging, and platoon coordination, are discussed. The diversity of conditions and approaches highlights the complexity of the lane change problem and the need for adaptable RL solutions.

The past decade has witnessed remarkable advances in RL algorithms, enabling agents to achieve superhuman performance across complex domains like games, robotics, and autonomous driving. This review synthesizes key developments in modern RL algorithms based on recent research papers in this growing field, including single agent RL, multi-agent RL, curriculum learning and representation learning.

Key findings highlight the similarities and differences in DRL formulations, training algorithms, simulations, and performance metrics. The review also identifies current limitations and best practices in the field, offering valuable insights to guide future research. By understanding these elements, researchers can enhance DRL applications, making autonomous vehicles more capable of handling complex traffic situations under uncertain conditions.

RL-based AV control, especially in highway conditions, has significant benefits to improve our society, such as enabling cooperative and altruistic behaviors, handling complicated dynamics and uncertainties.

The insights from this literature review can help guide future research toward realizing the full potential of RL for AV control in complex real-world conditions.

In future, the following key directions for future research include: Multi-agent reinforcement learning (MARL) algorithms that can handle varying numbers of agents. Increase testing in diverse and realistic traffic conditions. Development of standardized benchmarks for comparative evaluation. More sophisticated human driver models that capture complex interactive behaviors. Handling imperfect state observations from real-world sensors.

Reference

[1] Qie T, Wang W, Yang C, Li Y, Liu W, Xiang C. A path planning algorithm for autonomous flying vehicles in cross-country environments with a novel TF-RRT* method. Green Energy Intellig Transp 2022;1(3):100026.

[2] Dang F, Chen D, Chen J, Li Z. Event-triggered model predictive control with deep reinforcement learning. In: IEEE transactions on intelligent vehicles; October 2023. https://doi.org/10.1109/TIV.2023.3329785. in press.

[3] Jiang L, Xie Y, Evans NG, Wen X, Li T, Chen D. Reinforcement learning based cooperative longitudinal control for reducing traffic oscillations and improving platoon stability. Transport Res C Emerg Technol 2022;141:103744.

[4] Hwang S, Lee K, Jeon H, Kum D. Autonomous vehicle cut-in algorithm for lane-merging scenarios via policy-based reinforcement learning nested within finitestate machine. IEEE Trans Intell Transport Syst 2022;23(10):17. 594–17 606.

[5] Remonda A, Krebs S, Veas E, Luzhnica G, Kern R. Formula rl: deep reinforcement learning for autonomous racing using telemetry data. arXiv preprint arXiv:2104.11106. 2021.

Author: Ali Irshayyid, Jun Chen, Guojiang Xiong

Title of original paper: A review on reinforcement learning-based highway autonomous vehicle control

Article link: https://doi.org/10.1016/j.geits.2024.100156

Journal: Green Energy and Intelligent Transportation

https://www.sciencedirect.com/science/article/pii/S2773153724000082