Empowering Autonomous Driving with Large Language Models: A Safety Perspective

This paper addresses significant safety challenges in autonomous driving (AD) systems, particularly in long-tail unforeseen driving scenarios. The work explores integrating Large Language Models (LLMs) into AD systems to leverage their robust common-sense knowledge and reasoning abilities. The proposed methodologies employ LLMs as intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. The paper presents two key studies in simulated environments: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine, demonstrating superior performance and safety metrics compared to state-of-the-art approaches.

Authors: Yixuan Wang, Ruochen Jiao, Simon Zhan, Chengtian Lang, Chao Huang, Zhaoran Wang, Zhuoran Yang, Qi Zhu

@inproceedings{wang2024empowering, title={Empowering Autonomous Driving with Large Language Models: A Safety Perspective}, author={Wang, Yixuan and Jiao, Ruochen and Zhan, Simon and Lang, Chengtian and Huang, Chao and Wang, Zhaoran and Yang, Zhuoran and Zhu, Qi}, booktitle={LLMAgent Workshop at ICLR 2024}, year={2024}, url={https://arxiv.org/abs/2312.00812} }