Optimizing Autonomous Vehicle Performance Using Improved Proximal Policy Optimization

Sensors · Mart 2025

YÖKSİS Kayıtları

Sensors · 2025 SCI-Expanded

DOKTOR ÖĞRETİM ÜYESİ ONUR İNAN →

Makale Bilgileri

DergiSensors

Yayın TarihiMart 2025

Cilt / Sayfa25

DOI10.3390/s25061941

Scopus ID2-s2.0-105000983503

Erişim🔓 Açık Erişim

Özet Highlights: In this study, a new Lévy flight-integrated proximal policy optimization (LFPPO) algorithm for enhanced exploration and control in autonomous vehicles is introduced. This enables autonomous vehicles to overcome the exploration limitations of standard PPO algorithms, providing improved decision-making and enhanced control over speed and acceleration, especially in complex urban environments. Indeed, the LFPPO algorithm achieves superior performance and reliability in dynamic driving scenarios: the experimental results in the CARLA simulator show that the LFPPO algorithm significantly outperforms the standard PPO algorithm, achieving a success rate of 99% (vs. 81%), and exhibiting a robust and reliable autonomous driving performance with optimized speed and acceleration control in dynamic urban traffic scenarios. What are the main findings? The integration of Lévy flight into the proximal policy optimization (PPO) algorithm (LFPPO) significantly improves the algorithm’s exploration capabilities, allowing it to escape local minima and achieve better policy optimization. The experimental results in the CARLA simulator show that the LFPPO algorithm achieves a 99% success rate, compared to the 81% achieved by the standard PPO algorithm, demonstrating enhanced stability and higher rewards in autonomous vehicle decision-making. What is the implication of the main finding? The LFPPO algorithm enables autonomous vehicles to make more reliable and safer decisions in complex and dynamic traffic conditions, enhancing overall driving performance. The integration of real-time data streaming using Apache Kafka allows autonomous systems to process and react to dynamic environments more efficiently, improving real-time decision-making. Autonomous vehicles must make quick and accurate decisions to operate efficiently in complex and dynamic urban traffic environments, necessitating a reliable and stable learning mechanism. The proximal policy optimization (PPO) algorithm stands out among reinforcement learning (RL) methods for its consistent learning process, ensuring stable decisions under varying conditions while avoiding abrupt deviations during execution. However, the PPO algorithm often becomes trapped in a limited search space during policy updates, restricting its adaptability to environmental changes and alternative strategy exploration. To overcome this limitation, we integrated Lévy flight’s chaotic and comprehensive exploration capabilities into the PPO algorithm. Our method helped the algorithm explore larger solution spaces and reduce the risk of getting stuck in local minima. In this study, we collected real-time data such as speed, acceleration, traffic sign positions, vehicle locations, traffic light statuses, and distances to surrounding objects from the CARLA simulator, processed via Apache Kafka. These data were analyzed by both the standard PPO and our novel Lévy flight-enhanced PPO (LFPPO) algorithm. While the PPO algorithm offers consistency, its limited exploration hampers adaptability. The LFPPO algorithm overcomes this by combining Lévy flight’s chaotic exploration with Apache Kafka’s real-time data streaming, an advancement absent in state-of-the-art methods. Tested in CARLA, the LFPPO algorithm achieved a 99% success rate compared to the PPO algorithm’s 81%, demonstrating superior stability and rewards. These innovations enhance safety and RL exploration, with the LFPPO algorithm reducing collisions to 1% versus the PPO algorithm’s 19%, advancing autonomous driving beyond existing techniques.