Ganesh Venkataraman

Model-based RL

Deep reinforcement learning (RL) has achieved remarkable success in games and control tasks, but model-free algorithms such as PPO and SAC are sample-inefficient, requiring millions of environment steps. This makes them impractical for real-world domains like robotics, where interactions are expensive, whereas Model-based reinforcement learning (MBRL) offers improved sample efficiency but suffers from model bias, as prediction errors accumulate during planning. Probabilistic Ensembles with Trajectory Sampling (PETS) [1] addresses this by combining an ensemble of probabilistic dynamic models aimed at planning, enabling robust and uncertainty-aware decision-making. This project aims to implement and evaluate PETS, a model-based reinforcement learning that leverages the Cross-Entropy Method (CEM) and Random Shooting Algorithms for uncertainty-aware planning. PETS is known for its strong sample efficiency compared to model-free methods. Over our project tenure, we will progressively build PETS: starting with random data collection and deterministic models, extending to probabilistic ensembles, integrating planning, and enabling online learning.