An RL agent learning to fly a drone through a procedurally-grown forest in AirSim — and, eventually, the real thing. Can a drone using only a 20 dollar camera, learn to navigate around trees, with the same intention of doing that in real life? The project required drone building and operation, reinforcement learning in a simulator, and custom realistic creation of the simulation. No Lidar, no Stereoscopic cameras
The world is built in Unreal Engine on top of AirSim: large and medium redwoods, saplings, and tree ferns, spawned procedurally so every episode is a forest the agent has never seen before. Density ramps with a curriculum — sparse glades at first, tight canopies later — so the policy has somewhere to start.
The agent is a PPO policy (Stable-Baselines3) with a custom CNN feature extractor. Observation is a stack of monocular RGB frames fused with privileged state (position, velocity, heading, recent actions). Action space is continuous body-frame velocity commands — no waypoints, no map, just see and move.
Reward shaping is the heart of it: forward progress + clearance bonus, minus collision penalty, minus jitter, minus drift from the desired heading. Tuned across hundreds of runs until the agent stopped flying straight into the nearest trunk.
The video shows the trained policy doing its job in the real-life — weaving between trees, holding its forward velocity, and not running into anything.
In the video, my companion and the best teammate ever Elina Rosato is holding the drone with safety ropes — just in case the 1500 dollar drone does not crash.
The real drone is a Pixhawk 6C running PX4, with a Jetson onboard for inference and an Arducam staring forward. Protection itself was drawn in FreeCAD and printed in PLA.
Domain randomization on textures, lighting, and sensor noise gives the policy a fighting chance once it leaves the simulator.
The drone was assebmbled completely on our own. Custom connections for Jetson were accomodated by us
The full code lives on GitHub —
View the RL Forest code →
Download the thesis (PDF) →