← projects

World model: a walking robot and a robot brain

A neural network that predicts what happens next, written from scratch in C++20 (own matrix maths and neural nets, no Eigen), teaches a simulated 1.66 m humanoid to walk with far fewer real tries. A second part builds helpers around π0.5, a large robot "brain" that turns camera images and a sentence into arm motions.

view on github →
The simulated humanoid walking forward
The learned controller walking 18 m in 30 s.
~172×
fewer real tries to learn to walk than PPO
+16.5 pts
π0.5 success when the camera has moved
3.3 GB
4-bit π0.5, down from 7.7 GB, no worse
895
automated tests (596 C++, 299 Python)

// the idea

Robots learn by trying things, and real tries are slow and costly. A world model learns how the world responds ("if I move my legs like this, where will I be?"), so the robot can practise mostly inside its own imagination.

Part 1 · C++20

Learning to walk

Built bottom-up: matrices, a neural network, then world models, uncertainty from a group of 5 models, and planners. Proven first on a balancing cart-pole, then used to train a 12-motor biped in the MuJoCo physics simulator.

Part 2 · Python

A robot brain

Takes π0.5 as it is (never retrained) and tests add-ons on LIBERO-Plus, simulated kitchen tasks with disturbances such as a moved camera or sensor noise. Rented GPUs for testing; everything else built and tested on a laptop first.

// results

64,000 vs 11.0 M

Walking with ~172× fewer real tries

The world-model trainer (MBPO) taught the humanoid to walk 10 m without falling after a mean of 64,000 real simulator steps. PPO, a standard method that learns from real tries only, needed 11.0 million. Five training runs each, all five walked.

Honest caveat: the learner inside MBPO (SAC) is already ~61× better than PPO on its own; the world model adds ~2.7× on top. PPO is still faster in wall-clock hours.

Learning curves: MBPO reaches the walking goal with far fewer real steps than PPO
Real steps needed until each run first walked.

57% → 73%

Fixing a moved camera

π0.5 fails more when the camera is not where it was during training. Redrawing the image as seen from the training spot (a view-synthesis network) raised success by 16.5 points on 115 fresh paired tasks (95% range +2.1 to +30.5).

81.7% → 83.2%

A 4-bit π0.5 for a small robot computer

Storing most weights in 4 bits cut running memory from 7.7 GB to 3.3 GB, aimed at an 8 GB Jetson Orin Nano Super. Over 630 paired tries it was not worse (+1.4 points, 95% range −2.7 to +6.2).

Missed: the weights are 3.08 GB against a 2.8 GB goal, and each answer is 1.8× slower on a desktop GPU.

// how it's built

// honest limits

// what next

Questions about this project, or want to work together?

⎇ main
Toronto --:--