At a glance
What it does today
You give it a goal from a phone or laptop. It plans a route on its own map and drives there around furniture, boxes and people. An AI driver decides how fast to run each track about a dozen times a second, and a safety layer underneath can always overrule it: an emergency stop, sensor watchdogs, and a hard block on driving into anything.
Test runs: 200 runs at fixed difficulty in a cluttered room with partitions and alcoves, 3–5 goals each.
- Body
- Tracked base, 30 × 19 cmTop speed 0.36 m/s; drives at up to 0.31 m/s
- Senses
- 3D LiDAR and a RealSense depth cameraThe camera sits on a pan-tilt gimbal the robot aims itself
- Brain
- NVIDIA Jetson Orin Nano, on boardMapping, planning, the AI driver and the web console all run on the robot
- Software
- Python, PyTorch, ROS 2, FAST-LIO2, YOLO11Driver trained with reinforcement learning in a GPU simulator I wrote
The journey
From a bare chassis to real rooms
-
Early August 2026
The idea: go there, look, report back
Most inventory work comes down to a person walking to a spot and looking at something: counting a shelf, finding a box, checking that an aisle is clear. It's repetitive, it's everywhere, and it's exactly the kind of job a robot should take over.
The core skill is general: get to a place on your own, don't hit anything on the way, and look at what's there. I started by prototyping an AI driver in NVIDIA Isaac Lab and testing it in a phone-scanned copy of a real room it had never trained in.
What I foundCheck the test before blaming the model. Two unfair setups in my test were hiding real progress. Fixing them tripled the prototype's score in the held-out room.
-
August 8–12
Giving it senses
I built the robot around a small tracked chassis, a LiDAR on top and an NVIDIA Jetson as its brain. FAST-LIO2 fuses the LiDAR with an inertial sensor to give the robot a live map and its own position on it, and a route planner finds the way to any goal on that map. On its best early run, the path it drove was only 2% longer than a straight line.
Then the quiet bugs appeared: problems that produce believable output that's wrong. The LiDAR's 30° downward tilt hid obstacles from the planner. The position it reported was the sensor's, not the robot's, so every on-the-spot turn swung it by 11.6 cm. The sensor also sat 5° off the robot's axis.
What I foundMeasure constants from the data, on every scan. A mounting angle I measured once had drifted by 3–4°. Fitting the floor plane on every scan fixed it for good.
-
August 13–22
Teaching it to drive
Hand-written driving rules break the moment a room gets cluttered, so the robot learns to drive instead. I wrote my own GPU simulator in PyTorch, built from the robot's measured physics: its real top speed, how far it coasts, how its tracks pivot, how late its position arrives. The AI driver practises in endlessly varied rooms, sees the world the way the robot does (50 LiDAR rays across the front, 13 sectors behind) and makes a decision about 12 times a second.
Before any training run, a scripted expert driver has to pass the same test. If a hand-built driver can't do the task, a learned one won't either, and that gate catches a broken task before it wastes a training run.
What I foundWhere training settles is a design choice. Letting practice rooms get harder than the driver's skill made it bump into things in 77% of runs. Capping the difficulty brought that to about 17%, and produced the driver that runs on the robot today.
-
August 23–24
The hard part: from simulation to the real world
On the real robot, the driver that aced the simulator spun in circles, turning up to 439° for every metre it drove. Instead of retraining and hoping, I recorded everything the robot sensed and compared it, channel by channel, with what it saw in simulation.
Five faults surfaced, one after another. Its sense of its own turning was estimated from its commands rather than measured, and its position arrived about 300 ms late. The mapping software deleted everything within 30 cm. Phantom obstacles caged it inside its own map. Part of the obstacle sensing read the carpet as an obstacle 90% of the time. And the route planner was hogging the control loop, so for two-thirds of the drive the AI driver wasn't running at all.
With the carpet fixed, it reached 38% of its goals. With the planner fixed too, the same trained driver reached 100% of them that afternoon, with no retraining.
What I foundA planner on the control thread is a driver that isn't running. The simulator plans in zero time; the real robot doesn't. Moving the planner to its own thread did more than any retrain could have.
-
August 25 – September 2
Real rooms are narrower than practice rooms
The real test space was a corridor 2.44 m wide. The narrowest practice room in the simulator was 3 m, so not one training room looked like the place it actually had to drive. I added rooms down to 1.8 m wide, which took corridor-width rooms from 0% of practice to 29%.
I also moved the robot's safety layer, and its habit of clearing map cells it has driven over, into the simulator, so the driver practises under the same rules it lives by.
What I foundPut every input from a real drive next to the same input from simulation. That one habit found both the carpet problem and the corridor gap.
-
September 3–6
Seeing in 3D
With driving solid, I gave it better eyes. A RealSense depth camera adds colour 3D, and the LiDAR builds a live 3D map that erases things once they've moved away. A box detector trained on about 53,000 images from 27 datasets finds boxes with 93% mAP50 on held-out images. Frames from the robot's camera can also be rebuilt into a photo-realistic 3D model of the room (a Gaussian splat) that you can view on a phone.
All of it runs from a web console the robot serves itself. You can tap a spot to send it there, drive with a thumb joystick, fence off keep-out zones, let it explore unknown space on its own, and save rooms by name.
-
September 2026 – now
Aiming its own camera
The newest driver steers a pan-tilt gimbal as well as the tracks, so it decides where to look while it drives. In a harder narrow-corridor test it reaches every goal in 93.8% of runs.
Building it also exposed a bug that had been hiding in the simulator for two weeks: its map-clearing was mirrored. Fixing it lifted the scripted expert from 71.6% to 85.9% on a harder test level. The new driver is on the robot now, and tuning it on real floors is where the work is today.
What I foundA test that shares a bug's assumptions can't see the bug, and a test that doesn't exist never fails.
Where it's useful
One skill, many jobs
The core loop of going somewhere on its own, avoiding what's in the way and looking at what's there is general. Inventory is the first target.
On the robot Tested in software Next
- On the robotDrive to a spot and scan
Pick a spot on a saved map. It plans the route, drives there around whatever is in the way, and builds a coloured 3D map of what it sees.
- On the robotMap a space
Explore a room on its own and save the map for later missions, or rebuild it as a photo-realistic 3D model.
- On the robotRemote inspection
Drive it by hand from a phone to check a spot that's hard to reach or unsafe to walk to.
- Tested in softwareFind and inspect boxes
The detector spots boxes, the robot places each one on its map, plans views of up to five sides and drives to each.
- Tested in softwareGesture commands
"Follow me", "point to go", "scan and store" and "stop", read from a person's hand and body pose.
- NextInventory cycle counts
Drive a route of shelves on a schedule, read each label or barcode and reconcile the counts with the inventory system.
- NextShelf audits
Compare what's on each shelf with what should be there, and flag gaps, misplaced stock and damaged boxes.
- NextAisle and safety patrols
Walk the floor after hours and log blocked aisles and exits, spills and pallets left in walkways.
- NextBeyond the warehouse
Rack audits in data centres, out-of-stock checks in retail, supply runs in labs and hospitals, progress scans on construction sites and overnight security rounds.
What's next
Where it's going
Still in progress
I'm still working on this project
AMR is an active build. These are the next milestones, and this page will change as they land.
- Feed the depth camera into driving and route planning, so it catches low and thin obstacles the LiDAR can miss.
- Handle moving obstacles: people and carts crossing its path.
- Run box-finding, gesture commands and label reading on the real robot, for full inventory counts on its own.
- Navigate a whole building, room to room, from saved maps.
- Make long real-world sessions as dependable as the short tests.
Expect further improvements and changes to this project as it develops.