At a glance
Every box, checked from three sides
Boxes travel along a conveyor under an arch carrying three cameras, all pointed at the same spot on the belt, so the top and both sides of each box are inspected. An anomaly detector flags anything that looks wrong, an object detector finds where the defect is so the arm knows where to pick, and a vision-language model makes the last check. A confirmed reject is lifted off the belt by the arm and placed in a bin, while good boxes pass through untouched.
- Task
- Remove defective boxes from a conveyorA food-production line; good boxes pass, rejects go into a bin
- Cameras
- Three, on one archTop, left side and right side, all aimed at the same point on the belt
- AI pipeline
- EfficientAD, YOLO, Qwen3-VLAnomaly detection, then defect location, then a vision-language check
- Arm
- Universal Robots UR5eDriven over RTDE and URScript
- Validation
- NVIDIA Isaac SimPhysics validation of the arm model
The journey
From a missed defect to a robot that rejects it
-
The problem
A defect slips by on a line
On a food-production line, boxes stream past on a conveyor, and most of them are fine. A crushed corner, a torn flap or a dented side is easy to miss in that stream, and a defective box that gets through carries on to the next stage of the line.
Checking every box by eye, on every face, hour after hour, is exactly the kind of job a camera system should own. That is what this cell does: it looks at every box, and when one is bad, the arm takes it off the belt.
What I foundInspection only counts if it covers every box and every face. A check that catches most defects most of the time still lets some through.
-
The cameras
Three cameras, one point on the belt
A defect can sit on any face of a box, so one camera is never enough. I mounted three on a single arch over the conveyor, all aimed at the same point on the belt: one looking down at the top, one at each side.
As a box reaches that point, the top and both visible sides are captured, so three faces of every box are inspected before it moves on.
What I foundPut the cameras where the defects can be. Aiming all three at one point means every view of a box comes from one place, which keeps the rest of the pipeline simple.
-
The pipeline
An AI pipeline, in three stages
The images go through three models, each doing the job it is best at. EfficientAD, an anomaly detector, looks first. It flags anything that doesn't look like a normal box, so it can raise the alarm on damage nobody described in advance.
If a face is flagged, YOLO, an object detector, finds where the defect is, so the arm knows where to pick. Last, a vision-language model, Qwen3-VL, makes the final check on what was flagged.
What I foundGive each stage one job. A stage that only has to say “something looks off” can afford to be sensitive, because the stages after it separate real defects from false alarms.
-
The hand-off
From pixels to a pick
A box drawn around a defect in an image is no use to an arm. That location has to become a point on the real box on the belt, and then a path the arm can follow to reach it.
The arm is driven over RTDE and URScript, Universal Robots' real-time data interface and its motion scripting language, so the pick target goes straight to the controller as a motion.
What I foundA picture of a defect is not a plan. Every step between the image and the gripper has to make sense in real units, not pixels.
-
The arm
Sorting into the bin
When a box is confirmed defective, the UR5e picks it off the belt and places it in a reject bin. Good boxes aren't touched: they pass under the cameras and carry on down the line.
So the whole chain reads as one action. Three cameras look, three stages decide, and the arm rejects, with no one standing at the belt.
-
In simulation
Validating the arm in simulation
Before trusting a robot arm with a real motion, it helps to try the motion somewhere harmless. I validated the physics of the arm model in NVIDIA Isaac Sim, where the same joint motions can run without any risk to hardware.
The ghost arm in the twin is that simulated model, drawn over the arm while it does the pick.
What I foundA simulator is only useful once you have checked that it behaves like the thing it stands in for.
-
Now
From one box to every box
The goal is simple to state: every box on the line is checked, and every defective one ends up in the bin without anyone watching the belt. The work now is making the whole chain dependable from end to end, cameras, models and arm together.
I'm building this as an AI/ML engineer intern at L&T Technology Services, where I started in April 2026. This page grows as the cell does.
What's next
Where it's going
Still in progress
I'm still building this
The UR5e cell is an active build. These are the next milestones, and this page will grow as they land.
- Tune each stage so it catches real defects and leaves good boxes alone.
- Run the whole chain, from cameras to arm, continuously on the line.
- Keep validating the arm's motions in simulation before they reach the real arm.
- Add measured results here once I can report them properly.
Expect further improvements and changes to this project as it develops.