Robot Arm · Conveyor · Transformer Pick-and-Place

Overhead camera — model input (64×32)

This is the exact frame the transformer sees, plus joint feedback and the grasp flag.

Training loss (log scale)

Status

Mode	—
Red binned	0
Red missed	0
Dropped / wrong	0
Grasped now	no
Demo samples	0
Train steps	0
Loss	—
Policy score	—

Workflow: watch the scripted expert pick red spheres, then Record demos (runs at 5× until 4500 samples), Train the 48-dim / 2-layer / 2-head vision transformer (~45k params, trained right here in JS), then Run policy — the transformer now drives yaw, both arm joints and the gripper from the camera view alone. If it fumbles, press Refine (DAgger) a few times: the policy drives for 45 s while the expert labels every state, then it retrains on the aggregate. Or skip all of that with the Load pretrained… dropdown, which runs transformers snapshotted at different stages of offline training — from a barely-trained model to the fully DAgger-refined one — so you can watch competence emerge. They were trained on the synthetic overhead view, so the camera panel switches to that view while one is active. Drag to orbit, scroll to zoom.