The Handover Problem
When a robot hands you a tool, who is watching?
A robot that hands you a screwdriver has to watch two things at once: whether its own grip is about to fail, and whether your hand is really ready to take it. Watch only one, and a whole family of failures goes uncaught.
of failure trials caught by the dual-stream system
vs 0% for the unmonitored baseline
unsafe completions
a failing grip never reached the hand
human-side reaches caught
vs 8% for confidence-only , where the second stream earns its place
false-positive cost
within four points of the unmonitored baseline
Robots that can do the job still can't stand next to us.
When a Vision-Language-Action model meets conditions outside its training distribution it tends to continue confidently toward a wrong action rather than signalling doubt. In close collaboration that silent failure becomes the precondition for injury.
Industrial robots driven by Vision-Language-Action models can now interpret spoken instructions and manipulate unfamiliar objects, yet every documented commercial deployment still keeps these robots physically or temporally separated from human workers. The barrier is a safety one. When a Vision-Language-Action model meets conditions outside its training distribution it tends to continue confidently toward a wrong action rather than signalling doubt, and in close collaboration that silent failure becomes the precondition for injury. This dissertation designs and evaluates a dual-stream safety monitor for the most demanding collaborative case, robot-to-human tool handover. The first stream watches the robot, reading the policy's own internal state to estimate the chance of impending failure. The second stream watches the human, predicting reach intent from partial trajectory data. A rule-based fusion layer combines the two into a graduated response controller.
Watch both. Trust neither alone.
A safe handover means reading the robot and the human at the same time, then acting on whichever one is worried.
Stream One , watching the robot
Reads the policy's own internal state (the SAFE approach to VLA failure detection) and estimates the chance of an impending failure. Catches camera shift, lighting changes, and unfamiliar tool geometry , failures that corrupt what the robot perceives or how it acts.
Robot-side failures (camera shift, lighting, novel geometry)
Stream Two , watching the human
Predicts the human reach intent from a partial hand trajectory, reaching reliable prediction by the halfway point of the reach. Catches the human-side failure the robot can never see in its own state: an unexpected or premature approach.
Human-side failures (unexpected approach trajectory)
Fusion , a graduated response
A transparent, rule-based controller combines both streams into four graduated responses: run at full speed, slow to half speed, yield the workspace, or halt and request confirmation. Rule-based by design, so the system stays auditable for safety certification.
Combines both into an auditable, graduated controller
What the experiment showed
One thousand simulated handovers, four conditions, a criterion fixed before any data was collected.
Each stream covers a different failure family
On the three robot-side failures , camera shift, lighting, novel geometry , the confidence stream is close to perfect (98–100% intervention). On the human-side approach it catches only 8%, because that failure never shows up in the robot's own state. Only the dual-stream condition lifts the human-side number appreciably, from 8% to 28%.
Monitoring the robot adds no measurable delay
Among the conditions that proceed without consulting intent, confidence-only matches the unmonitored baseline almost exactly on handover timing. The act of watching the robot costs nothing on a handover that was going to succeed. Safety and fluency did not have to be traded against each other.
A pre-registered criterion, met in plain sight
Chapter Three committed the study to a three-part test, fixed before any data was collected. The dual-stream framework met that criterion where its single-stream and unmonitored variants did not. The intent stream is the honest weak point , and naming it is part of the contribution, because it tells the next researcher exactly where the effort should go.
| Condition | Safety intervention | Unsafe completion | Trial completed |
|---|---|---|---|
| Unmonitored baseline | 0.0% | 24.5% | 100.0% |
| Confidence-only | 76.5% | 0.0% | 23.5% |
| Intent-only | 4.5% | 28.0% | 95.5% |
| Dual-stream (full) | 81.5% | 0.0% | 18.5% |
| Condition | Completion | False positive |
|---|---|---|
| Unmonitored baseline | 100.0% | 0.0% |
| Confidence-only | 96.0% | 4.0% |
| Intent-only | 100.0% | 0.0% |
| Dual-stream (full) | 96.0% | 4.0% |
How it was tested
A deterministic PyBullet simulation, chosen because it is the only ethically defensible way to induce failure conditions near a human-shaped object.
“A robot is not safe or unsafe in the abstract. It is safe or unsafe doing a particular task next to a particular person, and the only way to know which is to monitor both the robot and the person and to act on what the monitoring says.”