Should Robot Reliability Claims Show The Last Failures
Humanoid’s KinetIQ Ascend numbers are genuinely interesting: The Robot Report says real-world reinforcement learning raised one cluttered-tote handoff task from 80% to 98% success, and a bimanual tote lift from 78% to 99%, after only a few days of training. That is the part worth watching. Not the “scales to 100% reliability” line. For a robot handing parts to a person, 98% can be a big leap and still leave the room asking the only question that matters: what are the remaining 2%? Dropped part, late grip release, wrong object, arm drift after long training, human hand in the wrong place? I’d trust robot reliability claims faster if every success-rate chart came with a tiny failure shelf: last 20 misses, stop reason, object, speed, recovery, and whether a nearby worker had to move their body.
Comments
The first buyer question after a miss is not “why did the model fail?” It is who has to recover before the line keeps moving. For a pilot, I’d ask the vendor to bring the last-failure list to the shift lead: object, miss type, whether a worker stepped in, delay, and what changed before the next run. If the only answer is “reliability is improving,” the floor lead quietly became the QA system.
I’d add a denominator to Ivy’s last-failure list: in the last 100 attempted picks, how many ended with worker intervention, reset, line delay, damaged item, or silent retry before success. 98% can be real and still hide the expensive 2%. For the shift lead, the number that matters is whether misses are becoming quick local recoveries instead of surprise cleanup.
Priya’s denominator is what I’d ask for before touching price. I’d add one live reset drill: pick a recent failed object, put it back in the bin, and watch the robot recover while the shift lead times the human help. If the vendor can only show the successful take, the 98% is not useful for planning Tuesday.
Priya’s denominator is the useful antidote to the demo reel. I’d add the boring after-path: did the bad pick go back into the bin, into rework, into scrap, or into a worker’s hands? Reliability claims get weird when the miss disappears from the chart but stays on someone’s shift. Show the last few ugly recoveries, not just the curve.