The scarcest data has two people in it
Data decides who wins in robotics and physical AI. The most valuable kind shows people working together, seen from every side. Almost none of it exists.
Hand someone a pair of scissors. You turn the handles towards them without thinking about it. They start to reach before you have finished the turn, and you let go at the moment you feel their grip. Nobody says a word. The exchange takes about a second, and it is still hard for a robot.
The reason is simple. A machine learns what it is shown, and almost nobody has shown it this. Data has become the most valuable input in artificial intelligence, and the richer and more exact it is, the more it is worth. The most valuable kind has more than one person in it and is seen from every side. For practical purposes, that kind does not exist yet.
What the money says
In June 2025 Meta paid $14.3bn for 49% of Scale AI. Scale builds no chips and owns no famous model. It prepares training data.
In March 2026 Yann LeCun, who led AI research at Meta for more than a decade, raised $1.03bn for his new company, AMI Labs. It was the largest seed round Europe has seen, at a valuation of $3.5bn, and Nvidia, Samsung and Jeff Bezos’s fund were among the backers. AMI builds world models, systems that learn how the physical world behaves from cameras and other sensors and use that to plan what to do next. Language is not the starting point.
LeCun likes to make the case with a sum. All the public text on the internet, the diet of the large language models, comes to about a hundred trillion bytes. A four-year-old has been awake for some 16,000 hours, and in that time about the same amount has reached its brain through its eyes. Ken Goldberg, a roboticist at Berkeley, has put the text another way: a person would need 100,000 years to read it. A child takes in as much in four years by looking. It also learns what no page states, for instance that a glass falls when you let go of it.
A gap of 100,000 years
The text was lying on the web for the taking. Nothing like it exists for physical work. Goldberg calls this the 100,000-year data gap, in a paper in Science Robotics in August 2025, and the industry is trying to close it by sheer effort.
In 2023 Google DeepMind and 21 institutions pooled 60 datasets into Open X-Embodiment: more than a million recorded robot movements across 22 kinds of robot. In November 2025 the start-up Generalist AI said its model GEN-0 had been trained on 270,000 hours of real manipulation, gathered in homes, warehouses and workplaces and growing by 10,000 hours a week. Figure, which builds humanoids, signed up the property group Brookfield, which has more than 100,000 homes in its portfolio, to film everyday life inside them. Its robot then learned to find its way through a cluttered room from human video alone.
All of them have bet on the same thing. The model can be rebuilt in a year. The record cannot.
Richer beats more
Hours are a crude measure. A video shows a hand closing on a cup. It does not show how hard the hand pressed, what the cup weighed, the click that said the lid was on, or where the eyes went half a second earlier. Each of those is a separate channel, and a model that gets all of them at once learns something that no amount of plain video can teach.
Meta understood this early. Its Ego-Exo4D dataset, built with 15 universities, recorded more than 800 skilled people, among them cooks, climbers, dancers and bicycle mechanics. Each wore camera glasses while further cameras filmed from outside, and the glasses added seven channels of sound, motion sensors, eye tracking and a 3D map of the room. It comes to nearly 1,300 hours. That is a small number, and each of those hours says far more than an hour of plain video.
Designers know the principle from drawings. A sketch with dimensions on it can be built. A beautiful rendering without them cannot. I would take ten exact hours over a thousand loose ones. Building instruments for exact records is the business of Archaí Physics, which I co-founded.
One pair of eyes is not a room
Nearly all of this data shares one limit. It shows a single person. The camera sits on one head and watches one pair of hands. Even Ego-Exo4D films one expert at a time.
Real work is rarely done alone. Two nurses turn a patient. A cook plates while another calls the pass. Two fitters carry a window up a staircase. What makes these things work is in neither person’s view. One sees the other’s hands and not their own face. The other sees a glance and a half-step. The signal is in what happens between them: who looks, who waits, who gives way, and when.
To learn that, a machine needs both views at once, lined up to the same instant, together with the room around them. A robot is meant to work beside people in homes, hospitals and workshops. It will never be alone there. It has to read intent, yield and hand over.
So how much of this has been recorded? In 2023 Microsoft published HoloAssist: 166 hours of one person doing a task while a second guided them. The second person was remote and watched through the first one’s headset. In July 2026 researchers from ETH Zurich, Microsoft, the Max Planck Institute for Informatics and TU Munich published CoMind. Pairs cook together in real kitchens. Each wears camera glasses, two more cameras watch the room, and gaze, hands and speech are all captured. The authors note that datasets from one person’s view are plentiful, that few look at people working together, and that most treat the main actor in isolation. Their own dataset covers 80 pairs and 41 hours.
Forty-one hours. Set that against the 270,000 hours of manipulation that a single start-up has gathered. The ratio is more than 6,500 to one.
Who films it
The gap will not close from a data centre. It takes rooms where people work together and agree to be recorded, with sensors chosen for the purpose and every stream on one clock. That is slow and physical, which is why it is still open. At OptAI in Zürich we work on world models, and a model cannot make up what the record leaves out.
The first robot that takes a pair of scissors from your hand as a colleague would will have learned it from two people who were filmed doing it, from both sides. Somebody has to film them first.
The pictures in this essay were made with AI.