Sim2Real

A robot learns to walk in a simulation. It takes thousands of steps without falling. Then someone puts it on a real floor, and it falls over.

What changed? Maybe the floor is more slippery. Maybe a motor responds a little slower. Maybe the camera is late. The robot learned to walk in a world that was close to ours, but different in ways that mattered.

Robotics calls this the sim-to-real problem: getting something that works in a simulated world to work in the real one.

I think the same idea helps explain a much broader problem. A product works in a demo. A government pilot gets good results. A defense prototype passes a test. Then it struggles to become something people can depend on every day.

The demo left something out.

Simulations are useful because they simplify the world. You can run experiments quickly, change one thing at a time, and start over after a failure. A simulated robot can fall thousands of times without anyone replacing a broken part.

But a simulation is still a model. Research on transferring robot control to real hardware shows how behavior can depend on the simulator’s particular version of physics. Training can produce a robot that works well under those conditions and struggles when they change.

A pilot can simplify the world too. It might use carefully prepared data, a few enthusiastic users, and a founder who fixes every problem. Those are useful conditions for learning. They also hide work that someone will eventually have to do.

This is where “edge cases” become important. The term makes a problem sound rare enough to deal with later. But a system used all day, in many places, gets many chances to encounter something unusual.

Imagine a robot that completes 99 out of 100 attempts without help. That sounds good. At 100 attempts a day, it needs help once on average. At 10,000 attempts, it needs help 100 times. If each intervention takes five minutes, that is more than eight hours of human work.

The robot has the same success rate in both cases. The job of keeping it running is very different. Try changing the daily workload below.

Recovery is part of the product. After dropping an object, a robot needs to notice, try again, or safely ask for help. A person resetting the experiment by hand can hide how much work recovery requires.

At UNIT, data records were pushed through SFTP, a secure way to transfer files, over VPN connections. The data synced on a ten-minute cycle. Failed transfers needed retries, and the system had to keep delivering records reliably under those conditions.

Staff also needed to understand that the data was not live. We had to explain the ten-minute sync limitation and help them build operating policies around it. That delay had to be accounted for in how people used the software. Making the system useful required reliable transfers, clear communication, and procedures that fit the way the data actually arrived.

In defense, one version of this gap is called the valley of death: the difficult transition from developing a technology to buying and using it.

A concrete example comes from GAO’s 2023 review of directed energy weapons. Laser prototypes had shot down drones in demonstrations. Yet moving the technology into acquisition programs still faced problems. GAO identified gaps in transition planning, including finding the partners who would take the technology forward and agreeing on their responsibilities.

That is the connection I see to Sim2Real. A prototype proves something under a particular set of conditions. Everyday use requires more: a budget that lasts beyond the pilot, someone responsible for the system, a way to buy and maintain it, and people who can use it in their actual work.

These are often called obstacles to deployment. But to the person who has to run the system, they are basic requirements. They were only edge cases from the demo’s point of view.

The comparison has limits. People can change a purchasing rule; they cannot negotiate with gravity. And some prototypes should stop because they are too expensive or do not solve a useful problem. Successful testing should help make that decision too.

One technique used in robotics is domain randomization: change conditions during training so the system gets practice handling variation. That can mean changing the appearance of a scene or, in dynamics randomization, how the simulated machine and environment behave. It helps reduce dependence on one perfect setup. Testing on real hardware still matters.

I would borrow that principle for pilots. Use messy data. Include ordinary staff. Try a shift without the founder standing by. Find out what happens when a connection fails or an approval is delayed. Agree early on who will own and fund the system if the pilot works.

The same question applies to an AI tool, a factory prototype, or a business plan: what have we assumed away that someone will have to handle later?

Getting to real means finding those assumptions early, testing them, and building for what happens when they fail. The useful measure of progress is how well the system keeps working on an ordinary day.