Sim-to-Real Gap

"Simulation said yes. Reality said maybe. The gap between yes and maybe is where the PhD thesis lives."

Eval, Sim2Real-Pessimist AI Agent

24.13.1 Anatomy of the Gap

The sim-to-real gap decomposes into four independent error sources. Knowing which dominates your task is the first step in closing the gap.

24.13.2 Domain Randomization, the Workhorse

The most reliable bridging technique is domain randomization (Tobin et al., 2017, arXiv:1703.06907): randomize every simulator parameter that could differ from reality, train across the union of these variations, and rely on the policy to learn features that are invariant to the randomized properties. The randomized parameters depend on the task. For manipulation, the standard set is lighting (brightness, color temperature, shadow directions), camera position (pose jitter), object texture, object mass and friction, gripper compliance, and controller PID gains.

import numpy as np

class DomainRandomizer:
    """Sample a randomized simulator config for each training episode."""

    def __init__(self, rng=None):
        self.rng = rng or np.random.default_rng()

    def sample(self):
        return {
            "lighting": {
                "intensity": self.rng.uniform(0.3, 2.0),
                "color_temp_k": self.rng.uniform(3000, 7000),
                "shadow_direction": self.rng.uniform(-np.pi, np.pi),
            },
            "camera": {
                "position_jitter_m": self.rng.uniform(-0.05, 0.05, size=3),
                "focal_length_jitter": self.rng.uniform(0.95, 1.05),
            },
            "object": {
                "mass_kg": self.rng.uniform(0.05, 2.0),
                "friction": self.rng.uniform(0.3, 1.2),
                "texture_id": self.rng.integers(0, 1000),
            },
            "controller": {
                "kp_jitter": self.rng.uniform(0.7, 1.3),
                "latency_ms": self.rng.uniform(5, 40),
            },
            "sensor_noise": {
                "force_torque_sigma": self.rng.uniform(0.0, 0.5),
                "depth_dropout": self.rng.uniform(0.0, 0.1),
            },
        }

24.13.3 System Identification

The complement to randomization is identification: measure your real robot's properties (joint friction, controller gains, camera-to-base calibration) and set the simulator to match. If domain randomization is "train across many possible worlds", system identification is "shrink the variation by knowing the real world more precisely". Modern toolchains automate this: you run a calibration procedure on the real robot, the toolchain fits simulator parameters to the measurements, and the trained policy starts much closer to the real robot's actual dynamics.

# Sketch of automated system identification for a robot arm.
import numpy as np
from scipy.optimize import minimize

def identify_joint_friction(real_trajectories, sim):
    """Fit simulator joint friction coefficients to real trajectory data.

    real_trajectories: list of (t, q, qdot, tau) on the real robot
    sim: simulator object with friction setter and step method
    """
    def loss(friction):
        sim.set_friction(friction)
        err = 0.0
        for t, q, qdot, tau in real_trajectories:
            sim.reset(q=q[0], qdot=qdot[0])
            sim_q = []
            for tau_step in tau:
                sim.step(tau_step)
                sim_q.append(sim.get_q())
            err += np.mean((np.array(sim_q) - q[1:]) ** 2)
        return err / len(real_trajectories)

    result = minimize(loss, x0=np.ones(7) * 0.5, bounds=[(0.0, 2.0)] * 7)
    return result.x

24.13.4 Real-World Fine-Tuning

The third pillar is fine-tuning on real-world demonstrations. The recipe is straightforward: take the sim-trained policy, collect 100-1,000 real teleop demonstrations on the deployment robot, fine-tune the policy on the real data with a small learning rate. The result is a policy that retains the broad capability from sim training but adapts to the specific quirks of the deployment hardware. For VLAs, the LoRA recipe from Section 24.2 applies directly. For LLM planners the fine-tuning step is usually unnecessary because the planner is already operating at the abstraction layer above the dynamics gap.

24.13.5 Hardware-in-the-Loop Validation

The final piece is the validation harness. A production deployment cannot push a new policy version directly to customer-facing robots; it needs a hardware-in-the-loop (HIL) stage where the policy runs on real hardware in a controlled lab environment against a regression test suite of tasks. The 2026 best practice is a CI/CD pipeline where: (a) each policy commit triggers a sim regression test, (b) sim-passing commits proceed to a HIL test on a single lab robot, (c) HIL-passing commits proceed to a canary deployment on 1-5 percent of fleet robots, (d) canary-passing commits roll out to the full fleet. This is the same canary-deployment pattern used in software engineering, transplanted to robotics.

# Sketch of a HIL regression test harness.
from dataclasses import dataclass

@dataclass
class HilTask:
    name: str
    instruction: str
    scene_setup_fn: callable          # physically configures the lab
    success_check_fn: callable        # observes the lab after the run
    timeout_s: float = 120.0

def run_hil_regression(policy, tasks: list[HilTask], required_pass_rate=0.85):
    results = []
    for task in tasks:
        task.scene_setup_fn()
        policy.run(task.instruction, timeout=task.timeout_s)
        success = task.success_check_fn()
        results.append((task.name, success))
    pass_rate = sum(s for _, s in results) / len(results)
    assert pass_rate >= required_pass_rate, f"HIL regression failed: {pass_rate:.2%} < {required_pass_rate:.2%}"
    return results

24.13.6 Deployment Patterns in 2026

The deployment playbook that has stabilized in 2026 has five stages: (1) develop the policy in pure simulation with aggressive domain randomization; (2) calibrate simulator parameters to the specific deployment hardware via system identification; (3) fine-tune on 100-1000 real demonstrations from the deployment robot; (4) validate on a HIL regression suite; (5) canary deploy and monitor a small fraction of the fleet before full rollout. The same five stages apply whether you are deploying a VLA, a SayCan planner, or a hybrid stack.

Key Takeaway

pip install transformers accelerate torch torchvision robosuite libero gymnasium

Chapter 39 closes the LLM-robotics arc that Chapter 24 began. Chapter 24 moves to world models, the next abstraction layer that lets a policy predict the consequences of its actions before executing them. World models, sim-to-real, and VLA all share a common substrate (next-frame prediction); seeing the connection is what makes 2026 robotics feel coherent rather than a grab bag of techniques.