Comparing VLA Models

"Six VLAs, eight metrics, one comparison table. Pick a column and you have picked a vendor."

Compass, Capability-Matrix-Operator AI Agent

24.5.1 The Capability Matrix

The single most useful artifact in this chapter is the cross-model capability table. The columns are the dimensions a practitioner cares about; the rows are the five families. Numbers reflect publicly reported metrics as of Q1 2026, normalized where possible to the BridgeData-V2 and LIBERO benchmarks.

24.5.2 Success Rates on the Public Benchmarks

BridgeData-V2 and LIBERO are the two most-cited public benchmarks in 2026. BridgeData-V2 measures real-robot success on a fixed WidowX arm across novel object placements; LIBERO is a simulator suite covering long-horizon tasks. Reporting on these two benchmarks lets you compare models that otherwise disclose different evaluation protocols.

24.5.3 Decision Tree: Which VLA Do You Pick

The following decision tree captures the working recommendations from the last 18 months of robotics-team interviews. It is not authoritative, but it matches what most production teams converge on.

def pick_vla(project):
    """Pragmatic decision tree for choosing a VLA in 2026."""
    if project.requires_humanoid_or_bimanual:
        if project.has_pi_access:
            return "pi-0.5"
        else:
            return "pi-0-fast + OpenPI training stack, accept the data gap"

    if project.vram_budget_gb < 12:
        return "TinyVLA-1.4B, accept the ~10 pp quality drop"

    if project.requires_open_weights:
        return "OpenVLA-7B with LoRA finetune on your robot"

    if project.latency_budget_hz >= 25:
        return "OpenVLA-7B + INT4 + TensorRT-LLM + speculative"

    if project.requires_long_horizon_planning:
        return "pi-0.5 if accessible, else OpenVLA + a separate planner LLM"

    return "OpenVLA-7B (the default working answer)"

24.5.4 The Action Vocabulary Axis

The most important architectural choice that distinguishes the families is how they represent actions. The three options in 2026 are discrete tokens (OpenVLA, RT-2-X, TinyVLA), diffusion (Octo), and flow matching (pi-0, pi-0.5). Each comes with predictable trade-offs:

24.5.5 Licensing and the Open-Weights Frontier

Robotics has a more polarized open-weights situation than text. The open-weights end (OpenVLA, Octo, TinyVLA) lets you finetune, redistribute, and ship in commercial products without restriction (Apache 2.0 and MIT licenses dominate). The closed-weights end (pi-0.5, RT-2-X, Figure's internal models, Tesla Optimus, 1X Neo) is gated behind partnerships, research programs, or fully proprietary stacks. The gap matters because robotics teams cannot run a closed-weights model behind their own private API the way a SaaS company can pay for a Claude or GPT-5 endpoint; the policy has to run on the robot, which means the weights have to live on the robot, which means somebody at the closed-weights vendor has to trust your deployment.

24.5.6 What Changed in the Last 12 Months

The state of this matrix as of Q1 2026 differs from Q1 2025 in three ways. First, pi-0.5 displaced RT-2-X as the unambiguous quality leader. Second, OpenVLA gained INT4 + speculative + TensorRT inference tooling that closed most of the latency gap to pi-0. Third, TinyVLA emerged as a viable embedded option for resource-constrained robots; the 1.4B variant runs on a Jetson Orin AGX at ~25 Hz with 5 GB VRAM, which a year ago was impossible.

Key Takeaway

Section 24.6 closes the chapter by stepping back to look at the limitations that all five families share: the sim-to-real gap, the dexterity ceiling, and the safety story that nobody in robotics has fully solved.