Multi-Robot Dispatch via Shared LLM

"One LLM, ten robots, and an open question about whose latency budget is whose."

Sched, Coordinator-In-Chief AI Agent

24.10.1 The Shared-LLM Dispatcher

The minimal multi-robot dispatch architecture has three components: a fleet of robots with individual VLA executors (Chapter 24), a shared LLM dispatcher that observes the global state, and a message bus over which the dispatcher streams natural-language task assignments to each robot. The dispatcher's job is to receive the human's high-level instruction ("set the table for four people"), the current state of each robot (position, what it is carrying, current sub-goal), and emit per-robot task assignments at each planning cycle.

from dataclasses import dataclass, field

@dataclass
class RobotState:
    robot_id: str
    position_xy: tuple[float, float]
    held_object: str | None = None
    current_task: str | None = None
    battery_pct: float = 100.0
    capabilities: list[str] = field(default_factory=list)

class FleetDispatcher:
    def __init__(self, llm):
        self.llm = llm

    def assign(self, goal: str, fleet: list[RobotState]) -> dict[str, str]:
        prompt = self._build_prompt(goal, fleet)
        response = self.llm.complete(prompt, response_format="json")
        # response is a dict mapping robot_id to a sub-instruction string
        return response

    def _build_prompt(self, goal, fleet):
        fleet_str = "\n".join(
            f"  {r.robot_id} at {r.position_xy}, holding={r.held_object}, can={r.capabilities}"
            for r in fleet
        )
        return f"""You are dispatching a fleet of {len(fleet)} robots.
Goal: {goal}
Current fleet state:
{fleet_str}

Assign each robot a one-sentence sub-task. Two robots must not target the same object.
If a robot has no useful task, assign "idle".
Output strictly as JSON mapping robot_id to sub-task string.
"""

24.10.2 When the LLM Dispatcher Breaks

The pure-LLM dispatch architecture has three structural failure modes that any real deployment must handle.

Fleet size. Above ~10 robots the LLM's context window starts to struggle. Each robot's state takes 50-100 tokens; ten robots is 500-1000 tokens; the working state plus the dispatcher's reasoning takes a non-trivial fraction of the context budget. At 50 robots the dispatch latency climbs to 5+ seconds per cycle, which is incompatible with reactive control.

Conflicting objectives. Multi-robot dispatch in practice has trade-offs: balance load across robots, minimize travel distance, respect battery levels, prefer specialist robots for specialized tasks. A pure LLM can reason about these qualitatively but does not produce Pareto-optimal assignments; classical assignment algorithms (Hungarian algorithm, Auction methods) do better when the trade-offs are well-defined.

Real-time constraints. If two robots are converging on the same cup and one will arrive 200 ms before the other, neither classical nor LLM dispatch can re-coordinate fast enough. The 1-2 Hz dispatcher cadence is fine for high-level task assignment but does not handle the reactive collision-avoidance problem.

24.10.3 The Hybrid Pattern: LLM at the Top, Classical Below

2026 production fleets settle on a layered architecture. The LLM dispatcher handles two responsibilities: parsing the human's natural-language goal into a structured task list, and assigning each task to a robot capability class ("this needs a mobile manipulator", "this needs a quadruped"). A classical optimizer then handles the within-class assignment, using Hungarian or Auction methods to minimize travel distance subject to per-robot capacity and battery constraints. The two layers communicate via a structured task description format (JSON, typically).

# Hybrid two-layer dispatch.
from scipy.optimize import linear_sum_assignment

class HybridDispatcher:
    def __init__(self, llm):
        self.llm = llm

    def dispatch(self, goal, fleet):
        # Layer 1: LLM parses goal into typed tasks.
        tasks = self.llm.complete(
            f"Decompose '{goal}' into atomic tasks. "
            f"Tag each with required_capability and target_object. JSON output.",
            response_format="json",
        )
        # tasks = [{"description": ..., "required_capability": ..., "target_object": ...}, ...]

        # Layer 2: classical Hungarian assignment within capability classes.
        assignments = {}
        for cap in _distinct_capabilities(tasks):
            cap_tasks = [t for t in tasks if t["required_capability"] == cap]
            cap_robots = [r for r in fleet if cap in r.capabilities]
            cost = _build_cost_matrix(cap_robots, cap_tasks)
            row, col = linear_sum_assignment(cost)
            for i, j in zip(row, col):
                assignments[cap_robots[i].robot_id] = cap_tasks[j]["description"]
        return assignments

24.10.4 The Coordination Protocol: Tokens on a Shared Channel

Beyond task assignment, robots in a fleet need to communicate runtime events: "I have picked up cup A", "I have failed to grasp", "I need to recharge". The 2026 pattern is to expose this as a structured event stream that all robots read and write, with the dispatcher LLM consuming the event log as part of its prompt context. Events are typed (TaskCompleted, TaskFailed, ObjectGrasped, ObjectReleased) and addressed (which robot emitted them).

# The event protocol for inter-robot coordination.
import enum

class EventType(enum.Enum):
    TASK_STARTED = "task_started"
    TASK_COMPLETED = "task_completed"
    TASK_FAILED = "task_failed"
    OBJECT_CLAIMED = "object_claimed"
    OBJECT_RELEASED = "object_released"
    HELP_REQUESTED = "help_requested"

@dataclass
class FleetEvent:
    event_type: EventType
    robot_id: str
    timestamp: float
    payload: dict            # task description, object id, etc.

# The dispatcher polls the event bus each cycle and re-prompts the LLM with new events.
def dispatch_cycle(dispatcher, event_bus, fleet, goal):
    events = event_bus.drain()
    new_assignments = dispatcher.assign(goal, fleet, recent_events=events)
    for robot_id, subtask in new_assignments.items():
        send_to_robot(robot_id, subtask)

24.10.5 Emergent Behaviors

An interesting empirical observation from 2024-2025 multi-robot LLM deployments: the dispatcher LLM exhibits emergent role-assignment behavior without being prompted for it. Given a fleet of three identical robots and the goal "set the table for dinner", the dispatcher consistently assigns one robot to "fetch plates", one to "fetch utensils", and one to "place napkins", even when the prompt does not mention role specialization. This emerges from the LLM's prior over how multi-agent tasks are normally divided in human descriptions of teamwork. It is empirically useful but somewhat unpredictable, and is one reason the "LLM at the top, classical below" hybrid is preferred over pure LLM dispatch for safety-critical fleets.

24.10.6 The Decentralized Alternative

The dispatcher architecture is centralized: a single LLM observes the whole fleet. The decentralized alternative is to run a small LLM on each robot, with peer-to-peer communication via natural-language messages. Each robot decides its own next action based on its local view plus messages from neighbors. This scales better to large fleets but loses the global-optimality property of centralized dispatch.

The decentralized pattern was pioneered by the Stanford SMART-LLM work (Kannan et al., 2024) and the RoCo system (Mandi et al., 2024). It works well for fleets above ~20 robots where centralized dispatch hits context-window limits, and for fleets that operate in poorly-connected environments where a central network is unavailable (search-and-rescue, agriculture). In 2026 the trade-off is well understood: centralized for less than 20 robots, decentralized above 50, hybrid in between.

Key Takeaway

Section 24.11 drops into the implementation details: how does an LLM agent actually talk to a ROS 2 robot? The integration patterns are surprisingly close to the function-calling patterns from Part VI, but with the wire format and timing semantics that ROS 2 brings.