How Substrate serves a model to the floor

A model on Substrate is not a file a developer runs on a workstation. It is a service the whole building can call. The appliance in the rack loads it, holds it ready, and answers requests from the floor over the facility network. Nothing about the request goes anywhere else.

This article follows one request from the moment a robot or an application makes it to the moment the answer comes back, and explains what the platform does in between.

The request arrives on the facility network

A caller on the floor, whether a pick arm, a mobile robot, a kiosk, or a line-of-business application, sends its request to Substrate's Inference service over the site's own network. The caller addresses one API. It does not know, and does not need to know, which node in the rack will do the work.

Each request carries the caller's declared latency budget. That budget is the central piece of information the serving system uses. A robot mid-reach has a short budget. A nightly report has a long one. Substrate treats these differently from the first instant, and it does so without the caller having to describe anything about the hardware it is talking to.

The pool decides where the work runs

Inside the appliance, or across a Czar Grid, the model the caller asked for is already loaded and warm on one or more nodes. The serving layer picks a node that can meet the budget given its current queue and places the request there. On a Grid, this is how a whole facility shares one pool rather than each cell owning its own compute.

Requests that can share a forward pass are grouped. The grouping is bounded by the tightest budget in the group, so a tolerant caller never delays an impatient one. The caller sees none of this. It sees an answer that arrives inside its budget, or a prompt refusal that lets it act on its own.

The answer comes back, and a record stays behind

The response returns over the same path. At the same time the Evaluation service writes a trace of the operation: which model and version answered, which node ran it, how long each stage took, and what the caller sent and received, subject to the retention policy the site sets. That record lives on the appliance. It is there for the engineer who tunes the pool and for the auditor who later asks what the system did and when.

The record is also how the site learns about its own workload. A model that is routinely close to its budget shows up in the traces before anyone on the floor notices a slow robot. A caller that has quietly changed its request pattern shows up too.

What the floor does not have to do

Because the model lives in the rack, the robot does not have to carry it. A vendor can ship a lighter machine, or keep its machine as it is and offload only the heavy perception or language work. The site can change the model behind the API without touching the fleet, and can retire or roll back a release in one place rather than across every machine on the floor.

The floor also does not have to trust a network path it does not control. Substrate has no default route to the public internet. A caller inside the building talks to an appliance inside the building, and the answer never crosses the property line. If the site has declared a route to a warehouse system through Local Bridge, that route is explicit and governed by policy; it is not something the serving path depends on.

When the pool is busy or absent

Serving is designed around the assumption that the pool will sometimes be unable to answer in time. When that happens the caller is told promptly, inside its budget, and falls back to whatever it carries on board. A robot that slows or stops is the intended outcome. A robot that waits indefinitely for the rack is not.

Safety functions are never routed through Substrate at all. They stay on the machine, where the vendor put them, and they do not change when a pool is added to the building. The pool improves what a robot can perceive and decide; it does not become the thing that keeps a person safe.

The pieces of this path each have their own article. The latency budget is explained in the budget each caller declares. Grouping work without missing a deadline is covered in batching without missing a deadline. The fallback path is the subject of when the pool cannot answer in time. The platform as a whole is described on the platform page.