The latency budget each caller declares

A pick arm pausing above a tote and a planner drafting next week's schedule both call the same pool. They cannot be served the same way. The difference between them is not the model they ask for. It is how long each can wait.

Substrate makes that difference explicit. Every request carries a latency budget, declared by the caller. The budget is the single most important thing the serving layer knows about a request, and almost every scheduling decision flows from it.

What the budget is

The budget is the time the caller can tolerate between sending a request and needing an answer. Past that point, the caller must act on what it carries locally, whether that is an on-board model, a conservative default, or a controlled stop.

It is not a wish or a preference. It is a statement of fact about the caller's situation. A robot that will drop a part if it holds position too long has a short budget. A dashboard refreshing a summary has a long one. A speech caller that is transcribing a radio channel sits somewhere in between.

Because only the caller knows what it can do without an answer, the caller is the one that declares the budget. The pool does not infer it from the model name or the size of the request.

How the pool uses it

When a request arrives, the serving layer reads the budget first. It then does three things with it.

It chooses a node. On a single Czar or across a Czar Grid, each node has a queue, and the layer places the request where the queue leaves enough room to answer in time. A short budget may skip a node that would be ideal for throughput but is momentarily busy.

It decides about batching. Requests to the same model can share a forward pass, which is how a pool stays efficient. But a group can only wait as long as its most impatient member. A short budget either joins a group that is about to run or runs alone. A long budget waits briefly so that more work can ride with it.

It decides whether to refuse. If no placement can meet the budget, the pool says so immediately rather than accepting work it will fail. This is deliberate. A refusal that arrives early leaves the caller with its whole budget to fall back. A late answer is worse than no answer.

Why an honest budget matters

The temptation for an integrator is to declare a short budget for everything, on the theory that faster is always better. It is not. A short budget takes priority placement and removes the request from batching, and both of those cost the rest of the floor. If every caller claims urgency, the pool cannot distinguish the arm from the dashboard, and the arm loses the advantage it should have.

The opposite mistake is quieter. A caller that declares a generous budget but actually needed the answer sooner will wait past the point where its fallback would have been the right move. Nothing in the trace will look wrong, because the pool met the budget it was given. The symptom appears on the floor as a machine that hesitates.

The right budget is the one that matches the caller's real fallback. Vendors who know their machines well tend to arrive at it quickly, and the traces confirm it.

Budgets across a mixed floor

A real facility has callers of every kind on one pool: perception for mobile robots, language models for a planning application, embeddings for the retrieval service indexing a new manual, and a vision-language-action model guiding a humanoid pilot. Their budgets span orders of magnitude in the ordinary sense of the phrase, and that is what lets them share a pool at all.

The tolerant work fills the gaps between the urgent work. The pool stays busy without the urgent callers noticing the tolerant ones. This is the economic argument for a facility pool as much as a technical one, and the budget is what makes it possible.

Seeing budgets in the record

Every request's budget, and how much of it the pool used, is part of the trace written by the Evaluation service. An engineer bringing up a new line can sort callers by how close they run to their limit and find the ones that need a different model, a different placement, or a more honest budget.

The same record answers a question an operator will eventually ask: when the robot slowed yesterday afternoon, was that the pool missing a deadline, or the robot's own fallback doing what it was built to do. The budget, the placement, and the outcome are all there.

The serving path as a whole is described in how Substrate serves a model to the floor. How grouping respects the budget is covered in batching without missing a deadline. What happens on refusal is in when the pool cannot answer in time.