Which model runs where on a Grid

A single Czar holds whatever models fit in its memory and answers from them. A Czar Grid has several nodes, and now the question arises of which models sit on which nodes. The answer is not that every node holds everything. Memory is finite, models compete for it, and the placement of each one shapes how the whole pool behaves.

Placement is a decision the site makes through the platform. This article explains what the decision trades off and how to read the record to make it well.

Three forces on a placement

The first force is batching. A model answers most efficiently when many requests share a forward pass. That happens more often when the model's requests converge on fewer nodes. A model with patient callers, such as the embedding model that indexes documents for Retrieval, benefits from being concentrated.

The second force is proximity to urgent callers. A request with a short budget needs a node that holds the model warm and has queue room right now. If the model is on one node and that node is busy, the request is refused even though other nodes are idle. A model with impatient callers, such as the perception model serving mobile robots, benefits from being spread.

The third force is resilience. A node leaves a Grid for a replacement, a rollback, or a fault. Any model held only on that node is unavailable until it is loaded elsewhere. A model the floor cannot do without should be held on more than one node.

These forces pull in different directions, and the right placement for a given model depends on who calls it and how.

Placement by caller profile

In practice the models on a facility pool fall into recognizable groups.

Perception and vision models called by robots mid-motion have short budgets and bursty demand. They are usually spread across many nodes, often all of them, so that a warm copy is near every burst. They batch opportunistically within a burst but are never held waiting for one.

Language models serving planning applications and the agent layer have longer budgets and steadier demand. They are often concentrated on a few nodes where their requests can be grouped. If they are large, they may be the main occupant of those nodes.

Embedding models have the longest budgets and the most regular demand. They can live wherever there is memory left over, and they batch well almost anywhere, because their callers can wait.

Vision-language-action models for robots combine the worst of both: heavy, and called with short budgets. They are placed deliberately, with more than one copy, near the nodes that serve the cells where those robots work. The article on vision-language-action models covers this case.

Placement and the caller

None of this is visible to the floor. A robot calls the same API whether its model is on one node or all of them. Changing a placement does not change any caller's configuration. That is what lets the site move models around during bring-up, or after the fleet changes, without a coordinated rollout across machines.

Routing reads the current placement and the current node states, and sends each request where it can be met. Placement sets the stage; routing acts on it request by request.

Reading the evidence

The Evaluation trace is where placement decisions are made and checked. Per model, it shows how often requests were refused and why, how large the batches were, how much budget callers typically had left, and which nodes served the work.

A model with frequent refusals and idle copies elsewhere is placed on the wrong nodes or too few of them. A model with tiny batches and tolerant callers is spread when it should be concentrated. A heavily used model held on a single node is a resilience problem waiting for a maintenance window to expose it. The traces make each of these patterns plain.

An integrator doing bring-up typically starts with a reasonable placement, runs the floor for a while, reads the traces, and adjusts. The second pass is usually close to right.

When the fleet changes

A new robot vendor arriving on the floor, a line being added, or a planning tool being adopted all change the caller mix. Placement that was right for the old mix may not be right for the new one. The site does not have to anticipate this. The traces show the new pattern, and the placement follows.

If the change is large enough that no placement satisfies every budget, the pool is telling the site it needs another node. The traces show which models and which callers are short, which is the evidence a proposal for Grid expansion is built on.

How routing acts on a placement is in routing work across a Czar Grid. Why concentration helps grouping is in batching without missing a deadline. The families of model being placed are described in open-weight models on the appliance.