RESOURCES
How Substrate serves models on the appliance: pools, routing, batching, and the latency budget.
Articles in Serving
- Serving
When the pool cannot answer in time
Substrate is built on the assumption that the pool will sometimes miss. When it cannot meet a caller's budget it refuses early, the caller falls back to what it carries, and safety never depended on the rack.
· 4 min read
- Serving
Routing work across a Czar Grid
A Czar Grid is several Czars serving one facility as one pool. Substrate decides which node takes each request, using the caller's budget, where models are warm, and what each node has already accepted.
· 4 min read
- Serving
Batching without missing a deadline
A serving pool stays efficient by grouping requests into one forward pass. Substrate groups work only as far as the tightest latency budget in the group allows, so efficiency never costs a caller its deadline.
· 4 min read
- Serving
The latency budget each caller declares
Every request to Substrate carries a budget: how long the caller can wait for an answer before it must act on its own. The budget shapes placement, batching, and refusal.
· 4 min read
- Serving
How Substrate serves a model to the floor
A model on Substrate is a service the whole building can call. This article follows one request from a robot on the floor into the rack and back, and explains what the platform does in between.
· 4 min read