The provenance record for a node

When an auditor asks what a particular machine is, the honest answer is a history, not a label. Where it was assembled, when it was sealed, what software it has run, when that software changed, and what has been observed about it since. For a Czar node, that history is the provenance record. It is kept on the appliance, and it is written for the person who will one day have to answer for the node.

This article describes what the record contains, how it grows, and how it is used.

Where the record begins

The provenance record starts before the node reaches the site. The integration entry describes the node's assembly as a configured appliance: the configuration the customer subscribed to, built and tested as a unit. The sealing entry marks the point at which the enclosure was closed, the tamper indicators armed, and the signed software image installed. From that moment the node is a sealed artifact with a known image.

These two entries are the foundation. Everything that follows is a change to a node whose starting state is documented, rather than a series of observations about a machine of uncertain origin.

Every software load

The only way a node's software changes is a signed release, applied as a bundle. Each such load is an entry in the provenance record: the release applied, verified by its signature, and the time. Updates forward are entries. Rollbacks to the previous signed release are entries. There is no path by which a node's software could change without a corresponding line.

Because each entry names a signed release rather than a free-text version, the record is checkable. The release a node claims to have loaded on a given date can be verified against the release's signature, and the attestation the node made afterward confirms that the load took.

Attestation and Watchman join the record

The node's attestations, each time it booted, joined a Grid, or resumed work, are part of its history. So are any deviations Watchman recorded: a mismatch in the image, a disturbed tamper indicator. These are not kept in separate systems that an investigator must reconcile. They are entries in the same per-node record, in order, alongside the software loads.

The result is a timeline. A node was integrated, sealed, loaded with a release, attested at the site, admitted to a Grid, updated to a newer release, attested again, flagged by Watchman for a storage fault, rolled back, attested again, and readmitted. Each step is a line, and an auditor can read the whole sequence without leaving the appliance.

How the site and an auditor use it

Day to day, the provenance record answers operational questions. Which release is each node on. When was this node last updated. Has this node ever been flagged. Is there a node that was rolled back and never brought forward again. An operator managing a Grid of several nodes reads the records to keep the fleet consistent.

During an incident, the record is where investigation starts. A node that began refusing work at a certain time has a provenance history that either explains it, with a Watchman entry or a recent load, or rules those causes out, pointing the investigator toward the Evaluation traces instead.

An auditor's question is usually of the form: on this date, what software was running on the machine that served this decision. The Evaluation trace names the node that served the request. The node's provenance record names the release it was running on that date and the attestation that verified it. The release's signature verifies the release. The chain is complete, and every link is on the appliance.

This is what distinguishes a provenance record from a maintenance log. A log says what someone wrote down. A provenance record built from signed loads and verified attestations says what can be shown.

Held on the appliance

The authoritative provenance record for a node is on the appliance. It is not held by Element 31, because the appliance never connects to the portal and nothing in the platform sends the record out. A site that wants a copy in its own archive can take one over a route it declares, and that is the site's decision.

For Czar Chassis deployments, where the appliance is shipped by a vendor under their own name, the provenance record is the same. The vendor's brand is on the enclosure; the integration, sealing, and software history are documented the same way, and the record belongs to the site that operates the node.

The proof a node makes at each admission is in attestation before a node takes work. The continuous checks whose deviations join the record are in what Watchman watches. The software load a site makes when it steps back a release is in rolling back to the previous signed release. The appliance itself is described at e31.io/czar.