Updates

Node agents update themselves, but only to a build signed with your release key. The panel carries the signed bundle to the nodes; every agent checks the signature against the key compiled into its own binary before it replaces itself. The panel itself is updated by hand (self-update of the panel is planned).

How trust works#

• There is one ed25519 key pair, the release key. The private half stays with you, offline. The public half is compiled into both binaries at build time (RELEASE_KEY).
• For every release you sign a manifest: the version, the build time, an expiry date and, for every binary, its name, OS, architecture, size and SHA-256.
• The agent checks, in this order: the signature against its compiled-in key, the manifest format, the expiry, that the release is newer than itself, and that the bundle has a file for its OS and architecture. Then it downloads the file from the panel over its mutual-TLS connection and checks the size and the SHA-256. Any failure leaves the installed binary untouched.
• The panel is only a courier: a compromised panel cannot make a node run code you did not sign.
• Releases are ordered by their build time (the Unix time of the source commit), not by the version string. A release with the same build time as the node is "already current"; an older one is refused as a downgrade.
• A binary built without RELEASE_KEY never updates itself. mistgate version and mistgate-node version print the fingerprint of the key a binary trusts, or say that it has none.

Publish a release#

1. Make the release key (once)#

mistgate release keygen --out ~/mistgate-release.key

It writes the private key to the file (mode 0600; an existing file is never overwritten) and prints the public key and its fingerprint (the first 16 hex characters of its SHA-256).

2. Build with the public key#

RELEASE_KEY=<public key> make build

This builds bin/mistgate-linux-{amd64,arm64} and bin/mistgate-node-linux-{amd64,arm64} and stamps into both the release key, the version (git describe --tags --always --dirty) and the build time (git log -1 --format=%ct). Build the panel with the key too: without it the panel cannot check a bundle and will not start a rollout.

3. Sign the node binaries#

mistgate release sign --key ~/mistgate-release.key \
  --version "$(git describe --tags --always)" \
  --built "$(git log -1 --format=%ct)" --expires 30d \
  bin/mistgate-node-linux-amd64 bin/mistgate-node-linux-arm64 --out dist/

• The binaries must be named <name>-<os>-<arch>, as make build names them.
• --built must be the build time stamped into the binaries: build and sign from the same commit. A new build whose own build time differs from the manifest rolls itself back after the update (built_mismatch).
• --expires is a number of days (30d) or a Go duration (720h); the default is 30d. After that the panel and the nodes refuse the manifest, so a captured old manifest cannot be replayed for long.
• The command writes dist/manifest.json and dist/manifest.sig, copies the binaries next to them, reads the result back and verifies it, then prints the version, the expiry, every file with its size and the key fingerprint.

4. Put the bundle on the panel#

scp dist/* panel.example.com:/var/lib/mistgate/dist/

The panel reads <data-dir>/dist and notices a change within a minute; Read the folder again on the Updates page reads it at once. Only regular files count (a symbolic link is not followed). Replace the whole bundle at once; changing it while a rollout runs pauses the rollout.

The Release bundle card shows the result:

The panel serves the bundle's files only to agents with a valid node certificate, through the agent endpoint, at most four downloads at a time, in resumable chunks.

The Updates page#

Everyone can view the page. Starting, pausing, resuming and cancelling a rollout, rolling a node back and Read the folder again need the owner and a fresh step-up (see Security).

• This panel: its version, build date and release key fingerprint (or "none: this build cannot check bundles"), with the steps to update the panel.
• Release bundle: the bundle in <data-dir>/dist.
• Nodes: every node with its agent version and build date, its state and its last update. The owner gets Update and Roll back per row.

"No guard against repeated crashes" under a node means its service file has no crash-loop guard: only the agent's own 5-minute self-rollback protects an update there. Run mistgate-node install with a new binary on it once to add the guard.

Rollouts#

Start#

Update all (canary first) updates every node in the "Update available" state; Update in a node's row updates one node. The dialog says which node goes first and how big the batches are.

• Canary. The first node is the one with the fewest people online, then the fewest profiles, then by name. It is updated alone.
• Batches. After the canary the rest go in batches: 1 node at a time while fewer than 5 nodes are to be updated, otherwise 2. A batch starts only when every node of the earlier ones is decided.
• One rollout runs at a time.
• A rollout ships the bundle as it was at the start. A node that is offline, cannot update itself or is already up to date when its turn comes is skipped.

One node's update#

1. The panel sends the signed manifest. The agent checks it, downloads its file, keeps its current binary as <binary>.prev, puts the new one in its place and restarts itself in place (systemd sees nothing). The node is offline for a few seconds; the admin shows it as updating, not as an outage.
2. The panel waits up to 10 minutes for the agent's answer (the download included), then up to 3 minutes for the node to come back with the new build.
3. The gate runs for up to 5 minutes after the node is back. It passes when all of this holds:
   • the new build has committed: it connected and applied its configuration without a failed profile;
   • the configuration the node applied is the panel's;
   • no profile that worked before the update has failed (a profile that stays failed for 30 seconds fails the gate at once);
   • every profile the panel can check has passed a client-eye check since the node came back (the panel starts a round at once).
4. When the gate passes, the step is done. When it fails, the panel sends the node a rollback (the previous binary comes back), the step ends as "rolled back" and the rollout pauses.

The agent's own safety net#

The agent does not rely on the panel to undo a bad update:

• A new build that has not connected and applied its configuration within 5 minutes of starting puts the previous binary back and restarts (not_committed).
• A new build whose own build time differs from the manifest rolls back at once (built_mismatch).
• The service file counts the starts while an update is pending; at the third start in a row it restores the previous binary before the agent runs (crash_loop).
• The previous build reports the outcome when it connects again.

Pause, resume, cancel#

• Pause: no new step starts. A step already in progress finishes and goes through its gate. Nodes already updated stay updated.
• Resume: the remaining waiting steps go on. Failed and rolled-back steps stay as they are; update those nodes with a new rollout after fixing the cause. If the bundle on disk changed, resume is refused: cancel and start again.
• Cancel rollout: waiting nodes are skipped; a node being updated finishes and is checked; updated nodes stay on the new version.

A rollout also pauses by itself:

The first, second and fourth also raise the alert "The node update stopped" until you resume, cancel or start a new rollout.

A finished rollout is Done, Cancelled or Finished with problems (at least one node failed or was rolled back). The page shows the active rollout, or the last one. Finished rollouts are kept: always the last 20, older ones for 90 days.

Common step errors:

Roll back by hand#

Roll back in a node's row puts the node's previous binary (<binary>.prev) back and restarts the agent. It works outside a rollout too. A node that was never updated has no previous version and refuses. If the node is part of an active rollout, its step is marked "rolled back" and the rollout pauses.

The Last update column says who rolled a node back: "Rolled back by you", "The panel put the previous version back: …" (the gate), or "The previous version came back on a command of the panel".

Old agents: update by hand once#

A node shows Update by hand when its agent cannot update itself: it predates self-update, it was built without a release key, or it runs under the first version of the service file (whose program directory is read-only). How to update in its row gives two commands, for example:

scp /var/lib/mistgate/dist/mistgate-node-linux-amd64 root@de1.example.com:/root/mistgate-node
ssh root@de1.example.com 'chmod +x /root/mistgate-node && /root/mistgate-node install'

install copies the binary to /usr/local/bin/mistgate-node, writes the current hardened service file (with the crash-loop guard) and restarts the agent. After that the node updates from the Updates page. Without a bundle on the panel, take mistgate-node of the same release as the panel.

Older agents keep working with a newer panel: changes to the agent protocol are additive. A feature an old agent lacks is reported as "agent too old" where it is needed.

Update the panel#

The panel does not update itself yet. Update it by hand:

1. Build the new version with the same RELEASE_KEY (see above). A panel built without it can no longer check bundles or start rollouts.
2. Do not update the panel while a rollout is running: pause it or let it finish. A restart in the middle of a step pauses the rollout ("the panel restarted during a step").
3. Copy the new binary next to the installed one, then stop the panel, back up the data directory, replace the binary and start it again. Here the binary is /usr/local/bin/mistgate and the systemd unit is mistgate.service, as in Install the panel; use your own names:

scp bin/mistgate-linux-amd64 panel.example.com:/usr/local/bin/mistgate.new
# on the panel server, as root
systemctl stop mistgate
tar czf /root/mistgate-backup-$(date +%F).tgz -C /var/lib mistgate
mv /usr/local/bin/mistgate.new /usr/local/bin/mistgate
systemctl start mistgate
journalctl -u mistgate -n 50

4. The database is migrated when the panel starts; nothing else is needed. The log line mistgate starting shows the new version, and so do This panel on the Updates page and mistgate version.

What happens around the restart:

• The node agents reconnect by themselves (they retry with a growing pause of 1 to 60 seconds). Their profiles keep running while the panel is away.
• During the first 90 seconds after the start, the panel opens no "Node is unreachable" alert.
• Admin sessions survive the restart.

Updates through the API and MCP#

Every API token profile can read the Updates page (GetUpdates). Starting, pausing, resuming and cancelling a rollout and rolling a node back are refused over the plain API; an MCP agent with the admin profile can only plan them, and the owner approves each plan in the admin. See API and MCP.