valis / Understanding the system

valis - Architecture

What valis is

valis is the userland process the founding doctrine calls "a Lisp Machine for network protocols." Linux's eBPF makes one IP address answer on every TCP/IP port and forwards the traffic to userland; valis is the process that receives it and multiplexes a single person's data and identity across the protocols they choose to run.

valis is not a static "one machine, one person" install. It is a host-agnostic, migratable service: a person's valis can move between valis-compatible hosts anywhere on the internet, because the thing that is durable is not the process but the person's namespace. The architecture reifies Plan 9 semantics (per-process namespaces, everything-as-a-file, network-transparent resources) with the 9P protocol as intrinsic substrate rather than one protocol plugin among many. Migration, and the ability to run a person's protocol modules across more than one machine, fall out of that namespace model.

The central abstraction: every port a protocol, every protocol a plugin

The protocol surface is organised around one idea:

A protocol is a plugin. It declares the ports it answers on and knows how to handle a connection by integrating, through common interfaces, with the user's own data and identity.

This is the seed in src/:

  • src/protocol.lisp: the protocol class and the handle-connection generic function every plugin specialises. The contract every protocol plugin implements. (A long-lived protocol may instead be driven from the event loop through session callbacks and define no handle-connection at all; see the execution model.)
  • src/registry.lisp: the port → protocol registry and the dispatch entry point. A connection arrives carrying its original destination port; the registry routes it to the protocol that claimed that port.
  • src/main.lisp: process lifecycle (run / daemon / dev).

The architecture is the moat: not any single protocol, but the substrate that makes every protocol a uniform, user-owned plugin.

One caveat rides with this framing, because it is the model a reader most easily over-reads: "it declares the ports it answers on" governs dispatch (which protocol handles a connection that arrives), not reachability, whether the internet may reach that port at all. On a deployed host those are two different decisions owned in two different places, and a port's public reachability is a privileged decision at the host agent, never a consequence of a plugin declaring it. The public surface settles which ports the internet reaches and who opens them.

The keystone: the 9P namespace fabric

The durable unit is the per-process namespace: a mount table of 9P resources naming the person's data, identity, names, and the mailboxes of their running protocol modules. A protocol module is an ephemeral viewer onto that namespace, not the owner of any irreplaceable state.

Organising the system this way buys three otherwise-hard properties from a single idea:

  • Migration is cheap. A module holds no state that cannot be rebuilt by re-attaching to the namespace, so moving a person's valis to another host is closer to "restart there and re-mount" than to "serialise a live process."
  • Sovereignty is enforceable. Content lives in the namespace; operations only ever move the viewer. The substrate and lifecycle are operable without the operator ever touching the content and meaning of the data.
  • Distribution is uniform. A module on another machine is just another 9P-reachable resource in the namespace: local threads and remote Lisp nodes are addressed the same way.

Everything below hangs off this keystone.

Data flow: how a packet reaches a protocol module

outside world
     │  packets to <user-ip>:<any-port>
     ▼
┌──────────────────────────────────────────────────────────┐
│  privileged host agent, external to valis                │
│                                                          │
│   binds the privileged ports    attaches the eBPF        │
│   (:53 udp+tcp, :443, :80,      sk_lookup catchall:      │
│    the owner terminus)          every port to one socket │
└─────────┬──────────────────────────────┬─────────────────┘
          │ inherited                    │ steered
          │ (passed in at exec)          ▼
          │                       ┌──────────────┐
          │                       │  Linux/eBPF  │
          │                       └──────┬───────┘
          ▼                              ▼
┌──────────────────────────────────────────────────────────┐
│  valis process (unprivileged)                             │
│   listener: one socket, many ports                        │
│     │  accept; recover original destination port          │
│     ▼                                                      │
│   registry: port ──▶ protocol module                      │
│                         │  dispatched over 9P              │
│                         ▼  (local thread or remote node)   │
│              the user's substrate                          │
│         (namespace: data · identity · names · crypto)      │
└──────────────────────────────────────────────────────────┘

What one picture flattens: two arrival paths, at different maturities. The inherited path on the left is what carries public traffic today, and it is the older of the two. The steered path is live: on a node whose catchall is attached it takes every TCP port at the unit's address, DNS over TCP included, and the two DNS arrival paths sets out what that costs. What remains target is the shape where one node declares many ports and serves them all through the single steered socket, since a boot declares at most one port today. Both arrivals converge on the same listener, which is why they are drawn together; arrival paths treats the distinction properly.

The steering layer: sklookup (validated)

The eBPF mechanism that makes "one IP answers on every port" concrete is BPF_PROG_TYPE_SK_LOOKUP (Linux 5.9+; kernel docs): a program that runs during the kernel's listener lookup for every inbound connection, before port demultiplexing. It calls bpf_sk_assign to bind the connection to one chosen socket regardless of destination port. That socket is valis's single listener, registered in a BPF_MAP_TYPE_SOCKMAP; the program is attached to a network namespace via a BPF link.

This is the real steering primitive, and it had to be established by investigation. An earlier proof-of-concept that hooked the inet_sock_set_state tracepoint only observed connections: it could not redirect them. sklookup is the program type that actually answers for every port.

Written in Lisp: the substrate stays introspectable

The steering program is compiled by Whistler, a Common Lisp s-expression → eBPF → ELF compiler with a pure-CL loader (no libbpf, no clang, no bpftool). Keeping the kernel-facing layer in Lisp is the Lisp-Machine thesis applied to the substrate itself: the owner reads and reshapes the steering logic in the same language as everything above it.

sklookup support and a catch-all redirector were validated end to end on Linux 7.0: compiled, accepted by the kernel verifier, and attached to a network namespace through the pure-CL loader. Minimal shape:

(defmap redir-sockmap :type :sockmap :key-size 4 :value-size 8 :max-entries 1)

(defprog catch-all (:type :sk-lookup :section "sk_lookup" :license "GPL")
  (when-let ((sk (map-lookup redir-sockmap 0)))
    (sk-assign (ctx-ptr) sk 0)
    (sk-release sk))          ; mandatory: see invariant below
  SK_PASS)

Invariants worth remembering:

  • A sockmap lookup in sklookup context returns a reference-counted socket; the program must sk-release it before exit or the verifier rejects it with "unreleased reference."
  • Two kernel constants that are easy to confuse: program type SK_LOOKUP = 30, attach type BPF_SK_LOOKUP = 36.

The Whistler additions are upstream (atgreen/whistler#43), so a stock Whistler carries them.

The listener: one socket, many ports

A connection steered by bpf_sk_assign keeps its original four-tuple, so the original destination port (the port the client dialed) is recoverable in userland with getsockname on the accepted socket, with no cooperation from the eBPF program. The recovery works identically whether the socket arrived by sklookup or by an ordinary bind, and it is the ordinary-bind case that keeps the userland path testable over loopback with no kernel in the loop. On its own that arrangement would leave the sklookup case argued rather than shown. It is shown: proving-ground/driver/scenarios/steered-port-delivery.lisp puts an off-host client on the far side of a real steer, and the resident dispatches each connection on the port the client dialed, including one port that appears in no configuration on either side.

The listener is an event-loop multiplexer, not a synchronous accept loop. It is built on iolib's multiplexer over epoll; the event source is kept behind valis's own interface so an io_uring backend can be slotted in later without changing any protocol plugin. (iouring is deliberately out of the first cut: it has no mature Common Lisp support today.) Each accepted connection has its original destination port recovered and is handed to the registry, which dispatches it to the owning protocol module.

The execution model: one loop, one seam, and how a protocol is driven

Everything in this section follows from a single constraint: the event-dispatch thread must never block. Stalling inside a readiness callback stalls every other source on the loop, so any protocol that can block must be run off it.

src/listener.lisp holds the one backend-selection point: *default-multiplexer-factory* names the epoll multiplexer, and nothing else in the tree chooses a backend. Swapping in a different one is a change to that variable, not a change to any protocol.

src/executor.lisp is the indirection that keeps the loop free. The executor's dispatch-connection generic schedules an accepted connection onto whatever execution context the executor manages, and returns at once.

Three strategies exist in the tree, and which one a port gets is a property of the transport rather than of the protocol:

  • Every bound edge port builds a budget executor. It is a thread per accepted connection with a per-port concurrent-connection cap: the cap-th accept deregisters the port's listening source, so further connections wait in the kernel backlog rather than being accepted and closed, and a slot freeing re-registers it. The default cap is 64 connections per port.
  • The connectionless datagram path takes an inline executor, which spawns nothing and runs the exchange on the loop thread itself. A datagram arrives whole, so there is nothing to block on, and spawning per datagram would hand an off-path packet flood an unbounded thread-creation lever.
  • A bare thread per connection, with no budget, is the third and is what the other two are variations on.

All three sit behind the one generic, so a pooled or coroutine executor is a further subclass and the swap touches neither the multiplexer nor any protocol handler.

A protocol may instead declare :raw-descriptor stream discipline, in which case its connections are driven from the loop itself through the session callbacks rather than handed to the executor. This is what makes long-lived protocols affordable: a connected but idle session costs no thread at all, so concurrency scales with the number of active clients and never with the number of connected ones. Sizing for such a protocol is therefore against a rate of activity rather than a count of peers. The connection-lifetime assumptions measure that claim, at zero OS threads and a few kilobytes retained per idle session.

Two boundaries govern that declaration:

  • An adapter that wraps or terminates the stream cannot honestly declare raw discipline, because the descriptor beneath it carries ciphertext while the wrapper holds the bytes the protocol wants. A terminating edge therefore keeps the thread shape. Driving a continuous protocol over a terminated port from the loop is unresolved: termination hands back a blocking stream bound to a thread, so such a session costs a thread even while idle. That question gates continuous protocols over TLS, and owner mail access with them.
  • A protocol that needs to make blocking calls while serving a session, such as a namespace round trip, must still get off the loop to make them. Connection dispatch cannot carry those workers. Its unit is a whole connection lifetime: it is keyed on a port an adopted session has already resolved, it reaches the protocol through the registry's handle-connection, which a session-driven protocol no longer defines, and where a strategy owns that lifetime it closes the connection when the work returns. A per-session worker runs many times over one session, so the first one to return would tear the session down.

    Neither entry point covers work that is not a whole connection lifetime, such as a per-session worker or a timer body. A task-level dispatch that takes a thunk, owns no lifecycle and closes nothing is owed here; connection dispatch is expressible on top of it. Until it exists, each protocol spawns its own threads, and the choice between unbounded workers and a bounded pool is settled independently in each one. The two are not interchangeable: a pool smaller than the simultaneously-active set couples clients whose latency is independent today.

Arrival paths: how a wire descriptor reaches a node

Wire descriptors reach a running node two ways. The distinction is in how a descriptor arrives, never between kinds of node:

  • Inherited. The privileged host agent binds a port outside the node and passes the descriptor in. The reason is privilege rather than protocol: valis runs unprivileged in its namespace, so a port it may not bind is bound for it and handed down before privilege is dropped. Four things arrive this way (src/main.lisp): :443 for the public TLS edge, :80 for the documentation every node serves in the clear, :53 on both transports for authoritative DNS, and the routable owner terminus on whatever port the agent's config names. Three of those are TCP streams, and the steer is perfectly capable of carrying them. Only a UDP reply must source from the socket the client addressed, which is why UDP is the one transport that has to be inherited to work at all; the two DNS arrival paths sets that out, along with what it costs that :53 is split across both arrivals. Of the four, :80 is the arrival the code has not grown into: the resident parses its descriptor as --edge-http-fd and hands it to no adapter, so a node inherits the port and answers nothing on it. What :80 is for states the design; TODO.org carries where the code stands.
  • Declared and pushed. The node mints a listening descriptor and hands a duplicate to the privileged agent over the existing control connection, which is what lets the agent steer every port onto that one socket. The push direction is the property that makes the arrangement safe, and the steered-fd handoff sets out why. Note separately, at the per-namespace steering constraint, that a declaration does not by itself bound the answered set on a routable address: the agent's default-deny firewall is what narrows it.

These compose: a node inherits the descriptors the agent bound for it and declares a port of its own, and a launch may use either arrival, both, or neither. A boot declares at most one port today, since src/main.lisp refuses a --ports set carrying more than one, so "declaring for everything else" is a shape the model allows and the code has not yet grown into. Declaring stays default-closed, so a launch that only inherits behaves as it always did.

They did not compose until recently. The boot shape that adopts inherited :53 descriptors never declared its socket to the agent, so on a node booted that way nothing entered the steering map and no other port was reachable regardless of what the firewall admitted. That was a defect in the code rather than a second kind of node.

The invariant the fix must restore: authoritative DNS is a service in the :names group and nothing more. It is not a mode the node runs in, and its presence must never suppress the general arrival path for other protocols. What that protects is a node routing an arbitrary number of protocols, the surfaces at the edge/core seam and others as they are adopted. Naming none of the others here is deliberate: the model must not need to know them. If adding a protocol requires a new branch in the boot path, the model has been drawn wrongly and the exception has been allowed to suppress the rule.

Identity, capability, and the namespace boundary

The connection-open moment is an authentication event. When bytes arrive on the IP, valis must transparently establish who is calling, because the answer decides which namespace is mounted for that connection. The same port serves different worlds by identity: an anonymous request to the web port sees the published-files view; the owner, proven by key on that same port, sees the management view or the operator interface entirely.

This is a full capability system (authority can be minted, attenuated, and delegated to external actors who hold no identity of their own), not a yes/no on a fixed identity. Every primitive is established prior art; the combination is what is new. It composes as layers:

  • Transport identity. A Noise-style handshake authenticates the channel to a peer public key (a decentralized identifier), autonegotiated and invisible by default, yielding a principal, or anonymous for a keyless public request. (Noise is also libp2p's secure channel, so it arrives with the content-addressing work.)
  • An auth agent, factotum-shaped. Following Plan 9's factotum (man page), an auth agent that is itself a 9P file system, holding the owner's keys, negotiating all security interactions for other programs, and moderating which identity applies, valis keeps all key handling in one agent. Protocol modules never see keys.
  • Capability tokens. Authority is carried as delegable, attenuatable, public-key-rooted capabilities in the UCAN mould, where the owner is the authorization server directly: no central server, offline-verifiable (which matters under migration). Delegation chains double as provenance.
  • Capabilities as names. Published data is named by a capability that embeds its own access right and key, in the manner of Tahoe-LAFS read/write/verify caps and capability URLs. Handing out the name is the grant; the recipient needs no identity of their own. This is how anonymous external actors are authorized.
  • Enforcement by mounting. The resolved authority decides what is mounted into the connection's namespace. A module can only name what is mounted; sovereignty is the absence of a name, set once at namespace construction rather than checked on every access.
  • No ambient authority. A module acts only on capabilities designated to it through its namespace, never on a global identity. This is the discipline that prevents the confused deputy: a semi-trusted, possibly remote module being induced to wield the owner's authority on an attacker's behalf.

Authority resolves in two phases. Key-based transport auth fixes the principal at connect time; but a capability presented inside the protocol (an unguessable URL path, say) is known only after the request is parsed. So a connect-time principal (often anonymous) establishes a base namespace, and a per-request capability may mount further within it.

The NoiseXX handshake

The connect-time authenticator for a 9P-native peer is a Noise Protocol Framework handshake in the NoiseXX25519ChaChaPolySHA256 pattern. XX is mutually authenticating without pre-shared knowledge: neither party needs the other's static public key in advance, and both recover the other's key during the exchange. That symmetry is what lets a fresh peer dial a fresh valis and walk away with a did:key for each other: no directory, no out-of-band setup.

The exchange is three messages and binds every byte into the same transcript. The initiator opens with an ephemeral X25519 public key in the clear. The responder answers with its own ephemeral and its static public key, the static encrypted under a key derived from the running transcript; the message ends with an authenticator (an AEAD of the empty string under the same key) that proves the responder also derived the same transcript. The initiator's third message carries its static the same way, again followed by an authenticator. Each derivation step folds the latest DH output into the chaining key, and every emitted byte is folded into the transcript hash, so a single bit-flip anywhere in the conversation breaks the next decryption. After the third message a Split derives two asymmetric transport CipherStates (one for each direction) that a post-handshake transport layer would use; valis's current use ends at admission and does not exercise transport-mode encryption.

Mutual authentication here means a key recovery, not a key challenge: the responder reads the initiator's static out of the third message and publishes it to the rest of the system as a principal. There is no separate "prove you hold this key" step, because the AEAD authenticator on the message a party encrypted with the keys derived from its own static is the proof; impersonating a key requires possessing it.

The cipher behind every encrypted token is RFC 8439 AEADCHACHA20POLY1305 with the IETF 96-bit nonce shape (four zero bytes followed by the counter as little-endian uint64) and the running transcript hash as additional data. The AEAD wrapper has its own RFC 8439 §2.8.2 known-answer test, so the cryptographic primitive is anchored before the protocol that uses it.

The state machine and seven-compatible auth node are in src/identity/noise.lisp; the AEAD wrapper is in src/identity/aead.lisp. See the Noise Protocol Framework entry in the Bibliography.

The factotum auth-conversation

The factotum is mounted at /id and is itself a 9P file tree, faithful in shape to Plan 9's factotum(4) (src/namespace/id.lisp). It exposes three children:

  • /id/ctl - key management: a Tread renders the owner's key descriptors, a Twrite carries control verbs. The interface deliberately mirrors factotum(4) so an operator (or a local module) drives it the way they would drive Plan 9's.
  • /id/rpc - the auth-conversation endpoint. Every open of /id/rpc mints a fresh conversation object holding its own auth node, so two concurrent callers cannot share state through it. The rpc surface is a generic auth-protocol multiplexer, not Noise-specific: its first message may carry a proto-selection line, and the endpoint really dispatches on it, building the Noise responder for noise or the challenge driver for ssh-reality, with noise the default when a caller begins its handshake without selecting. It is the factotum's mechanism for driving whichever proof the caller wants, and the seam a further authenticator plugs into by becoming one more dispatch arm.
  • /id/proto - a list of authenticator names available behind /id/rpc, one per line. It now advertises both noise and ssh-reality. An architect or an auditor reads it to see which proofs this image admits without opening source.

The same machinery serves a remote 9P peer through seven's auth seam. A Tauth against the root binds the afid to a fresh Noise auth node by way of the node-auth method on valis-root (src/namespace/root.lisp). Subsequent Twrite and Tread operations on that afid pump the three XX messages through seven's "dumb pipe" auth carrier: seven never inspects the bytes; it carries them between the peer and the node, and parks reads until a write completes so the challenge-response ordering is correct. When the third message lands and the node-auth predicate (node-auth-ok-p) reports completion, seven calls node-attach-identity on valis-root with the approved auth node; the specialiser returns the keyed principal the Noise responder resolved from the peer's static public key, and seven records that return verbatim on the live 9P session before admitting the Tattach.

Two paths (a remote afid and a local open of /id/rpc) exercise the same Noise auth node class. The symmetry is the whole point: a local module driving the factotum and a remote peer attaching through it authenticate against the same code.

See the factotum(4) entry in the Bibliography.

Principal resolution and the authenticator seam

Once a proof completes, it has to become something the rest of valis can read. That conversion is a single named operation: the authenticate generic function in src/identity/authenticator.lisp. Everything above the seam (the connection layer, the capability layer, the namespace mounter) only ever sees a principal; only the code below the seam knows what proof produced it.

NoiseXX is the first method on this seam (noise-authenticator); anonymous-authenticator is the second. Adding a future proof (an HTTP-edge signature authenticator, for example) means writing a new class that specialises authenticate to mint the same principal type from a different proof. No code above the seam changes. The Noise implementation does not need to be read or understood to add a new authenticator; the seam is the entire contract.

The principal value type itself lives in src/identity/principal.lisp and carries a did:key identifier and the raw public key bytes that identifier encodes. For a Noise admission, the responder recovers the peer's static public key from the third message, encodes it as a did:key (multicodec 0xec01 for X25519, base58btc per the did:key method spec, in src/identity/did-key.lisp), and constructs the principal. At Tattach the principal lands on the live 9P session through seven's node-attach-identity seam: valis-root's specialiser returns it; seven stores it verbatim on the session's identity slot. Backend code reads it back through current-identity (seven) or the thin wrapper current-principal (src/namespace/root.lisp) for the dynamic extent of request dispatch. A standard file node's node-read runs synchronously on the session thread, so the principal is in scope during node-open / node-read / node-write; a backend that parks reads off-thread (the bus pattern) captures it at the synchronous call site and closes over it for delivery.

For a keyless attach (an Tattach that asks no key, e.g. a NOFID Tattach), the seam binds a distinct anonymous sentinel, never nil. The distinction is load-bearing: nil is the unresolved state before admission, the sentinel is the resolved state for "no key presented". Module code can therefore test anonymous-p without conflating "not yet authenticated" with "authenticated as nobody in particular". The sentinel carries the literal DID string "anonymous" so it is safe to log and dispatch on; the boundary that no key material reaches a module is enforced on its principal-public-key slot, which is nil. The anonymous principal is autonegotiated by the absence of a handshake, never produced by an error path and never a login prompt.

See the W3C Decentralized Identifiers and Noise Protocol Framework entries in the Bibliography.

Key custody

The custody store in src/identity/custody.lisp is the sole holder of private key material in the running image. The owner's master key is an Ed25519 signing pair; the X25519 transport key Noise uses is derived from it deterministically, so one keyfile roots both the owner's signatures and the owner's channels. The master key is held in memory and persisted to a mode-0600 keyfile so the owner's did:key is stable across restarts: an in-memory-only key would mint a fresh owner DID on every boot and break every capability rooted in it. Real at-rest encryption (passphrase / OS keyring / HSM) belongs to a later phase when real steering and the IETF edge land; the dev keyfile is sufficient for the loopback substrate.

A protocol module receives only a resolved principal: a did:key identifier and a public key. It never calls into the custody store, never holds a private key object, and never has a name through which it could ask for one. This boundary is the key-custody enforcement point: the absence of any path from a module to a private key. The custody store is wired into the factotum's node-auth path by start-fabric (src/fabric.lisp), and stop-fabric clears it; an architect who needs to convince themselves the boundary is intact can read those two lifecycle hooks and confirm that nothing else is ever handed a custody reference.

The seam this creates around the rest of the system is the answer to the "confused deputy" risk that the capability discipline above is designed to head off: a module asked to act on a peer's behalf cannot be tricked into reaching for the owner's key, because it has no name for it.

Validation coverage

Each phase success criterion is anchored to a named test in tests/identity-test.lisp that proves it. An auditor reading this document can run the suite and see the claim row stand or fail without reconstructing the coverage from source.

Criterion Test File
NoiseXX interop: both parties derive identical transport keys noise-xx-two-party-interop tests/identity-test.lisp
AEAD: RFC 8439 §2.8.2 test vector encrypt/decrypt aead-encrypt-rfc-vector, aead-decrypt-rfc-vector, aead-tampered-tag-fails tests/identity-test.lisp
did:key: round-trip identity for a 32-byte X25519 key did-key-round-trip, did-key-z6ls-prefix tests/identity-test.lisp
Noise init: h and ck are the raw protocol name bytes noise-init-h-ck-known-answer tests/identity-test.lisp
Keyed attach: principal recorded on the 9P session, DID matches peer key, readable via current-identity noise-handshake-end-to-end-over-seven, node-attach-identity-keyed-returns-noise-principal tests/identity-test.lisp
Keyless attach: anonymous sentinel recorded on the session, readable via current-identity, no error anonymous-attach-nofid, node-attach-identity-nofid-returns-anonymous tests/identity-test.lisp
Factotum reachability: /id tree walkable, /id/proto lists noise factotum-9p-reachability tests/identity-test.lisp
Key stability: owner DID unchanged after keyfile reload key-stability-across-restart tests/identity-test.lisp
Seam decoupling: principal package imports no crypto symbols principal-seam-decoupling-structural tests/identity-test.lisp

One validation limitation is recorded here rather than papered over. The AEAD primitive carries its RFC 8439 §2.8.2 known-answer vector, but the NoiseXX handshake above it is anchored only by an initialisation known-answer (noise-init-h-ck-known-answer) and a two-party self-interop (noise-xx-two-party-interop) in which both parties are valis: that proves the two sides agree with each other, not that they conform to the wider Noise ecosystem. The handshake state machine is a from-scratch implementation of a standard protocol, so before the authenticator is trusted in production it should be checked against independent published NoiseXX vectors (Cacophony / noise-c) and given a short cryptographic review; a bug shared by both sides would pass self-interop silently. This weighs more heavily as the same state machine becomes the basis other consumers in the constellation interoperate with.

Wrapping the post-attach session

The NoiseXX handshake authenticates who is calling, but the 9P session it admits still ran in the clear: every Tread=/=Twrite=/=Tclunk after Tattach crossed the wire unencrypted, authenticated only by the handshake that preceded it. That leaves a transport man-in-the-middle free to read or rewrite a peer's file traffic once admission is past. Closing that gap means every post-attach message (request and reply) is encrypted and authenticated under the Noise transport keys, and a tampered byte is refused.

The split that makes this auditable is the design's defining choice: all transport crypto belongs to mercer; valis is pure plumbing. mercer, the validated auth agent, owns the transport session: the keys, the AEAD, the two per-direction monotone counters, the nonce construction (the sequence is folded into the nonce and never appears on the wire), the orientation baked in by role, and fail-closed exhaustion. valis never touches any of it. valis owns only the keyed accept loop (it binds the socket, accepts each connection, wraps the connected stream in a secure endpoint, and hands that endpoint to the serve thread) and a thin delegating endpoint that holds nothing but an opaque session handle. Its read-frame calls the session's unseal; its write-frame calls the session's seal; it performs no cryptography of its own. The whole question "is the transport crypto correct?" is therefore answered once, inside mercer; valis's audit reduces to "does the plumbing route every post-attach frame through the session, and never leak the handle?"

The session contract (seal, unseal, close-session, and a transport-session-error condition family) is deliberately provider-neutral. It is the general secure-transport seam, not a Noise interface. NoiseXX is its first provider; a TLS provider for the IETF HTTP edges and ACME-issued certs is the intended second, and the plumbing consumes the abstract contract so that edge reuses it unchanged. Only one operation, obtaining the session from a completed handshake, is Noise-specific, and it lives at the attach boundary where valis installs the handle.

On the wire, each post-attach record is a cleartext 4-byte length prefix outside the sealed body, followed by the sealed ciphertext-and-tag. The length sits outside the seal so the reader can delimit a record before it has any key to unseal it; the AEAD tag is computed over the exact ciphertext, so any tamper at the record boundary fails authentication (which is why an empty additional- data field is safe here). The 9P engine keeps its own size[4] message framing above this layer, untouched: the wrap is a transparent record cipher beneath the protocol, not a change to it.

The pre/post-attach flip is valis-owned plumbing. The handshake, Tauth, and Tattach must cross in the clear: there is no session yet to seal them. valis installs the handle at the attach boundary (node-attach-identity in src/namespace/root.lisp), so the very first sealed record is the Rattach reply; from there every frame is delegated. Tamper is terminal: when unseal signals transport-auth-failed, valis tears the session down. There is no in-band rekey and no renegotiation: an anomaly ends the session rather than recovering it.

This is the same architecture Plan 9's 9front uses for post-auth styx encryption: factotum holds the keys and yields an opaque secret, devtls applies a record cipher (seal/open) under it, and exportfs speaks plain 9P over that cipher without knowing it is there. The mapping is one-to-one onto mercer (the keys and cipher), valis's plumbing (the record framing and the endpoint substitution), and seven (the cipher-agnostic 9P engine). The 9front-validated rules carry over verbatim: a cleartext record header outside the sealed body, a monotone per-direction sequence folded into the nonce and never on the wire, no in-band rekey, and interposition done by substituting the endpoint in the serve path.

The thin delegating endpoint is in src/transport/secure-endpoint.lisp; the keyed accept loop is in src/transport/keyed-acceptor.lisp; the attach-boundary install is in src/namespace/root.lisp. The transport session itself (every byte of the crypto) lives in mercer at mercer/src/transport.

Validation coverage: transport wrap

Crypto correctness is mercer's, not valis's: the transport session's ciphertext is anchored byte-for-byte against mercer's own round-trip and the published Cacophony / noise-c vectors under mercer 0.3.0. valis's tests prove the plumbing: that the production keyed path routes every post-attach frame through seal/unseal, flips the pre/post boundary at the Rattach, refuses and tears down on tamper, frames records with the cleartext length prefix, closes and never exposes the opaque handle, round-trips orientation both ways, and holds the session cap and drain. Each row is anchored to a named test in tests/transport-wrap-test.lisp, and each test declares in its own docstring whether it observes the real production effect (DIRECT) or an equivalent property the production path guarantees where the live serve thread offers no external observation seam (PROXY), so an auditor sees exactly which dimension is proven how.

Criterion Test File
Confidentiality on the keyed path: a post-attach frame on the wire is ciphertext, via a real keyed attach (DIRECT) confidentiality-on-keyed-path tests/transport-wrap-test.lisp
Tamper-reject and teardown: a flipped post-attach byte fails authentication and tears the session down, no plaintext (DIRECT arm + PROXY arm) tamper-rejected-and-torn-down tests/transport-wrap-test.lisp
Pre/post-attach boundary: handshake/Tauth/Tattach cleartext, the first post-attach frame delegated (DIRECT) pre-post-attach-boundary tests/transport-wrap-test.lisp
Wire framing: the cleartext length prefix delimits the record outside the seal; a boundary tamper fails the tag (DIRECT) wire-framing-length-prefix tests/transport-wrap-test.lisp
Handle lifetime: the opaque session is closed on teardown and exposed by no public accessor (PROXY) handle-lifetime tests/transport-wrap-test.lisp
Orientation cross-check: what one side seals the other unseals, both directions, pinned to mercer's round-trip anchor (DIRECT) orientation-cross-check tests/transport-wrap-test.lisp
Accept-loop parity: the session cap and drain behave as the prior listener's, adding no DoS surface (DIRECT) accept-loop-parity tests/transport-wrap-test.lisp

Validation coverage: over-the-wire conformance

The transport wrap above proves valis seals every post-attach frame, but it proves it against valis's own initiator talking to valis's own responder, both sides our code. A bug shared symmetrically by both would pass that test silently. The remaining question is conformance: does the handshake valis admits, and the transport cipher it then runs, match an externally-defined standard rather than merely agreeing with itself? That question is answered by anchoring valis's real accept loop to a published NoiseXX transcript (bytes authored by the spec ecosystem, not by us) and confirming the wire reproduces them. Because the handshake valis sends carries no payload, the only published transcript its wire can byte-match is the empty-payload NoiseXX25519ChaChaPolySHA256 vector, dual- referenced (noise-c and an independent reference) and vendored under mercer. The replay fixes all four handshake keys to the vector's values so the wire is determined by the transcript, not by agreement between two pieces of our code.

Each criterion below rides the genuine production path (start-fabrickeyed-acceptornode-attach-identity, the same loop a real peer reaches), so the proof comes off the real socket, never a direct endpoint call. The named tests live in tests/over-the-wire-test.lisp.

Criterion Test File
Handshake conformance: on the real keyed wire, the handshake messages and the responder's final handshake hash match the published empty-payload NoiseXX vector byte-for-byte (DIRECT) published-vector-rides-the-real-wire tests/over-the-wire-test.lisp
First sealed record: the first post-attach record off the wire unseals over the client session to a valid Rattach, proving the real seal path carries a sealed reply end-to-end (DIRECT) published-vector-rides-the-real-wire tests/over-the-wire-test.lisp
Transport-cipher conformance: the session derived from the real-wire handshake, sealing the vector's transport payload at the first responder send, reproduces the vector's transport ciphertext byte-for-byte, so the transport-key derivation conforms, not just the handshake (PROXY, tied to the wire by the shared session) published-vector-rides-the-real-wire tests/over-the-wire-test.lisp
Principal-scoped sealed read: over the sealed session a Twalk=+=Tread of /id/owner returns the authenticated owner DID and the principal has not collapsed to anonymous: the initiator reaches its keyed view, not a shared one (DIRECT) sealed-read-resolves-owner-did tests/over-the-wire-test.lisp

One verification here is out-of-band rather than automated: an end-to-end wrapped attach against a real, non-loopback peer. The in-process and loopback dimensions all have automated in-suite coverage above; only driving a wrapped attach across a real network interface (which needs a remote peer and mount privileges the test image does not hold) is a manual test the operator runs, confirming the post-attach frames are ciphertext on the wire. The runbook below makes that check repeatable.

Runbook: a real-NIC sealed attach (manual, operator-run)

This is the one out-of-band confirmation the architecture asks for: a sealed 9P attach driven across a real network interface, from a peer holding mount privileges, observed to be ciphertext on the wire and to reach the caller's own principal-scoped view. The automated suite proves confidentiality and conformance over an ephemeral loopback socket; loopback cannot stand in for a real NIC, and a remote peer with mount privileges is not driveable from the Lisp image, so the operator runs this once, by hand, to close it. Running it successfully discharges the previously-open out-of-band wrapped-attach check.

The procedure reuses the test image's existing VALIS_TEST_HOST opt-in: the real-NIC drive stays skipped by default and engages only when the operator points that variable at the machine's actual non-loopback address, so the manual run never fires accidentally in an ordinary suite pass.

Steps:

  1. Bring up a responder on the real interface. On the responder host, start valis in the foreground (valis run). It binds the 9P fabric on an ephemeral port and prints valis: 9P fabric on port N; note N and the host's non-loopback address. Leave it running. It tears down cleanly on SIGINT.
  2. Opt the initiator in to the real NIC. On the peer that will attach, set VALIS_TEST_HOST to the responder's non-loopback address (not 127.0.0.1 / ::1; a loopback value leaves the drive skipped by design). This is the single switch that turns the otherwise-skipped real-NIC drive on.
  3. Drive a sealed attach from the remote peer. Using the reusable sealing initiator under tests/support/ (the same harness the automated conformance tests drive), complete the keyed handshake (Tauth → the NoiseXX exchange over the afid → Tattach) against VALIS_TEST_HOST:=N=, then promote the connection to the sealed client session and issue a Twalk=+=Tread of /id/owner.
  4. Observe the two outcomes that matter. First, capture the post-Tattach traffic on the wire (e.g. with a packet capture on the interface) and confirm it is ciphertext: no cleartext 9P frame, no readable Rattach, no readable file payload. Second, confirm the sealed /id/owner read returns the attaching peer's own owner DID, the initiator reaching its authenticated principal- scoped view across the real network, not a shared or anonymous one.

A clean run (ciphertext on the wire plus the correct principal-scoped read off a real interface) is the out-of-band confirmation the architecture names, and completing it once discharges the open real-NIC sealed-attach item.

Capability tokens: delegation and confinement

A capability token is the delegable form of authority. It carries the issuer's DID (the source of the authority), the audience's DID (the grantee), a list of mount grants (each a namespace designation with a rights set), an expiry, a nonce, and an Ed25519 signature by the issuer over a deterministic encoding of those fields. A delegated token carries its ancestry inline: the proof chain back to the owner's root token rides inside the token itself, so a verifier needs no external store to walk it. That self-containment is what makes verification offline: a property that matters under migration, when the owner's substrate may be mid-move and no authorization server is reachable. The value type lives in src/capability/token.lisp; the chain walk in src/capability/verifier.lisp.

Verification walks the chain from leaf to root. The root token must be issued by the owner and signed by the owner's Ed25519 key; every token in the chain must be unexpired and absent from the revocation store. Each delegation link is then held to four conditions: the child's rights must be a subset of the parent's, the child's designation must be a path extension of the parent's, the parent must itself carry the delegate right, and the child must not outlive its parent. Its expiry may equal the parent's but never exceed it. The third condition deliberately diverges from stock UCAN: re-delegation is a structural property of the chain, not a policy judgment made at verification time, so a grantee who was not given the delegate right cannot mint valid sub-grants at all. The fourth (expiry attenuation) is what keeps a delegate that legitimately holds the delegate right from wielding it to escape its own lifetime: because equality is the loosest a child's expiry may be, a delegate cannot mint itself a longer-lived token and outlast the grant it descends from. Extending a lifetime is therefore not a delegation act at all; it takes a fresh grant minted at the root, which only the owner can produce. Together with the issuer-to-audience continuity check at each hop, this closes the confused-deputy route through token forwarding: a chain that escalates rights, escapes its designation prefix, lengthens its own life, or passes through a non-delegable hop fails verification with a stated reason.

Both verification entry points are fail-closed: every condition raised during decoding or checking is caught inside the verifier and returned to the caller as a refusal with a reason string, never as an escaped condition. The 9P layer that fronts verification can therefore treat the verifier as total.

Capability names: bearer grants and the self-certifying name

Where a token grants authority to a principal, a capability name grants authority to its holder. A name encodes a designation, a rights bitmask, an expiry, and a nonce, followed by the owner's Ed25519 signature over those bytes, the whole rendered as base58btc behind the valis: URI prefix: URL-safe, so the same string can later ride in an HTTP path. The codec is src/capability/name.lisp; verification is the same src/capability/verifier.lisp described above.

The name is self-certifying in the Tahoe-LAFS sense: verifying it requires only the owner's public key and the revocation store: no directory, no central service, no identity for the presenter. Handing out the string is the grant. This is how anonymous external actors are authorized without ever acquiring an identity: possession designates. The two costs of bearer semantics are answered structurally. Replay within the validity window is bounded by mandatory expiry: a name without a positive expiry cannot be minted. And the revocation store is the kill switch: every verification consults it, for names exactly as for tokens, so a leaked name dies the moment the owner records its hash, before expiry, not at it. The store itself (src/capability/revocation.lisp) is an append-only set of canonical hashes persisted beside the owner's keyfile; revocation is irreversible by construction. Tahoe's further trick (embedding an encryption key in the name so the storage layer never sees plaintext) is deliberately deferred until the durable store exists for it to protect.

Capabilities as mount grants

Every verified presentation (token or name) resolves to the same internal authority value: the mount directive (src/capability/directive.lisp), a designation in the owner's canonical namespace plus a closed rights set (read, write, mount, delegate). A mount directive is an assembly instruction: it tells the namespace assembler which 9P-served subtree to bind into the requester's session view, with which rights. Recipients never interpret the designation string; they find the subtree present (or absent) in their assembled view. Authorization is thereby enforced by what is mounted, not checked per access: out-of-scope objects are not denied, they are unnameable.

The per-session view is the view-root (src/namespace/view-root.lisp), an attach root whose children come from a per-instance mount table. The binding happens at the attach moment: a bearer name presented as the 9P attach name is verified and the session receives a view containing exactly the designated subtree; a name that fails verification admits the session to an empty view: fail closed, never the shared tree. A plain attach continues to bind the development tree; the policy for what base namespace a keyed versus anonymous session receives there, and the per-request second phase that mounts further capabilities into a live view mid-session, is the namespace-assembly layer's work and lands with it.

Coverage over time is intrinsic to live mounts. A mount is a live 9P binding (walks reach the serving node at access time), so new children appearing under a granted subtree are covered the moment they exist, with no reissue traffic. The converse is also deliberate: new rights verbs are never implicitly covered, because the rights set is closed and a bitmask from an older grant simply has no bit for a verb that did not exist when it was minted.

The capability service itself is 9P-addressable at /cap (src/namespace/cap.lisp), mirroring the factotum discipline: a ctl file mints tokens and names and records revocations (signing always delegated to the custody store: key material never enters the capability layer), and a verify file turns a presented token or name into a verification verdict. Modules interact with authority through file operations, never through direct calls.

Owner-client trust: two privilege tiers and proof-of-reality enrolment

An owner does not always stand at the host. The durable way to operate a deployed node is through an owner-facing client (ubik) that attaches over the sealed 9P terminus from somewhere else entirely, and that never witnessed the node's genesis. The problem this section settles is how such a client comes to hold authority without the owner seed ever moving to it. The answer is two privilege tiers, each anchored to a proof matched to what it can do, and both expressed as ordinary revocable, key-bound capability delegations, never a copy of the root secret.

Tier Authority Anchor to obtain it Carrier
Operator-equivalent unrestricted; can destroy the instance proof of reality (real host control) a maximal, revocable OCAP grant bound to the client's key
Scoped client read /mail, edit one content axis, … ordinary OCAP delegation over the terminus an attenuated, revocable OCAP token

The anchor is matched to consequence. Operator-equivalence can destroy everything, so it demands the strongest gate a genesis-less client can be held to: a demonstration of real host control, strictly stronger than mere network reach. Scoped grants are bounded and revocable, so they ride the ordinary capability path with no host-access requirement; forcing proof-of-reality on every client action would defeat the point of a delegable capability system. Crucially, neither tier moves the owner seed. Even operator-equivalent access is a delegated, key-bound, revocable capability, which is exactly what lets a client be operator-equivalent without becoming an unrevocable clone of the owner key.

Tier 2 is already the capability layer above at work: an owner (or an operator-equivalent client bearing the delegate right) mints an attenuated token via /cap ctl (audience the client's own DID, a specific mount grant, a short expiry), and the client presents it over the terminus, where the assembler binds its audience to the Noise-authenticated session principal and verifies the chain to the owner root. No host access, no new mechanism; the trust model simply uses it.

The operator-equivalent grant shape

Operator-equivalence is expressed as a single maximal grant, not a union of control-surface designations. It is one owner-custody-signed capability token whose audience is the client's self-generated OCAP DID, whose designation is the namespace root / (which prefix-covers every subtree, so the grant is maximal by reach rather than by enumerating axes that would have to be maintained as new ones appear) and whose rights are the full closed set :read :write :mount :delegate. The issuer is the owner custody root, never the client: this is a root-minted grant, never a client→client maximal self-mint. It carries :delegate deliberately, so an operator-equivalent client can itself mint Tier-2 grants for other clients without a fresh proof-of-reality each time: the proof is a one-time bootstrap of the tier, not a per-action gate.

A grant that can destroy the instance is contained not by scoping it down but by two structural limits on its lifetime and its progeny:

  • Expiry attenuation. A child token can never outlive its parent (the fourth chain condition above). So even though the grant carries :delegate and the child ⊆ parent rule permits an equal-authority re-issue, an operator-equivalent client cannot self-extend: any token it mints for itself expires no later than the Tier-1 grant it descends from. Renewing the tier is not a delegation act: it requires a fresh root grant through the proof-of-reality gate, which only the owner can drive.
  • Cascade revocation. Every descendant chains through the Tier-1 grant to the owner root, and the verifier consults the revocation store for every token in a chain. So revoking the single Tier-1 grant hash kills it and every token minted beneath it at once: one hash retires the whole subtree of authority the enrolment created.

Its lifetime is moderate and revocation-primary: expiry is the backstop, revoking the hash is the real containment lever. (Scoped Tier-2 grants invert this: short-lived and expiry-primary, because re-issue from an operator-equivalent client or the owner is cheap.) The blast radius is thus bounded by revocability and TTL, never by pretending a scoped grant is operator-equivalent when it is not.

The proof-of-reality gate: the ssh-reality authenticator

Minting the Tier-1 grant is authorized by a demonstration of real host control, delivered as a first-class /id/proto authenticator named ssh-reality, driven behind /id/rpc exactly as noise is. It is the native place for a proof-of-reality: the driver lives in valis's /id factotum, and on a satisfied proof it mints the grant through the shared /cap mint path: the privileged host agent grows no auth-mint policy of its own.

The gate's defence in depth is structural, and it requires both of two independent anchors. Either alone is refused, and the driver weighs each on its own so a diagnostic can always name which one failed rather than returning a single undiscriminating boolean:

  • Host presence. The enrolment material must carry the un-forgeable provenance stamp that only fulcrum's root-only, AF_LOCAL, never-network-reachable host-local intake can apply. A terminus-facing path can fabricate an arbitrary stand-in object, but not that stamp, so this is a structural binding, not a checked field: a leaked operator SSH key presented over the network can never satisfy it.
  • Pinned-key reality. The operator must sign valis's issued challenge with the one pinned Ed25519 operator key, and that SSHSIG must verify (by mercer) against the pinned key and a fixed provisioning namespace.

valis mints a single-use, time-bounded challenge nonce for each attempt; the operator signs it out of band with ssh-keygen -Y sign; mercer verifies the SSHSIG blob; and only when both anchors hold does the driver return a satisfied result carrying the client's bootstrap public key, from which the audience DID of the minted grant is derived. Because host presence and a valid operator signature are both required, a stolen SSH key alone mints nothing remotely, and a host-access holder without the pinned key mints nothing either: there is no second network-reachable path to the owner surface. The client's bootstrap keypair is one-time: its only job is proving live possession during the challenge, and it is spent once the grant is minted to the client's durable OCAP DID.

A fully off-host owner (one who cannot demonstrate host control at all) is out of scope for this authenticator; that case is served later by a separate pairing-code /id/proto authenticator, and the SSH path is not weakened to accommodate it. ssh-reality is one proof plugged into the seam, not the only one it will ever carry.

Where the trust material rests

The outboard trust material for this path (the pinned Ed25519 operator key, the fixed provisioning namespace every proof is checked against, and the enrolled-client records [each an audience DID paired with its minted grant's revocation hash, kept for cascade-revocation lookup]) lives in its own cache directory beside the owner root seed, under the instance state root, and is never merged into the owner keyfile. Own directory, own lifecycle: this is outboard client trust, not the root secret, co-located with the root authority only for locality. The pin is host-local config the operator sets out of band and that is never mutated over the network. Every lifetime the gate honours (the nonce window, the Tier-1 and Tier-2 token TTLs) is a named, config-tunable parameter, not a magic number buried in a check; the defaults are deliberately generous for development and the initial go-live, to be tightened once the live client interaction is observed.

Threat model and dispositions

The dispositions this design commits to, recorded so an auditor can hold the code to them:

Threat Disposition
Pure-network attacker at the terminus mints operator access mitigated: Tier 1 requires host control, not reachability
Captured token replayed by a third party mitigated: audience is bound to the Noise-authenticated session principal
Enrolment message replayed to enrol a rogue key later mitigated: a single-use, time-bounded challenge nonce bound to the specific bootstrap key
Malicious client build enrols an attacker's key mitigated: the operator drives the delivery and confirms the key fingerprint
Operator-equivalent grant becomes an unrevocable owner clone mitigated: it is a revocable, key-bound OCAP token, never the seed
A second network-reachable auth path to the owner surface mitigated: the enrolment intake is host-local, uid/root-gated, off the public surface
Leaked operator SSH key alone mints operator-equivalence mitigated: the host-presence anchor is structural; a key over the network cannot stamp host provenance
Operator-equivalent client self-extends its own lifetime mitigated: expiry attenuation caps a child at its parent; renewal needs a fresh root grant
Host compromise accepted: already total compromise (the seed is on the box); anchoring Tier 1 on host access adds nothing to lose
Client dials a spoofed node mitigated (client-side) by the server-DID pin; the server-side lever is exposure-gating

Two residuals are named rather than papered over. The trust of a genesis-less client bottoms out at one owner-controlled vouch: here, host access; the design only makes that "once" require public material, reuse an existing anchor, and yield a revocable delegate. And explicit revocation is best-effort, so short token expiry is the real containment lever: bound the blast radius by lifetime, and re-issue.

Owner key rotation

The owner's identity method is did:key, which has no update or deactivate operation: the DID is the key. Rotation is therefore destructive and total: every grant rooted in the old key dies with it. The procedure, in order:

  1. Revoke all outstanding capability hashes by writing them to the revocation store, then stop the fabric. Revoking first means a copy of the old keyfile recovered later still cannot resurrect old grants within their expiry windows.
  2. Delete the dev keyfile at ~/.valis/dev-keyfile. The keyfile is the only persistence of the master key; removing it is the rotation.
  3. Start the fabric. A fresh Ed25519 master key is generated, the owner DID changes, and the derived X25519 transport key and every factotum-served descriptor follow it automatically.
  4. Re-mint every capability name the owner wishes to keep granting. Old names became unverifiable at step 3 (their signatures no longer match the owner key the verifier is handed), so bearer names held by external parties are invalid immediately, not merely at expiry.
  5. Reissue delegation tokens to every delegate. Old chains terminate at the old owner DID and fail the root-issuer check.

There is no migration utility in v1: the procedure assumes the loopback development phase, where no real authority is outstanding. When real external grants exist, rotation tooling must walk the revocation and reissue steps mechanically rather than trusting the operator's memory.

Two planes of authority, and why the system is a federation of instances

The record of how this position was reached, with the constraints that produced it and what it means for work built on valis, is The epistemology of fully connected symbolic computers. This section remains the statement of record; that one explains it.

The classical statement of this division is in docs/REPL-AXIS.org under The hierarchy, in the classical terms, and it is the one to read first: the Lisp runtime is this machine's operating system kernel, and the capability model is that machine's programming ABI. What follows here is that argument's consequence for how instances relate, which that note poses without resolving.

Authority in this system divides into two planes, and only one of them is described by the rights above.

  • The data plane. Reading and editing projections of the operator's own data. This is what :read, :write, :mount and :delegate describe, and it is where substantially all ordinary operation happens.
  • The execution context. Root access to the running Lisp machine itself, in the sense that term carries for any machine: loading code, installing a service, changing what the node is.

A question of the form "which right is evaluation" is malformed, and the answer is not a new bit. :write cannot carry evaluation, because :write means editing a projection of the data. A new right alongside the others is the same category error under another name: it would enter a vocabulary whose central promise is that a child grant is a subset of its parent, while meaning every right at once plus the power to mint more. A holder performing correct attenuation arithmetic on such a right would get a wrong answer, and nothing in the mechanism would object.

Evaluation is not attenuated. The context is.

Once a party can evaluate in a context, they hold everything in that context by definition. Evaluation reaches every symbol, every package, this capability layer, and any key in memory, so a restricted evaluator resembles a boundary without being one.

Evaluation is therefore the wrong thing to bound, and the context is the right one. The confinement this substrate applies is outward: it governs what a node may reach, never what the party holding that node may do inside it.

An instance that executes but holds no data has an execution context that is total over itself and confers exactly its outward reach. That reach is capability-mediated 9P to the instances that do hold the data, and those mounts attenuate exactly as the rest of this chapter describes: each carries a grant hash, revocation actively evicts it, an expiry applies, and a read-only grant is wrapped so the write operation has no name.

The attenuation happens at the 9P boundary, never at the evaluation boundary. A party holds total authority over a context that holds nothing, and reaches the operator's data only through grants the operator can withdraw. Holding that context was never the same as holding the data.

The consequence: cooperating instances, not one image

This is why the system is a collection of valis instances cooperating through the capability system rather than one image serving many parties. A second party is given a second instance, not a confined region of the first. A collaborator, a vendor, or a hosted service can therefore execute without any of them holding the execution context of an instance that holds data.

An in-image confinement is not an alternative that has been left unexplored. A restricted evaluator, a sandbox, a package boundary, or an audited language subset each resemble a boundary and none of them is one, for the reason above. A better answer to the irreducibility of evaluation would change this design and would be welcome; the shape here is what the constraint currently admits.

The operational shape

Three layers, carrying very different traffic:

  1. The owner-facing viewer to an instance, over capability-mediated 9P. This is the overwhelming majority of all operation.
  2. The execution context of each instance, reached individually by real host control, for update and service installation. It is a maintenance channel rather than a user-facing capability, which is why anchoring it to host access is proportionate: whoever holds the machine already holds everything the channel would grant.
  3. Administration and service installation across instances, intended to be mediated by consensus over material signed by other instances holding operator authority, including instances a vendor maintains for the benefit of its clients. What crosses that boundary is an attestation that an artifact is legitimate; what never crosses it is access to the context that installs the artifact. This layer is direction rather than built shape.

The seam for the third layer already exists. Module admission runs through a content hash plus an owner vouch carrying the :admit right. Federating administration changes the signer of that vouch from the owner alone to a quorum, and needs no new mechanism. :admit belongs to the execution context rather than to the data rights it is currently listed beside.

Open questions: bootstrap authority, and the owner's reach into a live image

These are stated rather than answered. They are recorded here because each one constrains the capability model above, and a reader who takes that model as complete would be misled. Each will be answered in this document when implementation settles it, not before.

The bootstrap problem, and why one obvious answer is rejected

Authority must exist before the system that grants authority is coherent. That window is real twice over: while an operator is still building an instance into the program they want, and on the recondense path, where control may be needed before the instance is whole.

The tempting answer is a credential minted into the genesis image. It is rejected in that shape, and the reasoning generalises past this instance:

  • It cannot be revoked. It lives in the artifact, therefore in every copy, so withdrawing it means rebuilding every node that carries it. Nothing in the eviction machinery can reach it.
  • It inverts what makes possession of the machine survivable. Possession is a fact that ends when the machine is lost. A credential inside the image is a secret that outlives seizure, is extractable without the owner present, and survives evacuation, so evaporating does not shed it.
  • It accretes dependents. Once recovery, deployment, and habit assume it, removing it is a breaking change, and it becomes permanent by drift rather than by decision.

⚠ The general rule, which applies well beyond this question: anything that sits beside the capability model rather than inside it is the thing that can never be hardened away. A bootstrap credential belongs inside the model, as a grant that carries an expiry and that revocation reaches, held by the operator rather than contained in the artifact, absent by default, and empirically demonstrable as absent from a build that excludes it. That last test is the hard one, and this project has one instance of how it fares in practice.

The example is the development listener, and it did not end with the test being run. Its absence from the delivery binary was argued from the build graph and never shown directly; then, on 2026-07-29, the absence was ruled away instead, and every node now carries the listener behind a gate. The demonstration was never owed again because the claim it would have supported was withdrawn.

⇒ The lesson survives the example and is worth stating on its own: an absence argued from the build graph is a claim about the build, not about the artifact, and until something measures the artifact it stays that. A bootstrap credential inside the model does not need that measurement, which is a further reason to prefer it over one that sits beside the model and must be argued away.

Why the owner's reach into a live image cannot be a canonical axis

An owner attaching receives the full canonical frame, and entries placed there by base policy carry no grant hash, so both eviction sweeps pass over them. Any axis mounted that way can be removed only by fencing the entire session, and ordinary revocation cannot reach it at all.

For a service that evaluates code on behalf of the owner, that is the wrong placement, and for the most consequential surface in the system it is the worst place to get it wrong. Such an axis must be capability-backed, established through the second phase and carrying a grant hash, so revocation reaches it on its own, an expiry applies to it, and it can be withheld from a delegate who legitimately holds other rights.

Note also that evaluation cannot be confined from the inside: evaluation reaches every symbol, every package, this capability layer, and any key in memory. A restricted evaluator resembles a boundary without being one. The boundary is therefore the designation, which the rule that a delegate's designation must extend its grantor's already supplies: a delegate holding publication rights alone cannot name such an axis, and is not refused it so much as unable to see it.

The design itself is carried in docs/REPL-AXIS.org, which is where the placement is settled, the shapes that were considered and rejected are recorded with the reasoning that rejected them, and the questions still genuinely open are listed.

Finding (2026-08-16): the first piece has been settled by implementation, and it is not the axis. What shipped is the listener the axis is proposed to be reached through, on the terms ruled on 2026-07-29: every node carries it, and it must be possible to turn off. A node that cannot be reached is a node that cannot be changed while it runs, and staying changeable while running is why this system is written in Lisp; withholding the listener from every node bought that property away on all of them to keep it on none.

The consequence that matters to this section is that an absence guarantee has become a gate guarantee, which is the strictly harder proof, and the cost is accepted rather than overlooked. Presence is not access: nothing starts at load time, the resident starts a listener only when the environment names a port, and a routable host is refused unconditionally rather than bound.

The axis itself remains unimplemented, and the two are easy to conflate. What exists is an ambient switch reached from the host, which is precisely the shape the axis is meant to replace: it is not capability-backed, carries no grant hash, and revocation does not reach it. Every argument above about placement therefore still stands undischarged, and the questions this document defers to REPL-AXIS.org are still open.

Recovering an identity without holding a durable secret

Reconstitution authority is modelled as a quorum whose parameters are presently one of one, satisfied today by an off-host secret; changing those parameters later moves nothing above the interface. Two properties recorded elsewhere in this document make the general case more tractable than it first appears: the durability cluster imports no capability, namespace, or identity symbol, so a store cannot read what it holds as a matter of construction rather than of policy; and embedding an encryption key in a capability name, so a storage layer never sees plaintext, was deferred only until a durable store existed to protect. That precondition is now met.

⚠ Two costs are not cryptographic and should not be discovered late. Content addressing conceals contents and does not conceal requests, so the act of recovering can disclose who is recovering, from where, and when. And a quorum that cannot be reached is a quorum that is not held, which trades a secret surrendered under seizure for a recovery denied under censorship, at exactly the moment recovery is wanted.

Validation coverage: capability layer

As with the identity layer above, each capability success criterion is anchored to a named test an auditor can run. All tests live in tests/capability-test.lisp.

Criterion Test File
Delegation chain verifies offline against the owner's key token-chain-valid tests/capability-test.lisp
Attenuation: child rights must not exceed the parent's token-chain-child-exceeds-parent-rights tests/capability-test.lisp
Re-delegation requires the delegate right token-chain-no-delegate-right tests/capability-test.lisp
Expired tokens are rejected token-chain-expired tests/capability-test.lisp
Revoked tokens are rejected within their expiry window token-chain-revoked tests/capability-test.lisp
Name round-trips designation, rights, expiry, nonce, signature cap-name-encode-decode-round-trip tests/capability-test.lisp
Valid name yields its mount directive; bad signature or expiry refuse cap-name-verify-valid, cap-name-verify-bad-sig, cap-name-verify-expired tests/capability-test.lisp
Revoked names are rejected within their expiry window cap-name-verify-revoked tests/capability-test.lisp
The name is the grant: attach with it, walk the designated subtree, out-of-scope axes unnameable cap-name-attach-and-walk tests/capability-test.lisp
The capability service is reachable over live 9P cap-9p-ctl-reachable tests/capability-test.lisp
Pure capability packages import zero crypto symbols directive-imports-no-ironclad, token-imports-no-ironclad, name-imports-no-ironclad tests/capability-test.lisp

Two-phase namespace resolution

A session's namespace resolves in two phases. At connection time, the principal established during the Noise handshake (or the anonymous sentinel for keyless connections) determines the base namespace the session receives. This is the connect-time assembly in src/namespace/assembler.lisp, the function build-base-view. A connection whose principal matches the owner's key receives the full canonical namespace: all axes present, all of /proto, /bus, /id, /cap, and /ctl named and reachable. The anonymous sentinel receives the read-only /pub published view and nothing else, the same surface an unauthenticated HTTP GET resolves to (see The anonymous published view across both edges, below). A keyed peer who is not the owner receives an empty namespace: the connection is admitted, but there are no names to reach for. Sovereignty is the absence of a name.

After the base namespace is established, a second phase runs when a capability is surfaced alongside a request. Writing a capability to the session's /ctl node (or calling the in-image assembler API directly) mounts the designated subtree into the live view. The capability is verified through the same path as bearer-name attaches: the designation is resolved, rights are applied, and an audience check is made for audience-bound tokens. A read-only grant produces a deep read-only proxy (see below); a write-capable grant mounts the raw node. The mount persists until the session detaches or the capability is explicitly unmounted.

Base-namespace policy

The base policy has exactly three cases, decided once at Tattach time:

  • Anonymous connection (NOFID or keyless): the read-only /pub published view, and only that. /pub is published (public by design), so an anonymous mount names it and traverses the published items beneath it, but the floor is wrapped in a read-only projection: every write, create, and publish is refused. The credential-gated axes (/proto, /bus, /id, /cap, /edge) have no name to reach for. If the publication substrate is not up the floor is simply empty: fail-closed, never an error. Capabilities mounted in the second phase still widen the view from here.
  • Keyed connection whose principal is not the owner: empty namespace. Knowing who a peer is does not grant them anything; authority arrives only as capabilities, not as identity alone. A non-owner keyed principal does not receive the /pub floor: only the anonymous sentinel does.
  • Connection whose principal's key matches the owner's key: full canonical namespace. The owner is the root of all authority: for the owner, every axis has a name by definition.

The policy is expressed as a function in src/namespace/assembler.lisp. The identity check compares the connecting principal's did:key identifier against the owner's DID from the key custody store (src/identity/custody.lisp). If the custody store has not been initialised (for example, in tests that exercise individual nodes without starting the full fabric), the owner-DID check is skipped and the connection receives an empty namespace: a deliberately fail-closed default.

Bearer-name attaches bypass the base policy: a capability presented as the 9P attach name is verified by the capability service and mounts exactly the designated subtree, regardless of the connecting principal's identity. The name is the grant.

The anonymous published view across both edges

The /pub published view is the one thing an unauthenticated caller can see, and it looks the same whichever edge they arrive through. An unauthenticated HTTP GET resolves to the read-only /pub view via the anonymous edge grant the seam holds; an unauthenticated 9P mount (access=any, no credential) resolves to the same read-only /pub floor through build-base-view's anonymous branch. Both are projected read-only, so neither path can write or publish. The credential-gated axes (/proto, /bus, /id, /cap, /edge) stay absent for the anonymous principal on either edge: an out-of-scope name fails the walk rather than returning a refusal, so a caller cannot even confirm the name exists. The owner's full frame is built by assemble-canonical-frame; the anonymous floor and the owner frame are the two ends of the same build-base-view policy, with the non-owner keyed principal resolving to absence between them.

Union directories and bind semantics

A session namespace is an ordered list of mount entries. When two grants cover the same leaf name, the entries form a union: walking the name tries entries in list order and returns the first successful resolution. Directory reads concatenate entries from all union members without deduplication: both entries appear in a listing even when their names collide, because the collision is resolved at walk time, not at list time. This follows the Plan 9 and 9Front kernel's union-directory semantics.

Three bind flags govern how a new mount is added to an existing name:

  • Replace: the new entry becomes the sole entry for that name; any prior members are removed.
  • Before: the new entry is prepended to the member list; it is searched first on walk.
  • After (default): the new entry is appended to the member list; it is searched last on walk. The existing members keep precedence on name collision.

The bind flag is a choice made by the caller at mount time: it belongs to the presentation, never to the grant. A grant carries a designation and rights only; the flag is supplied separately in the ctl grammar or the assembler API.

When a create request targets a union name, the first union member that has write rights receives the create. This is a simplified reading of the Plan 9 MCREATE rule, adequate for the current namespace structure where grant rights are known at assembly time.

The union semantics are implemented in src/namespace/union-node.lisp and exercised by the assembler; the seven server never interprets union structure: it invokes the backend's node-walk and node-entries methods and the union node responds appropriately.

Read-only projection enforcement

When a grant carries only read rights, the assembler wraps the designated subtree in a read-only projection node at mount time (src/namespace/projection.lisp). The projection is a deep proxy: walking through it returns further projections, so every node in the subtree seen through the projection is also read-only. Write, create, remove, and wstat operations on a projected node fail at open time: the write path has no name, not merely an error response.

The projection node passes through the inner node's QID unchanged. A QID identifies a resource object, not a view of it, so two sessions projecting the same underlying node present the same QID on the wire. This is correct 9P2000 behaviour: the projection is transparent at the protocol level.

Enforcement is by construction: the projection is what was mounted; there is no policy lookup on the hot path. The decision is made once when the grant is assembled, and thereafter no per-access check is needed or present.

Revocation eviction and live sessions

When a capability is revoked, the revocation takes effect immediately in all live sessions that have mounted it. The revocation path (src/capability/revocation.lisp) persists the revoked hash, then calls the assembler's eviction sweep. The sweep walks the live-session registry and removes every mount entry whose stored grant hash matches the revoked hash.

The registry is a weak-reference table keyed by the live namespace root, constructed via make-hash-table with :weakness :key (an SBCL extension). What it maps that root to is a live-view-entry, and the entry carries considerably more than the session's principal: the session id an operator has to be able to type, the auth conversation the session was admitted through, the Unix seconds at which it was admitted and at which it was last used, a fenced marker that is nil for a live session and the second of the fence otherwise, and a per-view lock. The id exists because the registry is keyed by object identity, which is unspeakable outside the image, so an operator looking at a session they want stopped needs a name for it. The same entry is also held on the view itself, so the request path stamps its activity mark from a slot read rather than a lookup on a weak table.

The lock ordering is a discipline rather than an aside. A sweep snapshots the registry under *live-view-registry-lock* and releases that lock before it takes any per-view lock, which maintains one acquisition order throughout: registry lock, then per-view lock. Two paths that take the same two locks in opposite orders deadlock the fabric, and a deadlocked fabric answers nothing at all. Within that order a sweep holds the session's own lock across the removal, so a session's namespace is consistent before and after; no session sees a partially-evicted state.

Mount entries created by the base-namespace policy carry no grant hash and are never affected by the revocation sweep. Revocation targets only capability-backed mounts established through the second phase. That immunity is correct for what the sweeps do and is depended on elsewhere, and it is also why withdrawing an owner's live authority needs a mechanism of its own: see owner session lifetime below.

Weak references are what make an ended session safe to forget. A session whose view root is no longer reachable by anyone is collected without an explicit deregistration hook, and that is a settled arrangement rather than an outstanding one. The gap worth naming is a different thing and it concerns sessions that are still live: a handle already open below the root of a session that has been fenced goes on working. The next section states that limit and what follows from it.

Owner session lifetime: fencing, idleness, and authority transfer

The owner was the one principal whose live authority could not be withdrawn. Authority that arrives by presenting a capability carries a grant hash, so both eviction sweeps can find it and take it back. Authority that arrives by proving who you are is base-namespace policy, carries no grant hash, and both sweeps pass over it by design. The owner's frame is base policy from end to end, so it was precisely the case the sweeps could not reach: short of restarting the fabric, an owner session admitted once went on naming the full frame for as long as its transport lived.

Fencing a session empties its view outright, base policy included. That asymmetry with the two eviction sweeps is deliberate: a mechanism that respected the usual immunity could not touch the very session an operator is trying to stop. What is left is a session that can name nothing, which is how this namespace refuses in general. An out-of-scope object has no name to reach for rather than a name that answers no.

A fence does not tear down the sealed transport, and it is not meant to. The connection stays up and the session on it can reach nothing, which is the same enforcement model the rest of this document describes: absence rather than denial.

A session is stopped three ways.

  • By name, at the identity door. An operator reads the live sessions, picks the one they want, and writes a fence command naming it. Fencing the session issuing the command is allowed and does exactly what it says, because an operator who has lost control of a session should not first have to work out whether they are about to cut themselves off, and an attacker sharing that session should not keep it by virtue of being the one still holding it.
  • By idleness, checked lazily. The bound is enforced by the operation that would have used the session, not by a sweeper. A dormant session issues no requests and reaches nothing; the case worth catching is someone picking up an abandoned one, and that arrives as an operation.
  • By an authority transfer. When write authority moves to another locus, every live owner session on the surrendering instance is fenced. This is a separate step from the capability sweep that runs beside it, and it decides by principal rather than by generation, because an owner's namespace is base policy and the sweep cannot reach it. Without this step the instance that gave up authority would go on serving its owner after an evacuation.

The idle bound is unset by default, and the default is a decision. Applying a bound to a deployment that never asked for one is a lockout nobody requested arriving with an upgrade. Fifteen minutes is the recommended value, short enough that an unattended browser tab stops being a live management console over a lunch break and long enough that ordinary editing is never interrupted, but it is advice a launcher or an operator adopts explicitly rather than a value that applies on its own.

Getting back in means authenticating again, and that is where the seam earns its place. A re-attach on the same transport is refused by conversation, so a client cannot simply ask for a fresh frame and hand itself back what was taken; re-running the handshake yields a conversation the fence has never seen, and that succeeds. This connects directly to the evaporate-and-recondense architecture described later in this document: an owner whose instance has relocated has to be able to ask whether the instance now answering is the one they meant, and a session that silently outlives a relocation is what makes that question unanswerable. Cutting the session is what forces it to be asked.

The limit: an open handle survives the fence

A fence does not revoke a handle already open below the session root. The read path serves an open object without re-consulting the root, so emptying a view stops walks and listings from the root while a client already holding a handle keeps using it. The same limit applies to idle expiry: a client working purely through handles it already holds never reaches the check that would fence it.

Two things follow, and both matter more than the limit itself.

The first is a rule for anyone checking a fence: walk from the session root. A probe that reuses a handle it opened before the fence will observe that handle still working and report a functioning fence as broken. The behaviour under test is reachability by name, so the test has to exercise naming.

The second is where the remedy belongs. The 9P library needs a generic per-session handle revoke, and it needs to know nothing about owners, fences, or capabilities: valis decides which session and why, and the library carries out the revocation and raises a distinct error to the client. The placement test is one line, and it settles questions like this one throughout the constellation: could another consumer of this library use the mechanism with no valis present? Here the answer is yes, so the mechanism belongs there.

That mechanism now exists upstream, as a forcible revocation of a session's handles with its own error class and a stable client-facing message string. valis does not yet call it, so the limit above is live in valis today and is described as it behaves, not as it will behave once the two are wired together.

The door an operator touches

The surface is a pair of files on the identity axis, following the split the capability axis already uses: one file performs and its sibling reports. It sits under identity rather than capability because a session is identity and transport, not a grant.

The listing is read-only and rendered afresh on every read. It advertises no length, because any length it advertised would be a number that was true a moment ago. Beside it is a control file that performs, and the result of a command is held per open, so one caller's result never surfaces on another caller's read.

It is deliberately not an evaluation surface. Command lines are split on whitespace with no reader involved, an unknown verb is ignored rather than diagnosed, and a malformed argument to a known verb produces an error line readable on the handle instead of a signal into the 9P layer. ⚠ The evaluation axis proposed in docs/REPL-AXIS.org gives that property up by its nature, which is exactly why it must be its own axis and must never be reachable from a grant that reaches a control file.

Enforcement validation coverage

Each success criterion for the enforcement-by-mounting layer is anchored to a named test that proves it. An auditor reading this document can run the suite and see each claim stand or fail without reconstructing the coverage from source.

Criterion Description Test File
Absence not refusal Out-of-scope resource has no name on the wire; walk raises an error as if the name never existed, not a permission-denied sc1-absence-of-name-over-socket tests/namespace-test.lisp
No per-access guard Walk and read paths contain no verifier calls (structural check of source) sc2-no-per-access-guard-structural tests/namespace-test.lisp
Mid-session widening Writing a capability to /ctl widens the live view; the name is absent before and present after sc3-ctl-mount-widens-live-view tests/namespace-test.lisp
Base policy: anonymous published floor Anonymous mount names read-only /pub, not the gated tree; empty and fail-closed when /pub is down anonymous-mount-sees-read-only-pub-floor, base-policy-anonymous-empty-view tests/substrate-test.lisp, tests/capability-test.lisp
Base policy: owner canonical Owner-keyed attach yields the full canonical namespace base-policy-owner-yields-canonical-frame tests/capability-test.lisp
Revocation eviction Revoking a hash removes matching live mount entries; base-policy entries with no hash survive revocation-evicts-live-mount tests/namespace-test.lisp
Self-narrowing unmount A session can shed grants; the /ctl node is irremovable by design self-narrowing-unmount tests/namespace-test.lisp
Projection denial A read-only grant's write path signals at open time projection-open-write-mode-denied, projection-write-denied tests/namespace-test.lisp
Union walk order First member wins on name collision; directory reads concatenate all members union-node-walk-order, union-node-walk-first-wins, union-node-entries-concatenated tests/namespace-test.lisp
Session has a nameable identity A live session carries an id an operator can read off the listing and type back session-carries-a-nameable-identity tests/fenceable-sessions-test.lisp
Fence reaches base policy Fencing empties the session's view including base-policy mounts, which no eviction sweep touches fence-removes-base-policy-mounts tests/fenceable-sessions-test.lisp
Fence is not undone by re-attach A second attach on the same auth conversation does not restore the frame; re-running the handshake does fenced-session-is-not-restored-by-attaching-again, authenticating-again-restores-the-frame tests/fenceable-sessions-test.lisp
Lazy idle expiry The operation that wakes an idle session fences it; with no bound configured, sessions are left alone idle-session-is-fenced-by-the-operation-that-wakes-it, an-unconfigured-idle-bound-leaves-sessions-alone tests/fenceable-sessions-test.lisp
Authority transfer fences owners Surrendering write authority fences live owner sessions and leaves non-owner sessions untouched authority-transfer-fences-owner-sessions, authority-transfer-leaves-non-owner-sessions-alone tests/fenceable-sessions-test.lisp
Fencing leaves the sweeps alone The capability eviction sweep still spares base-policy entries after the fence path exists the-capability-sweep-still-spares-base-policy tests/fenceable-sessions-test.lisp
The door reports and performs The listing names the live sessions, a fenced session reads as fenced, and a malformed command is refused rather than signalled the-door-lists-the-live-sessions, a-fenced-session-reads-as-fenced-in-the-listing, the-door-refuses-a-malformed-command tests/fenceable-sessions-test.lisp
Fence over a real socket Over a sealed client connection the door names a live session, the fence empties the session read from its root, and it takes only the session it names the-door-names-a-live-session-over-the-wire, a-fence-empties-a-live-session-read-from-its-root, a-fence-takes-the-session-it-names-and-leaves-the-other tests/session-fence-over-the-wire-test.lisp
Publication placed on a running node An owner-keyed client places a publication on a node already running, including a body spanning several messages publish-places-a-publication-on-a-running-node, publish-carries-a-body-larger-than-one-message tests/owner-publish-test.lisp

Publication substrate and wire-format adapters

The publishing category of the substrate begins here: one publication object, owned by the person, with the wire protocols as projections over it. HTTP and Gopher are the first two projections, chosen deliberately as a pair: two protocols with nothing in common but the object they render prove the object is protocol-agnostic in fact, not by assertion.

Publication object model

A publication is a directory node in the 9P namespace whose children are named attribute files (title, author, published-date, content-type, and an optional gemtext) plus a body file. The object is shaped as a Plan 9-faithful directory-of-files: each attribute is individually walkable, individually grantable, and independently readable by an adapter that needs only some fields. The five core attributes are the web-minimal metadata set; gemtext is a per-protocol representation used by the Gemini adapter when a publication wants to serve native gemtext while the generic body carries HTML for HTTP. The adapters are required to consult all present attributes when rendering: this faithfulness contract is what makes the projections equivalent views rather than competing copies.

Absent attribute files are tolerated at read time. The node layer returns defined defaults (Untitled for title, an empty author, a date derived from the publication's modification time, text/plain for content-type, an empty body), so a half-authored publication stays on the wire during creation instead of erroring until every field exists. The defaults are applied by the node layer, not the store: a store backend reports absence (a null attribute, an empty body), and the caller decides what absence means. The node classes live in src/substrate/publication.lisp; the /pub axis that roots them in the namespace is src/namespace/pub.lisp.

Protocol-agnosticism is an import invariant, not a convention. The store and publication packages import no protocol or registry package, and the adapter packages import no substrate package: an adapter's only path to a publication is a real 9P attach, walk, and read. A future adapter must preserve this boundary: if it can name a substrate symbol, it has bypassed the capability seam the design depends on.

Placing a publication on a running node

A publication reaches the published axis at any time. An owner-keyed client walks the axis over the sealed loopback fabric, creates the slug when it is not already there, and writes the content type and the body; a slug that already exists is replaced rather than refused, so a page can be revised as often as its author likes. The reason this matters is plain in the operator's terms: a node built before a page existed must still be able to receive that page.

Genesis is not a gate on content

Seeding the branded landing publication at genesis is one behaviour with two callers and no policy gate. There is no absence check, no reseeding at boot, and no record anywhere of whether something has been placed. The consequence is the point rather than an oversight: if the operator deletes a publication it stays deleted.

The principle generalises well past publications. Everything here is meant to be mutable by the operator while it runs, which is a large part of why the substrate is written in a language whose systems are patchable live. A system that restores what you removed is not one you control, and an ordering gate at genesis would contradict the whole arrangement.

The write discipline, and one case that still lacks it

The sealed session negotiates a message size of 8192 bytes, so a body larger than one message arrives as several writes at advancing offsets. Such a body must be accumulated across the writes and committed when the handle closes. Before that was done each write replaced the last, and an 11114-byte body read back as 2945 bytes, exactly its trailing remainder. Committing at close also makes the replacement atomic from a reader's side: the previous body is served until the new one is whole.

Any writable node whose content can exceed that ceiling needs the same discipline, and one does not yet have it. Attribute writes have the identical shape and persist each write on its own. Most attributes are far too short to reach the ceiling, but the per-protocol gemtext representation is stored as an attribute and could exceed it. No path writes one today, so the case is latent rather than live, and it is recorded here so that it is found by reading rather than rediscovered as a surprise.

Store seam

The store protocol (src/substrate/store.lisp) is the seam between the namespace node layer and backing storage. It is a narrow CLOS contract: eight generic functions (list, attribute read and write, body read and write, create, existence, and modification time) each taking a store instance as its first argument. There are no default methods; a backend that misses a method is a missing-method error, never a silently papered-over default. Reads are tolerant by contract: absence comes back as a null attribute, an empty octet body, or a null mtime, and the calling node layer applies the documented defaults. The filesystem backend (src/substrate/fs-store.lisp) is the v1 implementation; a future content-addressed store implements the same eight functions and no node class above the seam changes. This seam is the designated replacement point for the durable-storage work.

The filesystem layout mirrors the namespace: one directory per publication, one file per attribute, the slug as the directory name. The layout is inspectable with ordinary tools and is the same tree a kernel v9fs mount would see.

Writes are atomic via rename-into-place: the value is written to a temporary file in the publication's own directory, then renamed over the destination, so a reader never observes a torn write: the rename is atomic because source and destination are on the same filesystem. Temporary names are unique per write (process id plus an atomically incremented counter), so concurrent writers never collide on a shared temp path. Writes are durable as well as atomic: the temporary file's data is flushed to stable storage (fdatasync) before the rename, and the containing directory is flushed (fsync) after it, so a write that has returned survives a crash or power loss; creating a publication flushes the data directory the same way, so the new directory entry is itself durable. A write whose flush or rename fails signals store-error instead of reporting success. The cost is two device flushes per write (tens of milliseconds on rotating storage) which suits operator-paced publication writes; *durable-writes* can be bound to nil where durability is deliberately traded for throughput, such as bulk imports.

Slug validation is the path-injection guard, and it is enforced fail-closed at both ends of the object's life. A slug is a non-empty string of lowercase ASCII [a-z0-9-], checked at the create path (node-create on the /pub root, and again inside the store backend before any directory is made) and at the read path (node-walk refuses a non-conforming name even when a matching directory exists on disk). Because the walk side validates independently, no adapter bug or encoding trick can turn a wire request into filesystem traversal: the HTTP adapter additionally percent-decodes each path segment before validating it, so an encoded .. is rejected as the decoded text it really is, and the Gopher adapter rejects .. segments in the raw selector. The shared predicate lives in src/namespace/pub.lisp.

Durable content-addressed store backend

The durable backend (src/substrate/store-store.lisp) is the second implementation of the store seam, the content-addressed analog of the filesystem store. It implements the same eight generic functions over a content-addressed block stack, a Merkle tree codec, and a generation-fenced head, so no node class above the seam changes and the /pub path behaves identically to the filesystem backend. The seam, not a backend choice, is what the rest of the system sees; this backend exists so publications can become crash-consistent and content-addressed without disturbing the publication object model that sits on top of it.

The on-tree layout is the same shape the filesystem backend writes to disk, expressed through the tree codec rather than through directory entries. The head names a root tree; that tree carries a pub directory child; each publication slug is a subdirectory under it; each attribute is a file child named for the attribute; the body is a body file child. The publication node layer's slug / attribute / body model therefore carries over unchanged: the durability mechanism shifts, the addressing model does not.

Durability here is structural rather than a flush discipline bolted onto a rename. A block is content-addressed and written (and flushed) before the head names it, so the head never references a block that is not already durable. Advancing the head is a generation-fenced compare-and-swap, and that single advance is the only linearization point for a write: a write that carries a superseded generation is rejected at the commit and surfaces as store-error, never a silent overwrite and never a retry that could lose a concurrent change. This is the durable-before-referenced ordering the rest of the store stack already relies on, reused at the publication seam.

Writers within one image are serialized by an in-image lock the backend holds for the read-modify-write cycle, so the only way the compare-and-swap can fail is an out-of-band advance from outside this image, and that case fails closed as store-error. The lock removes the need for an unbounded retry loop; the compare-and-swap remains the authority on what actually committed.

One place the POSIX assumption behind the seam leaks through: the seam exposes a publication modification time, and there is no filesystem timestamp to report. The durable backend synthesizes that value from the head generation, which is monotone, so a later write always reports a later time. The value is used only as the published-date display fallback when a publication carries no explicit published-date attribute; it is a display proxy, not a wall-clock guarantee.

The backend is selected when the fabric starts (src/fabric.lisp, start-fabric). The durable store is the wired default; the filesystem store is retained as a selectable operator fallback for a conservative rollout or for continuity with an existing on-disk publication tree. A fresh durable store serves an empty /pub on first boot: there is no in-band migration of existing filesystem publications into the durable store, so an operator who needs continuity with a filesystem publication tree selects the filesystem backend until a later migration path lands. The durable store roots at its own data directory, distinct from the filesystem publications tree, resolved in a fixed order: an explicit operator-supplied store directory wins; otherwise the store roots at a sibling of a supplied publication directory; otherwise it falls to the XDG data home default. Both the durable store's data directory and the selected store are cleared on a failed start and on stop, so a fabric that fails to come up or that has been stopped leaves no dangling seam state behind.

Two boundaries make this backend auditable. First, it is the one file that bridges the store seam and the lower block, head, and tree codecs, and it deliberately reaches no identity, capability, namespace, or protocol symbol: the seam-crossing is one-directional and structurally enforced, so a capability bug cannot reach storage and a storage bug cannot reach the capability layer. Second, the store holds opaque blocks: it never interprets publication content and carries no key bytes. The durability layer sees ciphertext-or-cleartext blocks as bytes to address and name, never as material to read: the sovereignty boundary that keeps key custody out of the storage path holds here as it does everywhere else in the substrate.

Adapter attach and projection-over-capability

The wire-format adapters (the adopted c3po-http and c3po-gopher modules) are ordinary valis protocol plugins: each registers on its port and specialises handle-connection. What makes them adapters rather than owners is how they reach content. At fabric start (src/fabric.lisp), each adapter is handed a freshly minted bearer capability designating /pub with read-only rights, and opens a long-lived 9P session by attaching over an in-process channel, the same codec and the same enforcement path a remote 9P client would traverse, with no TCP loopback. The assembler resolves the capability to a mount directive and builds a view containing exactly one entry: a read-only projection wrapping the publication root. The adapter can walk and read; a write-mode open is denied at the server by the projection wrapper, not by the adapter declining to write. Because the attach path is the in-process form of the real transport, the adapter is structurally identical to a sandboxed out-of-process edge module. Moving it out of the image changes the transport endpoint and nothing else: capability, view assembly, and projection enforcement are already in their final shape.

Each wire request maps one-to-one onto a walk in the adapter's mounted subtree. A missing name is a walk failure, and the adapter surfaces it as the protocol-native absence: HTTP 404, Gopher error entity type 3. A principal with no /pub grant has no /pub name at all, so absence reaches the wire by construction, never by a permission check on an admitted request.

Responses and requests are size-bounded. Directory listings and file bodies are read with hard ceilings (one megabyte for a listing, sixteen megabytes for a body), so a pathological publication cannot balloon a response buffer. Request lines are byte-capped on read (4096 bytes for an HTTP request line, 1024 for a Gopher selector), and overflow is answered with HTTP 413 or a Gopher type-3 error rather than unbounded buffering. Connection occupancy is bounded the same way memory is: each adapter arms a per-connection read deadline when it accepts a request (*http-read-deadline-seconds* and *gopher-read-deadline-seconds*, ten seconds by default), so a client that connects and sends nothing (or dribbles bytes slower than the deadline) cannot hold a handler thread past it. Expiry is treated exactly like a malformed request: HTTP answers 408 Request Timeout, Gopher closes without a response per its one-transaction model, and the executor reclaims the thread and the socket. The mechanism lives behind the connection seam: the adapter arms the deadline through a transport-blind generic, and the epoll backend (the one component allowed to name the socket library) wakes the blocked reader by shutting down the socket's read side, which surfaces to the handler as ordinary end-of-file while leaving the write side open for the parting response. In-memory connections, and any transport with nothing to time out, accept the call as a no-op.

Adapter teardown is complete: stopping the fabric closes both ends of each adapter's in-process channel, and close means drain-then-EOF: frames already written still deliver, then both sides observe end-of-file. The teardown order is deliberate. The port is unregistered first, so no new connection lands in a dying session; the live fid handles are clunked while the channel still delivers; the channel is then closed, which wakes the server's blocked dispatch loop and lets it return; finally the stopped adapter joins its server thread. The client-side reply reader exits on the same EOF, with any outstanding waiters drained by a transport-closed condition rather than left blocked. The join is bounded: a server thread that fails to exit within the timeout is a regression in the channel's close path, and it surfaces as a loud warning naming the leaked thread rather than a stop that hangs forever: teardown must never trade a thread leak for a deadlock.

Validation coverage

Each success criterion and security property of the publication substrate is anchored to a named test. The substrate tests prove the object model and the 9P session path; the render tests drive handle-connection end to end against an in-memory connection (the same entry point a real TCP client reaches), so the request-parse, walk, and response-render paths are exercised exactly as shipped.

Criterion Test File
Store backend round-trips every store protocol operation substrate-store-protocol tests/substrate-test.lisp
Non-conforming slugs rejected fail-closed on create and on walk substrate-slug-charset-fail-closed tests/substrate-test.lisp
Absent attributes read as defined defaults substrate-tolerant-defaults tests/substrate-test.lisp
Owner creates and writes publications via 9P substrate-authoring-create-write tests/substrate-test.lisp
One publication object backs both adapters substrate-publication-shared-object tests/substrate-test.lisp
HTTP adapter reads only through the read-only projection substrate-http-read-through-projection tests/substrate-test.lisp
Gopher adapter reads through the read-only projection substrate-gopher-read-through-projection tests/substrate-test.lisp
No grant means no name: capless session sees absence on the wire substrate-no-mount-no-name tests/substrate-test.lisp
HTTP listing renders end to end and links the publication http-render-directory-listing tests/adapter-render-test.lisp
Listing links below the root carry the parent path http-render-nested-listing-links tests/adapter-render-test.lisp
HTTP serves the publication body end to end http-render-file-body tests/adapter-render-test.lisp
Percent-encoded traversal segments are rejected with 404 http-render-rejects-encoded-traversal tests/adapter-render-test.lisp
The /pub node refuses traversal names regardless of adapter pub-walk-refuses-traversal-slug tests/adapter-render-test.lisp
Duplicate create surfaces as a precise 9P error, not an internal one pub-create-duplicate-signals-clean-error tests/adapter-render-test.lisp
Gopher menu renders end to end with the RFC 1436 terminator gopher-render-menu tests/adapter-render-test.lisp
Gopher serves the publication body end to end gopher-render-file-body tests/adapter-render-test.lisp
An idle client is cut off at the read deadline, its thread reclaimed idle-client-cut-off-at-read-deadline tests/read-deadline-test.lisp
A prompt request is served normally with the deadline armed prompt-request-unaffected-by-armed-deadline tests/read-deadline-test.lisp
HTTP answers a deadline cut-off with 408 http-deadline-expiry-answers-408 tests/read-deadline-test.lisp
Gopher closes a deadline cut-off without a response gopher-deadline-expiry-closes-silently tests/read-deadline-test.lisp
Fabric stop reaps session threads and clears every seam variable adapter-lifecycle-reaps-threads tests/adapter-render-test.lisp

The bus: dispatch over 9P

Inter-module and inter-node dispatch rides 9P itself. 9P is already a network-transparent RPC over a namespace (it is a message bus), so a protocol module's mailbox is a file, and routing a connection to a module running in a local thread or on a remote Lisp node is the same file operation either way. A separate message-bus transport (e.g. ZeroMQ) is admitted only where a pub/sub fan-out pattern genuinely beats file semantics, not as the default carrier.

There is no usable Common Lisp 9P implementation in the wild, but 9P2000 is a small protocol (~13 message types), so a native CL client+server is tractable. Because the Linux kernel ships a 9P client (v9fs), a valis 9P server can additionally be mounted by the host kernel, bridging the namespace into ordinary Linux tools.

Migration: light process, durable namespace

A person's valis is migratable (it can be stood up on any valis-compatible host) because the durable thing is the namespace, not the process. Migration is therefore reassembly, not transfer: a fresh instance reconstructs the filesystem semantics from a small manifest of mounts plus the owner's keys, and the destination host's agent re-steers the designated ports to it. This demands a continuous discipline: keep the process boundary light, externalising anything reconstructable, so standing up an instance is cheap. The model is Plan 9's: a per-process namespace (assembled, local) distinct from a shared authoritative file server.

Where the namespace lives: tiered by access pattern and secrecy

  • Bulk and cold data is content-addressed and immutable: a block's hash is its name (the Venti model; IPFS as the transport). Immutable content is location-independent: an instance pulls it from wherever it is served.
  • Published data is content-addressed and unencrypted, served from many places at once. Because it lives outside the mutable store, high-fanout public reads never touch the private substrate.
  • Mutable state lives on a backing store the instance mounts; the mutable head is a small hash pointer over the immutable tree, in the manner of fossil over Venti or git refs over objects.
  • Secret keys are owner-held or held in a separate trust domain; they are never content-addressed and never live solely on a host that might be evacuated.
  • A hot working set is served by a read-side caching tier in front of the store, so a coincident instance keeps its reads off the single-writer head while still observing every committed write. It is two caches and a coherence primitive, each of which gets its correctness from a fact the store already carries (a block's hash is its name, and the head's generation is its version), so the cache adds speed without adding a way to be wrong. The read-side caching tier below is the mechanism.

The reassembly path in code: where to follow it

The reassembly is not a single function but an ordered handshake whose composition site is src/fabric.lispstart-fabric (the :evacuative t arm drives the security regime; see the two regimes below). An auditor can follow it through these named seams.

A fresh instance comes up only if it holds the owner's key out of band: the gate is src/fabric.lispowner-key-required, which fails closed when the keyfile is absent rather than standing up an unauthenticated substrate; the key itself is loaded through the custody seam (src/identity/custody.lispload-or-create-keyfile), so the key bytes never live on the durable store the instance mounts. With the key in hand, the instance reconstructs the filesystem semantics at src/namespace/assembler.lispassemble-canonical-frame, the single reassembly entry that rebuilds the view from the manifest of mounts; an unreachable mount is a fail-closed stand-up (reassembly-unreachable), never a silent partial view. The assembler reads the authoritative mutable head through an injected seam, *durable-head-reader* (bound at fabric start, src/fabric.lisp), so the same reassembly code path serves whether the head is read from the durable store or from a test double: the durability boundary is one named indirection, not a scattered set of direct reads.

The mutable head carries the generation that is both the consistency token and the fence (the two regimes section traces its security role). It advances through one linearization point: src/store/head.lispadvance-head, a compare-and-swap against the expected generation. A late writer that expected a superseded generation loses the swap and is told so by generation comparison alone: no wall clock, no lease deadline. The pre-commit fault seam *advance-head-pre-commit-hook* (src/store/head.lisp) lets the out-of-band crash harness freeze a writer exactly at the swap to prove the rejection survives a real pause.

Consistency across coincident instances

valis instances can be temporally coincident: several at once, to absorb load. Authority over the mutable namespace is a single writer per subtree (the fossil model), not a leaderless or eventually-consistent scheme: a sovereign namespace wants one authoritative truth, not merge. For one person's modest write load a single writer is ample; if write scaling ever bites, the namespace (already a tree of mounts) is partitioned into per-subtree authorities (the Ceph dynamic-subtree-partitioning model). Coincident instances each assemble their own namespace view and interpose caches; the authoritative writer serialises mutation; public read load is offloaded to the unencrypted content-addressed publishing path.

The read-side caching tier

A coincident instance must be able to read fast without consulting the single writer on every access, yet must never serve a stale answer once a write has committed. The read-side tier achieves both by deriving each cache's correctness from a value the store already carries, so caching adds no new way to be inconsistent. It has three parts.

The first is an immutable-block cache (src/store/block-cache.lisp), a read-mostly tier in front of the content-addressed block device, one per store handle and never a process-global. Because a block is named by the SHA-256 of its bytes, the name is the validator: a clean hit returns the admitted bytes verbatim and never re-hashes, yet can never be stale, because the same name always means the same bytes. Admission is verify-then-admit: the bytes are re-hashed against their requested name before they enter the cache, on the store's single digest seam, so a substituted or bit-rotted on-disk block is rejected at admission and can never become a hit. The corollary is that a cached hit stays correct even if the on-disk block is later corrupted: the content is immutable, so the in-memory copy is as authoritative as the disk ever was. Growth is doubly bounded (an explicit byte budget and an independent descriptor/slot cap, so a flood of tiny blocks cannot exhaust entries while staying under the byte budget) with CLOCK (second-chance) eviction, the right policy for an immutable read-mostly tier where eviction order does not move the hit rate. A reader that must hold a block across an operation pins it: with-pinned-block refcounts the slot and releases the pin on every exit path via unwind-protect, and eviction skips a pinned slot rather than force-evicting it: force-evicting a block a reader still holds would be a use-after-free of its byte vector.

The second is a generation-revalidated head-decode cache, one fixed record per store handle (src/substrate/store-store.lisp). Decoding the head's pub directory is the expensive part of a read; caching the decoded result would normally reintroduce staleness. It does not here, because the head already carries a monotonic generation token (the same token that fences a superseded writer), and that token doubles as the cache validator. Every read re-reads only the generation through a narrow reader (current-head-generation, src/store/head.lisp) that integrity-verifies the 124-byte HEAD record but decodes no tree, reuses the cached decode when the generation is unchanged, and reloads on any change. The generation probe, the compare, the decode, and the recorded generation all happen inside one critical section under the store lock, so the cache is always stamped with the generation that was actually current: a coincident writer cannot advance the head between the probe and the decode and leave the cache mislabelled. The staleness bound is therefore zero: the moment another handle commits, the next read here sees a moved generation and reloads. The write path is untouched (it reads the current head directly under the lock and never consults this read cache), so the cache cannot interfere with the compare-and-swap that is the sole linearization point.

The third is content-addressed qid.version, the coherence primitive a correct cache above the store needs. A served node's 9P qid.version is the low 32 bits of its content score (store-node-score / score-low-word), with 0 remapped to 1 so genesis, an absent slug, and an unwritten leaf all read 1, never 0 and never an error. The score is a pure function of the node's durable content (the Merkle property: a directory's score changes iff its listing changes, a leaf's iff its bytes change), so the version is correct across instances by construction: two instances over one data directory compute the same version for the same node with zero coordination, and a write to one slug never perturbs another slug's version because their scores are independent. The override is per node, with no shared mutable version counter, so there is nothing to serialize. This is what lets a 9P or HTTP cache one tier up validate correctly: the version it keys on is itself the content hash, which is exactly the change-detector a cache validator wants.

The three compose into the property the coincident regime needs: reads run off the per-handle caches at content-addressed speed, the generation token keeps the head-decode cache coherent with staleness zero, and the content-derived version lets every cache above the store (including an ordinary HTTP cache at the edge) track live content without a coordination protocol.

  • Validation coverage: the read-side caching tier

    Each property of the caching tier is anchored to a named test. The block-cache tests drive the cache directly; the store-store tests drive the generation-revalidated decode and the content-addressed version over the real durable store.

    Criterion Test File
    A clean cache hit returns the device bytes verbatim, with no re-hash block-cache-hit-returns-device-bytes tests/store-block-cache-test.lisp
    Admission verifies bytes against their score; a corrupt block is never admitted block-cache-verify-then-admit tests/store-block-cache-test.lisp
    A cached hit stays correct even after the on-disk block is corrupted block-cache-immutability-under-ondisk-corruption tests/store-block-cache-test.lisp
    Resident bytes never exceed the byte budget block-cache-honours-byte-budget tests/store-block-cache-test.lisp
    The descriptor count never exceeds the slot cap block-cache-slot-cap tests/store-block-cache-test.lisp
    A pinned block is never evicted while it is held pinned-block-never-evicted tests/store-block-cache-test.lisp
    The pin is released on every exit path from with-pinned-block with-pinned-block-releases-on-every-exit tests/store-block-cache-test.lisp
    The cache layer reaches no capability/namespace/identity symbol block-cache-import-clean tests/store-block-cache-test.lisp
    The narrow generation reader verifies the HEAD but decodes no tree current-head-generation-narrow-read tests/store-head-test.lisp
    An unchanged generation reuses the cached head decode store-store-unchanged-generation-reuses-decode tests/substrate-store-store-test.lisp
    Coincident handles over one store read each other's committed writes store-store-coincident-handles-read-your-writes tests/substrate-store-store-test.lisp
    The head-decode cache is per handle, never shared across instances store-store-head-cache-is-per-instance tests/substrate-store-store-test.lisp
    A node's version is non-zero after a write store-store-version-nonzero-after-write tests/substrate-store-store-test.lisp
    A write to one slug does not move another slug's version store-store-version-per-file-isolation tests/substrate-store-store-test.lisp
    Two instances over one data directory compute the same version for a node store-store-version-cross-instance-same tests/substrate-store-store-test.lisp
    Genesis and an absent slug read version 1, never 0 store-store-version-genesis-absent-slug-is-one tests/substrate-store-store-test.lisp

Two regimes, and evacuation under attack

The mutable head carries a monotonic generation token, which yields consistency and fencing from one mechanism. There are two migration regimes:

  • Coincident (scaling): several instances share the store; the generation/lease serialises who may advance the head.
  • Evacuative (security): fleeing a computational substrate under attack, assuming the host is compromised. Standing up the new instance bumps the generation, which fences the old, possibly-compromised instance: the store rejects writes from a superseded generation, the same shape as revoking a superseded capability. Because keys never lived solely on the evacuated host, the new instance comes up without trusting the old one.

In-flight connections drop and re-establish by default, acceptable because the namespace, not the process, is the source of truth. Live carry-over of established TCP connections (the kernel's TCP_REPAIR path) is a possible enhancement for protocols that cannot tolerate a reconnect, not a dependency. Re-steering the established sessions cleanly (so the old instance's clients reconnect to the new one rather than waiting on a passive TCP drop) is the job of a session-drain seam on the steering host's agent (the chartered drain-sessions boundary, proposed in the steering agent's repository). That seam is a latency and user-experience improvement on handover, never the fence: the fence is the store's, and it holds whether or not the old host cooperates, because the threat model assumes the old host may be compromised.

The fence and the sweep: where to follow them

The bump fences the superseded instance through the same one revocation mechanism that revokes a capability. The fence write is src/capability/revocation.lispfence-epoch-hash (the content-addressed name of a fenced generation) entered into the durable revocation set by revoke-hash (fdatasync-durable, so a fence survives a crash the instant it is written). A write capability is rejected if its bound generation is fenced: the verifier consults the revocation set at src/capability/verifier.lisp%write-cap-fenced-p, on the same path that checks an ordinary revoked token. Each capability carries the generation it was minted under as src/capability/token.lisptoken-bound-generation, so the consult is a pure membership test against the fenced generation: a superseded writer is rejected by construction, not by a host that must be trusted to police itself. Finally, live write-capable grants still held under the fenced generation are dropped by src/namespace/assembler.lispevict-by-fence-epoch, so a fenced instance keeps no live authority even in memory.

Evacuation walkthrough: one migration, end to end

One full evacuation traces these seams in order. An operator (or the new host's agent) stands up a fresh instance over the same durable data directory while the old, possibly-compromised instance is still running.

  1. Owner-key gate. The fresh instance refuses to start without the owner's key, held out of band (src/fabric.lispowner-key-required, loaded via src/identity/custody.lispload-or-create-keyfile). Because the key never lived solely on the evacuated host, the new instance comes up without trusting the old one.
  2. Reassembly. It rebuilds the namespace from the manifest of mounts (src/namespace/assembler.lispassemble-canonical-frame), reading the authoritative head through the injected *durable-head-reader* seam: reassembly, not transfer.
  3. Generation bump. It advances the head past the running instance's generation (src/store/head.lispadvance-head), the single compare-and-swap that is the store's only linearization point.
  4. Fence. The bump writes the superseded generation's fence-epoch-hash into the durable revocation set via revoke-hash (src/capability/revocation.lisp), fdatasync-durable.
  5. Sweep. Live write-capable grants under the fenced generation are evicted (src/namespace/assembler.lispevict-by-fence-epoch), so the old instance retains no live authority in memory.
  6. Rejection of the late writer. The old instance's next write loses on both paths: its capability is rejected at the verifier consult because its token-bound-generation is fenced (src/capability/verifier.lisp%write-cap-fenced-p), and its compare-and-swap is rejected because it expected a superseded generation (advance-head returns the stale sentinel). The rejection is clock-free: it holds across an arbitrarily long pause.
  7. Serving resumes. The new instance serves the reassembled namespace at the pinned generation; in-flight connections to the old instance drop and re-establish.

The whole handshake is composed at src/fabric.lispstart-fabric with the :evacuative t arm, the one site an auditor reads to see the steps wired together, rather than re-implemented in a test body.

Opaque-block audit: why an operations path reaches no content

The store holds opaque blocks, addressed and named by the hash of their bytes, never interpreted. The durability cluster (store-store, block-device, block-cache, block, head, tree, manifest) imports zero capability, namespace, or identity symbol, so a storage-side or operations-side path cannot become a confused deputy that reaches content or keys: the absence of the import is structural, proven over the whole cluster by store-cluster-imports-no-content-symbols, not a runtime check that could be skipped.

The table below audits the operations paths an operator would use to move or observe a substrate. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting: it sees ciphertext-or-cleartext blocks as bytes to address, never as material to read.

Operations path Seam it would use Why it reaches no content Anchoring test
Migrate / evacuate a substrate start-fabric :evacuative t → reassembly + bump Moves the head pointer and the addressed blocks; never decodes a block, never holds a key migration-reassemble-then-evacuate-end-to-end
Back up the durable store (subsystem TBD) the block-device / head / tree codecs Copies hash-named opaque blocks and the 124-byte head record; no content/key import path store-cluster-imports-no-content-symbols
Observe / meter the store (subsystem TBD) the store handle's counters and the narrow head reader Reads generation and block counts; the cluster imports no namespace/capability symbol store-cluster-imports-no-content-symbols
Revoke an evacuated instance's authority revoke-hash over fence-epoch-hash Writes a content-addressed fence into the revocation set; the set holds hashes, not data two-instances-only-one-write-accepted-fail-closed
Sweep live grants under a fenced gen evict-by-fence-epoch Drops in-memory write grants by generation; touches no block bytes and no key material bump-sweeps-live-write-grants

Validation coverage: migration and evacuation

Each success criterion of the migration-and-evacuation story is anchored to a named test. The end-to-end and fence tests drive the real start-fabric :evacuative t path over a durable store; the out-of-band proofs run in the standalone crash harness, where a real process is paused or killed across the bump.

Criterion Test File
A fresh instance reassembles and serves; the evacuated instance is fenced migration-reassemble-then-evacuate-end-to-end tests/migration-end-to-end-test.lisp
and its live write grants are swept    
Live write-capable grants under the fenced generation are swept by the bump bump-sweeps-live-write-grants tests/store-fence-revocation-test.lisp
A write paused across the bump is rejected at the verifier and at the CAS two-instances-only-one-write-accepted-fail-closed tests/store-fence-revocation-test.lisp
The store cluster imports no content, key, or namespace symbol: a confused store-cluster-imports-no-content-symbols tests/migration-end-to-end-test.lisp
deputy is impossible by construction    
Out of band: an old process SIGSTOP'd across the bump cannot land its CAS proof-evacuation-under-pause scripts/store-crash-test.lisp
Out of band: a recovered head never names a generation past an unfenced one proof-fence-precedes-head-advance scripts/store-crash-test.lisp

The mail transport spine

valis terminates mail as a first-class axis of the sovereign namespace. The wire engines that speak SMTP and IMAP to the hostile internet live in a separate process at the IETF edge; valis owns the substrate the mail moves through: the authenticated submission door, the durable transport queue, the router that decides local-versus-relay, the per-DID Maildir inbox, and the seam a wire adapter drains relayed mail through. The whole spine moves an opaque message body end to end: every layer below names and addresses the body by the hash of its bytes and never interprets it.

This chapter is the auditor's entry point to that spine. It traces one message end to end through the named code, states what the open-relay guard guarantees and why it cannot be turned off, and shows that no operations path reaches the content of a message. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement, not delivery onto a real network.

Mail flow walkthrough: one message, end to end

One submission traces these seams in order. The owner stands at /mail/outbox holding a write capability minted through the real /cap/ctl path; an anonymous or unauthenticated peer never reaches this door.

  1. Authenticated submission (the clunk commit). A capability-mounted 9P write into /mail/outbox stages octets; the clunk is the atomic commit (src/namespace/mail-outbox.lispnode-close on the outbox-submission file). The envelope and the submit provenance are stamped at the clunk, never parsed from the body; a fid abandoned without a clunk lands nothing: complete or absent by construction.
  2. The live seam. /mail is present in the running owner frame only because src/fabric.lispstart-fabric binds the three mail seams (*mail-dir-root*, *local-delivery-did-resolver*, *local-domains*), each gated on a durable store-backed substrate and reset fail-closed on teardown. The axis is mounted by src/namespace/assembler.lisp%ensure-mail-mount and reached at src/namespace/root.lispnode-walk on "mail".
  3. Ingest. The clunk hands the assembled octets, derived envelope, and stamped provenance to src/mail/seam.lispland-message, the single ingestion door through which both owner-submitted and peer-landed mail enter.
  4. Route. src/mail/router.lisproute-message classifies every recipient through route-recipient, which tests locality against *local-domains* and admits relay only on proven submit authority (see the open-relay guard below).
  5. Queue. Each routed recipient lands a durable per-recipient record via src/mail/transport-queue.lispplace-incoming: the ack follows the fdatasync, so the queue is durable before the submitter is told "committed". The relay path is admitted into a bounded active window with admit-active and the slot is returned on a terminal outcome with release-slot.
  6. Local delivery. A local recipient is written into the owner's Maildir at /mail/inbox/<did>/new/ by src/mail/local-deliver.lisplocal-deliver, keyed under the live owner DID the *local-delivery-did-resolver* returns from custody.
  7. Relay drain. A relay recipient is drained by src/mail/drain.lispstart-mail-drain / drain-active-once, which hands the opaque message to a registered mail-adapter. The wire engine is stubbed here; the adapter records a disposition and the drain frees the active slot.

The same path is driven against the live fabric (outbox capability, real router, real queue, real delivery) in tests/mail-end-to-end-test.lisp, so the walkthrough above is exercised, not merely asserted.

The open-relay guard: fail-closed by capability, not by flag

route-recipient reaches the relay path by exactly two yes-paths: the recipient is local (recipient-local-p true against *local-domains*, so the message is delivered into a local inbox and never relayed), or the message carries authenticated-submit provenance, an owner who stood at /mail/outbox under a keyed session. A peer-landed message (non-authenticated provenance) for a non-local recipient matches neither path: route-recipient signals mail-open-relay-refused and lands nothing, never silently forwarding.

What this guarantees is that valis cannot become an open relay. Why it holds by construction: there is no allow-relay flag to misconfigure. Relay authority is a capability the submitter either presents or does not; the refusal is the absence of that authority, not a policy toggle a host could flip. An auditor confirms the guard is live, not vestigial, in unauthenticated-non-local-relay-is-refused-fail-closed.

The content boundary: the body is an opaque block

A message body is an opaque block, content-addressed octets the store names by the hash of their bytes and never interprets, exactly like every other block in the durability cluster. Sovereignty here is absence: the operations-facing view of the mail spine has no name for any body. A read grant on /mail/queue resolves a body-free projection (src/namespace/mail-queue.lisp%synthetic-name) that exposes envelope and status only; it has no walkable name for any message body and none for /mail/inbox. The body score that keys a status record is an address, never invertible to content.

The table below audits the operations paths an operator would use to move or observe the mail spine. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting.

Operations path Seam it would use Why it reaches no content Anchoring test
Observe the mail queue (operator) the /mail/queue body-free projection (%synthetic-name) Names envelope + status only; no walkable name is a body, and none is /mail/inbox ops-queue-grant-resolves-no-message-body-name
Any ops path over the queue projection the queue projection package's imports Imports zero body-octet read (read-block / decode-file): a confused deputy is impossible ops-queue-projection-imports-no-body-read-symbol
Back up the mail store (subsystem TBD) the block-device / head / tree codecs Copies hash-named opaque blocks; no content or key import path store-cluster-imports-no-content-symbols
Observe / meter mail flow (subsystem TBD) the queue projection's status records and counters Reads envelope, status, and counts; the body score is an address, not content ops-grant-on-queue-has-no-name-for-the-other-leaves
Relay-drain a message (the transport itself) drain-active-once → a registered mail-adapter Hands opaque octets to the wire adapter; classifies and moves, never decodes for ops authenticated-relay-submission-drains-to-the-stub-adapter

Validation coverage: the mail transport spine

Each success criterion of the mail spine is anchored to a named test. The end-to-end tests drive the real start-fabric seam, outbox capability, router, queue, and delivery over a durable store with only the wire engine stubbed; the sovereignty tests prove the content boundary both behaviorally (over a really queued message) and by construction (a do-symbols import firewall).

Criterion Test File
A local-recipient outbox submission delivers into the owner inbox byte-identical local-recipient-submission-delivers-into-the-owner-inbox tests/mail-end-to-end-test.lisp
An authenticated non-local submission drains to a stubbed adapter; the slot frees authenticated-relay-submission-drains-to-the-stub-adapter tests/mail-end-to-end-test.lisp
An unauthenticated non-local relay is refused fail-closed and lands nothing unauthenticated-non-local-relay-is-refused-fail-closed tests/mail-end-to-end-test.lisp
An ops grant on /mail/queue resolves no name for any message body or the inbox ops-queue-grant-resolves-no-message-body-name tests/mail-sovereignty-audit-test.lisp
The ops-facing queue projection imports no body-octet read by construction ops-queue-projection-imports-no-body-read-symbol tests/mail-sovereignty-audit-test.lisp
An ops grant on /mail/queue has no walkable name for the other axis leaves ops-grant-on-queue-has-no-name-for-the-other-leaves tests/mail-axis-test.lisp

The DNS substrate: authoritative names off the answer hot path

valis terminates authoritative DNS as a first-class axis of the sovereign namespace, the same way it terminates mail. Two siblings sit between the wire and the data: the engine that speaks the RFC 1035 binary protocol on port 53 to the hostile internet (a separate process at the IETF edge), and runciter, the DNS service module (the logical nameserver that composes answers and holds the serving index). valis owns the data substrate those answers come from (the zone data held in PostgreSQL as the operator's system of record, the read seam runciter loads its index through, and the capability gate that admits a DNS wire module), and it dispatches a steered connection to the registered service module. valis holds no nameserver answer logic and no serving cache: it is the authoritative ndb DATA owner, and runciter is the nameserver. PostgreSQL is the operator's system of record, never a per-query dependency: runciter answers from its own in-memory index and keeps serving the last good index across a database outage, so the names do not go dark.

This chapter is the auditor's entry point to that substrate. It states the data read seam runciter binds over, the narrow answer-composition contract the registered serving handler honours, traces a decoded query end to end through the named code, locates the availability and answerability invariants (now runciter's, over the cutover facets valis owns), and shows the capability gate a DNS wire module passes before it can bind. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement of an answer, not encoding onto a real network.

The seam contract: a decoded query, a record set, and a result code

The contract a bound DNS wire engine sees is deliberately narrow and versioned. The engine hands a decoded query (a name, a type, and a class) to the registered serving handler, and the handler returns a record set and a result code. That handler is runciter's: valis registers (make-dns-service-handler <pg-zone-source>) as the adapter handler, so the answer is composed by the nameserver over valis's data, not by valis. The result code is one of three keywords: :noerror (the record set is the answer), :nxdomain (the name does not exist), or :nodata (the name exists but carries no record of the asked type). The contract keeps :nodata distinct from :nxdomain because they are different facts about the zone, and a correct resolver caches them differently; the wire maps :nodata onto a NOERROR message with the authority section carrying the zone's apex SOA.

Negative answers carry authority data, not just a code. On both :nxdomain and :nodata, the handler returns the zone's apex SOA record in the record set, so the wire has everything it needs to emit a spec-correct negative response and to compute the negative-cache lifetime as the minimum of the SOA MINIMUM field and the SOA record's own TTL (RFC 2308). A name outside every authoritative origin is :nxdomain with an empty record set (not-authoritative, no apex SOA), kept distinct from an in-zone negative. runciter owns what authority data accompanies each outcome; the wire stays a pure codec that encodes what it is handed.

Name synthesis is the nameserver's job, never the wire's. For an aliased name the returned record set already contains the CNAME record plus any in-zone records the alias resolves to (one in-zone, single-hop chase); for a name covered by a wildcard owner the returned set is the synthesized answer. The wire encodes what it is handed and never chases names or reaches below the seam for zone data. Everything that is purely a property of the wire stays wire-side and is never surfaced at the seam: EDNS0/OPT and the DO bit, truncation, response-rate limiting, recursion flags, the message ID, name compression, the authoritative-answer bit, and the transport-generated result codes (FORMERR/SERVFAIL/NOTIMP/REFUSED). Delegations and DNSSEC are not in the v1 contract; they are additive later through a contract-version bump, gated by the same fail-closed version check described next.

The version check is fail-closed at register time. src/plugin/dns-adapter.lisp holds the seam: the *dns-adapters* registry, register-dns-adapter, and a contract-version constant +dns-adapter-contract-version+ (1) with the set of admitted versions in *supported-dns-contract-versions*. An engine that declares a contract version outside that set is refused before any registration mutates the table: register-dns-adapter signals dns-adapter-version-mismatch and binds nothing, so a wire engine built against a future, incompatible contract can never silently skew against this core.

The data read seam: valis's PostgreSQL zone source

valis exposes its zone data to the nameserver through a narrow read protocol, not a serving cache. pg-zone-source (src/operator-state/zones.lisp) implements runciter's zone-data-source protocol over the dns_zone / dns_record rows: zone-version (the zone's SOA serial as an opaque RFC-1982-comparable token), zone-record-set (the runciter resource-record structs of a zone), and lookup-records (records owned by a name). Two further reads let the nameserver build its serving index and gate answerability without a serving view living in valis: source-origins (the held-inclusive set of canonical origins valis holds authoritatively now) and zone-cutover-status (a zone's (cutover-state, expire-deadline): the facets a secondary's answerability turns on). Each method queries the ambient pooled connection, so a caller wraps a read in with-operator-state-connection; the methods open no per-query socket on any hot path because the hot path is runciter's, not valis's.

The in-memory serving index and the answer composition live in runciter, the nameserver. runciter loads its index from this read seam, holds it as an immutable snapshot, and composes every answer (the RFC-2308 decision tree: exact match, one in-zone single-hop CNAME chase, single-label *.<parent> wildcard synthesis, NODATA-versus-NXDOMAIN against an existence set, apex SOA on in-zone negatives, no apex SOA on a not-authoritative name) over its own snapshot. valis composes no answer and caches no zone. This is the Option-A split: the serving cache lives with the nameserver; valis is the authoritative DATA owner over PostgreSQL.

Availability is runciter's, by construction over this seam. runciter rebuilds its index build-then-swap under a lock and serves last good: a rebuild whose source read raises (PostgreSQL unreachable, a row malformed) is contained, leaving the prior snapshot answering, so a database outage degrades to serve-last-good rather than to an empty answer. PostgreSQL is the operator's system of record, never the per-query dependency. valis's contribution to that guarantee is the data seam itself plus the cutover facets the nameserver gates on; the serve-last-good property is verified in runciter's suite (failed-rebuild-keeps-prior-index, source-outage-after-load-keeps-answering).

The commit fires a change signal; the nameserver pulls

valis installs nothing into a serving view on commit: it owns no view. The capability-gated :names door (src/namespace/names.lisp) takes a zone's master text and commits an atomic full-zone replace at the 9P clunk: node-close on the submission file is the commit, exactly as the mail outbox clunk is. After the import resolves the new serial, the commit fires the in-process zone-change signal (fire-zone-change, origin and serial) so an in-process listener (the outbound zone-change feed, and a wire feed) observes the change. The nameserver refreshes by pull: runciter re-checks zone-version per origin (and source-origins for set changes) on a throttled, query-triggered poll and rebuilds its index when the version is newer, so a committed write is picked up without valis pushing into runciter. The committed write is durable in PostgreSQL regardless; runciter's next successful pull reflects it. (A lower-latency optional force-refresh seam exists on the runciter side, refresh-dns-service, that a commit hook may call; valis leaves it unwired by default, the throttled poll being the correctness floor.)

The record boundary: valis owns the envelope, runciter owns the rdata

valis stores zone records in typed PostgreSQL columns (owner, TTL, class, type, and the record's rdata as presentation text) and is the authority on that envelope. It is not the authority on the per-type rdata grammar. Parsing, canonicalizing, and rendering the rdata of each record type is runciter's job, the sibling that owns the DNS protocol knowledge. Only the rdata crosses the boundary: on the write path record->rdata-columns renders a record's rdata to canonical text through runciter:render-rdata, and on the load path row->resource-record (src/operator-state/zones.lisp) builds the resource-record envelope from valis's own typed columns while handing the rdata text to runciter:parse-rdata to type it. valis re-parses no resource-record syntax of its own. This keeps a single authority for the wire grammar (the same authority the nameserver composes through and the real wire engine encodes through), so the bytes the nameserver serves and the bytes a zone transfer would carry cannot drift apart. The published master-file text an operator reads back is rendered from these same structs through runciter's canonical-line renderer, so a round trip through the substrate is byte-faithful.

Admission: a DNS wire module is a capability decision

A DNS wire engine does not bind by being named in a config file; it binds by passing the same owner-vouched content-hash admission gate every other pluggable module passes. admit-and-bind-dns-module (src/edge/dns-controller.lisp) delegates admission whole to the module-admission chokepoint (the content hash, the owner :admit vouch, and the revocation/fence check) and introduces no new trust path of its own. Only on a successful admission does it bind the adapter over valis's data source; an unvouched or revoked hash is refused fail-closed, the module loader is never called, and the adapter registry is left byte-identical to its pre-call state.

The bind registers the nameserver's serving handler: bind-dns-view (src/edge/dns-controller.lisp) takes valis's pg-zone-source, builds runciter's handler over it via (make-dns-service-handler <source>) (runciter's builder symbol resolved late by find-symbol in its package, the same late-binding idiom the serving seam uses, so valis carries no hard import of a serving symbol), and registers that handler under the adapter name. The serving contract version is asserted equal to valis's +dns-adapter-contract-version+ at bind, fail-closed: a missing builder or a version skew signals rather than binding a wrong-shaped or absent handler. Because the handler holds its own source and pulls its own refresh, the bind is a single registry mutation (there is no separate serving view to offer), and it preserves the no-partial-bind discipline on that mutation: a post-bind self-check funcalls the handler once, and on any failure bind-dns-view restores whatever adapter held the bound name before the attempt rather than blind-deleting it, while admit-and-bind-dns-module restores the whole registry it snapshotted before admission. The result code from a refused admission is the absence of a binding, not a half-built one. Handler construction runs inside a pooled operator-state connection when the pool is armed (the eager index build reads PostgreSQL) and bare otherwise, so an in-memory test bind needs no database.

A decoded query therefore traces these seams in order: the wire adapter is fetched from *dns-adapters* by name and its handler invoked with (name, type, class); the handler is runciter's serving handler, which composes the answer over its own index (loaded from valis's pg-zone-source) and returns (record-set, rcode); the adapter hands that back to the wire for encoding. No port 53 and no binary codec runs inside valis on that path, and no answer logic runs inside valis at all: the proof drives the whole chain through the registered runciter handler in dns-bind-resolves-query-through-runciter-handler, so what is exercised is the production wiring through the real nameserver, not a view in isolation.

Forward scope: what is stubbed and what is open

The binary wire engine (the RFC 1035 codec, port-53 serving with response-rate limiting and non-recursion, zone-transfer and NOTIFY, and the resolver client) is the sibling DNS engine's responsibility and is stubbed here; the conformant wire transcript is that engine's acceptance gate, not this substrate's. The nameserver's index warms at handler construction (an eager build over source-origins + zone-record-set) and is kept current by its throttled pull; valis's job is to keep the data seam answering and to fire the change signal on commit. Deeper wildcard closest-encloser synthesis, delegations with glue, and DNSSEC signing are additive through a contract-version bump and the fail-closed version check, deliberately out of the v1 flat-authoritative-zone scope.

Validation coverage: the DNS substrate

Each property is anchored to a named test. valis-side coverage proves the data read seam, the cutover facets answerability turns on, the capability gate and its no-partial-state unwind, and that the bind dispatches a decoded query through the real runciter handler. The answer-composition decision tree and the serve-last-good availability property are runciter's (composition relocated to runciter's compose-answer parity suite [parity-checked branch-for-branch against valis's prior zone-view-lookup before the shed], serve-last-good to runciter's refresh suite) and are cited below so the auditor can follow the data/answer split to its other half.

valis-side:

Criterion Test File
The PG source surfaces a held zone's cutover-state as :held held-zone-surfaces-held-state tests/cutover-status-source-test.lisp
A secondary surfaces its persisted absolute epoch expire-deadline answer-as-secondary-surfaces-absolute-epoch-deadline tests/cutover-status-source-test.lisp
A primary forces a NIL deadline (it never expires) primary-zone-forces-nil-deadline tests/cutover-status-source-test.lisp
An unstored origin reads (nil nil): fail-closed unstored-origin-fails-closed tests/cutover-status-source-test.lisp
source-origins includes held zones (the held-inclusive authoritative set) source-origins-includes-held-zones tests/cutover-status-source-test.lisp
A held secondary commit reads :held and so is never answerable held-secondary-commit-reads-held-cutover-status tests/secondary-ingest-test.lisp
A vouched DNS module admits through the chokepoint and self-registers vouched-dns-module-admits-and-self-registers tests/dns-admission-controller-test.lisp
A refused module leaves no partial adapter state refused-dns-module-leaves-no-partial-state tests/dns-admission-controller-test.lisp
The controller-bound adapter resolves a query through the runciter handler controller-binds-admitted-adapter-and-resolves-query tests/dns-admission-controller-test.lisp
A bind failure reverse-unwinds to the exact pre-attempt registry forced-bind-failure-restores-pre-attempt-state tests/dns-admission-controller-test.lisp
An adapter declaring an unsupported contract version is refused at register time dns-adapter-version-fail-closed tests/dns-adapter-registry-test.lisp
A decoded query resolves end to end through the registered runciter handler dns-bind-resolves-query-through-runciter-handler tests/dns-adapter-registry-test.lisp

Relocated to runciter (the nameserver's suite), cited for the auditor:

Criterion Home
The full RFC-2308 answer-composition decision tree (present/absent/NODATA/NXDOMAIN, CNAME chase, wildcard synthesis, not-authoritative) runciter compose-answer parity suite (parity vs valis's prior zone-view-lookup)
A failed rebuild keeps the prior index answering (serve-last-good) failed-rebuild-keeps-prior-index (runciter tests/dns-service-test.lisp)
A source outage after load keeps the nameserver answering source-outage-after-load-keeps-answering (runciter tests/dns-service-test.lisp)

The substrate and the engine: where the cutover logic lives

The primary-zone story above is valis answering names it authored. The substrate also answers names it did not author (a legacy zone fed in from an upstream master during a migration onto valis) and feeds its own zones out to downstream secondaries. That secondary operation is a clean split between two owners. valis owns the durable state (the zone rows, the provenance and lifecycle facets, the crash-safe timer deadlines) and the operator door that flips a zone through its lifecycle. The sibling library runciter owns the pure logic (whether a transfer is admissible, what the served-type set is, how an SOA refresh-timer advances, which cutover transitions are legal, and whether an entry may answer right now) and the serving index that answers from it. valis never re-implements any of that.

valis reaches the logic through a single late-binding seam, src/operator-state/runciter-serving-seam.lisp, and never through a runciter symbol directly. The seam declares the contract (accept-zone-transfer, advance-refresh-timer, cutover-transition-legal-p, may-answer-p, effective-status, out-of-set-records) and resolves each real runciter symbol at call time, delegating directly. runciter is a hard dependency that delegates every target live, so the seam carries no valis-side fallback: a pinned-but-missing symbol fails closed (runciter-serving-unavailable) rather than serving a stale local answer. The point of the seam is that the DNS-semantic vocabulary has exactly one home: the secondary door and the cutover door speak to this package, so the day a newer runciter ships a sharper policy the substrate picks it up with no caller change and no second copy of the rule to drift.

The general naming database: a zone is the first record class

:names is not a DNS table that happens to be called a naming database: it is a general naming database (a Plan 9 ndb analog) whose first record class is a DNS zone. The cross-cutting facets that describe any named entry (who controls it versus who authored its data [=authority-state=, provenance-kind=], the lifecycle position [=cutover-state=], the monotonic version, and the crash-safe SOA refresh-timer state) live on the =ndb-entry anchor (src/operator-state/ndb-entry.lisp), keyed by (name, kind) and agnostic to the record class. The DNS-typed dns-zone projection is the typed face of those facets for kind "dns-zone": its origin is the anchor's name.

This is a deliberate altitude choice, not incidental layering. Modelling provenance, write-authority, and version at the ndb-general altitude rather than as DNS-only columns means a later record class (an identity or federation entry) attaches to the same anchor with no schema migration. A one-time backfill seeds every existing zone as a primary / operator anchor, so the general model is populated from the DNS class that motivated it without a flag day.

Accepting a secondary: authorize the master, never the session

A secondary transfer arrives in-process from the wire engine as a flat sequence of already-decoded, runciter-typed records. ingest-secondary-zone (src/operator-state/secondary.lisp) authorizes it against the upstream-master principal (an operator-curated allowlist of public master references, one per zone) fail-closed, and it does so before the instance fence and before any write. The session or owner principal that happens to drive the ingest never enters the authority decision: control of a zone is not authorship of its data, and an in-process transfer has no meaningful mounting principal to vouch for the bytes. A master that is not on the zone's allowlist signals upstream-master-not-allowed and commits nothing: the refusal precedes the transaction, so a refused or un-allowlisted master cannot leave a partial zone behind.

An allowlisted transfer still has to be admissible. The DNS-semantic decision (the served-type set, SOA presence and uniqueness, serial sanity) is runciter's, consumed through the serving seam's accept-zone-transfer. A record whose type falls outside the served set refuses the whole transfer atomically (:refused-out-of-set-type); the offending (owner . type) pairs are named for the operator and the audit surface through the seam's out-of-set-records helper, single-sourced in runciter's own admission predicate so valis never re-derives or re-scans the served set on its own. An accepted transfer commits through the shared commit-zone-records door with a :secondary provenance and a :held cutover state, so the same fence-first, RFC 1982-monotonic, full-zone-replace transaction the operator text path uses runs here too, and no serving-view install runs, because a held secondary is tracked but does not answer.

The cutover lifecycle: held, answering, primary (operator-gated)

A migrated zone moves through three lifecycle states (:held, :answer-as-secondary, :primary), and valis flips it only on an explicit operator write. The two forward transitions ride the capability-gated :names /ctl control surface (src/namespace/names.lisp): an axis-level declare-secondary verb bootstraps a cold held zone (creating the held shell and the master allowlist entry in foreign-key order so the very first transfer can self-authorize), then a per-zone answer-as-secondary verb makes the zone answer from the upstream's data, and finally promote-to-primary makes valis the system of record for it. Each verb is an ordinary owner-gated 9P write whose clunk is the commit, exactly like a zone import.

Legality is runciter's, not valis's. commit-cutover-transition asks cutover-transition-legal-p through the seam whether the requested flip is a legal forward step from the persisted state; an illegal or out-of-order flip signals illegal-cutover-transition and persists nothing, and valis never advances a zone's state on its own initiative. A held secondary never answers: it surfaces :held through zone-cutover-status, and the nameserver refuses to answer it; the gate that enforces this is structural, described next, not a convention each caller must remember.

Answerability and expiry: the safety core

Two invariants keep a non-authoritative zone from ever answering when it should not, and under the data/answer split they straddle the seam: valis owns the facts, runciter owns the gate, and the flip door owns a write-time refusal.

The data valis owns is the cutover-status pair. zone-cutover-status surfaces a zone's (cutover-state, expire-deadline): :held for a not-yet-answerable secondary, :answer-as-secondary with its absolute epoch-second expire-deadline, or :primary with a NIL deadline (a primary never expires; valis is the system of record). The deadline is persisted as an absolute instant (the ndb-entry timer columns, written through persist-refresh-timer-state), never an uptime offset, and the read is fail-closed: an unstored origin, and a secondary with no recorded deadline, surface no answerable deadline. The nameserver gates on this (runciter's may-answer-p treats a :held zone, an expired secondary, or one with no deadline as not answering, and an expired owning zone as not owning the name at all), so a :held or expired secondary cannot answer regardless of which path loaded it. These choices are load-bearing: a restart must not silently extend a secondary's lifetime past the expiry it inherited from its last transfer, and a master gone unreachable must eventually cause the secondary to stop answering rather than serve stale data forever. The valis side of this guarantee is verified in tests/cutover-status-source-test.lisp (the :held, absolute-deadline, primary-NIL, and fail-closed cases); the gate itself is runciter's may-answer-p parity.

The flip door adds a write-time refusal: a flip to :answer-as-secondary is refused (secondary-timer-not-ready) unless a populated, non-expired deadline already exists, checked against the live clock (*zone-view-clock*, the surviving serve-time absolute clock in src/operator-state/zone-view.lisp, Unix epoch-seconds), so a secondary can never even be made answerable with a stale or missing timer.

The outbound zone-change feed: one signal off the shared commit

valis is also a source of zones for downstream secondaries, and that outbound side is pure in-process data. src/operator-state/zone-feed.lisp holds an in-process listener registry (*zone-change-listeners*, mirroring the DNS-adapter registry with the same register-time fail-closed contract-version check) that fires off the shared zone-commit point: commit-zone-records invokes its post-commit hook after a durable commit, and the feed installs fire-zone-change onto it. Because both the operator text-import path and the secondary-ingest path converge on that one door, a committed change on either path emits a single change signal with no per-door wiring; a cutover flip emits the same signal from its own commit point. A downstream secondary-peer allowlist (zone-feed-peers, the secondary_peer table) gates the data side, so the feed never offers a zone to an un-allowlisted peer.

The registry is fault-isolated by construction. fire-zone-change runs each listener under its own handler so a listener that signals is warned and skipped, its siblings still fire, and the error never unwinds out of the post-commit hook: an out-of-tree consumer throwing cannot turn a durably committed transfer into an apparent failure. This is strictly the data side: whether a given change warrants an actual wire NOTIFY, and the binary AXFR-out that carries the zone, are the sister DNS engine's concern, not the substrate's.

Boundary hygiene: what this substrate is not

The secondary substrate holds no wire and no secret. There is no port-53 serving loop, no binary AXFR codec, and (by construction) no transport-secret or TSIG column anywhere in its schema: every reference it stores is a public master or peer name or address, and the transport secrets that authenticate a real transfer are the factotum's domain (mercer), never :names's. The record boundary is the same one the primary path observes: the typed envelope (owner, TTL, class, type) is valis's, the per-type rdata grammar is runciter's, and only the rdata text crosses between them. A record whose stored rdata is malformed fails closed at the node read boundary: %render-zone maps runciter's typed rdata-parse error to a clean 9P protocol error scoped to that one zone, rather than letting an arbitrary Lisp error escape past the read into the serve path.

Validation coverage: secondary operation and cutover

Each guarantee of the secondary substrate is anchored to a named suite. Together they pin the authorization boundary, the held-never-serves invariant, serve-time expiry, the outbound feed's fault isolation, restart-safe deadlines, and the read-path typed guard.

Suite Property it pins
tests/secondary-ingest-test.lisp A transfer is authorized on the master allowlist fail-closed (never the session); an out-of-set type is refused and the offender named; a held secondary commits and reads :held cutover-status (so the nameserver never answers it).
tests/zone-serial-test.lisp RFC 1982 serial monotonicity on both the text and the secondary path: an advancing serial is honoured, a stale one refused as a regression, an equal one auto-bumps.
tests/operator-state-fence-test.lisp A secondary ingest under a superseded generation writes nothing: the split-brain fence stops the write.
tests/operator-state-zone-audit-test.lisp The ndb-general facet columns pass the confused-deputy audit and no operator-state table carries a secret column; the existing zone backfills as a primary anchor.
tests/cutover-lifecycle-test.lisp The held -> answering -> primary flips are operator-gated and observable; an illegal or unknown verb is refused; a non-owner cannot flip; declare bootstraps a cold transfer.
tests/secondary-timer-test.lisp Timer deadlines persist as absolute instants so a restart does not extend expiry; a held secondary never answers; a missing anchor signals rather than dropping the deadline.
tests/cutover-status-source-test.lisp The cutover facets the nameserver gates on: a held zone reads :held, a secondary surfaces its absolute expire-deadline, a primary forces NIL, an unstored origin is fail-closed. (The serve-time gate over these [held/expired/no-deadline never answers] is runciter's may-answer-p parity.)
tests/zone-feed-test.lisp The change signal fires from both ingest paths; a throwing listener is isolated from its siblings and the durable commit; only an allowlisted peer is fed.
tests/names-import-test.lisp An operator zone imports, lands rows, reads back, and re-imports/refuses under the shared commit door; a fenced write commits nothing.
tests/names-axis-test.lisp Writes are owner-gated while reads are open; a malformed master file and a corrupt stored record each return a clean Rerror at the node boundary.

Privilege: unprivileged valis, external steering

The eBPF program load, the sockmap population, and the netns attach are privileged operations performed by a host agent external to valis. valis itself runs unprivileged and simply serves on the listening socket it is given. Holding no privileged kernel state is the same property that makes valis migratable: there is nothing host-specific for it to carry.

The one host-coupled obligation is a contract: a valis unit declares the designated ports it answers, and the host's privileged agent undertakes that those ports reach the unit's socket. It does not undertake to steer only those. The steer is a catchall and carries every port at the unit's address, so the narrowing is the agent's firewall rather than the declaration (the public surface). That contract is what "valis-compatible infrastructure" concretely means, and on migration it is the destination host's agent that re-establishes both halves so the arriving unit answers its ports again.

The IETF edge and the sovereign core

A core promise of the doctrine is that a person's valis answers the existing internet protocols on their designated ports (SMTP on 25, HTTP on 80 and 443, DNS on 53, IMAP on 143) and services them as a faithful, standards-compliant server. Those callers are the anonymous, hostile internet; they know nothing of valis's namespaces, capabilities, or 9P. This sits in natural tension with valis's evolution toward its own identity-first, capability-scoped, 9P-native services. valis resolves the tension by carrying the split inside one migratable unit, as two tiers that meet at the namespace:

  • The IETF edge is port-addressed. A connection's recovered destination port selects the protocol module that answers it: this is exactly what the registry (src/registry.lisp) does. Edge modules face hostile input and are anonymous-first: a foreign peer with no proven identity is the anonymous principal, reaching a deliberately narrow capability-scoped namespace (an inbound mail spool, a public file view).
  • The sovereign core is name-addressed. Native services are reached by 9P paths in the namespace, never by port. They are identity-first and grow as valis evolves.

The IETF protocols are first-class native modules, not a compatibility skin bolted onto a "real" inner system: keeping them first-class is how the doctrine's promise is kept. An edge module reaches the core only by carrying its connection's principal and capabilities into the namespace, exactly like any other module; the tier distinction is one of trust posture and hardening, not of mechanism.

The seam is 9P-shaped, so isolation is a transport choice

The hostile edge is the part eating untrusted input, so the long-term target is to run edge modules as sandboxed OS processes that reach the core only over 9P with their narrow capabilities: a compromised edge process then holds nothing but its own anonymous-scoped authority. The near-term implementation keeps edge and core co-located in one image with a logical boundary (fast to build), hardening the anonymous path first.

The durable commitment that makes this an evolution rather than a rewrite is that the edge↔core boundary is a 9P/capability interface from the start, never a shared-memory assumption. Co-location is then 9P over an in-process transport; isolation is the same 9P over a pipe between processes. This is the same discipline applied to the event multiplexer (epoll now, iouring later) and to module placement (local thread or remote node): design the seam at the eventual boundary, implement the cheap version first.

The edge↔core seam contract

The seam is a single module (src/edge/seam.lisp) and it is the only place an edge module reaches the sovereign core. An adapter never touches the namespace directly; it asks the seam for the view its connection is entitled to and walks that. The entitlement is the connection's principal and its request-scoped capabilities, carried across a real 9P interface: the same attach, walk, and read a remote 9P client performs, with no privileged shortcut.

The seam holds one live attach per principal class, not per connection. Every anonymous connection shares a single cached attach to the public view, because they are all the same principal and opening a fresh 9P session per request would be ruinous on a busy port; a keyed principal, when keyed edge service arrives, gets its own attach through the identical mechanism. The cache key is the principal class rather than the principal object, because the anonymous sentinel and a keyed principal are the same Lisp type: only the class of authority distinguishes the slots.

A second, deliberately separate path serves the request-scoped phase: the with-request-capability entry point mounts a capability surfaced inside a request into a fresh private per-request view, runs the request body bound to that view, and unmounts on the way out, including on a non-local exit. The private view is not the shared cached attach, and the reason is a concrete leak it forecloses: mounting into a shared view would make one connection's request-scoped grant briefly visible to every other connection on the same principal. A per-request view is structurally incapable of that cross-connection disclosure: the mount lives and dies inside one request's dynamic extent.

The transport under the seam is a rebindable factory: *edge-channel-factory* is a function returning the two endpoints of a channel, defaulting to an in-process channel served against the assembler root. Rebinding it swaps the transport with no change to any adapter: the co-located case (an in-process channel), a TCP loopback to the fabric's own 9P listener, and a real inter-process socket pair have all been driven through this one point, each serving the same publication byte-for-byte. An adapter names no transport symbol at all; it imports only view-for from the seam. That import boundary is what makes "co-located now, sandboxed later" a binding rebind rather than a rewrite.

The anonymous-grant lifecycle and self-healing

The public edge runs on exactly one bearer grant. At fabric start (src/fabric.lisp), start-fabric mints a single edge-wide capability name designating the public view with read-only rights and installs it as the anonymous grant; the seam attaches against it on first use. There is no per-adapter grant and no per-connection minting: one name is the entire public authority, and an auditor can find its single mint site and its single designation.

Because that one attach is long-lived, the seam has to survive losing it, and it discriminates why the view was lost before deciding what to do. The discriminator is membership in the revocation store. If the grant's hash is in the store, the loss was a deliberate unpublish: the view goes dark and stays dark (walks fail, adapters serve absence) until the operator re-establishes the public view. The seam never silently re-mints a revoked grant. Any other loss (a closed session, a dropped channel, an expired grant) self-heals: the heal path consults the revocation store first and re-checks it on every retry (so a revocation landing mid-heal still wins), re-attaches with a bounded inline retry, and re-mints through the grant's mint hook only on genuine expiry. The re-mint can therefore never resurrect a revoked grant, because the store is consulted before the mint is reached.

Sovereignty on the wire is the absence of a name, not a policy refusal. When an anonymous caller names anything outside the public view, the walk simply fails, and the adapter surfaces that failure as the protocol's own absence response: an HTTP 404-class answer, a Gopher type-3 error item. There is no "permission denied" code path at the edge to disclose that a thing exists but is forbidden; the owner's axes have no name in the anonymous view, so the only honest answer is that they are not there.

Edge wiring and the fail-closed bind order

The edge controller (src/edge/controller.lisp) binds and unbinds the edge ports through start-edge / stop-edge, riding the listener's already-running event loop rather than owning a listener of its own. Dispatch is purely port-addressed: a connection's recovered destination port selects its protocol module through the registry, exactly as the central abstraction promises.

The bind order is fail-closed, and the gate is a real resolution, not a presence check. Before start-edge binds a single port it asserts the fabric is up, the anonymous grant is wired, and a real view-for for the anonymous principal can attach and walk the public view through the seam. A grant string that decodes but no longer resolves (revoked, or pointing at a view that has gone dark) fails this gate, so the edge refuses to come up serving a public face that cannot actually be reached. No namespace means no open port. Binding a port here opens it only for dispatch, though: whether that port is reachable from off the host is a separate, privileged decision at the host agent's firewall, not a consequence of the edge binding it: see The public surface. Teardown runs in strict reverse: stop-edge closes the ports before the seam resets, so the wire stops feeding a session before that session is torn down, and a mid-start failure unwinds whatever bound in reverse order.

The one construction point a later phase rebinds is named and isolated: *edge-source-constructor* builds each port's event source, defaulting to a plain bind, and is the seam where a steered file descriptor handed over by the privileged host agent arrives. The resident boot already rebinds it (src/main.lisp), so the edge binds a source around a passed-in fd instead of opening its own, with nothing else in the controller changing. Alongside the wiring, the controller publishes a read-only /edge status subtree (src/namespace/edge.lisp): a directory per bound port carrying live connection, budget, and deadline-cut counters rendered fresh on every read, plus an anonymous file reporting whether the public view is unwired, dark, undetermined, or serving. Undetermined is kept distinct from dark because the two want different repairs: dark means the grant was found revoked and is answered by minting a fresh one, while undetermined means the node could not establish whether the grant is revoked at all, refuses to serve on that, and would be no better off with a fresh grant. The subtree is observable as ordinary files, and it is named only in the owner's canonical frame: an anonymous attach cannot walk to /edge at all, so the diagnostic surface is itself sovereign.

The edge abuse posture

The edge faces the open internet, so abuse defense is layered, and the layering is deliberate: each defense lives at the tier that can express it most cheaply, and none of it lives in the sovereign core.

At the adapter tier, each connection carries a read deadline and a hard cap on input size: a client that connects and sends nothing, or dribbles bytes slower than the deadline, cannot hold a handler thread, and an oversized request line is refused rather than buffered. At the edge tier, each bound port carries its own concurrent-connection budget. When a port reaches its budget the controller stops accepting on that port: it deregisters the listening source, so overflow connections wait in the kernel's listen backlog and no valis thread or write is ever spent refusing an over-budget client. A connection's whole lifetime is wrapped in one place in the controller (never in adapter code), so the live count is decremented on every exit path (normal, error, or deadline cut) and the port re-arms its accept as soon as a slot frees. A flood on one protocol's port cannot starve another, because the budgets are per-port. The third tier (per-IP tracking, rate limiting, and volumetric or amplification defense) belongs to the privileged host agent that sits in front of valis, outside the process entirely.

The rationale for the split is the sovereignty invariant. Per-IP and volumetric defense need a view of the network the unprivileged, migratable valis process deliberately does not hold, and the sovereign core must carry no abuse logic at all: the core moves and mounts views, it does not reason about hostile traffic. Pushing the outermost defenses to the host agent and keeping only connection-occupancy and per-port budgets inside valis keeps the core clean while still bounding what a single open port can consume.

Connection lifetime: what the edge assumes, and what it no longer does

Nearly everything the edge serves today is request shaped: a client connects, says what it wants, is answered, and goes away. The connection exists to carry one exchange and has no reason to outlive it. For that traffic the shape described above is the right one and is deliberately unchanged. A thread per connection is cheap when the thread lives as long as one short exchange; a per-port concurrency budget is a true measure of work in flight when every live connection is work in flight; and a wall-clock deadline is a sound liveness test when silence past the deadline means a client that was never going to finish.

A protocol whose connections are long lived, mostly idle, and written to unprompted breaks all three at once, and it breaks them for one reason rather than three. The thread cost, the per-port budget and the one-shot deadline read as separate limits, but each is an expression of a single assumption nobody had written down: a connection lives exactly as long as the request that created it. The table below states that assumption and its relatives plainly and disposes of each. The disposition is rarely a removal. Most are SPLIT: the request-shaped path keeps the shape it has, and a protocol that declares itself continuous gets the other one. A protocol declares which it is; the seam never guesses.

Assumption Where it is made Disposition Reason
A connection lives exactly as long as the request that created it src/protocol.lisp:52, where handle-connection runs to completion and the exchange ends with it REPLACED A continuous session is opened by an exchange it outlives, so its lifetime belongs to the session rather than to a handler's dynamic extent.
Each accepted connection costs one OS thread for its whole life src/edge/controller.lisp:312, one thread spawned per accept SPLIT Kept for a request-shaped connection, where a thread that lives as long as one short exchange is the cheapest place to hold its state. Replaced for a continuous session, whose thread is idle for nearly all of its life, so the cost is paid per connection instead of per unit of work.
Only a listening socket is an event source the two source classes in src/backends/epoll.lisp (:350, :456) REPLACED An established connection becomes a registered source as well, which is what lets the loop drive it without a thread of its own.
Readiness means a connection is waiting to be accepted source-ready in src/multiplexer.lisp:171 REPLACED Readiness now also means an established connection has bytes to read or room to write, so one notification serves two kinds of source.
A source is monitored for readability only the :read registration in src/backends/epoll.lisp:268 REPLACED A queued write that could not complete needs writability notification to finish; without it, a slow reader can only be served by parking a thread on it.
Output needs no buffering because a handler writes straight to the stream no output queue exists anywhere in the edge layer: src/protocol.lisp:52 hands the adapter the connection with nothing in between REPLACED A queue is the only place a bound can be enforced, so the queue has to exist before a cap can be placed on it. Without one, a peer that reads slowly is served out of whatever the handler happens to allocate, which is bounded by nothing.
Liveness is a wall-clock one-shot armed by the handler set-read-deadline in src/connection.lisp:249 SPLIT Kept for a request-shaped exchange, where silence past the deadline really does mean a client that will not finish. Replaced for a continuous session, where silence is the normal state and a clock alone cannot separate a healthy idle peer from a dead one; there the exchange itself has to supply the evidence.
A live connection is in-flight work, so one per-port count bounds abuse the default budget at src/edge/controller.lisp:158 REPLACED A session that is idle by design holds a slot while consuming nothing, so a count of live connections stops measuring work and starts refusing legitimate peers long before any resource is scarce.
Bytes move through the connection's buffered stream view connection-stream at src/connection.lisp:193, vending the accepted socket's own buffered stream SPLIT Kept for a request-shaped exchange, where reading ahead is exactly what makes line and header parsing cheap. Replaced for a connection driven from the event loop: the buffered view reads ahead into a userspace buffer, so bytes already drained out of the kernel are invisible to readiness notification, and a connection driven from the loop must therefore never take that view.
The worker closes the connection when its handler returns src/executor.lisp:107 and src/edge/controller.lisp:263 SPLIT Kept where the handler's return really is the end of the connection. Replaced for a session that outlives the exchange which opened it: closing becomes an explicit act ordered after the last queued bytes have left, and the party that closes is whoever ends the session, not whoever ran the last handler.
Every adapter can be driven the same way, because one budget executor serves them all src/edge/tls-serve.lisp:450, src/edge/mail-serve.lisp:286, src/edge/dns-serve.lisp:1005 and the controller's own ports all build the same executor REPLACED An adapter is driven from the event loop only if it positively declares that it drives the raw descriptor and hands nothing above it. Anything that terminates, decodes or wraps the stream keeps the thread shape, because readiness is asked of the descriptor while the wrapper holds transformed bytes above it, and the two disagree. This is a safety property rather than a performance note: on a port where the stream is terminated, driving the raw descriptor would serve the connection in cleartext.
A protocol module never holds a socket or a descriptor the module contract itself, held by every adapter and by no single file KEPT This is what makes a module killable and rebuildable from durable state plus a client reconnect. A module that held a descriptor could not be rebuilt without the connection dying with it, and the substrate would stop being movable.
Every outbound connection leaves through the one dial verb the single egress seam KEPT Nothing this work adds opens a socket of its own. A continuous session's own reconnection, where it has one, is an outbound connection like any other and leaves the same way, so the egress policy that governs the process keeps governing it.

The dispositions above leave a set of design questions that each have exactly one answer. They are recorded here so the implementation does not re-decide them one file at a time.

Where the send-queue cap is tested
The cap is tested against the length the queue would have once the message is added, before the message is buffered. The reason is arithmetic rather than taste: testing the already-queued length bounds the queue at the cap plus one maximum message, and no maximum message exists on this path, so that placement bounds nothing at all. src/transport/client-secure-endpoint.lisp:148 is the precedent already in the tree, rejecting an over-long declared length before any body is read.
A message larger than the whole cap
A single message that exceeds the entire cap is refused outright rather than split across several sends. The reason: splitting does not reduce the bytes the queue holds, and the seam has no producer it can pause part way through a message, so splitting would relabel an unbounded hold rather than bound it. That refusal is reported distinctly from an ordinary over-budget refusal, because an over-budget send can succeed on a later attempt once the queue drains and this one can never succeed at all.
Who closes on a breach
Closing a connection that breaches the cap is the edge's act and never the protocol module's. The reason: the queue and the cap belong to the edge, and a component cannot enforce a bound it does not hold. What this leaves with the module is an obligation rather than a power. It must notice that a send was rejected, and must never assume a send lands.
The cap is not the only backpressure
A bulk producer can read how full the queue is and suspend itself at a fraction of the cap, which makes the close a last resort instead of the only one. The reason: a producer that can pause is far cheaper to stop than a connection is to lose.
A registration timeout is not an idle timeout
These are two different mechanisms and are kept apart deliberately. The registration timeout runs from the moment the connection was established and is not reset by activity, while the idle timeout runs from the last activity and is paired with a protocol-level probe that asks the peer to prove it is still there.
A session with no liveness probe
A session whose protocol offers no way to probe the peer gets no idle cut at all, and only the registration timeout applies to it. The reason has to be stated out loud because the opposite is the tempting default: cutting an idle session that cannot be probed reinstates the wall-clock cut on exactly the connection that is idle by design, which is the mechanism this work exists to replace. Such a session ends when its peer goes away or when the edge tears its port down, never on a clock.
The send path copies
What a caller hands the send path is copied at the moment it is handed over, and never retained by reference. The reason: the per-session read buffer is reused on the next read, and a short write leaves queued bytes waiting across callback boundaries, so a caller echoing straight from a read into a send would otherwise ship whatever arrived next. The cost is one allocation per enqueued message, and it is reported as a measured per-unit figure rather than left implicit.
Every sizing constant here is provisional
Each sizing constant this seam introduces says so in its own docstring, and each is derived from a bound measured in this tree rather than adopted from another implementation. The reason: asymptotic shape is a design-time decision while constants come from profiling this system, so a number borrowed from a daemon carrying different message sizes would be a guess wearing the authority of someone else's measurement.

What the shape costs, per unit

A design that cannot say what one unit costs cannot say how many units a machine holds, and that question was among the grounds on which the dispositions above were chosen. The figures below are what the shape actually costs. They are produced by a named probe in the suite rather than reasoned from the source, and that probe reports every reading whether or not it asserts on it.

Unit Measured cost
One instance at rest, a loop up and no sessions 132,603,808 bytes resident
Bytes retained per idle session 4,802, of which 4,096 is the session's own read buffer
The same figure gross, with the peer end held in the measuring image 6,488
OS threads per idle session 0, against one per connection on the request-shaped path
The two liveness clocks a session arms 504 bytes per session at 16 sessions, 536 at 32
Allocation per queued and drained message, 256 octets of payload 304 bytes

At 4,802 bytes a session, ten thousand quiet peers hold about 46 MiB of session state. That is why the per-port ceiling on live sessions is a density decision rather than an abuse bound, and it is the figure a sizing pass should start from.

The per-message figure includes the copy the send path makes of the caller's bytes at the moment it accepts them. That copy is the decision recorded above, not queue overhead, and a sizing pass that read the figure as pure overhead would re-derive a question this document has already answered.

Conditions, without which none of the figures means anything. They were taken at commit f464a62, in a freshly started image with a warm compilation cache, nothing else running in it, one process at a time. The instrument's own noise floor, measured first from repeated readings of the instance at rest, is 256 bytes, and every per-unit figure above stands at least thirty times clear of it. Each figure is the median of three readings, and each session reading is a delta against an at-rest reading taken moments before it rather than against a baseline read at the start of the run. The figures move by tens of bytes between runs, so a reading differing from these in its last two digits is the same reading.

What the probe detects, and what it does not
It detects a per-session cost that grows worse than linearly as sessions accumulate. That is the defect invisible both to reading the code and to a green suite, and it is the one demonstrated: an injected cost growing with the number of sessions already adopted drove the cost of a doubling to 5.90 and 4.64 times against an asserted ceiling of 3, and the probe went red. It does not detect a constant that simply grew. Raising the per-session read buffer sixteenfold moved the per-session figure from 4,816 bytes to 66,288 and left every ratio at 2.00, so every assertion stayed green while the cost rose by a factor of fourteen. A grown constant shows only by comparing the figures one run reports against an earlier run's, which is why the probe prints every reading rather than only asserting on it.
How far the probe has been falsified
The assertions do not all carry the same weight, and the difference matters more than the figures do. Two have been observed failing: the ceiling on the cost of a doubling, and the whole thread-growth control, the latter by adopting the connections so that no worker was ever spawned, which is exactly how the instrument would stop discriminating in practice. The liveness and queued-message bands, and every structural assertion, have only ever been observed passing. Two are weaker still and are named here so that nobody credits them with more than they do. The lower bound on a doubling can fail only if a doubling costs less than 1.25 times as much, which no defect in this seam produces, so it is a rail against a broken measurement rather than a defect detector. And the liveness bound compares a figure against twice one measured moments earlier in the same test, so a defect inflating both counts equally passes it: it catches per-unit growth between 16 and 32 sessions and nothing else.
What would falsify these figures
A run of the same probe at the same commit, in a freshly started image with nothing else live, reporting a per-session cost outside the band the measured noise floor allows.
What is not covered at all
The cost of a namespace read parked on behalf of a session. That belongs to the namespace layer and was measured there, in an image rather than over a socket held open for a long period, so nothing here says what such a read costs once the connection under it has been quiet for hours.
Every sizing constant here is still provisional
None of them has been profiled under load, and the figures above are what a later profiling pass will move them against. The per-session read buffer of 4,096 octets, the send-queue cap at four times the carrier frame ceiling, the suspend fraction of one half, the idle, probe and registration intervals of 120, 30 and 30 seconds, and the per-port ceiling of 400 live sessions are each provisional and each says so in its own docstring. A number recorded with no statement of what it is provisional against reads as settled to the next person.

Validation coverage: the edge↔core seam

Each edge success criterion and security property is anchored to a named test an auditor can run. The acceptance suite drives plain loopback TCP clients against fully shipped wiring; the seam and controller suites exercise the seam internals and the budget machinery directly.

Criterion Test File
Recovered destination port alone selects the protocol module port-alone-selects-protocol-module tests/edge-acceptance-test.lisp
An anonymous caller names nothing outside the public view; absence is the wire answer anonymous-names-nothing-outside-public-view tests/edge-acceptance-test.lisp
The same adapter serves byte-identical over a second transport (TCP loopback) same-adapter-serves-over-second-transport tests/edge-acceptance-test.lisp
The same adapter serves byte-identical over an inter-process socket pair same-adapter-serves-over-socketpair-transport tests/edge-acceptance-test.lisp
Adapters name no transport symbol; the transport split is seam-internal adapter-code-unchanged-between-transports, adapters-name-no-transport-symbols tests/edge-acceptance-test.lisp, tests/edge-seam-test.lisp
A request-scoped capability never leaks across connections on the shared principal request-capability-never-leaks-across-connections, request-capability-unmounts-on-nonlocal-exit tests/edge-seam-test.lisp
A revoked grant stays dark; loss self-heals; expiry re-mints only after the store check revoked-grant-stays-dark, view-loss-self-heals, expired-grant-re-mints-after-store-check tests/edge-seam-test.lisp
The edge refuses to bind without a resolvable public view start-edge-refuses-without-namespace, start-edge-refuses-dark-anonymous-view tests/edge-controller-test.lisp
A mid-start failure unwinds every bound port mid-start-failure-unwinds-bound-ports tests/edge-controller-test.lisp
Teardown closes ports before the seam resets stop-edge-closes-ports-before-seam tests/edge-controller-test.lisp
The connection budget stops accepting at the cap; a queued connection is served when a slot frees budget-cap-stops-accepting, queued-connection-served-after-re-register tests/edge-controller-test.lisp
The live count is restored on every exit path and never leaks a slot counter-decrements-on-error-exit tests/edge-controller-test.lisp
The /edge status surface reports live counts and the anonymous dark state, and is unnameable anonymously edge-status-reports-live-counts, edge-status-tracks-anonymous-dark-state, anonymous-view-cannot-name-edge-status tests/edge-controller-test.lisp

The owner-proof flow: a live signature, never a token

The edge faces the anonymous internet, but one caller on the same port is not anonymous: the owner. The owner reaches a management view by proving possession of the custody master key on every request, not by presenting a token. Each request the owner makes is an HTTP Message Signature (RFC 9421): the owner signs the canonical request material with the same Ed25519 custody key that roots all valis authority, and the edge verifies that signature against the owner's public key. This is the deliberate choice that separates owner authentication from the anonymous grant. A bearer token is replayable by anyone who holds it (which is exactly the anonymous /pub model, an unscoped grant pointed at a public view), and pointing a bearer model at the management view would let a token-holder act as the owner without ever holding the owner's key. A live per-request signature cannot be lifted and reused, and it carries no server-held session state to leak or to outlive its purpose: the proof is the key-possession itself, re-demonstrated each time.

Verification routes through the same single seam the rest of identity admission already uses. The authenticate generic in src/identity/authenticator.lisp is the one named operation that turns a completed proof into a principal; the Noise handshake is the first method on it and the anonymous resolution is the second. The seam was built reserving exactly a third method for an HTTP-edge proof, and this is what fills it: a signature authenticator specialises authenticate to mint the owner principal from a valid signature, with no code above the seam changing: the connection layer, the capability layer, and the namespace mounter still see only a principal and never learn what proved it. The verifier itself lives in src/identity/http-signature.lisp: it reconstructs the RFC 9421 signature base over the covered components (the request method, the request authority, and the request path) through the same base builder the owner's signer used, so the bytes signed and the bytes verified are identical by construction and cannot drift apart into a canonicalization mismatch. It checks the signature's key identifier against the owner DID and verifies the Ed25519 signature against the owner public key.

The custody boundary holds at the edge exactly as it does everywhere else: the edge module relays only the signed request material (the method, the authority, the path, and the two signature header values) to the seam, and never sees a key. Verification needs only the owner public key, which is already in the image on the factotum side; the private custody key never crosses into the edge tier, and no ambient authority is granted to a module that eats hostile input. The replay defence is twofold and lives with the verifier: a bounded acceptance window over the signature's created / expires stamps rejects a signature presented too early, too late, or long after it was minted, and a bounded, expiry-pruned nonce cache rejects a second presentation of a nonce already seen inside the window. A signature that is absent, malformed, signed by the wrong key, expired, or replayed does not raise an error and does not reach management: every such failure resolves to the anonymous principal, so the management view is reachable only by a valid, live, first-use signature. Falling open to anonymous, never to an error and never to elevated access, is the fail-closed posture carried to the auth path.

Identity-selected views on one URL space

Anonymous and owner walk the same URLs on the same port. What differs between them is not the path they ask for but what is mounted under it. The anonymous caller resolves to the published view: the /pub scope, the same public face the anonymous grant has always served. The owner, once a valid signature has resolved to the owner principal, resolves to the full sovereign canonical frame (/pub /id /cap /edge /bus) rendered as a browsable read projection. The discrimination happens at one site: view-for (src/edge/seam.lisp) hands every non-anonymous principal to build-base-view (src/namespace/assembler.lisp), which dispatches the owner DID to assemble-canonical-frame server-side; the seam re-implements no identity comparison of its own. The HTTP adapter then walks an identity-dependent root: the anonymous request walks under the /pub scope, the owner request walks the canonical frame directly, because for the owner that frame is the root. The principal is re-resolved for each request and never cached on the connection, so under a persistent connection a signed request and a later unsigned request on the same socket each resolve their own principal and their own view; a signed request cannot infect an unsigned one that follows it.

This is sovereignty expressed as absence, carried now to the HTTP wire. An anonymous caller does not receive a permission refusal when it names a management axis: it has no name for one. The management names are simply not mounted in the anonymous view, so a walk toward them fails the way a walk toward any non-existent name fails, and the adapter surfaces that failure as the protocol's own not-found answer. There is no code path at the edge that says a thing exists but is forbidden, because such a path would itself disclose that the thing exists. The honest answer to an anonymous caller reaching for the owner's axes is that, in its view, they are not there.

The HTTP/1.1 conformance surface

The published view is served by a real HTTP/1.1 server, not a thin compatibility skin, so an ordinary client treats valis as an ordinary origin. The surface is deliberately bounded to the read path. A Host header is required and validated: an HTTP/1.1 request that omits it is answered 400, per the messaging specification. GET and HEAD are served, with HEAD returning the computed headers and no body; any other method is answered 405 with an Allow header. Connections are persistent, framed by Content-Length, with Date and Server on every response and the real status taxonomy (400 / 404 / 405 / 408 / 411 / 505 alongside 200 / 304). Conditional requests are answered from the substrate's own version stamp: the 9P qid.version of the addressed leaf becomes a strong ETag, and an If-None-Match that matches it yields 304 Not Modified with the validator and no body. That mapping is the substrate-alignment win: an ordinary HTTP cache works correctly against the sovereign namespace because the namespace already carries the exact per-node version a validator needs. That version is now derived from the node's content score (the low 32 bits of its Merkle hash; see the read-side caching tier), so the ETag is itself a content hash rather than an opaque counter, the content-addressed norm a strong validator is meant to be. A body mutation changes the leaf's score, so its ETag changes with it, and a conditional GET that no longer matches gets a fresh 200 carrying the new validator. The write-path long tail of HTTP/1.1 (chunked request bodies, =Expect=/100-continue, request pipelining) is out of scope by design: it bites almost entirely on request bodies and writes, and this is a read server.

The conformance surface composes cleanly with the edge abuse posture documented above rather than weakening it. A persistent connection holds exactly one of its port's concurrent-connection budget slots for its whole lifetime, while the per-request read deadline re-arms at the top of every request on that connection. The two together give the right behavior under keep-alive: a slow or dribbling client is cut by the deadline on the request it is starving, without a healthy persistent connection being dropped between requests, and the per-port budget continues to bound a flood of connections regardless of how many requests each one carries. The existing guards remain in force on the new surface as well (every response header value passes through the response-splitting guard before it is emitted, and every request path passes through the traversal-safe decode before it is walked), so the broadened method, header, and conditional-request handling adds no new injection or traversal surface.

Validation coverage: the HTTP edge

Each HTTP-edge criterion is anchored to a named test an auditor can run. The conformance cases drive plain loopback HTTP clients against fully shipped wiring; the owner-proof cases drive an in-image RFC 9421 signer over the same loopback; the signature unit verifies the cryptographic round-trip directly.

Criterion Test File
A faithful HTTP/1.1 GET returns a well-formed response over the published view ordinary-get-returns-wellformed-http11 tests/http-conformance-test.lisp
HEAD returns the headers with no body head-returns-headers-no-body tests/http-conformance-test.lisp
A Host-less HTTP/1.1 request is answered 400 missing-host-is-400 tests/http-conformance-test.lisp
Keep-alive serves two requests on one connection, each re-arming its deadline keep-alive-two-requests-one-connection tests/http-conformance-test.lisp
Conditional GET (If-None-Match against the qid.version ETag) yields 304 conditional-get-if-none-match-304 tests/http-conformance-test.lisp
A GET after a body mutation returns 200 with a fresh, differing ETag conditional-get-200-after-mutation-bumps-etag tests/http-conformance-test.lisp
An unsupported method is answered 405 with Allow post-returns-405-with-allow tests/http-conformance-test.lisp
Same port, identity-selected views: anonymous reaches the published view, a valid signature reaches management same-port-anonymous-vs-signed-views tests/http-owner-proof-test.lisp
The anonymous byte stream names nothing of the management axes; absence is the wire answer (404, not 403) anonymous-names-nothing-of-management tests/http-owner-proof-test.lisp
A valid owner signature reaches the management view valid-signature-reaches-management tests/http-owner-proof-test.lisp
An unsigned, invalid, expired, or replayed request resolves to anonymous; management is absent and /pub still reads unsigned-request-is-anonymous, invalid-signature-is-anonymous, expired-signature-is-anonymous, replayed-signature-is-anonymous tests/http-owner-proof-test.lisp
The signature base round-trips: the signer and verifier reconstruct identical bytes and a valid signature verifies signature-base-round-trip, verify-accepts-valid-rejects-tampered tests/http-signature-test.lisp
The replay window rejects a replayed or stale signature replay-window-rejects-replay-and-stale tests/http-signature-test.lisp

The per-namespace steering constraint

sklookup selects a socket per network namespace. The kernel runs the steering program only for traffic in the namespace the program's link is attached to, and it will only select a socket that lives in that same namespace. valis's listening socket and the agent's link must therefore share one namespace, or the steer silently never fires: load succeeds, the socket map is populated, the link attaches, and connections are still refused, with nothing in any log to point at.

This constraint is also the unit's boundary of authority, though it bounds what the steer can offer rather than what the unit answers. The steering program is a catchall: it fans every port on the unit's IP out to valis's one socket, so the namespace and the IP it carries fix the traffic that can be offered at all. That is not the answered set. A valis unit declares the ports it intends to serve, and on a routable IP the catchall does not narrow what it offers to that declaration. What narrows the reachable set back to the ports the agent admits is a separate control, the agent's default-deny firewall inside the namespace, and not the namespace itself. On a loopback-only island the two coincided and the namespace was the whole boundary; once the namespace carries a routable address they are two mechanisms, and the surface control is the firewall. This is the subject of The public surface, including why the allowlist must stay the agent's and never a module's. The first-cut topology resolves the shared-namespace requirement the simplest correct way: the agent creates the namespace and the unit's IP, then valis is launched into that namespace so its socket is born in the right place.

Privilege drawn as a repository boundary

Every capability that needs the kernel's trust lives in the external host agent and only there. CAP_BPF to load and attach the steering program, and CAP_NET_ADMIN (with CAP_SYS_ADMIN) to create the namespace and assign the unit's address, are held by the agent, a separate component in a separate repository. The migratable valis unit holds none of them. The privilege boundary is therefore not a runtime check that could be misconfigured; it is a boundary drawn in the filesystem, where an auditor who wants to know what runs with elevated privilege reads one repository and a compromised valis unit cannot reach for authority it was never granted.

This is the same property described in the edge abuse posture (the outermost, network-aware defenses belong to the agent because the unprivileged unit deliberately cannot see what they need), stated here as a privilege invariant rather than an abuse one. The steered socket needs only LISTEN state, nothing privileged: valis binds loopback or an ephemeral high port inside the agent's namespace and never binds the designated public ports. That valis holds no privileged kernel state is the very property that makes it migratable, because there is nothing host-specific for it to carry; on migration the destination host's agent re-establishes the steer so the arriving unit answers its ports again. The dialed port survives the steer because the kernel preserves the original four-tuple under bpf_sk_assign, so getsockname on the accepted socket recovers the real port the client dialed, the same recovery the listener already proves over loopback (see the fail-closed bind order and the four-tuple commitment in the listener section).

Validation coverage: the steered-fd handoff

The steering data path cannot be exercised from the Lisp image, since it needs a real kernel, a network namespace, and CAP_BPF=/=CAP_NET_ADMIN. The image-testable glue (descriptor framing, port declaration, the seam's recovered-port wrap) carries cold in-suite coverage. Past that there are two out-of-band gates rather than one, and they cross different boundaries. make steer-test runs in the agent's repository on the privileged side, signed off out of band exactly as the v9fs mount interop test is. The proving ground's steered-delivery gate builds a node and then drives it from a separate host, which is what puts a real off-host client on the far side of the steer and lets it read the delivery off accept queues rather than off an answer. The agent repository's docs/steering.org carries the same coverage table from the privileged side.

Criterion Test File
The steered seam wraps an inherited LISTEN descriptor and recovers the dialed port steered-fd-recovers-dialed-port tests/steered-fd-test.lisp
A received descriptor is validated as a LISTEN-state socket before it is steered received-listen-fd-is-validated-and-returned, non-listening-fd-is-rejected-fail-closed fulcrum tests/handoff-test.lisp
The descriptor handoff is refused until a valid port set is declared (sequencing) handoff-refused-before-ready-for-fd, handoff-advances-only-after-valid-port-set fulcrum tests/handoff-test.lisp, fulcrum tests/control-test.lisp
A declared port set is recorded exactly; an out-of-range port is rejected fail-closed valid-port-set-is-recorded-exactly, out-of-range-port-is-rejected-fail-closed fulcrum tests/control-test.lisp
The full steer lands the connection, recovers the dialed port, proves valis never bound it, and confines the steer to the namespace make steer-test (operator-run privileged gate) fulcrum spike/steer-test.{sh,lisp}
An off-host client's connection is delivered through the steer and dispatched on the port it dialed, including a port named in no configuration on either side steered-port-delivery scenario (operator-run gate) proving-ground/driver/scenarios/steered-port-delivery.lisp, proving-ground/drive-steered-delivery-gate.sh
TCP on the served port enters the steered socket's accept queue while the inherited listener's queue stays at zero, which is what separates the two hypotheses a DNS answer cannot steered-port-delivery scenario (operator-run gate) proving-ground/driver/scenarios/steered-port-delivery.lisp

The substrate as namespace

The doctrine's eight categories are not eight flat peers; they are a dependency stack, exposed as subtrees of the user's 9P namespace that protocol modules reach through their capabilities.

Semantic objects, not protocol silos

The substrate models the semantic object (a message, a contact, a publication, a contract), and the wire protocols are projections over it. This is the mechanism behind the doctrine's claim that "messaging is one thing": SMTP, IMAP, and NNTP are not three stores but three skins over a single message substrate; a message is the same object whether it arrived by SMTP or was posted by NNTP. Publishing (web, forum, and social protocols) likewise projects one publication substrate. The substrate de-duplicates reality; protocols are views onto it. This is how the protocols are "given back": the user owns the object and chooses which protocols answer for it.

A protocol module (the edge tier) is therefore a wire-format adapter between an IETF protocol on its port and a semantic subtree of the namespace, gated by the connection's capabilities. The substrate is protocol-agnostic; the protocol is storage-agnostic; they meet at the namespace.

The dependency stack

  • Foundation - crypto (1), names (2), identity (3). Every other category reaches these. Crypto is the autonegotiated, invisible-by-default layer the capability model already rests on: the factotum-shaped agent, content encryption, transport security, key custody. Names are the user's sovereign namespace of people, places, and things; identity is the user's own, canonical and transient. Identity is where the principals the connection layer authenticates come from, and names and identity are who capabilities are granted to.
  • Data - data (4): the relational and reactive store over the migration backing store; the queryable substrate the semantic domains build on.
  • Semantic domains - messaging (5), publishing (6), contracts (7). Each is one substrate with many protocol projections. Publishing's public face is the unencrypted, content-addressed path served from many places; contracts (services and payments) is the most forward category, capability- and crypto-adjacent.
  • Composite - composite protocols (8) such as IPFS and GNU Social are assemblies of the layers below (content addressing; identity, publishing, and messaging, federated). Their place at the top confirms the stack.

Content-addressed storage (IPFS) doubles as how a migratable instance stays coherent across hosts. There is no native Common Lisp libp2p/IPFS stack; the intended path is CFFI bindings to a libp2p built as a C-ABI shared library, with an external-daemon HTTP bridge as an interim stopgap and a fully native implementation as a long-horizon ideal.

What is framed and what is open

Framed: the layering, and that the substrate models semantic objects with protocols as projections. Open, and recorded as research questions: how richly a semantic object must be modelled so a protocol projection is faithful rather than lossy (email threading, NNTP groups, social-graph semantics); what the data engine actually is in Common Lisp; whether names is one model with two faces (a sovereign address book that also answers DNS); and the shape of the contracts category. These are named so each domain is built against shared substrate interfaces rather than re-inventing storage and identity per protocol, the discipline that keeps "their computer, their data" true.

Plugin composition: protocols as content-addressed, capability-gated modules

The central abstraction promises "every protocol a plugin." This section gives the second half of that sentence a concrete noun: a plugin is a module (a registered, lifecycled unit of code), and a protocol is a module that fills the edge-adapter role. The mechanism is Shinmera's modularize (a library that makes a package first-class: its own metadata store and load/delete hooks), adopted as valis's plugin lifecycle. The fit was validated empirically before it was committed: a throwaway module proved, cold, that modularize composes with valis's package-inferred discipline, and surfaced the one trap that would otherwise read as incompatibility (the grouping form is not idempotent under ASDF's compile-then-load and must be guarded). Composition governs what code runs and what it may reach within the substrate; whether a module's port is reachable from off the host is a separate, privileged decision at the surface, never derived from the module (The public surface). Three decisions define the model.

A plugin is a module, and a module is one combined system

A protocol module is a single ASDF system that is both an asdf:package-inferred-system (valis's one-package-per-file discipline, where each file's defpackage declares its own dependencies and the build graph derives from :import-from) and a modularize:virtual-module, a named, discoverable unit with a metadata store and a teardown lifecycle. The two are not rivals: one class is both, package-inferred derivation still fires, and the module's :packages grouping gathers the inferred sub-packages under one name. valis-core stays an ordinary system; only the pluggables become modules.

The core's one structural change is to stop naming concrete adapters. The edge controller's hard-coded adapter list (%adapter-specs) becomes an *edge-adapters* registry that modules install into on load, the same "downstream installs the hook" idiom valis already uses for *edge-source-constructor*, *ctl-node-factory*, and *bearer-view-decorate-fn*. A module declares its adapter in one line of module storage (an :edge-adapter spec); a load hook registers it; a delete hook drains then retires it. Drain-before-delete is not optional: a live protocol has in-flight connections, so the delete hook runs the ordered edge teardown (deregister the source, drain in-flight work, unregister) before the package is unbound. That drain is the same primitive evacuation uses: the session drain on the 9P listener. Two correctness disciplines fall out of the lifecycle: registration must be idempotent (the grouping form runs twice under compile-then-load), and a protocol's macros stay module-local: cross a module boundary with functions, not macros, because a reloaded producer leaves a consumer's already-expanded macro call site running the stale expansion, silently and without error.

Module load is a capability decision, never a hash fetch

Content addressing answers which module: a module package, held in the content-addressed store or fetched over IPFS, is named by its hash, so an instance pulls exactly the bytes it asked for and verifies them on read. But a hash answers identity, not trust. Loading a module means compiling and running code, and code pulled by hash from the network is, unqualified, a supply-chain hole: running code validated only by its hash is a recipe for pain. So which modules an instance will load and compile is a capability decision, gated by the same owner-rooted authority that gates every mount: a module is admitted only when the owner, or an authority the owner delegated to, has vouched for its hash, exactly as a capability vouches for a subtree. The hash is the name; the owner's signature is the permission. This keeps the "evaporate and condense elsewhere" malleability without turning a recondensing instance into an open code-execution endpoint, the discipline that matters most before the system is opened to outside contributors.

An instance's identity is its namespace manifest plus its module manifest

A valis instance is defined by two content-addressed manifests, not one. The namespace manifest records what is mounted: the data side, which migration already reassembles. A module manifest records what the instance is composed of: the protocol and plugin set it answers with. Both are durable, content-addressed, and reassembled on recondensation: a fleeing instance does not carry its code, it records the capability-vouched hashes of the modules it ran, and the destination pulls, verifies, and compiles them to stand up functionally equal to the instance that evaporated. Because the substrate already content-addresses data, it content-addresses the modules too, and IPFS becomes the distribution layer for protocols, not only a backend for bytes. modularize's in-image list-modules plus per-module storage is the live projection of this manifest; the durable module manifest is its persisted, migratable form. This closes the migration story for code the way the namespace manifest closed it for data: a valis instance is its mounts and its modules, both reassembled from hashes the owner vouched for.

The module-manifest record shape

The module manifest is a durable, content-addressed record distinct from the namespace manifest, and the distinction is the whole point. The namespace manifest records mount roots: what an instance has mounted, the data side. The module manifest records the module set as code: each entry is a system name, the module's ordered source files as paths relative to the module root, and a 32-byte content score over those bytes. Names and relative paths and a score: never a key, never a secret, never the vouch itself. The record is built once, its entries sorted by system name, so the same module set serializes to the same bytes regardless of discovery order; a content-addressed record demands that determinism. Decode is fail-closed: a bad width, an unknown version, a malformed entry, or a missing referenced byte stream aborts the read rather than yielding a partial set.

The two manifests are siblings (a manifest child and a modules child) under one durable head tree. At genesis they are written by a single combined publish that obtains both envelopes without advancing, builds one root tree naming both children, and advances the head exactly once. This is not a convenience: it is the only correct sequence. Two separate publishes cannot work, because the first advance moves the head off the genesis generation and the second's generation-fenced compare-and-set fails closed. So the manifests are atomic siblings by construction, and the evacuative reassembly that reads the modules child is correct precisely because the genesis publish guaranteed that child exists under the same root.

The record stores no vouch and no key, and that absence is deliberate. A credential frozen into a durable record would carry the authority lifetime of the moment it was written into a record that outlives it: a revoked owner key still vouching for code months later. Instead trust is decided fresh at reassembly: the owner re-vouches each module's score against live custody and live revocation/fence state at the moment of condensation. The record's job is to name which modules; the live authority decides whether to admit them. Authority lifetime is decoupled from record lifetime.

The reassembly flow

A successor instance stands its module set up from the durable record the same way the namespace assembler stands a namespace up from its manifest: reassembly, not transfer. The source instance does not hand its running code to the destination: it records the vouched hashes of what it ran, evaporates, and the destination re-derives equivalent running code from the record. This is the code-side symmetry of the data-side migration: an instance's identity is its mounts and its modules, both reassembled from hashes the owner vouched for.

The reassembly is two-pass and all-or-nothing. The first pass reads the manifest fail-closed. The second pass, over every record before any module loads, resolves the recorded relative paths against the destination's source root, pulls the bytes, re-derives the content score, and asserts it equals the recorded score: a wrong-hash record aborts the whole reassembly here, in front of the first load. The pull is transport-agnostic: a rebindable puller seam means the bytes can come from the local store today and from a content-addressed network (IPFS/libp2p) tomorrow without reshaping the record: the IPFS gate stays open. Then each vouch is located: the owner re-vouches each score against live state, and a missing vouch aborts the set. Only after the whole set passes does the third pass load, routing every module through the same capability-gated admission chokepoint that gates an interactive load, never calling the loader directly, so the gate is structurally in front of compilation. If any module is refused at the gate, the adapter registry is restored to its pre-reassembly snapshot, so an earlier-admitted module's adapter cannot leak past a later refusal. A partial module set never boots.

The resident default set (HTTP and Gopher) is handled honestly rather than forced through this path: it is recorded name-only with a zero score, is already present as part of valis itself, and is skipped by the reassembly. Only genuine non-resident modules (those carrying a real content score) run the pull-verify-vouch-admit path.

The honest scope of resident-set parity

One claim must not be overstated. The resident default adapters are recorded name-only with a zero-score sentinel, because they are part of the valis system and are already loaded: there is no separate bundle to content-address. So the migration guarantee for the resident set is naming parity: a successor's recorded module set names exactly the source's set, and the resident edge registry the successor serves through names that same set. It is not content-score parity for those entries, and the documentation does not pretend it is.

The genuine content path (re-derive the score, match it against the record, locate an owner vouch, route through the admission gate) is exercised and proven by the non-resident scratch-module reassembly, not by the resident default. A reader who wants to see the score-to-admit guarantee should look at the non-resident reassembly test, where a module with a real content score is pulled, verified, vouched, and admitted, with an unvouched control proving the successor fails closed. The resident set proves the set-naming parity; the non-resident set proves the content gate.

The reassembly security boundary

The trust boundary the reassembly defends is the one between the durable record and the live authority that admits its modules. The record is published data (anyone who can read the store can read which modules an instance named), so the record holds no secret, and the decision to run those modules is made fresh against live state. The threats and the assertions that pin them:

Threat STRIDE Direct assertion
Hash substitution: a record names another module's bytes Tampering the destination re-derives module-bundle-score over the resolved bytes and aborts unless it equals the recorded score
Unvouched code execution: recondensing as an open code endpoint Elevation of Privilege every module is routed through admit-module, structurally in front of the loader; a refusal never reaches compile or load
Revoked or fenced module re-admitted from a stale record Elevation of Privilege trust is decided at condensation against live revocation and fence state (the re-mint posture), not a stored credential
A partial or unverified module set on a mid-reassembly failure Tampering / DoS two-pass reachability and vouch precheck before any load; the adapter registry is snapshot and restored on any abort
A key or secret leaking into the durable record Information Disclosure the record holds scores, names, and relative paths only; a byte-absence regression and the store-cluster import firewall pin it
A custodial or single-transport assumption baked into the pull Tampering (supply chain) the pull is a rebindable seam; a content-addressed network transport drops in without reshaping the record
A two-step genesis publish leaving one manifest unwritten Tampering both manifests are written under one root advanced once; a second advance would fail the generation fence closed

Validation coverage: plugin composition

The record shape and the reassembly are anchored to named tests an auditor can run and watch stand or fail. The record codec and its fail-closed decode are pinned in tests/module-manifest-test.lisp; the reassembly chokepoint's fail-closed, all-or-nothing contract is pinned in the same file's condensation suite; the end-to-end migration parity (a source instance records its module set, evaporates, and a successor reassembles it through the production path) is pinned in tests/migration-end-to-end-test.lisp.

Criterion Test File
Record codec round-trips; fixed-width envelope; order-invariant; pins a generation record-codec-round-trips, envelope-is-fixed-width, module-order-is-canonical, pinned-generation-is-independent tests/module-manifest-test.lisp
Record carries no key bytes; the manifest package imports no content symbols record-carries-no-secret-key-bytes, store-cluster-imports-no-content-symbols tests/module-manifest-test.lisp, tests/migration-end-to-end-test.lisp
Genesis writes both manifests under one root advanced once genesis-publishes-both-manifests-one-advance tests/module-manifest-test.lisp
Reassembly admits a vouched set; the loader fires exactly once condensation-admits-vouched-set tests/module-manifest-test.lisp
Unvouched / wrong-hash / revoked module fails closed; no partial set condensation-refuses-unvouched-set-no-load, condensation-refuses-wrong-hash, condensation-refuses-revoked, condensation-all-or-nothing-multi-record tests/module-manifest-test.lisp
Reassembled module set names exactly the source's set (functional parity) resident-module-set-survives-reassembly tests/migration-end-to-end-test.lisp
Non-resident module pulled, content-verified, vouched, admitted; unvouched control fails closed vouched-non-resident-module-condenses-unvouched-refused tests/migration-end-to-end-test.lisp

What is framed and what is open

Framed: that a plugin is a combined package-inferred module; that the edge controller iterates an installed *edge-adapters* registry rather than naming adapters; that module load is a capability decision over a content-addressed hash; that the active module set is migration-recorded alongside the namespace; and now the record shape itself: names, relative source paths, and a content score, sibling to the namespace manifest under one genesis root, carrying no stored vouch because trust is re-decided against live authority at reassembly. The modularize fit with the package-inferred discipline is settled and cold-verified, the idempotency trap mapped, and the reassembly proven end to end. Open, and the work the next arc takes up: the protocol adapters move to their own sibling repository, c3po (the home for every wire-format module a valis consumes) extracted on the discipline of real need rather than anticipation, with the existing HTTP and Gopher adapters relocated so the conversion proves the seam against their already-green tests; the production capture path that records a real non-resident module set at publish time (the reassembly and content gate are proven, the publish-side capture of a live non-resident set is the remaining wiring); and whether runtime unload is exercised in production or modules are load-once per image. These are named so the plugin layer is built against one uniform module abstraction (discoverable, lifecycled, content-addressed, capability-gated) rather than a bespoke registry reinvented per plugin kind.

Per-axis sovereignty: one principle, three boundaries

This chapter is the auditor's map of the data-classification model that holds across the three operator-state axes: mail, DNS, and the social graph. It states one principle and then walks the three boundaries that make it concrete, each as a threat (what leak it prevents), an enforcement point (where in the code it is guaranteed), and an audit proof (the test that keeps it honest).

The principle is reach-by-classification, never reach-by-content. Operator state that lives in PostgreSQL holds pointers and metadata (a content-hash pointer to a body, an envelope field, a DID, a lifecycle disposition) and never a message body or a header. An infrastructure or operations path reaches a datum by what class of thing it is, not by reading its content; the content of a message stays opaque on the content-addressed store and is reached only through an authorized, owner-facing view, never through an operations seam.

That invariant is enforced structurally, not by review discipline, because a future migration is exactly where a body column could otherwise slip in. The confused-deputy schema audit (audit-migration-schema, src/operator-state/audit.lisp) introspects the live committed information_schema after a migration's DDL has applied and before COMMIT: forbidden-schema-columns rejects any column across the operator-state tables that is not in the explicit +permitted-operator-state-columns+ allowlist or that matches a body/header shape. The migration runner (run-pending-migrations, src/operator-state/migrate.lisp) calls the audit in-transaction on every schema migration and rolls back fail-closed on any finding, so a migration that would introduce a body or header column cannot land: the runner cannot be bypassed, and asserting over the live schema rather than the dao-class definitions catches hand-written DDL too. The proof that the gate is live and non-vacuous is run-pending-migrations-aborts-a-body-bearing-migration and operator-state-schema-has-no-body-or-header-column (tests/operator-state-sovereignty-audit-test.lisp).

The content boundary: mail bodies never reach PostgreSQL

A message body never enters PostgreSQL. It stays an opaque content-addressed block on the store, and the mail-transport-state row names it only by a content-hash pointer, body_ptr: an address, never invertible to content (the opaque-block view of the mail spine is detailed at the content boundary).

Threat. A confused deputy (an operations or migration path that could name, decode, or copy a body byte through the queue or state seam) would turn a substrate-management capability into a content read. That is the leak this boundary forecloses: operations moves and meters the spine and reaches no body.

Enforcement point. mail-transport-state (src/operator-state/mail-state.lisp) carries only body_ptr (a 64-char content-hash pointer) plus envelope and lifecycle metadata (no body or header column), and its package imports no body-octet reader (read-block / decode-file). The schema audit above bans any body/header column across the operator-state tables on every migration, so the row shape cannot regress.

Audit proof. The import-firewall sweeps ops-queue-projection-imports-no-body-read-symbol and ops-queue-grant-resolves-no-message-body-name (tests/mail-sovereignty-audit-test.lisp) prove the operations view reaches no body both by construction and over a really-queued message; the in-runner gate a-body-bearing-mail-migration-aborts and mail-state-migration-unit-passes-the-in-runner-audit (tests/operator-state-sovereignty-audit-test.lisp) prove the schema audit accepts the real mail unit and rejects a body-bearing one.

Write-authority and public serving: the DNS zone axis

DNS zone data is published by design: the records are meant to be world-readable. The protected asset is therefore not secrecy but write-authority: who may mutate a zone. valis is the authoritative ndb data owner over PostgreSQL; the nameserver reads through a narrow read seam and composes answers, holding no write authority of its own.

Threat. Unauthorised zone mutation (a session or a confused deputy committing a zone change it does not own, or smuggling its claimed authority in through the submitted bytes) is the leak this boundary forecloses.

Enforcement point. The :names write gate stamps the owning principal from the session, never from the message bytes: node-create reads the session principal at submission (%current-submission-did, src/namespace/names.lisp) and node-close commits it as the zone's owner_did at the 9P clunk. owner_did is an allowlisted operator-state column carrying the zone's write-authority owner principal, and serving is a read-only projection over the committed data.

Audit proof. dns-schema-has-no-forbidden-column, ndb-facet-columns-pass-confused-deputy-audit, and existing-zone-backfilled-as-primary-on-ndb-anchor (tests/operator-state-zone-audit-test.lisp) prove the zone schema passes the confused-deputy audit and that the write-authority facet is carried as metadata, not content.

Capability-scoped access: the social graph axis

A read of the operator's social graph is gated by a dedicated, independently grantable and revocable /social capability, not an ambient operator-wide power and not piggybacked on another grant. The graph is private operator state; access to it must be designated, never assumed.

Threat. An ambient or confused read of the graph (a module or operator tool that reaches the adjacency without holding a designated /social authority) is the leak this boundary forecloses.

Enforcement point. The raw recursive-CTE traversal traverse-reachable (src/operator-state/social-edge.lisp) is unauthenticated by design and capability-clean: it imports nothing from the capability or identity clusters and is not caller-facing. The single gated entry is query-reachable (src/plugin/social-admission.lisp), which calls verify-social-vouch first (an exact /social designation [=+social-designation+=] plus the :read right, decided fail-closed through the shared capability verifier) and only then applies the traversal. A source-level one-caller assertion pins query-reachable as the sole caller of the raw primitive, so the gate cannot be flanked.

Audit proof. social-read-vouch-admits-query proves a valid /social :read vouch admits; the four fail-closed refusals non-social-designation-refused, mount-only-vouch-over-social-refused, revoked-social-vouch-refused, and absent-vouch-refused-fail-closed prove every bad-vouch case is refused; and query-reachable-is-the-only-caller-facing-entry-to-the-raw-traversal (tests/social-capability-scope-test.lisp) pins the gate non-bypassable.

Validation coverage: per-axis sovereignty

Each boundary is anchored to its enforcement point and the test that proves it. The common principle (pointers and metadata, never bodies or headers) is the schema audit the migration runner runs before every COMMIT.

Boundary Threat Enforcement point (file:fn) Audit proof
The principle: reach by classification, not content a future migration adds a body/header column src/operator-state/audit.lispaudit-migration-schema / forbidden-schema-columns, run in-transaction by src/operator-state/migrate.lisprun-pending-migrations run-pending-migrations-aborts-a-body-bearing-migration (tests/operator-state-sovereignty-audit-test.lisp)
Mail bodies → content boundary a confused deputy decodes or copies a body byte src/operator-state/mail-state.lisp (body_ptr only, no body-octet import); the schema audit bans body/header columns ops-queue-projection-imports-no-body-read-symbol (tests/mail-sovereignty-audit-test.lisp); a-body-bearing-mail-migration-aborts (tests/operator-state-sovereignty-audit-test.lisp)
DNS zones → write-authority + public serving unauthorised zone mutation src/namespace/names.lispnode-create / node-close stamp owner_did from %current-submission-did (the session principal, not the bytes) dns-schema-has-no-forbidden-column, ndb-facet-columns-pass-confused-deputy-audit (tests/operator-state-zone-audit-test.lisp)
Social graph → capability-scoped access ambient / confused read of the graph src/plugin/social-admission.lispquery-reachable gates the raw traverse-reachable behind verify-social-vouch (/social designation + :read) query-reachable-is-the-only-caller-facing-entry-to-the-raw-traversal and the four bad-vouch refusals (tests/social-capability-scope-test.lisp)

Sovereignty as an architectural invariant

Operations manages the substrate and lifecycle; it never touches the content and meaning of a user's data. The namespace fabric is where this line is drawn: operability is expressed as moving and mounting viewers, never as reading content. The capability discipline above is the enforcement: a module (or an operator's tool) holds only the authority designated into its namespace, never an ambient authority over all of a user's data. Where code could still violate the line (observability, backup, multi-tenant isolation) it must be confined the same way, by what is mounted, not left to operational good behaviour.

What this invariant does not claim

The paragraph above is a statement about operations, and a reader is owed its limits in the same place, because the gap between them is where a wrong decision gets made.

Ordering. valis is first a way for a person to hold their own data, against the capture of a network architecture that did not have to end up this way. Resistance to a well resourced adversary is a real secondary capability and it is not the headline. Where the two conflict, holding the data wins. Anyone weighing a feature that trades availability of a user's own data for resistance to seizure should know that trade has already been decided in that direction.

The ceiling, stated rather than implied. A machine that is seized and carried off yields its processor, memory and storage state to an adversary who can then work on it without time pressure. There is no defence at that scale and none is claimed. What is achievable is narrower: notice physical interference, and make the relevant data unreadable afterwards.

That last part is harder than it sounds, and the reason is structural. Overwriting does not reliably erase modern persistent storage, because wear levelling leaves the controller deciding which physical cells a write lands on. Destroying a key is the only fast and checkable way to make data unreadable. So the arrangement has to put a small secret in a position where destroying it is sufficient, with no plaintext having reached swap, hibernation, or any store that outlives the key. That is a property of how the system is laid out. It cannot be added later as a panic button.

One tension that does not dissolve. A node that stays online without a person attending it must be able to restart on its own, so its operating key has to be reachable by the machine with nobody present. Putting that key behind a person's presence defeats what the system is for. So a node that can restart unattended will surrender its operating key to whoever takes the machine. That is a consequence of the goal rather than a defect awaiting a fix, and it is why authority to reconstitute a node elsewhere is worth separating from the key the node runs on.

Sealed backup is built; splitting the secret is not. What exists today is passphrase-sealed off-host backup (src/backup.lisp). Splitting a secret across several holders, so that no single seized machine holds enough to reconstitute anything, is design intent and is not implemented: src/ contains no secret-sharing of any kind. Read any description of reconstitution elsewhere in this document with that distinction in hand.

The public surface: what the internet reaches, and who opens it

A sovereign node answers the public on a routable address, and the question this chapter settles is the one a reader most needs before placing new behaviour: which ports the internet can reach, and who decides. Two mechanisms are involved, not one, and neither is the namespace by itself. This is worth stating plainly because an earlier framing ("the designated-ports boundary is the namespace itself") was true only while the node lived on loopback and stops being true the moment it carries a public IP.

The steer fans every port; the firewall selects them

The privileged host agent's sk_lookup program is a catchall (fulcrum/src/steering.lisp): it stashes valis's one socket in a single-slot map and redirects every TCP and UDP port on the unit's IP to it, transport- and port-agnostic by construction. On a loopback-only island that was harmless: nothing off-host could reach the namespace, so the ports the node answered were exactly the ports anything bothered to connect to. Once the namespace carries a routable address, the same catchall would fan every port on the public IP into valis.

What narrows the public surface back to the ports the agent admits is a second mechanism: a default-deny firewall the agent installs inside the namespace (fulcrum/src/firewall.lisp). Its input chain accepts loopback, established and related return traffic, :53 on both transports, and then, each one only when the agent's own config carries it, the owner port and the public TLS edge port. Everything else drops. The steer says "all ports to valis"; the firewall says "only these enter." They are complementary and both required, and the firewall (not the namespace) is the public-surface boundary. The distinction is not pedantic: a namespace boundary is structural and cannot be misconfigured open, whereas a firewall is a maintained rule that must be installed correctly, before any traffic, and that can in principle fail open. On a public node the surface control is therefore a security-critical, order-sensitive invariant of the same rank as the write fence, documented here rather than treated as an afterthought.

The port allowlist is the host agent's, never a module's

Because the boundary is a policy and not a topology, who writes the policy is the load-bearing security question, and the invariant is that the allowlist is the privileged host agent's alone. A valis module declares the ports it intends to serve, and that declaration reaches the agent as a contract (fulcrum/src/control.lisp's recorded port set), but it gates the descriptor handoff only; it opens nothing. The firewall ruleset takes no valis-supplied input at all: it is a pure function over the agent's own configuration, fed to a privileged nft inside the namespace. Widening it takes a privileged act at the agent, and there are two of them rather than one. A change to the agent's source is the obvious one. The other is a change to the agent's config store: :open-owner-port and :edge-port (fulcrum/src/launcher.lisp, read through ubiquitous) each add an accept line with no source change at all. An operator auditing what a live node exposes has to read both, and the config is the one that is easy to miss.

This decoupling is what keeps a node safe when it hosts code it does not fully trust. valis is the default home for every process not delegated to a sister, including the active-module archetype's untrusted executors; if a module's declaration could open a public port, registering a hostile module would expose an arbitrary listener on a routable IP. It cannot, because reachability is granted by a privileged act at the agent, admitting a port in the agent's own ruleset, that a module can request but never perform. Registration wires dispatch; the agent grants reach; the two must never be collapsed. The pattern for adding a protocol (the constellation map) is written so they are not: a new public port is a host-agent change, deliberately.

The set the firewall admits and the set the steer carries are disjoint, and one word doing duty for both is how a reader gets this backwards. A module's declaration reaches the agent as its designated port, and designated-port is precisely the key the firewall does not read: grep fulcrum/src/firewall.lisp for the word and it is not there. What the ruleset reads is owner-port and edge-port, both drawn from the agent's own config. A declared port is therefore steered inward and still dropped at the public IP unless the agent's config independently admits that same number, and that gap is the whole safety property. Read "designated" into the firewall and you will believe it admits what the steer carries.

DNS arrives by two paths: UDP inherited, TCP steered

The carve-out is UDP only, and it is physical rather than a preference. sk_lookup chooses which socket receives an inbound packet but does not govern the reply: a UDP answer is a fresh datagram that must source from the address and port the client sent to, or the client rejects it. A TCP connection escapes that, because once the steer selects the listener the kernel tracks the four-tuple and sources replies correctly, whereas valis answering UDP from its generic steered socket would source from the wrong address. The argument covers UDP replies and nothing else. I have ruled that it must not be stretched to cover TCP: DNS over TCP is required in a nameserver, and opening a second special case beside the UDP one buys nothing, so the steer carrying :53 TCP is the correct path.

The kernel program matches that ruling because it never had a port or protocol test in it. fulcrum/bpf/sk-lookup-catchall.lisp is a lookup of slot zero in a single-entry sockmap, an sk-assign, and SK_PASS: one map, one program, no port table, no protocol constant, and the compiled object is byte-identical everywhere it ships. There is no build variant in which it declines a port, so "transport- and port-agnostic by construction" above is the whole truth about it.

The privileged agent still binds the unit-IP :53 sockets on both transports and hands the descriptors down, for the reason that covers every inherited descriptor: binding a privileged port is the agent's to do, since valis runs unprivileged in the namespace. What differs by transport is what becomes of those descriptors on a node whose catchall is live. The inherited UDP socket serves. The inherited TCP socket is bound, handed down, adopted, and then never receives a connection, because the steer takes TCP on that port at the unit's address away from it.

What the transport split costs: one service, two independent failure modes

Read this before placing anything else on the DNS path, because it is the price the ruled design charges and nothing in the running system reports it. DNS over TCP rides the steered socket: its accept backlog, its per-port budget, its lifetime. DNS over UDP rides the inherited one. The two transports of one service share a name and share nothing else. A steered socket that saturates, that the edge budget deregisters, or that is torn down takes DNS over TCP with it while UDP keeps answering, and no counter, log line or check anywhere signals the split.

A nameserver answering UDP but not TCP is broken while presenting as healthy. A client that receives a truncated answer retries over TCP and gets nothing, so truncation fallback has no fallback. Zone transfers to a secondary fail, so a secondary serves its last good copy until the zone expires and then goes lame. Both failures surface at a resolver or at a secondary's operator, never at the node.

⚠ A green DNS answer is not evidence about which socket served it, in either direction. The resident recovers the destination port per accepted client rather than from the listening socket's own bind, so a hijacked connection's child carries local port 53 and reaches the DNS handler exactly as an inherited one would. A dig +tcp returns a perfect authoritative answer under both hypotheses. What separates them is which listening socket's accept queue the connections enter, the one measurement a DNS answer cannot give. That is what proving-ground/driver/scenarios/steered-port-delivery.lisp is built to read: it aims a burst at the inherited listener's own endpoint and attributes the resulting queue depth to one listening socket or the other.

The UDP fall-through is REASONED and has not been measured directly. With a live catchall the program attempts an assign for UDP too, and inherited UDP :53 keeps working because the kernel does not complete a cross-protocol assign and falls through to its own socket lookup. That mechanism is inferred from the program's shape and from UDP continuing to answer on a steered node. No gate observes the kernel declining the assign, so read it as the reasoning it is rather than as a measurement.

The owner control plane: a keyed terminus, shipped closed

The owner manages the node (mints and revokes capabilities, edits zones, enrols keys, drives the mail spine) through the 9P namespace, as a client that mounts the owner-proven management view. That control plane is deliberately not on the public surface: run-daemon binds the fabric on the namespace's loopback, the firewall drops the fabric port at the public IP and drops the owner port too unless the agent's config deliberately opens it, and an attach is admitted only after a NoiseXX handshake proves the owner's key (fabric.lisp's node-auth responder, seven's Tauth=/=Tattach path, mercer's sealed transport). Reachability and authorization are separate axes: the fabric is never open, it is authenticated, and loopback-binding is what makes "reachable" require first arriving on that namespace's loopback. The firewall reserves the loopback path ("the owner's loopback fabric path") for exactly this.

Reaching that loopback from off the host is a second keyed listener, not a hole in the first. When the resident inherits an owner-port descriptor it adopts it as the routable owner terminus (--owner-fd, threaded into start-fabric), and that adoption asserts the auth seam is engaged before it binds: a routable terminus that admitted an attach without the Noise handshake would be an open owner surface, so it fails closed exactly as the non-loopback bind assert does. *fabric-auth-enabled-p* is what lets the fabric bind a non-loopback host at all, and it does so because the handshake guards the attach rather than in spite of it.

The port is shipped closed. The host agent's launcher carries :open-owner-port, defaulting to nil, so the firewall drops the owner port and no owner socket is bound until an operator deliberately opens it. Opening it is the highest-value application of the host-agent allowlist above, though not the first: the public TLS edge port already carries its own accept line under the same mechanism.

A same-host proof and a cross-host proof are separate propositions, and neither stands in for the other. Driving the fence, the session listing, and a publication from a client over a real sealed socket proves that the behaviour meets a client as opposed to an in-image caller. It proves nothing about reach through the namespace and firewall boundary, which is a different path. Read any "verified live" against which of the two it actually crossed. TODO.org carries where that stands.

⚠ One detail of that path is easy to get backwards. A session admitted over the wire registers under the transport key's DID, because a Noise handshake proves the transport key and derives the principal from it, not from the signing key custody holds for the owner. The owner check therefore accepts either DID. A test that constructs principals in the image never crosses a handshake and so cannot observe this at all; only a test driving a real client does.

What :80 is for: the manual a node carries about itself

The design is that every valis serves its own documentation on :80, in the clear, with that port enabled on every node. I rank it with the owner control plane rather than with the protocol adapters because it is the surface a person meets first: before a certificate exists, before a name resolves anywhere, and before anything else on the node is working. A node whose manual you cannot read is not one you own, so the manual travels with the node and is answered by the node.

Plain HTTP is the reason rather than a shortcut. Point a phone, a handheld, a decade-old browser, or a text-mode client on a borrowed machine at the address you were given, and you get the documentation, because nothing in that path asks for a modern cipher suite, a trusted root, or a credential the node must first go and obtain from somebody else. A surface that comes up only after the credential machinery has succeeded cannot be the surface that tells you how to make the credential machinery succeed. That is a sovereignty property before it is a convenience: the one door that is open on a node you have just stood up is the one that explains the rest of it.

The content is the published documentation site the tree already carries (site/, landing/), served as an ordinary publication. The descriptor for :80 is inherited from the privileged host agent for the same privilege reason :443 is, and it is the same port an ACME http-01 responder would answer a challenge on: that use is real and it is secondary, never the reason the port is open.

This is the design, and the node is behind it. The resident parses the inherited descriptor and hands it to no adapter, so a node that inherits :80 answers nothing on it. TODO.org carries the item that closes the gap and the shape it copies.

Direction: unifying the boundary in the kernel

The two-mechanism split (a catchall steer plus a separate firewall) is the current shape, not the settled end state, and this section records the intended direction without asserting it as built. The unification is to make the sk_lookup program itself the allowlist: a map of admitted ports, written by the agent under privilege, that the program consults, redirecting only admitted ports to valis and letting the rest fall through to closed. The word matters: the map holds what the agent admits, which is not what a module designates. The boundary then lives in one place, in the kernel, and the separate firewall becomes defence-in-depth rather than the primary control. Two constraints survive the unification and are worth stating so the direction is not mistaken for a cure-all: the allowlist map must remain the agent's, never derived from a module's declaration (the invariant above is independent of mechanism), and UDP keeps its socket-activation carve-out regardless, since the steer cannot source a UDP reply. That second constraint carries a consequence worth naming here rather than rediscovering later: with UDP inherited and TCP admitted through the map, a service offering both transports still has them arriving on separate sockets with separate fates, so unifying the boundary does not close the transport split. Any move here changes the privileged kernel path that decides what the internet reaches, so it is gated on a steering test harness before the program is touched.

Deployment: condense-from-genesis on a real host

Standing up a valis node on a fresh host and condensing an evaporated unit back into existence are the same operation. Deployment is the genesis case of the reassembly path above: there is no separate installer to maintain, and deploy, evacuate (condense at a new location), and restore (condense from a durable head) unify onto one mechanism. The durable thing is the namespace, so a deploy is a reassembly whose predecessor happens to be empty: the only genesis-specific step is minting the owner Ed25519 seed rather than acquiring it from a predecessor.

The target is a stock Debian host; the resident runtime is a bare save-lisp-and-die executable supervised by a thin Type=notify systemd unit, isolated with ip netns, receiving its privileged wire descriptors by inheritance or SCM_RIGHTS. No container runtime runs on the resident. The contract splits at the point where condense can first run: adopt below it (the run account, a writable durable-state directory the account owns, the staged serving binary, a reachable operator-state database: ordinary host bootstrap), and reuse above it (the netns, the descriptor handoff, the fail-closed serve loop, the generation fence: valis's own machinery, driven, never re-implemented by a YAML control plane that would drift from the live image).

The bring-up order and the fail-closed invariants

The resident's production entry is run-daemon (src/main.lisp), which shares run-foreground's fail-closed bring-up: the fabric comes up and the anonymous public view resolves before the edge binds. For the public :53 service, fulcrum (privileged) creates the netns and moves a dedicated public NIC wholesale into it (the launcher's mode :dedicated, with the netns default route via that interface's own gateway), binds :53 UDP and TCP in the netns, clears CLOEXEC, and execs the unprivileged valis (ip netns exec → setpriv → env → valis) with the inherited descriptors; run-daemon adopts them, constructs a pg-zone-source over the operator-state rows, and drives the sibling serve loop over the inherited fds, binding no privileged port itself. (Both :53 descriptors are inherited because binding a privileged port is the agent's to do, but on a node whose catchall is live only the UDP one goes on to serve: the steer takes :53 TCP away from the inherited listener, so DNS over TCP arrives by the same route as the steered push-boot handoff and carries the split set out at what the transport split costs.) The invariants a deploy must preserve are the substrate's own: unprivileged valis dies rather than privileged-bind; a binary without the serving codec fails closed at boot; only the exposed ports are reachable behind fulcrum's default-deny (the surface control, the boundary that admits them); and two instances sharing one database are held to a single writer by the generation fence (the fence and the sweep). The written contract is the host-deployment contract.

Public TLS: the credential a name is answered under

A certificate is obtained through a dns-01 challenge, and the authority that answers that challenge is the node itself. I made the :443 edge cert-gated rather than required because of that circularity: a node must already be serving :53 before it can ever be issued a certificate, so a certless genesis boot has to be a healthy state and not a failure. With no usable certificate in custody for the configured name, the public edge stays dark and the resident still comes up, and the boot output says which case held. A silently dark port is indistinguishable from a descriptor that never arrived, which is the confusion that costs an operator an evening.

The edge holds one credential per name it serves and resolves it per connection, so a request for one name is never answered under another name's certificate. A renewal swaps the live leaf on the same cell the connection path reads, and the listening socket is never touched.

Two constraints carry into any operation on this path. Both are irreversible.

  • One certificate per registrable domain. Never SAN-pack. Certificate Transparency records the set of names on a certificate permanently and publicly. Pack several of the owner's domains into one credential and you have published, for good, the fact that one party holds them all. Nothing retracts it afterwards.
  • The ACME account key shares the certificate store path. Rename or move that store and the next order registers a fresh account, spending production rate limits that do not come back.

The proving ground: the migration thesis under test

Because deployment is the genesis case of condense, the harness that exercises a deploy is the same harness that exercises evacuate and restore: the migration-thesis regression harness. It lives under proving-ground/ as committed, declarative tooling: a Nix devshell pins the verification fabric (the guests it boots stay stock Debian, and the core is built impurely from the working tree, never as a Nix derivation); Terraform-libvirt boots Debian guests on two distinct VLAN-tagged network locations so a successor and predecessor sit at genuinely different locations: the topology a single-VM harness cannot express, and the precondition for the fence-and-reconnect path; and cloud-init provisions each guest to the adopt-below seam, staging the serving binary rather than rebuilding it. An in-image Lisp assertion layer (valis-proving-ground, its own ASDF system, never entering the resident binary) drives three scenarios over one mechanism (deploy-from-genesis, evacuate-cross-location, restore-from-backup) through three real interfaces: SSH for unit/netns/fd state, off-host dig for authoritative :53, and a fabric-liveness probe on the owner-proven loopback path. Each assertion is exit-coded and fail-closed; a scenario reduces to one go/no-go verdict. The scenarios assert the effects of the shipped condense and fence path, they do not re-implement it. See proving-ground/README.org.

The restore scenario exercises a passphrase-escrowed backup of the irreplaceable durable state, and a deployed resident reports substrate-only health so a silent failure surfaces before it becomes an outage. Both surfaces (the backup/restore and passphrase-escrow flow, and the logging/liveness/certificate-expiry observability surface) are documented for the operator in Backup and observability.

System shape (Lisp)

valis is an umbrella of cooperating Common Lisp systems, not a monolith. The core valis system stays focused (listener, registry, protocol contract, substrate interfaces) and depends on separate repositories for the heavy, independently-useful pieces:

  • seven: the 9P library carrying the namespace fabric and the bus.
  • mercer: the factotum. Transport authentication, key custody, and the ACME certificate lifecycle.
  • runciter: the authoritative-DNS answer logic the :53 serving path binds to.
  • c3po: the wire engines, adopted here as content-addressed edge modules.
  • fulcrum: the privileged host agent. eBPF steering and the descriptor handoff.
  • Whistler: the eBPF compiler and pure-CL loader fulcrum drives.
  • libp2p/IPFS bindings: content addressing, and the one dependency the Common Lisp ecosystem does not yet answer. CONSTELLATION.org is the map of which repo owns what.

Within the core:

  • package-inferred-system: each file's defpackage declares its own dependencies; there is no central component list. Adding a file under src/ and importing it is how you extend the system.
  • A new protocol plugin is, minimally: a new src/<proto>.lisp that defines a protocol subclass (or instance), specialises handle-connection, and registers itself on its ports.
  • Cold-verify every change (fresh image, cleared fasls): a warm REPL hides missing :import-from declarations.

Status

This document says what valis is and what it is meant to be. It carries no status, deliberately. A design record that also tracks what has landed goes stale every time something ships, and when a reader catches one wrong paragraph they discount the rest of the document with it.

TODO.org is the status board: where valis stands, what has landed, what is built but not yet deployed, what remains, and the order it assembles in. It also separates what has run on the live node from what is merely on main, which is the distinction this section used to blur.

Bibliography

External resources this design draws on, grouped by the part of the architecture they ground.

eBPF steering

Listener and event multiplexing

  • iolib - the Common Lisp I/O library whose epoll multiplexer the listener is built on.
  • Efficient IO with iouring (Jens Axboe) - the eventual multiplexer backend behind valis's own interface.

Authoritative DNS

  • RFC 1035 - domain names: the message format and master-file presentation the substrate serves and the wire engine encodes.
  • RFC 2308 - negative caching: the apex SOA on a negative answer and the min(SOA.MINIMUM, SOA.TTL) negative-cache lifetime the seam contract requires.
  • RFC 4592 - the role of wildcards: the synthesis-and-existence rules the nameserver's lookup tree follows.
  • RFC 3597 - handling of unknown RR types: the generic rdata form the record boundary round-trips through.

Plan 9 and 9P

  • The Use of Name Spaces in Plan 9 - per-process namespaces and 9P, the model the substrate reifies.
  • 9P specification and papers - the protocol that is both substrate and bus.
  • v9fs (Linux 9P client) - lets the host kernel mount a valis namespace.
  • factotum(4) - Plan 9's auth agent (a 9P file system holding keys); the model for valis's auth agent.
  • c9 - a small 9P client and server; reference for a Common Lisp implementation.
  • Interim - a Lisp environment where "everything is a file is a symbol"; closest prior art.
  • awesome-plan9 - curated index of Plan 9 / 9P software.

Storage, consistency, and migration

Identity, capabilities, and cryptography

Content addressing

  • IPFS - content-addressed storage for bulk and published data.
  • libp2p - the networking stack a future Common Lisp CFFI binding would expose.
  • cl-ipfs-api2 - an existing CL HTTP-API client to an external IPFS daemon (the interim bridge).

🄯 Brian O'Reilly <fade@deepsky.com>, 2026