valis / Understanding the system
valis - Architecture
What valis is
valis is the userland process the founding doctrine calls "a Lisp Machine for network protocols." Linux's eBPF makes one IP address answer on every TCP/IP port and forwards the traffic to userland; valis is the process that receives it and multiplexes a single person's data and identity across the protocols they choose to run.
valis is not a static "one machine, one person" install. It is a host-agnostic, migratable service: a person's valis can move between valis-compatible hosts anywhere on the internet, because the thing that is durable is not the process but the person's namespace. The architecture reifies Plan 9 semantics (per-process namespaces, everything-as-a-file, network-transparent resources) with the 9P protocol as intrinsic substrate rather than one protocol plugin among many. Migration, and the ability to run a person's protocol modules across more than one machine, fall out of that namespace model.
The central abstraction: every port a protocol, every protocol a plugin
The protocol surface is organised around one idea:
A protocol is a plugin. It declares the ports it answers on and knows how to handle a connection by integrating, through common interfaces, with the user's own data and identity.
This is the seed in src/:
src/protocol.lisp: theprotocolclass and thehandle-connectiongeneric function every plugin specialises. The contract every protocol plugin implements. (A long-lived protocol may instead be driven from the event loop through session callbacks and define nohandle-connectionat all; see the execution model.)src/registry.lisp: the port → protocol registry and the dispatch entry point. A connection arrives carrying its original destination port; the registry routes it to the protocol that claimed that port.src/main.lisp: process lifecycle (run / daemon / dev).
The architecture is the moat: not any single protocol, but the substrate that makes every protocol a uniform, user-owned plugin.
One caveat rides with this framing, because it is the model a reader most easily over-reads: "it declares the ports it answers on" governs dispatch (which protocol handles a connection that arrives), not reachability, whether the internet may reach that port at all. On a deployed host those are two different decisions owned in two different places, and a port's public reachability is a privileged decision at the host agent, never a consequence of a plugin declaring it. The public surface settles which ports the internet reaches and who opens them.
The keystone: the 9P namespace fabric
The durable unit is the per-process namespace: a mount table of 9P resources naming the person's data, identity, names, and the mailboxes of their running protocol modules. A protocol module is an ephemeral viewer onto that namespace, not the owner of any irreplaceable state.
Organising the system this way buys three otherwise-hard properties from a single idea:
- Migration is cheap. A module holds no state that cannot be rebuilt by re-attaching to the namespace, so moving a person's valis to another host is closer to "restart there and re-mount" than to "serialise a live process."
- Sovereignty is enforceable. Content lives in the namespace; operations only ever move the viewer. The substrate and lifecycle are operable without the operator ever touching the content and meaning of the data.
- Distribution is uniform. A module on another machine is just another 9P-reachable resource in the namespace: local threads and remote Lisp nodes are addressed the same way.
Everything below hangs off this keystone.
Data flow: how a packet reaches a protocol module
outside world
│ packets to <user-ip>:<any-port>
▼
┌──────────────────────────────────────────────────────────┐
│ privileged host agent, external to valis │
│ │
│ binds the privileged ports attaches the eBPF │
│ (:53 udp+tcp, :443, :80, sk_lookup catchall: │
│ the owner terminus) every port to one socket │
└─────────┬──────────────────────────────┬─────────────────┘
│ inherited │ steered
│ (passed in at exec) ▼
│ ┌──────────────┐
│ │ Linux/eBPF │
│ └──────┬───────┘
▼ ▼
┌──────────────────────────────────────────────────────────┐
│ valis process (unprivileged) │
│ listener: one socket, many ports │
│ │ accept; recover original destination port │
│ ▼ │
│ registry: port ──▶ protocol module │
│ │ dispatched over 9P │
│ ▼ (local thread or remote node) │
│ the user's substrate │
│ (namespace: data · identity · names · crypto) │
└──────────────────────────────────────────────────────────┘
⚠ What one picture flattens: two arrival paths, at different maturities. The inherited path on the left is what carries public traffic today, and it is the older of the two. The steered path is live: on a node whose catchall is attached it takes every TCP port at the unit's address, DNS over TCP included, and the two DNS arrival paths sets out what that costs. What remains target is the shape where one node declares many ports and serves them all through the single steered socket, since a boot declares at most one port today. Both arrivals converge on the same listener, which is why they are drawn together; arrival paths treats the distinction properly.
The steering layer: sklookup (validated)
The eBPF mechanism that makes "one IP answers on every port" concrete is
BPF_PROG_TYPE_SK_LOOKUP (Linux 5.9+; kernel docs): a program that runs during the
kernel's listener lookup for every inbound connection, before port
demultiplexing. It calls bpf_sk_assign to bind the connection to one
chosen socket regardless of destination port. That socket is valis's
single listener, registered in a BPF_MAP_TYPE_SOCKMAP; the program is
attached to a network namespace via a BPF link.
This is the real steering primitive, and it had to be established by
investigation. An earlier proof-of-concept that hooked the
inet_sock_set_state tracepoint only observed connections: it could not
redirect them. sklookup is the program type that actually answers for
every port.
Written in Lisp: the substrate stays introspectable
The steering program is compiled by Whistler, a Common Lisp s-expression → eBPF → ELF compiler with a pure-CL loader (no libbpf, no clang, no bpftool). Keeping the kernel-facing layer in Lisp is the Lisp-Machine thesis applied to the substrate itself: the owner reads and reshapes the steering logic in the same language as everything above it.
sklookup support and a catch-all redirector were validated end to end on Linux 7.0: compiled, accepted by the kernel verifier, and attached to a network namespace through the pure-CL loader. Minimal shape:
(defmap redir-sockmap :type :sockmap :key-size 4 :value-size 8 :max-entries 1)
(defprog catch-all (:type :sk-lookup :section "sk_lookup" :license "GPL")
(when-let ((sk (map-lookup redir-sockmap 0)))
(sk-assign (ctx-ptr) sk 0)
(sk-release sk)) ; mandatory: see invariant below
SK_PASS)
Invariants worth remembering:
- A sockmap lookup in sklookup context returns a reference-counted
socket; the program must
sk-releaseit before exit or the verifier rejects it with "unreleased reference." - Two kernel constants that are easy to confuse: program type
SK_LOOKUP= 30, attach typeBPF_SK_LOOKUP= 36.
The Whistler additions are upstream (atgreen/whistler#43), so a stock Whistler
carries them.
The listener: one socket, many ports
A connection steered by bpf_sk_assign keeps its original four-tuple, so
the original destination port (the port the client dialed) is
recoverable in userland with getsockname on the accepted socket, with no
cooperation from the eBPF program. The recovery works identically whether
the socket arrived by sklookup or by an ordinary bind, and it is the
ordinary-bind case that keeps the userland path testable over loopback with
no kernel in the loop. On its own that arrangement would leave the sklookup
case argued rather than shown. It is shown:
proving-ground/driver/scenarios/steered-port-delivery.lisp puts an off-host
client on the far side of a real steer, and the resident dispatches each
connection on the port the client dialed, including one port that appears in
no configuration on either side.
The listener is an event-loop multiplexer, not a synchronous accept
loop. It is built on iolib's multiplexer over epoll; the event source is
kept behind valis's own interface so an io_uring backend can be slotted
in later without changing any protocol plugin. (iouring is deliberately
out of the first cut: it has no mature Common Lisp support today.) Each
accepted connection has its original destination port recovered and is
handed to the registry, which dispatches it to the owning protocol module.
The execution model: one loop, one seam, and how a protocol is driven
Everything in this section follows from a single constraint: the event-dispatch thread must never block. Stalling inside a readiness callback stalls every other source on the loop, so any protocol that can block must be run off it.
src/listener.lisp holds the one backend-selection point:
*default-multiplexer-factory* names the epoll multiplexer, and nothing else in
the tree chooses a backend. Swapping in a different one is a change to that
variable, not a change to any protocol.
src/executor.lisp is the indirection that keeps the loop free. The executor's
dispatch-connection generic schedules an accepted connection onto whatever
execution context the executor manages, and returns at once.
Three strategies exist in the tree, and which one a port gets is a property of the transport rather than of the protocol:
- Every bound edge port builds a budget executor. It is a thread per accepted connection with a per-port concurrent-connection cap: the cap-th accept deregisters the port's listening source, so further connections wait in the kernel backlog rather than being accepted and closed, and a slot freeing re-registers it. The default cap is 64 connections per port.
- The connectionless datagram path takes an inline executor, which spawns nothing and runs the exchange on the loop thread itself. A datagram arrives whole, so there is nothing to block on, and spawning per datagram would hand an off-path packet flood an unbounded thread-creation lever.
- A bare thread per connection, with no budget, is the third and is what the other two are variations on.
All three sit behind the one generic, so a pooled or coroutine executor is a further subclass and the swap touches neither the multiplexer nor any protocol handler.
A protocol may instead declare :raw-descriptor stream discipline, in which case
its connections are driven from the loop itself through the session callbacks
rather than handed to the executor. This is what makes long-lived protocols
affordable: a connected but idle session costs no thread at all, so concurrency
scales with the number of active clients and never with the number of connected
ones. Sizing for such a protocol is therefore against a rate of activity rather
than a count of peers. The connection-lifetime assumptions measure that
claim, at zero OS threads and a few kilobytes retained per idle session.
Two boundaries govern that declaration:
- An adapter that wraps or terminates the stream cannot honestly declare raw discipline, because the descriptor beneath it carries ciphertext while the wrapper holds the bytes the protocol wants. A terminating edge therefore keeps the thread shape. Driving a continuous protocol over a terminated port from the loop is unresolved: termination hands back a blocking stream bound to a thread, so such a session costs a thread even while idle. That question gates continuous protocols over TLS, and owner mail access with them.
A protocol that needs to make blocking calls while serving a session, such as a namespace round trip, must still get off the loop to make them. Connection dispatch cannot carry those workers. Its unit is a whole connection lifetime: it is keyed on a port an adopted session has already resolved, it reaches the protocol through the registry's
handle-connection, which a session-driven protocol no longer defines, and where a strategy owns that lifetime it closes the connection when the work returns. A per-session worker runs many times over one session, so the first one to return would tear the session down.Neither entry point covers work that is not a whole connection lifetime, such as a per-session worker or a timer body. A task-level dispatch that takes a thunk, owns no lifecycle and closes nothing is owed here; connection dispatch is expressible on top of it. Until it exists, each protocol spawns its own threads, and the choice between unbounded workers and a bounded pool is settled independently in each one. The two are not interchangeable: a pool smaller than the simultaneously-active set couples clients whose latency is independent today.
Arrival paths: how a wire descriptor reaches a node
Wire descriptors reach a running node two ways. The distinction is in how a descriptor arrives, never between kinds of node:
- Inherited. The privileged host agent binds a port outside the node and passes
the descriptor in. The reason is privilege rather than protocol: valis runs
unprivileged in its namespace, so a port it may not bind is bound for it and
handed down before privilege is dropped. Four things arrive this way
(
src/main.lisp)::443for the public TLS edge,:80for the documentation every node serves in the clear,:53on both transports for authoritative DNS, and the routable owner terminus on whatever port the agent's config names. Three of those are TCP streams, and the steer is perfectly capable of carrying them. Only a UDP reply must source from the socket the client addressed, which is why UDP is the one transport that has to be inherited to work at all; the two DNS arrival paths sets that out, along with what it costs that:53is split across both arrivals. Of the four,:80is the arrival the code has not grown into: the resident parses its descriptor as--edge-http-fdand hands it to no adapter, so a node inherits the port and answers nothing on it. What:80is for states the design; TODO.org carries where the code stands. - Declared and pushed. The node mints a listening descriptor and hands a duplicate to the privileged agent over the existing control connection, which is what lets the agent steer every port onto that one socket. The push direction is the property that makes the arrangement safe, and the steered-fd handoff sets out why. Note separately, at the per-namespace steering constraint, that a declaration does not by itself bound the answered set on a routable address: the agent's default-deny firewall is what narrows it.
These compose: a node inherits the descriptors the agent bound for it and declares
a port of its own, and a launch may use either arrival, both, or neither. A boot
declares at most one port today, since src/main.lisp refuses a --ports set
carrying more than one, so "declaring for everything else" is a shape the model
allows and the code has not yet grown into. Declaring stays default-closed, so a
launch that only inherits behaves as it always did.
They did not compose until recently. The boot shape that adopts inherited :53
descriptors never declared its socket to the agent, so on a node booted that way
nothing entered the steering map and no other port was reachable regardless of
what the firewall admitted. That was a defect in the code rather than a second
kind of node.
The invariant the fix must restore: authoritative DNS is a service in the :names
group and nothing more. It is not a mode the node runs in, and its presence must
never suppress the general arrival path for other protocols. What that protects is
a node routing an arbitrary number of protocols, the surfaces at the edge/core seam
and others as they are adopted. Naming none of the others here is deliberate: the
model must not need to know them. If adding a protocol requires a new branch in the boot path,
the model has been drawn wrongly and the exception has been allowed to suppress
the rule.
Identity, capability, and the namespace boundary
The connection-open moment is an authentication event. When bytes arrive on the IP, valis must transparently establish who is calling, because the answer decides which namespace is mounted for that connection. The same port serves different worlds by identity: an anonymous request to the web port sees the published-files view; the owner, proven by key on that same port, sees the management view or the operator interface entirely.
This is a full capability system (authority can be minted, attenuated, and delegated to external actors who hold no identity of their own), not a yes/no on a fixed identity. Every primitive is established prior art; the combination is what is new. It composes as layers:
- Transport identity. A Noise-style handshake authenticates the channel to a peer public key (a decentralized identifier), autonegotiated and invisible by default, yielding a principal, or anonymous for a keyless public request. (Noise is also libp2p's secure channel, so it arrives with the content-addressing work.)
- An auth agent, factotum-shaped. Following Plan 9's
factotum(man page), an auth agent that is itself a 9P file system, holding the owner's keys, negotiating all security interactions for other programs, and moderating which identity applies, valis keeps all key handling in one agent. Protocol modules never see keys. - Capability tokens. Authority is carried as delegable, attenuatable, public-key-rooted capabilities in the UCAN mould, where the owner is the authorization server directly: no central server, offline-verifiable (which matters under migration). Delegation chains double as provenance.
- Capabilities as names. Published data is named by a capability that embeds its own access right and key, in the manner of Tahoe-LAFS read/write/verify caps and capability URLs. Handing out the name is the grant; the recipient needs no identity of their own. This is how anonymous external actors are authorized.
- Enforcement by mounting. The resolved authority decides what is mounted into the connection's namespace. A module can only name what is mounted; sovereignty is the absence of a name, set once at namespace construction rather than checked on every access.
- No ambient authority. A module acts only on capabilities designated to it through its namespace, never on a global identity. This is the discipline that prevents the confused deputy: a semi-trusted, possibly remote module being induced to wield the owner's authority on an attacker's behalf.
Authority resolves in two phases. Key-based transport auth fixes the principal at connect time; but a capability presented inside the protocol (an unguessable URL path, say) is known only after the request is parsed. So a connect-time principal (often anonymous) establishes a base namespace, and a per-request capability may mount further within it.
The NoiseXX handshake
The connect-time authenticator for a 9P-native peer is a Noise Protocol Framework handshake in the NoiseXX25519ChaChaPolySHA256 pattern. XX is mutually authenticating without pre-shared knowledge: neither party needs the other's static public key in advance, and both recover the other's key during the exchange. That symmetry is what lets a fresh peer dial a fresh valis and walk away with a did:key for each other: no directory, no out-of-band setup.
The exchange is three messages and binds every byte into the same transcript. The initiator opens with an ephemeral X25519 public key in the clear. The responder answers with its own ephemeral and its static public key, the static encrypted under a key derived from the running transcript; the message ends with an authenticator (an AEAD of the empty string under the same key) that proves the responder also derived the same transcript. The initiator's third message carries its static the same way, again followed by an authenticator. Each derivation step folds the latest DH output into the chaining key, and every emitted byte is folded into the transcript hash, so a single bit-flip anywhere in the conversation breaks the next decryption. After the third message a Split derives two asymmetric transport CipherStates (one for each direction) that a post-handshake transport layer would use; valis's current use ends at admission and does not exercise transport-mode encryption.
Mutual authentication here means a key recovery, not a key challenge: the responder reads the initiator's static out of the third message and publishes it to the rest of the system as a principal. There is no separate "prove you hold this key" step, because the AEAD authenticator on the message a party encrypted with the keys derived from its own static is the proof; impersonating a key requires possessing it.
The cipher behind every encrypted token is RFC 8439 AEADCHACHA20POLY1305 with the IETF 96-bit nonce shape (four zero bytes followed by the counter as little-endian uint64) and the running transcript hash as additional data. The AEAD wrapper has its own RFC 8439 §2.8.2 known-answer test, so the cryptographic primitive is anchored before the protocol that uses it.
The state machine and seven-compatible auth node are in
src/identity/noise.lisp; the AEAD wrapper is in src/identity/aead.lisp.
See the Noise Protocol Framework entry in the Bibliography.
The factotum auth-conversation
The factotum is mounted at /id and is itself a 9P file tree, faithful in
shape to Plan 9's factotum(4) (src/namespace/id.lisp). It exposes three
children:
/id/ctl- key management: a Tread renders the owner's key descriptors, a Twrite carries control verbs. The interface deliberately mirrors factotum(4) so an operator (or a local module) drives it the way they would drive Plan 9's./id/rpc- the auth-conversation endpoint. Every open of/id/rpcmints a fresh conversation object holding its own auth node, so two concurrent callers cannot share state through it. The rpc surface is a generic auth-protocol multiplexer, not Noise-specific: its first message may carry a proto-selection line, and the endpoint really dispatches on it, building the Noise responder fornoiseor the challenge driver forssh-reality, withnoisethe default when a caller begins its handshake without selecting. It is the factotum's mechanism for driving whichever proof the caller wants, and the seam a further authenticator plugs into by becoming one more dispatch arm./id/proto- a list of authenticator names available behind/id/rpc, one per line. It now advertises bothnoiseandssh-reality. An architect or an auditor reads it to see which proofs this image admits without opening source.
The same machinery serves a remote 9P peer through seven's auth seam. A
Tauth against the root binds the afid to a fresh Noise auth node by way
of the node-auth method on valis-root (src/namespace/root.lisp).
Subsequent Twrite and Tread operations on that afid pump the three XX
messages through seven's "dumb pipe" auth carrier: seven never inspects
the bytes; it carries them between the peer and the node, and parks reads
until a write completes so the challenge-response ordering is correct.
When the third message lands and the node-auth predicate
(node-auth-ok-p) reports completion, seven calls
node-attach-identity on valis-root with the approved auth node;
the specialiser returns the keyed principal the Noise responder
resolved from the peer's static public key, and seven records that
return verbatim on the live 9P session before admitting the Tattach.
Two paths (a remote afid and a local open of /id/rpc) exercise the
same Noise auth node class. The symmetry is the whole point: a local
module driving the factotum and a remote peer attaching through it
authenticate against the same code.
See the factotum(4) entry in the Bibliography.
Principal resolution and the authenticator seam
Once a proof completes, it has to become something the rest of valis can
read. That conversion is a single named operation: the authenticate
generic function in src/identity/authenticator.lisp. Everything above
the seam (the connection layer, the capability layer, the namespace
mounter) only ever sees a principal; only the code below the seam
knows what proof produced it.
NoiseXX is the first method on this seam (noise-authenticator);
anonymous-authenticator is the second. Adding a future proof (an
HTTP-edge signature authenticator, for example) means writing a new
class that specialises authenticate to mint the same principal type
from a different proof. No code above the seam changes. The Noise
implementation does not need to be read or understood to add a new
authenticator; the seam is the entire contract.
The principal value type itself lives in src/identity/principal.lisp
and carries a did:key identifier and the raw public key bytes that
identifier encodes. For a Noise admission, the responder recovers the
peer's static public key from the third message, encodes it as a
did:key (multicodec 0xec01 for X25519, base58btc per the did:key
method spec, in src/identity/did-key.lisp), and constructs the
principal. At Tattach the principal lands on the live 9P session
through seven's node-attach-identity seam: valis-root's specialiser
returns it; seven stores it verbatim on the session's identity slot.
Backend code reads it back through current-identity (seven) or the
thin wrapper current-principal (src/namespace/root.lisp) for the
dynamic extent of request dispatch. A standard file node's node-read
runs synchronously on the session thread, so the principal is in scope
during node-open / node-read / node-write; a backend that parks
reads off-thread (the bus pattern) captures it at the synchronous call
site and closes over it for delivery.
For a keyless attach (an Tattach that asks no key, e.g. a NOFID
Tattach), the seam binds a distinct anonymous sentinel, never nil.
The distinction is load-bearing: nil is the unresolved state before
admission, the sentinel is the resolved state for "no key presented".
Module code can therefore test anonymous-p without conflating "not yet
authenticated" with "authenticated as nobody in particular". The
sentinel carries the literal DID string "anonymous" so it is safe to
log and dispatch on; the boundary that no key material reaches a
module is enforced on its principal-public-key slot, which is
nil. The anonymous principal is autonegotiated by the absence of a
handshake, never produced by an error path and never a login prompt.
See the W3C Decentralized Identifiers and Noise Protocol Framework entries in the Bibliography.
Key custody
The custody store in src/identity/custody.lisp is the sole holder of
private key material in the running image. The owner's master key is an
Ed25519 signing pair; the X25519 transport key Noise uses is derived from
it deterministically, so one keyfile roots both the owner's signatures
and the owner's channels. The master key is held in memory and persisted
to a mode-0600 keyfile so the owner's did:key is stable across restarts:
an in-memory-only key would mint a fresh owner DID on every boot and
break every capability rooted in it.
Real at-rest encryption (passphrase / OS keyring / HSM) belongs to a
later phase when real steering and the IETF edge land; the dev keyfile
is sufficient for the loopback substrate.
A protocol module receives only a resolved principal: a did:key
identifier and a public key. It never calls into the custody store, never
holds a private key object, and never has a name through which it could
ask for one. This boundary is the key-custody enforcement point: the
absence of any path from a module to a private key. The custody store is
wired into the factotum's node-auth path by start-fabric
(src/fabric.lisp), and stop-fabric clears it; an architect who needs
to convince themselves the boundary is intact can read those two
lifecycle hooks and confirm that nothing else is ever handed a custody
reference.
The seam this creates around the rest of the system is the answer to the "confused deputy" risk that the capability discipline above is designed to head off: a module asked to act on a peer's behalf cannot be tricked into reaching for the owner's key, because it has no name for it.
Validation coverage
Each phase success criterion is anchored to a named test in
tests/identity-test.lisp that proves it. An auditor reading this
document can run the suite and see the claim row stand or fail without
reconstructing the coverage from source.
| Criterion | Test | File |
|---|---|---|
| NoiseXX interop: both parties derive identical transport keys | noise-xx-two-party-interop |
tests/identity-test.lisp |
| AEAD: RFC 8439 §2.8.2 test vector encrypt/decrypt | aead-encrypt-rfc-vector, aead-decrypt-rfc-vector, aead-tampered-tag-fails |
tests/identity-test.lisp |
| did:key: round-trip identity for a 32-byte X25519 key | did-key-round-trip, did-key-z6ls-prefix |
tests/identity-test.lisp |
| Noise init: h and ck are the raw protocol name bytes | noise-init-h-ck-known-answer |
tests/identity-test.lisp |
| Keyed attach: principal recorded on the 9P session, DID matches peer key, readable via current-identity | noise-handshake-end-to-end-over-seven, node-attach-identity-keyed-returns-noise-principal |
tests/identity-test.lisp |
| Keyless attach: anonymous sentinel recorded on the session, readable via current-identity, no error | anonymous-attach-nofid, node-attach-identity-nofid-returns-anonymous |
tests/identity-test.lisp |
Factotum reachability: /id tree walkable, /id/proto lists noise |
factotum-9p-reachability |
tests/identity-test.lisp |
| Key stability: owner DID unchanged after keyfile reload | key-stability-across-restart |
tests/identity-test.lisp |
| Seam decoupling: principal package imports no crypto symbols | principal-seam-decoupling-structural |
tests/identity-test.lisp |
One validation limitation is recorded here rather than papered over. The AEAD
primitive carries its RFC 8439 §2.8.2 known-answer vector, but the NoiseXX
handshake above it is anchored only by an initialisation known-answer
(noise-init-h-ck-known-answer) and a two-party self-interop
(noise-xx-two-party-interop) in which both parties are valis: that proves the
two sides agree with each other, not that they conform to the wider Noise
ecosystem. The handshake state machine is a from-scratch implementation of a
standard protocol, so before the authenticator is trusted in production it should
be checked against independent published NoiseXX vectors (Cacophony / noise-c)
and given a short cryptographic review; a bug shared by both sides would pass
self-interop silently. This weighs more heavily as the same state machine becomes
the basis other consumers in the constellation interoperate with.
Wrapping the post-attach session
The NoiseXX handshake authenticates who is calling, but the 9P session it
admits still ran in the clear: every Tread=/=Twrite=/=Tclunk after Tattach
crossed the wire unencrypted, authenticated only by the handshake that preceded
it. That leaves a transport man-in-the-middle free to read or rewrite a peer's
file traffic once admission is past. Closing that gap means every post-attach
message (request and reply) is encrypted and authenticated under the Noise
transport keys, and a tampered byte is refused.
The split that makes this auditable is the design's defining choice: all
transport crypto belongs to mercer; valis is pure plumbing. mercer, the
validated auth agent, owns the transport session: the keys, the AEAD, the
two per-direction monotone counters, the nonce construction (the sequence is
folded into the nonce and never appears on the wire), the orientation baked in
by role, and fail-closed exhaustion. valis never touches any of it. valis owns
only the keyed accept loop (it binds the socket, accepts each connection, wraps
the connected stream in a secure endpoint, and hands that endpoint to the serve
thread) and a thin delegating endpoint that holds nothing but an opaque
session handle. Its read-frame calls the session's unseal; its write-frame
calls the session's seal; it performs no cryptography of its own. The whole
question "is the transport crypto correct?" is therefore answered once, inside
mercer; valis's audit reduces to "does the plumbing route every post-attach
frame through the session, and never leak the handle?"
The session contract (seal, unseal, close-session, and a
transport-session-error condition family) is deliberately provider-neutral.
It is the general secure-transport seam, not a Noise interface. NoiseXX is its
first provider; a TLS provider for the IETF HTTP edges and ACME-issued certs is
the intended second, and the plumbing consumes the abstract contract so that
edge reuses it unchanged. Only one operation, obtaining the session from a
completed handshake, is Noise-specific, and it lives at the attach boundary
where valis installs the handle.
On the wire, each post-attach record is a cleartext 4-byte length prefix
outside the sealed body, followed by the sealed ciphertext-and-tag. The length
sits outside the seal so the reader can delimit a record before it has any key
to unseal it; the AEAD tag is computed over the exact ciphertext, so any tamper
at the record boundary fails authentication (which is why an empty additional-
data field is safe here). The 9P engine keeps its own size[4] message framing
above this layer, untouched: the wrap is a transparent record cipher beneath
the protocol, not a change to it.
The pre/post-attach flip is valis-owned plumbing. The handshake, Tauth, and
Tattach must cross in the clear: there is no session yet to seal them. valis
installs the handle at the attach boundary (node-attach-identity in
src/namespace/root.lisp), so the very first sealed record is the Rattach
reply; from there every frame is delegated. Tamper is terminal: when unseal
signals transport-auth-failed, valis tears the session down. There is no
in-band rekey and no renegotiation: an anomaly ends the session rather than
recovering it.
This is the same architecture Plan 9's 9front uses for post-auth styx
encryption: factotum holds the keys and yields an opaque secret, devtls
applies a record cipher (seal/open) under it, and exportfs speaks plain 9P
over that cipher without knowing it is there. The mapping is one-to-one onto
mercer (the keys and cipher), valis's plumbing (the record framing and the
endpoint substitution), and seven (the cipher-agnostic 9P engine). The
9front-validated rules carry over verbatim: a cleartext record header outside
the sealed body, a monotone per-direction sequence folded into the nonce and
never on the wire, no in-band rekey, and interposition done by substituting
the endpoint in the serve path.
The thin delegating endpoint is in src/transport/secure-endpoint.lisp; the
keyed accept loop is in src/transport/keyed-acceptor.lisp; the attach-boundary
install is in src/namespace/root.lisp. The transport session itself (every
byte of the crypto) lives in mercer at mercer/src/transport.
Validation coverage: transport wrap
Crypto correctness is mercer's, not valis's: the transport session's ciphertext
is anchored byte-for-byte against mercer's own round-trip and the published
Cacophony / noise-c vectors under mercer 0.3.0. valis's tests prove the
plumbing: that the production keyed path routes every post-attach frame
through seal/unseal, flips the pre/post boundary at the Rattach, refuses and
tears down on tamper, frames records with the cleartext length prefix, closes
and never exposes the opaque handle, round-trips orientation both ways, and
holds the session cap and drain. Each row is anchored to a named test in
tests/transport-wrap-test.lisp, and each test declares in its own docstring
whether it observes the real production effect (DIRECT) or an equivalent
property the production path guarantees where the live serve thread offers no
external observation seam (PROXY), so an auditor sees exactly which dimension is
proven how.
| Criterion | Test | File |
|---|---|---|
| Confidentiality on the keyed path: a post-attach frame on the wire is ciphertext, via a real keyed attach (DIRECT) | confidentiality-on-keyed-path |
tests/transport-wrap-test.lisp |
| Tamper-reject and teardown: a flipped post-attach byte fails authentication and tears the session down, no plaintext (DIRECT arm + PROXY arm) | tamper-rejected-and-torn-down |
tests/transport-wrap-test.lisp |
| Pre/post-attach boundary: handshake/Tauth/Tattach cleartext, the first post-attach frame delegated (DIRECT) | pre-post-attach-boundary |
tests/transport-wrap-test.lisp |
| Wire framing: the cleartext length prefix delimits the record outside the seal; a boundary tamper fails the tag (DIRECT) | wire-framing-length-prefix |
tests/transport-wrap-test.lisp |
| Handle lifetime: the opaque session is closed on teardown and exposed by no public accessor (PROXY) | handle-lifetime |
tests/transport-wrap-test.lisp |
| Orientation cross-check: what one side seals the other unseals, both directions, pinned to mercer's round-trip anchor (DIRECT) | orientation-cross-check |
tests/transport-wrap-test.lisp |
| Accept-loop parity: the session cap and drain behave as the prior listener's, adding no DoS surface (DIRECT) | accept-loop-parity |
tests/transport-wrap-test.lisp |
Validation coverage: over-the-wire conformance
The transport wrap above proves valis seals every post-attach frame, but it proves it against valis's own initiator talking to valis's own responder, both sides our code. A bug shared symmetrically by both would pass that test silently. The remaining question is conformance: does the handshake valis admits, and the transport cipher it then runs, match an externally-defined standard rather than merely agreeing with itself? That question is answered by anchoring valis's real accept loop to a published NoiseXX transcript (bytes authored by the spec ecosystem, not by us) and confirming the wire reproduces them. Because the handshake valis sends carries no payload, the only published transcript its wire can byte-match is the empty-payload NoiseXX25519ChaChaPolySHA256 vector, dual- referenced (noise-c and an independent reference) and vendored under mercer. The replay fixes all four handshake keys to the vector's values so the wire is determined by the transcript, not by agreement between two pieces of our code.
Each criterion below rides the genuine production path (start-fabric →
keyed-acceptor → node-attach-identity, the same loop a real peer reaches), so
the proof comes off the real socket, never a direct endpoint call. The named
tests live in tests/over-the-wire-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Handshake conformance: on the real keyed wire, the handshake messages and the responder's final handshake hash match the published empty-payload NoiseXX vector byte-for-byte (DIRECT) | published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
First sealed record: the first post-attach record off the wire unseals over the client session to a valid Rattach, proving the real seal path carries a sealed reply end-to-end (DIRECT) |
published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
| Transport-cipher conformance: the session derived from the real-wire handshake, sealing the vector's transport payload at the first responder send, reproduces the vector's transport ciphertext byte-for-byte, so the transport-key derivation conforms, not just the handshake (PROXY, tied to the wire by the shared session) | published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
Principal-scoped sealed read: over the sealed session a Twalk=+=Tread of /id/owner returns the authenticated owner DID and the principal has not collapsed to anonymous: the initiator reaches its keyed view, not a shared one (DIRECT) |
sealed-read-resolves-owner-did |
tests/over-the-wire-test.lisp |
One verification here is out-of-band rather than automated: an end-to-end wrapped attach against a real, non-loopback peer. The in-process and loopback dimensions all have automated in-suite coverage above; only driving a wrapped attach across a real network interface (which needs a remote peer and mount privileges the test image does not hold) is a manual test the operator runs, confirming the post-attach frames are ciphertext on the wire. The runbook below makes that check repeatable.
Runbook: a real-NIC sealed attach (manual, operator-run)
This is the one out-of-band confirmation the architecture asks for: a sealed 9P attach driven across a real network interface, from a peer holding mount privileges, observed to be ciphertext on the wire and to reach the caller's own principal-scoped view. The automated suite proves confidentiality and conformance over an ephemeral loopback socket; loopback cannot stand in for a real NIC, and a remote peer with mount privileges is not driveable from the Lisp image, so the operator runs this once, by hand, to close it. Running it successfully discharges the previously-open out-of-band wrapped-attach check.
The procedure reuses the test image's existing VALIS_TEST_HOST opt-in: the
real-NIC drive stays skipped by default and engages only when the operator points
that variable at the machine's actual non-loopback address, so the manual run
never fires accidentally in an ordinary suite pass.
Steps:
- Bring up a responder on the real interface. On the responder host, start
valis in the foreground (
valis run). It binds the 9P fabric on an ephemeral port and printsvalis: 9P fabric on port N; noteNand the host's non-loopback address. Leave it running. It tears down cleanly onSIGINT. - Opt the initiator in to the real NIC. On the peer that will attach, set
VALIS_TEST_HOSTto the responder's non-loopback address (not127.0.0.1/::1; a loopback value leaves the drive skipped by design). This is the single switch that turns the otherwise-skipped real-NIC drive on. - Drive a sealed attach from the remote peer. Using the reusable sealing
initiator under
tests/support/(the same harness the automated conformance tests drive), complete the keyed handshake (Tauth→ the NoiseXX exchange over the afid →Tattach) againstVALIS_TEST_HOST:=N=, then promote the connection to the sealed client session and issue aTwalk=+=Treadof/id/owner. - Observe the two outcomes that matter. First, capture the post-
Tattachtraffic on the wire (e.g. with a packet capture on the interface) and confirm it is ciphertext: no cleartext 9P frame, no readableRattach, no readable file payload. Second, confirm the sealed/id/ownerread returns the attaching peer's own owner DID, the initiator reaching its authenticated principal- scoped view across the real network, not a shared or anonymous one.
A clean run (ciphertext on the wire plus the correct principal-scoped read off a real interface) is the out-of-band confirmation the architecture names, and completing it once discharges the open real-NIC sealed-attach item.
Capability tokens: delegation and confinement
A capability token is the delegable form of authority. It carries the
issuer's DID (the source of the authority), the audience's DID (the
grantee), a list of mount grants (each a namespace designation with a
rights set), an expiry, a nonce, and an Ed25519 signature by the issuer
over a deterministic encoding of those fields. A delegated token carries
its ancestry inline: the proof chain back to the owner's root token rides
inside the token itself, so a verifier needs no external store to walk it.
That self-containment is what makes verification offline: a property
that matters under migration, when the owner's substrate may be mid-move
and no authorization server is reachable. The value type lives in
src/capability/token.lisp; the chain walk in
src/capability/verifier.lisp.
Verification walks the chain from leaf to root. The root token must be issued by the owner and signed by the owner's Ed25519 key; every token in the chain must be unexpired and absent from the revocation store. Each delegation link is then held to four conditions: the child's rights must be a subset of the parent's, the child's designation must be a path extension of the parent's, the parent must itself carry the delegate right, and the child must not outlive its parent. Its expiry may equal the parent's but never exceed it. The third condition deliberately diverges from stock UCAN: re-delegation is a structural property of the chain, not a policy judgment made at verification time, so a grantee who was not given the delegate right cannot mint valid sub-grants at all. The fourth (expiry attenuation) is what keeps a delegate that legitimately holds the delegate right from wielding it to escape its own lifetime: because equality is the loosest a child's expiry may be, a delegate cannot mint itself a longer-lived token and outlast the grant it descends from. Extending a lifetime is therefore not a delegation act at all; it takes a fresh grant minted at the root, which only the owner can produce. Together with the issuer-to-audience continuity check at each hop, this closes the confused-deputy route through token forwarding: a chain that escalates rights, escapes its designation prefix, lengthens its own life, or passes through a non-delegable hop fails verification with a stated reason.
Both verification entry points are fail-closed: every condition raised during decoding or checking is caught inside the verifier and returned to the caller as a refusal with a reason string, never as an escaped condition. The 9P layer that fronts verification can therefore treat the verifier as total.
Capability names: bearer grants and the self-certifying name
Where a token grants authority to a principal, a capability name grants
authority to its holder. A name encodes a designation, a rights
bitmask, an expiry, and a nonce, followed by the owner's Ed25519
signature over those bytes, the whole rendered as base58btc behind the
valis: URI prefix: URL-safe, so the same string can later ride in an
HTTP path. The codec is src/capability/name.lisp; verification is the
same src/capability/verifier.lisp described above.
The name is self-certifying in the Tahoe-LAFS sense: verifying it
requires only the owner's public key and the revocation store: no
directory, no central service, no identity for the presenter. Handing out
the string is the grant. This is how anonymous external actors are
authorized without ever acquiring an identity: possession designates.
The two costs of bearer semantics are answered structurally. Replay
within the validity window is bounded by mandatory expiry: a name
without a positive expiry cannot be minted. And the revocation store is
the kill switch: every verification consults it, for names exactly as for
tokens, so a leaked name dies the moment the owner records its hash,
before expiry, not at it. The store itself
(src/capability/revocation.lisp) is an append-only set of canonical
hashes persisted beside the owner's keyfile; revocation is irreversible
by construction. Tahoe's further trick (embedding an encryption key in
the name so the storage layer never sees plaintext) is deliberately
deferred until the durable store exists for it to protect.
Capabilities as mount grants
Every verified presentation (token or name) resolves to the same
internal authority value: the mount directive
(src/capability/directive.lisp), a designation in the owner's canonical
namespace plus a closed rights set (read, write, mount, delegate). A
mount directive is an assembly instruction: it tells the namespace
assembler which 9P-served subtree to bind into the requester's session
view, with which rights. Recipients never interpret the designation
string; they find the subtree present (or absent) in their assembled
view. Authorization is thereby enforced by what is mounted, not checked
per access: out-of-scope objects are not denied, they are unnameable.
The per-session view is the view-root
(src/namespace/view-root.lisp), an attach root whose children come from
a per-instance mount table. The binding happens at the attach moment:
a bearer name presented as the 9P attach name is verified and the session
receives a view containing exactly the designated subtree; a name that
fails verification admits the session to an empty view: fail closed,
never the shared tree. A plain attach continues to bind the development
tree; the policy for what base namespace a keyed versus anonymous session
receives there, and the per-request second phase that mounts further
capabilities into a live view mid-session, is the namespace-assembly
layer's work and lands with it.
Coverage over time is intrinsic to live mounts. A mount is a live 9P binding (walks reach the serving node at access time), so new children appearing under a granted subtree are covered the moment they exist, with no reissue traffic. The converse is also deliberate: new rights verbs are never implicitly covered, because the rights set is closed and a bitmask from an older grant simply has no bit for a verb that did not exist when it was minted.
The capability service itself is 9P-addressable at /cap
(src/namespace/cap.lisp), mirroring the factotum discipline: a ctl
file mints tokens and names and records revocations (signing always
delegated to the custody store: key material never enters the
capability layer), and a verify file turns a presented token or name
into a verification verdict. Modules interact with authority through
file operations, never through direct calls.
Owner-client trust: two privilege tiers and proof-of-reality enrolment
An owner does not always stand at the host. The durable way to operate a deployed node is through an owner-facing client (ubik) that attaches over the sealed 9P terminus from somewhere else entirely, and that never witnessed the node's genesis. The problem this section settles is how such a client comes to hold authority without the owner seed ever moving to it. The answer is two privilege tiers, each anchored to a proof matched to what it can do, and both expressed as ordinary revocable, key-bound capability delegations, never a copy of the root secret.
| Tier | Authority | Anchor to obtain it | Carrier |
|---|---|---|---|
| Operator-equivalent | unrestricted; can destroy the instance | proof of reality (real host control) | a maximal, revocable OCAP grant bound to the client's key |
| Scoped client | read /mail, edit one content axis, … |
ordinary OCAP delegation over the terminus | an attenuated, revocable OCAP token |
The anchor is matched to consequence. Operator-equivalence can destroy everything, so it demands the strongest gate a genesis-less client can be held to: a demonstration of real host control, strictly stronger than mere network reach. Scoped grants are bounded and revocable, so they ride the ordinary capability path with no host-access requirement; forcing proof-of-reality on every client action would defeat the point of a delegable capability system. Crucially, neither tier moves the owner seed. Even operator-equivalent access is a delegated, key-bound, revocable capability, which is exactly what lets a client be operator-equivalent without becoming an unrevocable clone of the owner key.
Tier 2 is already the capability layer above at work: an owner (or an
operator-equivalent client bearing the delegate right) mints an attenuated
token via /cap ctl (audience the client's own DID, a specific mount
grant, a short expiry), and the client presents it over the terminus, where
the assembler binds its audience to the Noise-authenticated session
principal and verifies the chain to the owner root. No host access, no new
mechanism; the trust model simply uses it.
The operator-equivalent grant shape
Operator-equivalence is expressed as a single maximal grant, not a union
of control-surface designations. It is one owner-custody-signed capability
token whose audience is the client's self-generated OCAP DID, whose
designation is the namespace root / (which prefix-covers every subtree,
so the grant is maximal by reach rather than by enumerating axes that
would have to be maintained as new ones appear) and whose rights are the
full closed set :read :write :mount :delegate. The issuer is the owner
custody root, never the client: this is a root-minted grant, never a
client→client maximal self-mint. It carries :delegate deliberately, so an
operator-equivalent client can itself mint Tier-2 grants for other clients
without a fresh proof-of-reality each time: the proof is a one-time
bootstrap of the tier, not a per-action gate.
A grant that can destroy the instance is contained not by scoping it down but by two structural limits on its lifetime and its progeny:
- Expiry attenuation. A child token can never outlive its parent (the
fourth chain condition above). So even though the grant carries
:delegateand thechild ⊆ parentrule permits an equal-authority re-issue, an operator-equivalent client cannot self-extend: any token it mints for itself expires no later than the Tier-1 grant it descends from. Renewing the tier is not a delegation act: it requires a fresh root grant through the proof-of-reality gate, which only the owner can drive. - Cascade revocation. Every descendant chains through the Tier-1 grant to the owner root, and the verifier consults the revocation store for every token in a chain. So revoking the single Tier-1 grant hash kills it and every token minted beneath it at once: one hash retires the whole subtree of authority the enrolment created.
Its lifetime is moderate and revocation-primary: expiry is the backstop, revoking the hash is the real containment lever. (Scoped Tier-2 grants invert this: short-lived and expiry-primary, because re-issue from an operator-equivalent client or the owner is cheap.) The blast radius is thus bounded by revocability and TTL, never by pretending a scoped grant is operator-equivalent when it is not.
The proof-of-reality gate: the ssh-reality authenticator
Minting the Tier-1 grant is authorized by a demonstration of real host
control, delivered as a first-class /id/proto authenticator named
ssh-reality, driven behind /id/rpc exactly as noise is. It is the
native place for a proof-of-reality: the driver lives in valis's /id
factotum, and on a satisfied proof it mints the grant through the shared
/cap mint path: the privileged host agent grows no auth-mint policy of
its own.
The gate's defence in depth is structural, and it requires both of two independent anchors. Either alone is refused, and the driver weighs each on its own so a diagnostic can always name which one failed rather than returning a single undiscriminating boolean:
- Host presence. The enrolment material must carry the un-forgeable
provenance stamp that only fulcrum's root-only,
AF_LOCAL, never-network-reachable host-local intake can apply. A terminus-facing path can fabricate an arbitrary stand-in object, but not that stamp, so this is a structural binding, not a checked field: a leaked operator SSH key presented over the network can never satisfy it. - Pinned-key reality. The operator must sign valis's issued challenge with the one pinned Ed25519 operator key, and that SSHSIG must verify (by mercer) against the pinned key and a fixed provisioning namespace.
valis mints a single-use, time-bounded challenge nonce for each attempt;
the operator signs it out of band with ssh-keygen -Y sign; mercer verifies
the SSHSIG blob; and only when both anchors hold does the driver return a
satisfied result carrying the client's bootstrap public key, from which the
audience DID of the minted grant is derived. Because host presence and a
valid operator signature are both required, a stolen SSH key alone mints
nothing remotely, and a host-access holder without the pinned key mints
nothing either: there is no second network-reachable path to the owner
surface. The client's bootstrap keypair is one-time: its only job is proving
live possession during the challenge, and it is spent once the grant is
minted to the client's durable OCAP DID.
A fully off-host owner (one who cannot demonstrate host control at all)
is out of scope for this authenticator; that case is served later by a
separate pairing-code /id/proto authenticator, and the SSH path is not
weakened to accommodate it. ssh-reality is one proof plugged into the
seam, not the only one it will ever carry.
Where the trust material rests
The outboard trust material for this path (the pinned Ed25519 operator key, the fixed provisioning namespace every proof is checked against, and the enrolled-client records [each an audience DID paired with its minted grant's revocation hash, kept for cascade-revocation lookup]) lives in its own cache directory beside the owner root seed, under the instance state root, and is never merged into the owner keyfile. Own directory, own lifecycle: this is outboard client trust, not the root secret, co-located with the root authority only for locality. The pin is host-local config the operator sets out of band and that is never mutated over the network. Every lifetime the gate honours (the nonce window, the Tier-1 and Tier-2 token TTLs) is a named, config-tunable parameter, not a magic number buried in a check; the defaults are deliberately generous for development and the initial go-live, to be tightened once the live client interaction is observed.
Threat model and dispositions
The dispositions this design commits to, recorded so an auditor can hold the code to them:
| Threat | Disposition |
|---|---|
| Pure-network attacker at the terminus mints operator access | mitigated: Tier 1 requires host control, not reachability |
| Captured token replayed by a third party | mitigated: audience is bound to the Noise-authenticated session principal |
| Enrolment message replayed to enrol a rogue key later | mitigated: a single-use, time-bounded challenge nonce bound to the specific bootstrap key |
| Malicious client build enrols an attacker's key | mitigated: the operator drives the delivery and confirms the key fingerprint |
| Operator-equivalent grant becomes an unrevocable owner clone | mitigated: it is a revocable, key-bound OCAP token, never the seed |
| A second network-reachable auth path to the owner surface | mitigated: the enrolment intake is host-local, uid/root-gated, off the public surface |
| Leaked operator SSH key alone mints operator-equivalence | mitigated: the host-presence anchor is structural; a key over the network cannot stamp host provenance |
| Operator-equivalent client self-extends its own lifetime | mitigated: expiry attenuation caps a child at its parent; renewal needs a fresh root grant |
| Host compromise | accepted: already total compromise (the seed is on the box); anchoring Tier 1 on host access adds nothing to lose |
| Client dials a spoofed node | mitigated (client-side) by the server-DID pin; the server-side lever is exposure-gating |
Two residuals are named rather than papered over. The trust of a genesis-less client bottoms out at one owner-controlled vouch: here, host access; the design only makes that "once" require public material, reuse an existing anchor, and yield a revocable delegate. And explicit revocation is best-effort, so short token expiry is the real containment lever: bound the blast radius by lifetime, and re-issue.
Owner key rotation
The owner's identity method is did:key, which has no update or
deactivate operation: the DID is the key. Rotation is therefore
destructive and total: every grant rooted in the old key dies with it.
The procedure, in order:
- Revoke all outstanding capability hashes by writing them to the revocation store, then stop the fabric. Revoking first means a copy of the old keyfile recovered later still cannot resurrect old grants within their expiry windows.
- Delete the dev keyfile at
~/.valis/dev-keyfile. The keyfile is the only persistence of the master key; removing it is the rotation. - Start the fabric. A fresh Ed25519 master key is generated, the owner DID changes, and the derived X25519 transport key and every factotum-served descriptor follow it automatically.
- Re-mint every capability name the owner wishes to keep granting. Old names became unverifiable at step 3 (their signatures no longer match the owner key the verifier is handed), so bearer names held by external parties are invalid immediately, not merely at expiry.
- Reissue delegation tokens to every delegate. Old chains terminate at the old owner DID and fail the root-issuer check.
There is no migration utility in v1: the procedure assumes the loopback development phase, where no real authority is outstanding. When real external grants exist, rotation tooling must walk the revocation and reissue steps mechanically rather than trusting the operator's memory.
Two planes of authority, and why the system is a federation of instances
⇒ The record of how this position was reached, with the constraints that produced it and what it means for work built on valis, is The epistemology of fully connected symbolic computers. This section remains the statement of record; that one explains it.
⇒ The classical statement of this division is in docs/REPL-AXIS.org under The hierarchy, in the classical terms, and it is the one to read first: the Lisp runtime is this machine's operating system kernel, and the capability model is that machine's programming ABI. What follows here is that argument's consequence for how instances relate, which that note poses without resolving.
Authority in this system divides into two planes, and only one of them is described by the rights above.
- The data plane. Reading and editing projections of the operator's own
data. This is what
:read,:write,:mountand:delegatedescribe, and it is where substantially all ordinary operation happens. - The execution context. Root access to the running Lisp machine itself, in the sense that term carries for any machine: loading code, installing a service, changing what the node is.
⛔ A question of the form "which right is evaluation" is malformed, and the
answer is not a new bit. :write cannot carry evaluation, because :write
means editing a projection of the data. A new right alongside the others is the
same category error under another name: it would enter a vocabulary whose
central promise is that a child grant is a subset of its parent, while meaning
every right at once plus the power to mint more. A holder performing correct
attenuation arithmetic on such a right would get a wrong answer, and nothing in
the mechanism would object.
Evaluation is not attenuated. The context is.
Once a party can evaluate in a context, they hold everything in that context by definition. Evaluation reaches every symbol, every package, this capability layer, and any key in memory, so a restricted evaluator resembles a boundary without being one.
⇒ Evaluation is therefore the wrong thing to bound, and the context is the right one. The confinement this substrate applies is outward: it governs what a node may reach, never what the party holding that node may do inside it.
An instance that executes but holds no data has an execution context that is total over itself and confers exactly its outward reach. That reach is capability-mediated 9P to the instances that do hold the data, and those mounts attenuate exactly as the rest of this chapter describes: each carries a grant hash, revocation actively evicts it, an expiry applies, and a read-only grant is wrapped so the write operation has no name.
⇒ The attenuation happens at the 9P boundary, never at the evaluation boundary. A party holds total authority over a context that holds nothing, and reaches the operator's data only through grants the operator can withdraw. Holding that context was never the same as holding the data.
The consequence: cooperating instances, not one image
This is why the system is a collection of valis instances cooperating through the capability system rather than one image serving many parties. A second party is given a second instance, not a confined region of the first. A collaborator, a vendor, or a hosted service can therefore execute without any of them holding the execution context of an instance that holds data.
⚠ An in-image confinement is not an alternative that has been left unexplored. A restricted evaluator, a sandbox, a package boundary, or an audited language subset each resemble a boundary and none of them is one, for the reason above. A better answer to the irreducibility of evaluation would change this design and would be welcome; the shape here is what the constraint currently admits.
The operational shape
Three layers, carrying very different traffic:
- The owner-facing viewer to an instance, over capability-mediated 9P. This is the overwhelming majority of all operation.
- The execution context of each instance, reached individually by real host control, for update and service installation. It is a maintenance channel rather than a user-facing capability, which is why anchoring it to host access is proportionate: whoever holds the machine already holds everything the channel would grant.
- Administration and service installation across instances, intended to be mediated by consensus over material signed by other instances holding operator authority, including instances a vendor maintains for the benefit of its clients. What crosses that boundary is an attestation that an artifact is legitimate; what never crosses it is access to the context that installs the artifact. This layer is direction rather than built shape.
⚠ The seam for the third layer already exists. Module admission runs through a
content hash plus an owner vouch carrying the :admit right. Federating
administration changes the signer of that vouch from the owner alone to a
quorum, and needs no new mechanism. :admit belongs to the execution context
rather than to the data rights it is currently listed beside.
Validation coverage: capability layer
As with the identity layer above, each capability success criterion is
anchored to a named test an auditor can run. All tests live in
tests/capability-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Delegation chain verifies offline against the owner's key | token-chain-valid |
tests/capability-test.lisp |
| Attenuation: child rights must not exceed the parent's | token-chain-child-exceeds-parent-rights |
tests/capability-test.lisp |
| Re-delegation requires the delegate right | token-chain-no-delegate-right |
tests/capability-test.lisp |
| Expired tokens are rejected | token-chain-expired |
tests/capability-test.lisp |
| Revoked tokens are rejected within their expiry window | token-chain-revoked |
tests/capability-test.lisp |
| Name round-trips designation, rights, expiry, nonce, signature | cap-name-encode-decode-round-trip |
tests/capability-test.lisp |
| Valid name yields its mount directive; bad signature or expiry refuse | cap-name-verify-valid, cap-name-verify-bad-sig, cap-name-verify-expired |
tests/capability-test.lisp |
| Revoked names are rejected within their expiry window | cap-name-verify-revoked |
tests/capability-test.lisp |
| The name is the grant: attach with it, walk the designated subtree, out-of-scope axes unnameable | cap-name-attach-and-walk |
tests/capability-test.lisp |
| The capability service is reachable over live 9P | cap-9p-ctl-reachable |
tests/capability-test.lisp |
| Pure capability packages import zero crypto symbols | directive-imports-no-ironclad, token-imports-no-ironclad, name-imports-no-ironclad |
tests/capability-test.lisp |
Two-phase namespace resolution
A session's namespace resolves in two phases. At connection time, the principal
established during the Noise handshake (or the anonymous sentinel for keyless
connections) determines the base namespace the session receives. This is the
connect-time assembly in src/namespace/assembler.lisp, the function
build-base-view. A connection whose principal matches the owner's key receives
the full canonical namespace: all axes present, all of /proto, /bus, /id,
/cap, and /ctl named and reachable. The anonymous sentinel receives the
read-only /pub published view and nothing else, the same surface an
unauthenticated HTTP GET resolves to (see The anonymous published view across
both edges, below). A keyed peer who is not the owner receives an empty
namespace: the connection is admitted, but there are no names to reach for.
Sovereignty is the absence of a name.
After the base namespace is established, a second phase runs when a capability is
surfaced alongside a request. Writing a capability to the session's /ctl node (or
calling the in-image assembler API directly) mounts the designated subtree into the
live view. The capability is verified through the same path as bearer-name attaches:
the designation is resolved, rights are applied, and an audience check is made for
audience-bound tokens. A read-only grant produces a deep read-only proxy (see below);
a write-capable grant mounts the raw node. The mount persists until the session
detaches or the capability is explicitly unmounted.
Base-namespace policy
The base policy has exactly three cases, decided once at Tattach time:
- Anonymous connection (NOFID or keyless): the read-only
/pubpublished view, and only that./pubis published (public by design), so an anonymous mount names it and traverses the published items beneath it, but the floor is wrapped in a read-only projection: every write, create, and publish is refused. The credential-gated axes (/proto,/bus,/id,/cap,/edge) have no name to reach for. If the publication substrate is not up the floor is simply empty: fail-closed, never an error. Capabilities mounted in the second phase still widen the view from here. - Keyed connection whose principal is not the owner: empty namespace. Knowing who a
peer is does not grant them anything; authority arrives only as capabilities, not as
identity alone. A non-owner keyed principal does not receive the
/pubfloor: only the anonymous sentinel does. - Connection whose principal's key matches the owner's key: full canonical namespace. The owner is the root of all authority: for the owner, every axis has a name by definition.
The policy is expressed as a function in src/namespace/assembler.lisp. The identity
check compares the connecting principal's did:key identifier against the owner's DID
from the key custody store (src/identity/custody.lisp). If the custody store has not
been initialised (for example, in tests that exercise individual nodes without
starting the full fabric), the owner-DID check is skipped and the connection receives
an empty namespace: a deliberately fail-closed default.
Bearer-name attaches bypass the base policy: a capability presented as the 9P attach name is verified by the capability service and mounts exactly the designated subtree, regardless of the connecting principal's identity. The name is the grant.
The anonymous published view across both edges
The /pub published view is the one thing an unauthenticated caller can see, and it
looks the same whichever edge they arrive through. An unauthenticated HTTP GET resolves
to the read-only /pub view via the anonymous edge grant the seam holds; an
unauthenticated 9P mount (access=any, no credential) resolves to the same read-only
/pub floor through build-base-view's anonymous branch. Both are projected read-only,
so neither path can write or publish. The credential-gated axes (/proto, /bus,
/id, /cap, /edge) stay absent for the anonymous principal on either edge: an
out-of-scope name fails the walk rather than returning a refusal, so a caller cannot
even confirm the name exists. The owner's full frame is built by
assemble-canonical-frame; the anonymous floor and the owner frame are the two ends of
the same build-base-view policy, with the non-owner keyed principal resolving to
absence between them.
Union directories and bind semantics
A session namespace is an ordered list of mount entries. When two grants cover the same leaf name, the entries form a union: walking the name tries entries in list order and returns the first successful resolution. Directory reads concatenate entries from all union members without deduplication: both entries appear in a listing even when their names collide, because the collision is resolved at walk time, not at list time. This follows the Plan 9 and 9Front kernel's union-directory semantics.
Three bind flags govern how a new mount is added to an existing name:
- Replace: the new entry becomes the sole entry for that name; any prior members are removed.
- Before: the new entry is prepended to the member list; it is searched first on walk.
- After (default): the new entry is appended to the member list; it is searched last on walk. The existing members keep precedence on name collision.
The bind flag is a choice made by the caller at mount time: it belongs to the presentation, never to the grant. A grant carries a designation and rights only; the flag is supplied separately in the ctl grammar or the assembler API.
When a create request targets a union name, the first union member that has write rights receives the create. This is a simplified reading of the Plan 9 MCREATE rule, adequate for the current namespace structure where grant rights are known at assembly time.
The union semantics are implemented in src/namespace/union-node.lisp and exercised
by the assembler; the seven server never interprets union structure: it invokes the
backend's node-walk and node-entries methods and the union node responds
appropriately.
Read-only projection enforcement
When a grant carries only read rights, the assembler wraps the designated subtree in a
read-only projection node at mount time (src/namespace/projection.lisp). The
projection is a deep proxy: walking through it returns further projections, so every
node in the subtree seen through the projection is also read-only. Write, create,
remove, and wstat operations on a projected node fail at open time: the write path
has no name, not merely an error response.
The projection node passes through the inner node's QID unchanged. A QID identifies a resource object, not a view of it, so two sessions projecting the same underlying node present the same QID on the wire. This is correct 9P2000 behaviour: the projection is transparent at the protocol level.
Enforcement is by construction: the projection is what was mounted; there is no policy lookup on the hot path. The decision is made once when the grant is assembled, and thereafter no per-access check is needed or present.
Revocation eviction and live sessions
When a capability is revoked, the revocation takes effect immediately in all live
sessions that have mounted it. The revocation path (src/capability/revocation.lisp)
persists the revoked hash, then calls the assembler's eviction sweep. The sweep walks
the live-session registry and removes every mount entry whose stored grant hash
matches the revoked hash.
The registry is a weak-reference table keyed by the live namespace root, constructed
via make-hash-table with :weakness :key (an SBCL extension). What it maps that
root to is a live-view-entry, and the entry carries considerably more than the
session's principal: the session id an operator has to be able to type, the auth
conversation the session was admitted through, the Unix seconds at which it was
admitted and at which it was last used, a fenced marker that is nil for a live session
and the second of the fence otherwise, and a per-view lock. The id exists because the
registry is keyed by object identity, which is unspeakable outside the image, so an
operator looking at a session they want stopped needs a name for it. The same entry is
also held on the view itself, so the request path stamps its activity mark from a slot
read rather than a lookup on a weak table.
The lock ordering is a discipline rather than an aside. A sweep snapshots the registry
under *live-view-registry-lock* and releases that lock before it takes any per-view
lock, which maintains one acquisition order throughout: registry lock, then per-view
lock. Two paths that take the same two locks in opposite orders deadlock the fabric,
and a deadlocked fabric answers nothing at all. Within that order a sweep holds the
session's own lock across the removal, so a session's namespace is consistent before
and after; no session sees a partially-evicted state.
Mount entries created by the base-namespace policy carry no grant hash and are never affected by the revocation sweep. Revocation targets only capability-backed mounts established through the second phase. That immunity is correct for what the sweeps do and is depended on elsewhere, and it is also why withdrawing an owner's live authority needs a mechanism of its own: see owner session lifetime below.
Weak references are what make an ended session safe to forget. A session whose view root is no longer reachable by anyone is collected without an explicit deregistration hook, and that is a settled arrangement rather than an outstanding one. The gap worth naming is a different thing and it concerns sessions that are still live: a handle already open below the root of a session that has been fenced goes on working. The next section states that limit and what follows from it.
Owner session lifetime: fencing, idleness, and authority transfer
The owner was the one principal whose live authority could not be withdrawn. Authority that arrives by presenting a capability carries a grant hash, so both eviction sweeps can find it and take it back. Authority that arrives by proving who you are is base-namespace policy, carries no grant hash, and both sweeps pass over it by design. The owner's frame is base policy from end to end, so it was precisely the case the sweeps could not reach: short of restarting the fabric, an owner session admitted once went on naming the full frame for as long as its transport lived.
Fencing a session empties its view outright, base policy included. That asymmetry with the two eviction sweeps is deliberate: a mechanism that respected the usual immunity could not touch the very session an operator is trying to stop. What is left is a session that can name nothing, which is how this namespace refuses in general. An out-of-scope object has no name to reach for rather than a name that answers no.
A fence does not tear down the sealed transport, and it is not meant to. The connection stays up and the session on it can reach nothing, which is the same enforcement model the rest of this document describes: absence rather than denial.
A session is stopped three ways.
- By name, at the identity door. An operator reads the live sessions, picks the one they want, and writes a fence command naming it. Fencing the session issuing the command is allowed and does exactly what it says, because an operator who has lost control of a session should not first have to work out whether they are about to cut themselves off, and an attacker sharing that session should not keep it by virtue of being the one still holding it.
- By idleness, checked lazily. The bound is enforced by the operation that would have used the session, not by a sweeper. A dormant session issues no requests and reaches nothing; the case worth catching is someone picking up an abandoned one, and that arrives as an operation.
- By an authority transfer. When write authority moves to another locus, every live owner session on the surrendering instance is fenced. This is a separate step from the capability sweep that runs beside it, and it decides by principal rather than by generation, because an owner's namespace is base policy and the sweep cannot reach it. Without this step the instance that gave up authority would go on serving its owner after an evacuation.
The idle bound is unset by default, and the default is a decision. Applying a bound to a deployment that never asked for one is a lockout nobody requested arriving with an upgrade. Fifteen minutes is the recommended value, short enough that an unattended browser tab stops being a live management console over a lunch break and long enough that ordinary editing is never interrupted, but it is advice a launcher or an operator adopts explicitly rather than a value that applies on its own.
Getting back in means authenticating again, and that is where the seam earns its place. A re-attach on the same transport is refused by conversation, so a client cannot simply ask for a fresh frame and hand itself back what was taken; re-running the handshake yields a conversation the fence has never seen, and that succeeds. This connects directly to the evaporate-and-recondense architecture described later in this document: an owner whose instance has relocated has to be able to ask whether the instance now answering is the one they meant, and a session that silently outlives a relocation is what makes that question unanswerable. Cutting the session is what forces it to be asked.
The limit: an open handle survives the fence
A fence does not revoke a handle already open below the session root. The read path serves an open object without re-consulting the root, so emptying a view stops walks and listings from the root while a client already holding a handle keeps using it. The same limit applies to idle expiry: a client working purely through handles it already holds never reaches the check that would fence it.
Two things follow, and both matter more than the limit itself.
The first is a rule for anyone checking a fence: walk from the session root. A probe that reuses a handle it opened before the fence will observe that handle still working and report a functioning fence as broken. The behaviour under test is reachability by name, so the test has to exercise naming.
The second is where the remedy belongs. The 9P library needs a generic per-session handle revoke, and it needs to know nothing about owners, fences, or capabilities: valis decides which session and why, and the library carries out the revocation and raises a distinct error to the client. The placement test is one line, and it settles questions like this one throughout the constellation: could another consumer of this library use the mechanism with no valis present? Here the answer is yes, so the mechanism belongs there.
That mechanism now exists upstream, as a forcible revocation of a session's handles with its own error class and a stable client-facing message string. valis does not yet call it, so the limit above is live in valis today and is described as it behaves, not as it will behave once the two are wired together.
The door an operator touches
The surface is a pair of files on the identity axis, following the split the capability axis already uses: one file performs and its sibling reports. It sits under identity rather than capability because a session is identity and transport, not a grant.
The listing is read-only and rendered afresh on every read. It advertises no length, because any length it advertised would be a number that was true a moment ago. Beside it is a control file that performs, and the result of a command is held per open, so one caller's result never surfaces on another caller's read.
It is deliberately not an evaluation surface. Command lines are split on whitespace with no reader involved, an unknown verb is ignored rather than diagnosed, and a malformed argument to a known verb produces an error line readable on the handle instead of a signal into the 9P layer. ⚠ The evaluation axis proposed in docs/REPL-AXIS.org gives that property up by its nature, which is exactly why it must be its own axis and must never be reachable from a grant that reaches a control file.
Enforcement validation coverage
Each success criterion for the enforcement-by-mounting layer is anchored to a named test that proves it. An auditor reading this document can run the suite and see each claim stand or fail without reconstructing the coverage from source.
| Criterion | Description | Test | File |
|---|---|---|---|
| Absence not refusal | Out-of-scope resource has no name on the wire; walk raises an error as if the name never existed, not a permission-denied | sc1-absence-of-name-over-socket |
tests/namespace-test.lisp |
| No per-access guard | Walk and read paths contain no verifier calls (structural check of source) | sc2-no-per-access-guard-structural |
tests/namespace-test.lisp |
| Mid-session widening | Writing a capability to /ctl widens the live view; the name is absent before and present after |
sc3-ctl-mount-widens-live-view |
tests/namespace-test.lisp |
| Base policy: anonymous published floor | Anonymous mount names read-only /pub, not the gated tree; empty and fail-closed when /pub is down |
anonymous-mount-sees-read-only-pub-floor, base-policy-anonymous-empty-view |
tests/substrate-test.lisp, tests/capability-test.lisp |
| Base policy: owner canonical | Owner-keyed attach yields the full canonical namespace | base-policy-owner-yields-canonical-frame |
tests/capability-test.lisp |
| Revocation eviction | Revoking a hash removes matching live mount entries; base-policy entries with no hash survive | revocation-evicts-live-mount |
tests/namespace-test.lisp |
| Self-narrowing unmount | A session can shed grants; the /ctl node is irremovable by design |
self-narrowing-unmount |
tests/namespace-test.lisp |
| Projection denial | A read-only grant's write path signals at open time | projection-open-write-mode-denied, projection-write-denied |
tests/namespace-test.lisp |
| Union walk order | First member wins on name collision; directory reads concatenate all members | union-node-walk-order, union-node-walk-first-wins, union-node-entries-concatenated |
tests/namespace-test.lisp |
| Session has a nameable identity | A live session carries an id an operator can read off the listing and type back | session-carries-a-nameable-identity |
tests/fenceable-sessions-test.lisp |
| Fence reaches base policy | Fencing empties the session's view including base-policy mounts, which no eviction sweep touches | fence-removes-base-policy-mounts |
tests/fenceable-sessions-test.lisp |
| Fence is not undone by re-attach | A second attach on the same auth conversation does not restore the frame; re-running the handshake does | fenced-session-is-not-restored-by-attaching-again, authenticating-again-restores-the-frame |
tests/fenceable-sessions-test.lisp |
| Lazy idle expiry | The operation that wakes an idle session fences it; with no bound configured, sessions are left alone | idle-session-is-fenced-by-the-operation-that-wakes-it, an-unconfigured-idle-bound-leaves-sessions-alone |
tests/fenceable-sessions-test.lisp |
| Authority transfer fences owners | Surrendering write authority fences live owner sessions and leaves non-owner sessions untouched | authority-transfer-fences-owner-sessions, authority-transfer-leaves-non-owner-sessions-alone |
tests/fenceable-sessions-test.lisp |
| Fencing leaves the sweeps alone | The capability eviction sweep still spares base-policy entries after the fence path exists | the-capability-sweep-still-spares-base-policy |
tests/fenceable-sessions-test.lisp |
| The door reports and performs | The listing names the live sessions, a fenced session reads as fenced, and a malformed command is refused rather than signalled | the-door-lists-the-live-sessions, a-fenced-session-reads-as-fenced-in-the-listing, the-door-refuses-a-malformed-command |
tests/fenceable-sessions-test.lisp |
| Fence over a real socket | Over a sealed client connection the door names a live session, the fence empties the session read from its root, and it takes only the session it names | the-door-names-a-live-session-over-the-wire, a-fence-empties-a-live-session-read-from-its-root, a-fence-takes-the-session-it-names-and-leaves-the-other |
tests/session-fence-over-the-wire-test.lisp |
| Publication placed on a running node | An owner-keyed client places a publication on a node already running, including a body spanning several messages | publish-places-a-publication-on-a-running-node, publish-carries-a-body-larger-than-one-message |
tests/owner-publish-test.lisp |
Publication substrate and wire-format adapters
The publishing category of the substrate begins here: one publication object, owned by the person, with the wire protocols as projections over it. HTTP and Gopher are the first two projections, chosen deliberately as a pair: two protocols with nothing in common but the object they render prove the object is protocol-agnostic in fact, not by assertion.
Publication object model
A publication is a directory node in the 9P namespace whose children are
named attribute files (title, author, published-date,
content-type, and an optional gemtext) plus a body file. The
object is shaped as a Plan 9-faithful directory-of-files: each attribute
is individually walkable, individually grantable, and independently
readable by an adapter that needs only some fields. The five core
attributes are the web-minimal metadata set; gemtext is a per-protocol
representation used by the Gemini adapter when a publication wants to
serve native gemtext while the generic body carries HTML for HTTP. The
adapters are required to consult all present attributes when rendering:
this faithfulness contract is what makes the projections equivalent views
rather than competing copies.
Absent attribute files are tolerated at read time. The node layer
returns defined defaults (Untitled for title, an empty author, a date
derived from the publication's modification time, text/plain for
content-type, an empty body), so a half-authored publication stays on
the wire during creation instead of erroring until every field exists.
The defaults are applied by the node layer, not the store: a store
backend reports absence (a null attribute, an empty body), and the
caller decides what absence means. The node classes live in
src/substrate/publication.lisp; the /pub axis that roots them in the
namespace is src/namespace/pub.lisp.
Protocol-agnosticism is an import invariant, not a convention. The store and publication packages import no protocol or registry package, and the adapter packages import no substrate package: an adapter's only path to a publication is a real 9P attach, walk, and read. A future adapter must preserve this boundary: if it can name a substrate symbol, it has bypassed the capability seam the design depends on.
Placing a publication on a running node
A publication reaches the published axis at any time. An owner-keyed client walks the axis over the sealed loopback fabric, creates the slug when it is not already there, and writes the content type and the body; a slug that already exists is replaced rather than refused, so a page can be revised as often as its author likes. The reason this matters is plain in the operator's terms: a node built before a page existed must still be able to receive that page.
Genesis is not a gate on content
Seeding the branded landing publication at genesis is one behaviour with two callers and no policy gate. There is no absence check, no reseeding at boot, and no record anywhere of whether something has been placed. The consequence is the point rather than an oversight: if the operator deletes a publication it stays deleted.
The principle generalises well past publications. Everything here is meant to be mutable by the operator while it runs, which is a large part of why the substrate is written in a language whose systems are patchable live. A system that restores what you removed is not one you control, and an ordering gate at genesis would contradict the whole arrangement.
The write discipline, and one case that still lacks it
The sealed session negotiates a message size of 8192 bytes, so a body larger than one message arrives as several writes at advancing offsets. Such a body must be accumulated across the writes and committed when the handle closes. Before that was done each write replaced the last, and an 11114-byte body read back as 2945 bytes, exactly its trailing remainder. Committing at close also makes the replacement atomic from a reader's side: the previous body is served until the new one is whole.
Any writable node whose content can exceed that ceiling needs the same discipline, and one does not yet have it. Attribute writes have the identical shape and persist each write on its own. Most attributes are far too short to reach the ceiling, but the per-protocol gemtext representation is stored as an attribute and could exceed it. No path writes one today, so the case is latent rather than live, and it is recorded here so that it is found by reading rather than rediscovered as a surprise.
Store seam
The store protocol (src/substrate/store.lisp) is the seam between the
namespace node layer and backing storage. It is a narrow CLOS contract:
eight generic functions (list, attribute read and write, body read and
write, create, existence, and modification time) each taking a store
instance as its first argument. There are no default methods; a backend
that misses a method is a missing-method error, never a silently
papered-over default. Reads are tolerant by contract: absence comes back
as a null attribute, an empty octet body, or a null mtime, and the
calling node layer applies the documented defaults. The filesystem
backend (src/substrate/fs-store.lisp) is the v1 implementation; a
future content-addressed store implements the same eight functions and
no node class above the seam changes. This seam is the designated
replacement point for the durable-storage work.
The filesystem layout mirrors the namespace: one directory per publication, one file per attribute, the slug as the directory name. The layout is inspectable with ordinary tools and is the same tree a kernel v9fs mount would see.
Writes are atomic via rename-into-place: the value is written to a
temporary file in the publication's own directory, then renamed over the
destination, so a reader never observes a torn write: the rename is
atomic because source and destination are on the same filesystem.
Temporary names are unique per write (process id plus an atomically
incremented counter), so concurrent writers never collide on a shared
temp path. Writes are durable as well as atomic: the temporary file's
data is flushed to stable storage (fdatasync) before the rename, and
the containing directory is flushed (fsync) after it, so a write that
has returned survives a crash or power loss; creating a publication
flushes the data directory the same way, so the new directory entry is
itself durable. A write whose flush or rename fails signals
store-error instead of reporting success. The cost is two device
flushes per write (tens of milliseconds on rotating storage) which
suits operator-paced publication writes; *durable-writes* can be
bound to nil where durability is deliberately traded for throughput,
such as bulk imports.
Slug validation is the path-injection guard, and it is enforced
fail-closed at both ends of the object's life. A slug is a non-empty
string of lowercase ASCII [a-z0-9-], checked at the create path
(node-create on the /pub root, and again inside the store backend
before any directory is made) and at the read path (node-walk refuses
a non-conforming name even when a matching directory exists on disk).
Because the walk side validates independently, no adapter bug or
encoding trick can turn a wire request into filesystem traversal: the
HTTP adapter additionally percent-decodes each path segment before
validating it, so an encoded .. is rejected as the decoded text it
really is, and the Gopher adapter rejects .. segments in the raw
selector. The shared predicate lives in src/namespace/pub.lisp.
Durable content-addressed store backend
The durable backend (src/substrate/store-store.lisp) is the second
implementation of the store seam, the content-addressed analog of the
filesystem store. It implements the same eight generic functions over a
content-addressed block stack, a Merkle tree codec, and a
generation-fenced head, so no node class above the seam changes and the
/pub path behaves identically to the filesystem backend. The seam, not
a backend choice, is what the rest of the system sees; this backend exists
so publications can become crash-consistent and content-addressed without
disturbing the publication object model that sits on top of it.
The on-tree layout is the same shape the filesystem backend writes to
disk, expressed through the tree codec rather than through directory
entries. The head names a root tree; that tree carries a pub directory
child; each publication slug is a subdirectory under it; each attribute is
a file child named for the attribute; the body is a body file child.
The publication node layer's slug / attribute / body model therefore
carries over unchanged: the durability mechanism shifts, the addressing
model does not.
Durability here is structural rather than a flush discipline bolted onto
a rename. A block is content-addressed and written (and flushed)
before the head names it, so the head never references a block that is
not already durable. Advancing the head is a generation-fenced
compare-and-swap, and that single advance is the only linearization point
for a write: a write that carries a superseded generation is rejected at
the commit and surfaces as store-error, never a silent overwrite and
never a retry that could lose a concurrent change. This is the
durable-before-referenced ordering the rest of the store stack already
relies on, reused at the publication seam.
Writers within one image are serialized by an in-image lock the backend
holds for the read-modify-write cycle, so the only way the
compare-and-swap can fail is an out-of-band advance from outside this
image, and that case fails closed as store-error. The lock removes the
need for an unbounded retry loop; the compare-and-swap remains the
authority on what actually committed.
One place the POSIX assumption behind the seam leaks through: the seam exposes a publication modification time, and there is no filesystem timestamp to report. The durable backend synthesizes that value from the head generation, which is monotone, so a later write always reports a later time. The value is used only as the published-date display fallback when a publication carries no explicit published-date attribute; it is a display proxy, not a wall-clock guarantee.
The backend is selected when the fabric starts (src/fabric.lisp,
start-fabric). The durable store is the wired default; the filesystem
store is retained as a selectable operator fallback for a conservative
rollout or for continuity with an existing on-disk publication tree. A
fresh durable store serves an empty /pub on first boot: there is no
in-band migration of existing filesystem publications into the durable
store, so an operator who needs continuity with a filesystem publication
tree selects the filesystem backend until a later migration path lands.
The durable store roots at its own data directory, distinct from the
filesystem publications tree, resolved in a fixed order: an explicit
operator-supplied store directory wins; otherwise the store roots at a
sibling of a supplied publication directory; otherwise it falls to the
XDG data home default. Both the durable store's data directory and the
selected store are cleared on a failed start and on stop, so a fabric
that fails to come up or that has been stopped leaves no dangling seam
state behind.
Two boundaries make this backend auditable. First, it is the one file that bridges the store seam and the lower block, head, and tree codecs, and it deliberately reaches no identity, capability, namespace, or protocol symbol: the seam-crossing is one-directional and structurally enforced, so a capability bug cannot reach storage and a storage bug cannot reach the capability layer. Second, the store holds opaque blocks: it never interprets publication content and carries no key bytes. The durability layer sees ciphertext-or-cleartext blocks as bytes to address and name, never as material to read: the sovereignty boundary that keeps key custody out of the storage path holds here as it does everywhere else in the substrate.
Adapter attach and projection-over-capability
The wire-format adapters (the adopted c3po-http and c3po-gopher
modules) are ordinary valis protocol plugins: each registers on its port and
specialises handle-connection. What makes them adapters rather than
owners is how they reach content. At fabric start
(src/fabric.lisp), each adapter is handed a freshly minted bearer
capability designating /pub with read-only rights, and opens a
long-lived 9P session by attaching over an in-process channel, the same
codec and the same enforcement path a remote 9P client would traverse,
with no TCP loopback. The assembler resolves the capability to a mount
directive and builds a view containing exactly one entry: a read-only
projection wrapping the publication root. The adapter can walk and read;
a write-mode open is denied at the server by the projection wrapper, not
by the adapter declining to write. Because the attach path is the
in-process form of the real transport, the adapter is structurally
identical to a sandboxed out-of-process edge module. Moving it out of
the image changes the transport endpoint and nothing else: capability,
view assembly, and projection enforcement are already in their final
shape.
Each wire request maps one-to-one onto a walk in the adapter's mounted
subtree. A missing name is a walk failure, and the adapter surfaces it
as the protocol-native absence: HTTP 404, Gopher error entity type 3.
A principal with no /pub grant has no /pub name at all, so absence
reaches the wire by construction, never by a permission check on an
admitted request.
Responses and requests are size-bounded. Directory listings and file
bodies are read with hard ceilings (one megabyte for a listing, sixteen
megabytes for a body), so a pathological publication cannot balloon a
response buffer. Request lines are byte-capped on read (4096 bytes for
an HTTP request line, 1024 for a Gopher selector), and overflow is
answered with HTTP 413 or a Gopher type-3 error rather than unbounded
buffering. Connection occupancy is bounded the same way memory is: each
adapter arms a per-connection read deadline when it accepts a request
(*http-read-deadline-seconds* and *gopher-read-deadline-seconds*,
ten seconds by default), so a client that connects and sends nothing
(or dribbles bytes slower than the deadline) cannot hold a handler
thread past it. Expiry is treated exactly like a malformed request:
HTTP answers 408 Request Timeout, Gopher closes without a response per
its one-transaction model, and the executor reclaims the thread and the
socket. The mechanism lives behind the connection seam: the adapter
arms the deadline through a transport-blind generic, and the epoll
backend (the one component allowed to name the socket library) wakes
the blocked reader by shutting down the socket's read side, which
surfaces to the handler as ordinary end-of-file while leaving the write
side open for the parting response. In-memory connections, and any
transport with nothing to time out, accept the call as a no-op.
Adapter teardown is complete: stopping the fabric closes both ends of each adapter's in-process channel, and close means drain-then-EOF: frames already written still deliver, then both sides observe end-of-file. The teardown order is deliberate. The port is unregistered first, so no new connection lands in a dying session; the live fid handles are clunked while the channel still delivers; the channel is then closed, which wakes the server's blocked dispatch loop and lets it return; finally the stopped adapter joins its server thread. The client-side reply reader exits on the same EOF, with any outstanding waiters drained by a transport-closed condition rather than left blocked. The join is bounded: a server thread that fails to exit within the timeout is a regression in the channel's close path, and it surfaces as a loud warning naming the leaked thread rather than a stop that hangs forever: teardown must never trade a thread leak for a deadlock.
Validation coverage
Each success criterion and security property of the publication
substrate is anchored to a named test. The substrate tests prove the
object model and the 9P session path; the render tests drive
handle-connection end to end against an in-memory connection (the
same entry point a real TCP client reaches), so the request-parse, walk,
and response-render paths are exercised exactly as shipped.
| Criterion | Test | File |
|---|---|---|
| Store backend round-trips every store protocol operation | substrate-store-protocol |
tests/substrate-test.lisp |
| Non-conforming slugs rejected fail-closed on create and on walk | substrate-slug-charset-fail-closed |
tests/substrate-test.lisp |
| Absent attributes read as defined defaults | substrate-tolerant-defaults |
tests/substrate-test.lisp |
| Owner creates and writes publications via 9P | substrate-authoring-create-write |
tests/substrate-test.lisp |
| One publication object backs both adapters | substrate-publication-shared-object |
tests/substrate-test.lisp |
| HTTP adapter reads only through the read-only projection | substrate-http-read-through-projection |
tests/substrate-test.lisp |
| Gopher adapter reads through the read-only projection | substrate-gopher-read-through-projection |
tests/substrate-test.lisp |
| No grant means no name: capless session sees absence on the wire | substrate-no-mount-no-name |
tests/substrate-test.lisp |
| HTTP listing renders end to end and links the publication | http-render-directory-listing |
tests/adapter-render-test.lisp |
| Listing links below the root carry the parent path | http-render-nested-listing-links |
tests/adapter-render-test.lisp |
| HTTP serves the publication body end to end | http-render-file-body |
tests/adapter-render-test.lisp |
| Percent-encoded traversal segments are rejected with 404 | http-render-rejects-encoded-traversal |
tests/adapter-render-test.lisp |
The /pub node refuses traversal names regardless of adapter |
pub-walk-refuses-traversal-slug |
tests/adapter-render-test.lisp |
| Duplicate create surfaces as a precise 9P error, not an internal one | pub-create-duplicate-signals-clean-error |
tests/adapter-render-test.lisp |
| Gopher menu renders end to end with the RFC 1436 terminator | gopher-render-menu |
tests/adapter-render-test.lisp |
| Gopher serves the publication body end to end | gopher-render-file-body |
tests/adapter-render-test.lisp |
| An idle client is cut off at the read deadline, its thread reclaimed | idle-client-cut-off-at-read-deadline |
tests/read-deadline-test.lisp |
| A prompt request is served normally with the deadline armed | prompt-request-unaffected-by-armed-deadline |
tests/read-deadline-test.lisp |
| HTTP answers a deadline cut-off with 408 | http-deadline-expiry-answers-408 |
tests/read-deadline-test.lisp |
| Gopher closes a deadline cut-off without a response | gopher-deadline-expiry-closes-silently |
tests/read-deadline-test.lisp |
| Fabric stop reaps session threads and clears every seam variable | adapter-lifecycle-reaps-threads |
tests/adapter-render-test.lisp |
The bus: dispatch over 9P
Inter-module and inter-node dispatch rides 9P itself. 9P is already a network-transparent RPC over a namespace (it is a message bus), so a protocol module's mailbox is a file, and routing a connection to a module running in a local thread or on a remote Lisp node is the same file operation either way. A separate message-bus transport (e.g. ZeroMQ) is admitted only where a pub/sub fan-out pattern genuinely beats file semantics, not as the default carrier.
There is no usable Common Lisp 9P implementation in the wild, but 9P2000 is
a small protocol (~13 message types), so a native CL client+server is
tractable. Because the Linux kernel ships a 9P client (v9fs), a valis 9P
server can additionally be mounted by the host kernel, bridging the
namespace into ordinary Linux tools.
Migration: light process, durable namespace
A person's valis is migratable (it can be stood up on any valis-compatible host) because the durable thing is the namespace, not the process. Migration is therefore reassembly, not transfer: a fresh instance reconstructs the filesystem semantics from a small manifest of mounts plus the owner's keys, and the destination host's agent re-steers the designated ports to it. This demands a continuous discipline: keep the process boundary light, externalising anything reconstructable, so standing up an instance is cheap. The model is Plan 9's: a per-process namespace (assembled, local) distinct from a shared authoritative file server.
Where the namespace lives: tiered by access pattern and secrecy
- Bulk and cold data is content-addressed and immutable: a block's hash is its name (the Venti model; IPFS as the transport). Immutable content is location-independent: an instance pulls it from wherever it is served.
- Published data is content-addressed and unencrypted, served from many places at once. Because it lives outside the mutable store, high-fanout public reads never touch the private substrate.
- Mutable state lives on a backing store the instance mounts; the mutable head is a small hash pointer over the immutable tree, in the manner of fossil over Venti or git refs over objects.
- Secret keys are owner-held or held in a separate trust domain; they are never content-addressed and never live solely on a host that might be evacuated.
- A hot working set is served by a read-side caching tier in front of the store, so a coincident instance keeps its reads off the single-writer head while still observing every committed write. It is two caches and a coherence primitive, each of which gets its correctness from a fact the store already carries (a block's hash is its name, and the head's generation is its version), so the cache adds speed without adding a way to be wrong. The read-side caching tier below is the mechanism.
The reassembly path in code: where to follow it
The reassembly is not a single function but an ordered handshake whose composition
site is src/fabric.lisp → start-fabric (the :evacuative t arm drives the
security regime; see the two regimes below). An auditor can follow
it through these named seams.
A fresh instance comes up only if it holds the owner's key out of band: the gate is
src/fabric.lisp → owner-key-required, which fails closed when the keyfile is
absent rather than standing up an unauthenticated substrate; the key itself is loaded
through the custody seam (src/identity/custody.lisp → load-or-create-keyfile), so
the key bytes never live on the durable store the instance mounts. With the key in
hand, the instance reconstructs the filesystem semantics at
src/namespace/assembler.lisp → assemble-canonical-frame, the single reassembly
entry that rebuilds the view from the manifest of mounts; an unreachable mount is a
fail-closed stand-up (reassembly-unreachable), never a silent partial view. The
assembler reads the authoritative mutable head through an injected seam,
*durable-head-reader* (bound at fabric start, src/fabric.lisp), so the same
reassembly code path serves whether the head is read from the durable store or from a
test double: the durability boundary is one named indirection, not a scattered set of
direct reads.
The mutable head carries the generation that is both the consistency token and the
fence (the two regimes section traces its security role). It advances
through one linearization point: src/store/head.lisp → advance-head, a
compare-and-swap against the expected generation. A late writer that expected a
superseded generation loses the swap and is told so by generation comparison alone:
no wall clock, no lease deadline. The pre-commit fault seam *advance-head-pre-commit-hook*
(src/store/head.lisp) lets the out-of-band crash harness freeze a writer exactly at
the swap to prove the rejection survives a real pause.
Consistency across coincident instances
valis instances can be temporally coincident: several at once, to absorb load. Authority over the mutable namespace is a single writer per subtree (the fossil model), not a leaderless or eventually-consistent scheme: a sovereign namespace wants one authoritative truth, not merge. For one person's modest write load a single writer is ample; if write scaling ever bites, the namespace (already a tree of mounts) is partitioned into per-subtree authorities (the Ceph dynamic-subtree-partitioning model). Coincident instances each assemble their own namespace view and interpose caches; the authoritative writer serialises mutation; public read load is offloaded to the unencrypted content-addressed publishing path.
The read-side caching tier
A coincident instance must be able to read fast without consulting the single writer on every access, yet must never serve a stale answer once a write has committed. The read-side tier achieves both by deriving each cache's correctness from a value the store already carries, so caching adds no new way to be inconsistent. It has three parts.
The first is an immutable-block cache (src/store/block-cache.lisp), a
read-mostly tier in front of the content-addressed block device, one per store
handle and never a process-global. Because a block is named by the SHA-256 of
its bytes, the name is the validator: a clean hit returns the admitted bytes
verbatim and never re-hashes, yet can never be stale, because the same name
always means the same bytes. Admission is verify-then-admit: the bytes are
re-hashed against their requested name before they enter the cache, on the
store's single digest seam, so a substituted or bit-rotted on-disk block is
rejected at admission and can never become a hit. The corollary is that a
cached hit stays correct even if the on-disk block is later corrupted: the
content is immutable, so the in-memory copy is as authoritative as the disk
ever was. Growth is doubly bounded (an explicit byte budget and an
independent descriptor/slot cap, so a flood of tiny blocks cannot exhaust
entries while staying under the byte budget) with CLOCK (second-chance)
eviction, the right policy for an immutable read-mostly tier where eviction
order does not move the hit rate. A reader that must hold a block across an
operation pins it: with-pinned-block refcounts the slot and releases the
pin on every exit path via unwind-protect, and eviction skips a pinned slot
rather than force-evicting it: force-evicting a block a reader still holds
would be a use-after-free of its byte vector.
The second is a generation-revalidated head-decode cache, one fixed record
per store handle (src/substrate/store-store.lisp). Decoding the head's pub
directory is the expensive part of a read; caching the decoded result would
normally reintroduce staleness. It does not here, because the head already
carries a monotonic generation token (the same token that fences a
superseded writer), and that token doubles as the cache validator. Every read
re-reads only the generation through a narrow reader (current-head-generation,
src/store/head.lisp) that integrity-verifies the 124-byte HEAD record but
decodes no tree, reuses the cached decode when the generation is unchanged, and
reloads on any change. The generation probe, the compare, the decode, and the
recorded generation all happen inside one critical section under the store
lock, so the cache is always stamped with the generation that was actually
current: a coincident writer cannot advance the head between the probe and the
decode and leave the cache mislabelled. The staleness bound is therefore zero:
the moment another handle commits, the next read here sees a moved generation
and reloads. The write path is untouched (it reads the current head directly
under the lock and never consults this read cache), so the cache cannot
interfere with the compare-and-swap that is the sole linearization point.
The third is content-addressed qid.version, the coherence primitive a
correct cache above the store needs. A served node's 9P qid.version is the
low 32 bits of its content score (store-node-score / score-low-word), with
0 remapped to 1 so genesis, an absent slug, and an unwritten leaf all read
1, never 0 and never an error. The score is a pure function of the node's
durable content (the Merkle property: a directory's score changes iff its
listing changes, a leaf's iff its bytes change), so the version is correct
across instances by construction: two instances over one data directory
compute the same version for the same node with zero coordination, and a write
to one slug never perturbs another slug's version because their scores are
independent. The override is per node, with no shared mutable version counter,
so there is nothing to serialize. This is what lets a 9P or HTTP cache one tier
up validate correctly: the version it keys on is itself the content hash, which
is exactly the change-detector a cache validator wants.
The three compose into the property the coincident regime needs: reads run off the per-handle caches at content-addressed speed, the generation token keeps the head-decode cache coherent with staleness zero, and the content-derived version lets every cache above the store (including an ordinary HTTP cache at the edge) track live content without a coordination protocol.
- Validation coverage: the read-side caching tier
Each property of the caching tier is anchored to a named test. The block-cache tests drive the cache directly; the store-store tests drive the generation-revalidated decode and the content-addressed version over the real durable store.
Criterion Test File A clean cache hit returns the device bytes verbatim, with no re-hash block-cache-hit-returns-device-bytestests/store-block-cache-test.lispAdmission verifies bytes against their score; a corrupt block is never admitted block-cache-verify-then-admittests/store-block-cache-test.lispA cached hit stays correct even after the on-disk block is corrupted block-cache-immutability-under-ondisk-corruptiontests/store-block-cache-test.lispResident bytes never exceed the byte budget block-cache-honours-byte-budgettests/store-block-cache-test.lispThe descriptor count never exceeds the slot cap block-cache-slot-captests/store-block-cache-test.lispA pinned block is never evicted while it is held pinned-block-never-evictedtests/store-block-cache-test.lispThe pin is released on every exit path from with-pinned-blockwith-pinned-block-releases-on-every-exittests/store-block-cache-test.lispThe cache layer reaches no capability/namespace/identity symbol block-cache-import-cleantests/store-block-cache-test.lispThe narrow generation reader verifies the HEAD but decodes no tree current-head-generation-narrow-readtests/store-head-test.lispAn unchanged generation reuses the cached head decode store-store-unchanged-generation-reuses-decodetests/substrate-store-store-test.lispCoincident handles over one store read each other's committed writes store-store-coincident-handles-read-your-writestests/substrate-store-store-test.lispThe head-decode cache is per handle, never shared across instances store-store-head-cache-is-per-instancetests/substrate-store-store-test.lispA node's version is non-zero after a write store-store-version-nonzero-after-writetests/substrate-store-store-test.lispA write to one slug does not move another slug's version store-store-version-per-file-isolationtests/substrate-store-store-test.lispTwo instances over one data directory compute the same version for a node store-store-version-cross-instance-sametests/substrate-store-store-test.lispGenesis and an absent slug read version 1, never0store-store-version-genesis-absent-slug-is-onetests/substrate-store-store-test.lisp
Two regimes, and evacuation under attack
The mutable head carries a monotonic generation token, which yields consistency and fencing from one mechanism. There are two migration regimes:
- Coincident (scaling): several instances share the store; the generation/lease serialises who may advance the head.
- Evacuative (security): fleeing a computational substrate under attack, assuming the host is compromised. Standing up the new instance bumps the generation, which fences the old, possibly-compromised instance: the store rejects writes from a superseded generation, the same shape as revoking a superseded capability. Because keys never lived solely on the evacuated host, the new instance comes up without trusting the old one.
In-flight connections drop and re-establish by default, acceptable because the
namespace, not the process, is the source of truth. Live carry-over of
established TCP connections (the kernel's TCP_REPAIR path) is a possible
enhancement for protocols that cannot tolerate a reconnect, not a dependency.
Re-steering the established sessions cleanly (so the old instance's clients
reconnect to the new one rather than waiting on a passive TCP drop) is the job of
a session-drain seam on the steering host's agent (the chartered drain-sessions
boundary, proposed in the steering agent's repository). That seam is a latency and
user-experience improvement on handover, never the fence: the fence is the store's,
and it holds whether or not the old host cooperates, because the threat model
assumes the old host may be compromised.
The fence and the sweep: where to follow them
The bump fences the superseded instance through the same one revocation mechanism
that revokes a capability. The fence write is src/capability/revocation.lisp →
fence-epoch-hash (the content-addressed name of a fenced generation) entered into
the durable revocation set by revoke-hash (fdatasync-durable, so a fence survives
a crash the instant it is written). A write capability is rejected if its bound
generation is fenced: the verifier consults the revocation set at
src/capability/verifier.lisp → %write-cap-fenced-p, on the same path that checks
an ordinary revoked token. Each capability carries the generation it was minted under
as src/capability/token.lisp → token-bound-generation, so the consult is a pure
membership test against the fenced generation: a superseded writer is rejected by
construction, not by a host that must be trusted to police itself. Finally, live
write-capable grants still held under the fenced generation are dropped by
src/namespace/assembler.lisp → evict-by-fence-epoch, so a fenced instance keeps
no live authority even in memory.
Evacuation walkthrough: one migration, end to end
One full evacuation traces these seams in order. An operator (or the new host's agent) stands up a fresh instance over the same durable data directory while the old, possibly-compromised instance is still running.
- Owner-key gate. The fresh instance refuses to start without the owner's key,
held out of band (
src/fabric.lisp→owner-key-required, loaded viasrc/identity/custody.lisp→load-or-create-keyfile). Because the key never lived solely on the evacuated host, the new instance comes up without trusting the old one. - Reassembly. It rebuilds the namespace from the manifest of mounts
(
src/namespace/assembler.lisp→assemble-canonical-frame), reading the authoritative head through the injected*durable-head-reader*seam: reassembly, not transfer. - Generation bump. It advances the head past the running instance's generation
(
src/store/head.lisp→advance-head), the single compare-and-swap that is the store's only linearization point. - Fence. The bump writes the superseded generation's
fence-epoch-hashinto the durable revocation set viarevoke-hash(src/capability/revocation.lisp),fdatasync-durable. - Sweep. Live write-capable grants under the fenced generation are evicted
(
src/namespace/assembler.lisp→evict-by-fence-epoch), so the old instance retains no live authority in memory. - Rejection of the late writer. The old instance's next write loses on both paths:
its capability is rejected at the verifier consult because its
token-bound-generationis fenced (src/capability/verifier.lisp→%write-cap-fenced-p), and its compare-and-swap is rejected because it expected a superseded generation (advance-headreturns the stale sentinel). The rejection is clock-free: it holds across an arbitrarily long pause. - Serving resumes. The new instance serves the reassembled namespace at the pinned generation; in-flight connections to the old instance drop and re-establish.
The whole handshake is composed at src/fabric.lisp → start-fabric with the
:evacuative t arm, the one site an auditor reads to see the steps wired together,
rather than re-implemented in a test body.
Opaque-block audit: why an operations path reaches no content
The store holds opaque blocks, addressed and named by the hash of their bytes, never
interpreted. The durability cluster (store-store, block-device, block-cache,
block, head, tree, manifest) imports zero capability, namespace, or identity
symbol, so a storage-side or operations-side path cannot become a confused deputy that
reaches content or keys: the absence of the import is structural, proven over the whole
cluster by store-cluster-imports-no-content-symbols, not a runtime check that could be
skipped.
The table below audits the operations paths an operator would use to move or observe a substrate. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting: it sees ciphertext-or-cleartext blocks as bytes to address, never as material to read.
| Operations path | Seam it would use | Why it reaches no content | Anchoring test |
|---|---|---|---|
| Migrate / evacuate a substrate | start-fabric :evacuative t → reassembly + bump |
Moves the head pointer and the addressed blocks; never decodes a block, never holds a key | migration-reassemble-then-evacuate-end-to-end |
| Back up the durable store (subsystem TBD) | the block-device / head / tree codecs | Copies hash-named opaque blocks and the 124-byte head record; no content/key import path | store-cluster-imports-no-content-symbols |
| Observe / meter the store (subsystem TBD) | the store handle's counters and the narrow head reader | Reads generation and block counts; the cluster imports no namespace/capability symbol | store-cluster-imports-no-content-symbols |
| Revoke an evacuated instance's authority | revoke-hash over fence-epoch-hash |
Writes a content-addressed fence into the revocation set; the set holds hashes, not data | two-instances-only-one-write-accepted-fail-closed |
| Sweep live grants under a fenced gen | evict-by-fence-epoch |
Drops in-memory write grants by generation; touches no block bytes and no key material | bump-sweeps-live-write-grants |
Validation coverage: migration and evacuation
Each success criterion of the migration-and-evacuation story is anchored to a named
test. The end-to-end and fence tests drive the real start-fabric :evacuative t path
over a durable store; the out-of-band proofs run in the standalone crash harness, where
a real process is paused or killed across the bump.
| Criterion | Test | File |
|---|---|---|
| A fresh instance reassembles and serves; the evacuated instance is fenced | migration-reassemble-then-evacuate-end-to-end |
tests/migration-end-to-end-test.lisp |
| and its live write grants are swept | ||
| Live write-capable grants under the fenced generation are swept by the bump | bump-sweeps-live-write-grants |
tests/store-fence-revocation-test.lisp |
| A write paused across the bump is rejected at the verifier and at the CAS | two-instances-only-one-write-accepted-fail-closed |
tests/store-fence-revocation-test.lisp |
| The store cluster imports no content, key, or namespace symbol: a confused | store-cluster-imports-no-content-symbols |
tests/migration-end-to-end-test.lisp |
| deputy is impossible by construction | ||
Out of band: an old process SIGSTOP'd across the bump cannot land its CAS |
proof-evacuation-under-pause |
scripts/store-crash-test.lisp |
| Out of band: a recovered head never names a generation past an unfenced one | proof-fence-precedes-head-advance |
scripts/store-crash-test.lisp |
The mail transport spine
valis terminates mail as a first-class axis of the sovereign namespace. The wire engines that speak SMTP and IMAP to the hostile internet live in a separate process at the IETF edge; valis owns the substrate the mail moves through: the authenticated submission door, the durable transport queue, the router that decides local-versus-relay, the per-DID Maildir inbox, and the seam a wire adapter drains relayed mail through. The whole spine moves an opaque message body end to end: every layer below names and addresses the body by the hash of its bytes and never interprets it.
This chapter is the auditor's entry point to that spine. It traces one message end to end through the named code, states what the open-relay guard guarantees and why it cannot be turned off, and shows that no operations path reaches the content of a message. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement, not delivery onto a real network.
Mail flow walkthrough: one message, end to end
One submission traces these seams in order. The owner stands at /mail/outbox
holding a write capability minted through the real /cap/ctl path; an anonymous
or unauthenticated peer never reaches this door.
- Authenticated submission (the clunk commit). A capability-mounted 9P write
into
/mail/outboxstages octets; the clunk is the atomic commit (src/namespace/mail-outbox.lisp→node-closeon the outbox-submission file). The envelope and the submit provenance are stamped at the clunk, never parsed from the body; a fid abandoned without a clunk lands nothing: complete or absent by construction. - The live seam.
/mailis present in the running owner frame only becausesrc/fabric.lisp→start-fabricbinds the three mail seams (*mail-dir-root*,*local-delivery-did-resolver*,*local-domains*), each gated on a durable store-backed substrate and reset fail-closed on teardown. The axis is mounted bysrc/namespace/assembler.lisp→%ensure-mail-mountand reached atsrc/namespace/root.lisp→node-walkon "mail". - Ingest. The clunk hands the assembled octets, derived envelope, and stamped
provenance to
src/mail/seam.lisp→land-message, the single ingestion door through which both owner-submitted and peer-landed mail enter. - Route.
src/mail/router.lisp→route-messageclassifies every recipient throughroute-recipient, which tests locality against*local-domains*and admits relay only on proven submit authority (see the open-relay guard below). - Queue. Each routed recipient lands a durable per-recipient record via
src/mail/transport-queue.lisp→place-incoming: the ack follows thefdatasync, so the queue is durable before the submitter is told "committed". The relay path is admitted into a bounded active window withadmit-activeand the slot is returned on a terminal outcome withrelease-slot. - Local delivery. A local recipient is written into the owner's Maildir at
/mail/inbox/<did>/new/bysrc/mail/local-deliver.lisp→local-deliver, keyed under the live owner DID the*local-delivery-did-resolver*returns from custody. - Relay drain. A relay recipient is drained by
src/mail/drain.lisp→start-mail-drain/drain-active-once, which hands the opaque message to a registered mail-adapter. The wire engine is stubbed here; the adapter records a disposition and the drain frees the active slot.
The same path is driven against the live fabric (outbox capability, real router,
real queue, real delivery) in tests/mail-end-to-end-test.lisp, so the
walkthrough above is exercised, not merely asserted.
The open-relay guard: fail-closed by capability, not by flag
route-recipient reaches the relay path by exactly two yes-paths: the recipient
is local (recipient-local-p true against *local-domains*, so the message is
delivered into a local inbox and never relayed), or the message carries
authenticated-submit provenance, an owner who stood at /mail/outbox under a
keyed session. A peer-landed message (non-authenticated provenance) for a
non-local recipient matches neither path: route-recipient signals
mail-open-relay-refused and lands nothing, never silently forwarding.
What this guarantees is that valis cannot become an open relay. Why it holds by
construction: there is no allow-relay flag to misconfigure. Relay authority is a
capability the submitter either presents or does not; the refusal is the absence
of that authority, not a policy toggle a host could flip. An auditor confirms the
guard is live, not vestigial, in
unauthenticated-non-local-relay-is-refused-fail-closed.
The content boundary: the body is an opaque block
A message body is an opaque block, content-addressed octets the store names by
the hash of their bytes and never interprets, exactly like every other block in
the durability cluster. Sovereignty here is absence: the operations-facing view
of the mail spine has no name for any body. A read grant on /mail/queue resolves
a body-free projection (src/namespace/mail-queue.lisp → %synthetic-name) that
exposes envelope and status only; it has no walkable name for any message body and
none for /mail/inbox. The body score that keys a status record is an address,
never invertible to content.
The table below audits the operations paths an operator would use to move or observe the mail spine. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting.
| Operations path | Seam it would use | Why it reaches no content | Anchoring test |
|---|---|---|---|
| Observe the mail queue (operator) | the /mail/queue body-free projection (%synthetic-name) |
Names envelope + status only; no walkable name is a body, and none is /mail/inbox |
ops-queue-grant-resolves-no-message-body-name |
| Any ops path over the queue projection | the queue projection package's imports | Imports zero body-octet read (read-block / decode-file): a confused deputy is impossible |
ops-queue-projection-imports-no-body-read-symbol |
| Back up the mail store (subsystem TBD) | the block-device / head / tree codecs | Copies hash-named opaque blocks; no content or key import path | store-cluster-imports-no-content-symbols |
| Observe / meter mail flow (subsystem TBD) | the queue projection's status records and counters | Reads envelope, status, and counts; the body score is an address, not content | ops-grant-on-queue-has-no-name-for-the-other-leaves |
| Relay-drain a message (the transport itself) | drain-active-once → a registered mail-adapter |
Hands opaque octets to the wire adapter; classifies and moves, never decodes for ops | authenticated-relay-submission-drains-to-the-stub-adapter |
Validation coverage: the mail transport spine
Each success criterion of the mail spine is anchored to a named test. The
end-to-end tests drive the real start-fabric seam, outbox capability, router,
queue, and delivery over a durable store with only the wire engine stubbed; the
sovereignty tests prove the content boundary both behaviorally (over a really
queued message) and by construction (a do-symbols import firewall).
| Criterion | Test | File |
|---|---|---|
| A local-recipient outbox submission delivers into the owner inbox byte-identical | local-recipient-submission-delivers-into-the-owner-inbox |
tests/mail-end-to-end-test.lisp |
| An authenticated non-local submission drains to a stubbed adapter; the slot frees | authenticated-relay-submission-drains-to-the-stub-adapter |
tests/mail-end-to-end-test.lisp |
| An unauthenticated non-local relay is refused fail-closed and lands nothing | unauthenticated-non-local-relay-is-refused-fail-closed |
tests/mail-end-to-end-test.lisp |
An ops grant on /mail/queue resolves no name for any message body or the inbox |
ops-queue-grant-resolves-no-message-body-name |
tests/mail-sovereignty-audit-test.lisp |
| The ops-facing queue projection imports no body-octet read by construction | ops-queue-projection-imports-no-body-read-symbol |
tests/mail-sovereignty-audit-test.lisp |
An ops grant on /mail/queue has no walkable name for the other axis leaves |
ops-grant-on-queue-has-no-name-for-the-other-leaves |
tests/mail-axis-test.lisp |
The DNS substrate: authoritative names off the answer hot path
valis terminates authoritative DNS as a first-class axis of the sovereign
namespace, the same way it terminates mail. Two siblings sit between the wire and
the data: the engine that speaks the RFC 1035 binary protocol on port 53 to the
hostile internet (a separate process at the IETF edge), and runciter, the DNS
service module (the logical nameserver that composes answers and holds the
serving index). valis owns the data substrate those answers come from (the zone
data held in PostgreSQL as the operator's system of record, the read seam runciter
loads its index through, and the capability gate that admits a DNS wire module),
and it dispatches a steered connection to the registered service module. valis
holds no nameserver answer logic and no serving cache: it is the authoritative
ndb DATA owner, and runciter is the nameserver. PostgreSQL is the operator's
system of record, never a per-query dependency: runciter answers from its own
in-memory index and keeps serving the last good index across a database outage,
so the names do not go dark.
This chapter is the auditor's entry point to that substrate. It states the data read seam runciter binds over, the narrow answer-composition contract the registered serving handler honours, traces a decoded query end to end through the named code, locates the availability and answerability invariants (now runciter's, over the cutover facets valis owns), and shows the capability gate a DNS wire module passes before it can bind. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement of an answer, not encoding onto a real network.
The seam contract: a decoded query, a record set, and a result code
The contract a bound DNS wire engine sees is deliberately narrow and versioned.
The engine hands a decoded query (a name, a type, and a class) to the
registered serving handler, and the handler returns a record set and a result
code. That handler is runciter's: valis registers (make-dns-service-handler
<pg-zone-source>) as the adapter handler, so the answer is composed by the
nameserver over valis's data, not by valis. The result code is one of three
keywords: :noerror (the record set is the answer), :nxdomain (the name does
not exist), or :nodata (the name exists but carries no record of the asked
type). The contract keeps :nodata distinct from :nxdomain because they are
different facts about the zone, and a correct resolver caches them differently;
the wire maps :nodata onto a NOERROR message with the authority section carrying
the zone's apex SOA.
Negative answers carry authority data, not just a code. On both :nxdomain and
:nodata, the handler returns the zone's apex SOA record in the record set, so
the wire has everything it needs to emit a spec-correct negative response and to
compute the negative-cache lifetime as the minimum of the SOA MINIMUM field and
the SOA record's own TTL (RFC 2308). A name outside every authoritative origin is
:nxdomain with an empty record set (not-authoritative, no apex SOA), kept
distinct from an in-zone negative. runciter owns what authority data accompanies
each outcome; the wire stays a pure codec that encodes what it is handed.
Name synthesis is the nameserver's job, never the wire's. For an aliased name the returned record set already contains the CNAME record plus any in-zone records the alias resolves to (one in-zone, single-hop chase); for a name covered by a wildcard owner the returned set is the synthesized answer. The wire encodes what it is handed and never chases names or reaches below the seam for zone data. Everything that is purely a property of the wire stays wire-side and is never surfaced at the seam: EDNS0/OPT and the DO bit, truncation, response-rate limiting, recursion flags, the message ID, name compression, the authoritative-answer bit, and the transport-generated result codes (FORMERR/SERVFAIL/NOTIMP/REFUSED). Delegations and DNSSEC are not in the v1 contract; they are additive later through a contract-version bump, gated by the same fail-closed version check described next.
The version check is fail-closed at register time. src/plugin/dns-adapter.lisp
holds the seam: the *dns-adapters* registry, register-dns-adapter, and a
contract-version constant +dns-adapter-contract-version+ (1) with the set of
admitted versions in *supported-dns-contract-versions*. An engine that declares
a contract version outside that set is refused before any registration mutates the
table: register-dns-adapter signals dns-adapter-version-mismatch and binds
nothing, so a wire engine built against a future, incompatible contract can never
silently skew against this core.
The data read seam: valis's PostgreSQL zone source
valis exposes its zone data to the nameserver through a narrow read protocol, not
a serving cache. pg-zone-source (src/operator-state/zones.lisp) implements
runciter's zone-data-source protocol over the dns_zone / dns_record rows:
zone-version (the zone's SOA serial as an opaque RFC-1982-comparable token),
zone-record-set (the runciter resource-record structs of a zone), and
lookup-records (records owned by a name). Two further reads let the nameserver
build its serving index and gate answerability without a serving view living in
valis: source-origins (the held-inclusive set of canonical origins valis holds
authoritatively now) and zone-cutover-status (a zone's (cutover-state,
expire-deadline): the facets a secondary's answerability turns on). Each method
queries the ambient pooled connection, so a caller wraps a read in
with-operator-state-connection; the methods open no per-query socket on any hot
path because the hot path is runciter's, not valis's.
The in-memory serving index and the answer composition live in runciter, the
nameserver. runciter loads its index from this read seam, holds it as an immutable
snapshot, and composes every answer (the RFC-2308 decision tree: exact match,
one in-zone single-hop CNAME chase, single-label *.<parent> wildcard synthesis,
NODATA-versus-NXDOMAIN against an existence set, apex SOA on in-zone negatives,
no apex SOA on a not-authoritative name) over its own snapshot. valis composes no
answer and caches no zone. This is the Option-A split: the serving cache lives
with the nameserver; valis is the authoritative DATA owner over PostgreSQL.
Availability is runciter's, by construction over this seam. runciter rebuilds its
index build-then-swap under a lock and serves last good: a rebuild whose source
read raises (PostgreSQL unreachable, a row malformed) is contained, leaving the
prior snapshot answering, so a database outage degrades to serve-last-good rather
than to an empty answer. PostgreSQL is the operator's system of record, never the
per-query dependency. valis's contribution to that guarantee is the data seam
itself plus the cutover facets the nameserver gates on; the serve-last-good
property is verified in runciter's suite (failed-rebuild-keeps-prior-index,
source-outage-after-load-keeps-answering).
The commit fires a change signal; the nameserver pulls
valis installs nothing into a serving view on commit: it owns no view. The
capability-gated :names door (src/namespace/names.lisp) takes a zone's master
text and commits an atomic full-zone replace at the 9P clunk: node-close on the
submission file is the commit, exactly as the mail outbox clunk is. After the
import resolves the new serial, the commit fires the in-process zone-change signal
(fire-zone-change, origin and serial) so an in-process listener (the outbound
zone-change feed, and a wire feed) observes the change. The nameserver refreshes
by pull: runciter re-checks zone-version per origin (and source-origins for
set changes) on a throttled, query-triggered poll and rebuilds its index when the
version is newer, so a committed write is picked up without valis pushing into
runciter. The committed write is durable in PostgreSQL regardless; runciter's next
successful pull reflects it. (A lower-latency optional force-refresh seam exists on
the runciter side, refresh-dns-service, that a commit hook may call; valis
leaves it unwired by default, the throttled poll being the correctness floor.)
The record boundary: valis owns the envelope, runciter owns the rdata
valis stores zone records in typed PostgreSQL columns (owner, TTL, class, type,
and the record's rdata as presentation text) and is the authority on that
envelope. It is not the authority on the per-type rdata grammar. Parsing,
canonicalizing, and rendering the rdata of each record type is runciter's job, the
sibling that owns the DNS protocol knowledge. Only the rdata crosses the boundary:
on the write path record->rdata-columns renders a record's rdata to canonical
text through runciter:render-rdata, and on the load path row->resource-record
(src/operator-state/zones.lisp) builds the resource-record envelope from
valis's own typed columns while handing the rdata text to runciter:parse-rdata
to type it. valis re-parses no resource-record syntax of its own. This keeps a
single authority for the wire grammar (the same authority the nameserver composes
through and the real wire engine encodes through), so the bytes the nameserver
serves and the bytes a zone transfer would carry cannot drift apart. The published
master-file text an operator
reads back is rendered from these same structs through runciter's canonical-line
renderer, so a round trip through the substrate is byte-faithful.
Admission: a DNS wire module is a capability decision
A DNS wire engine does not bind by being named in a config file; it binds by
passing the same owner-vouched content-hash admission gate every other pluggable
module passes. admit-and-bind-dns-module (src/edge/dns-controller.lisp)
delegates admission whole to the module-admission chokepoint (the content hash,
the owner :admit vouch, and the revocation/fence check) and introduces no new
trust path of its own. Only on a successful admission does it bind the adapter
over valis's data source; an unvouched or revoked hash is refused fail-closed, the
module loader is never called, and the adapter registry is left byte-identical to
its pre-call state.
The bind registers the nameserver's serving handler: bind-dns-view
(src/edge/dns-controller.lisp) takes valis's pg-zone-source, builds runciter's
handler over it via (make-dns-service-handler <source>) (runciter's builder
symbol resolved late by find-symbol in its package, the same late-binding idiom
the serving seam uses, so valis carries no hard import of a serving symbol), and
registers that handler under the adapter name. The serving contract version is
asserted equal to valis's +dns-adapter-contract-version+ at bind, fail-closed:
a missing builder or a version skew signals rather than binding a wrong-shaped or
absent handler. Because the handler holds its own source and pulls its own
refresh, the bind is a single registry mutation (there is no separate serving
view to offer), and it preserves the no-partial-bind discipline on that mutation:
a post-bind self-check funcalls the handler once, and on any failure
bind-dns-view restores whatever adapter held the bound name before the attempt
rather than blind-deleting it, while admit-and-bind-dns-module restores the whole
registry it snapshotted before admission. The result code from a refused admission
is the absence of a binding, not a half-built one. Handler construction runs
inside a pooled operator-state connection when the pool is armed (the eager index
build reads PostgreSQL) and bare otherwise, so an in-memory test bind needs no
database.
A decoded query therefore traces these seams in order: the wire adapter is fetched
from *dns-adapters* by name and its handler invoked with (name, type, class);
the handler is runciter's serving handler, which composes the answer over its own
index (loaded from valis's pg-zone-source) and returns (record-set, rcode);
the adapter hands that back to the wire for encoding. No port 53 and no binary
codec runs inside valis on that path, and no answer logic runs inside valis at all:
the proof drives the whole chain through the registered runciter handler in
dns-bind-resolves-query-through-runciter-handler, so what is exercised is the
production wiring through the real nameserver, not a view in isolation.
Forward scope: what is stubbed and what is open
The binary wire engine (the RFC 1035 codec, port-53 serving with response-rate
limiting and non-recursion, zone-transfer and NOTIFY, and the resolver client) is
the sibling DNS engine's responsibility and is stubbed here; the conformant wire
transcript is that engine's acceptance gate, not this substrate's. The nameserver's
index warms at handler construction (an eager build over source-origins +
zone-record-set) and is kept current by its throttled pull; valis's job is to
keep the data seam answering and to fire the change signal on commit. Deeper
wildcard closest-encloser synthesis, delegations with glue, and DNSSEC signing are
additive through a contract-version bump and the fail-closed version check,
deliberately out of the v1 flat-authoritative-zone scope.
Validation coverage: the DNS substrate
Each property is anchored to a named test. valis-side coverage proves the data
read seam, the cutover facets answerability turns on, the capability gate and its
no-partial-state unwind, and that the bind dispatches a decoded query through the
real runciter handler. The answer-composition decision tree and the
serve-last-good availability property are runciter's (composition relocated to
runciter's compose-answer parity suite [parity-checked branch-for-branch against
valis's prior zone-view-lookup before the shed], serve-last-good to runciter's
refresh suite) and are cited below so the auditor can follow the data/answer split
to its other half.
valis-side:
| Criterion | Test | File |
|---|---|---|
The PG source surfaces a held zone's cutover-state as :held |
held-zone-surfaces-held-state |
tests/cutover-status-source-test.lisp |
| A secondary surfaces its persisted absolute epoch expire-deadline | answer-as-secondary-surfaces-absolute-epoch-deadline |
tests/cutover-status-source-test.lisp |
| A primary forces a NIL deadline (it never expires) | primary-zone-forces-nil-deadline |
tests/cutover-status-source-test.lisp |
An unstored origin reads (nil nil): fail-closed |
unstored-origin-fails-closed |
tests/cutover-status-source-test.lisp |
source-origins includes held zones (the held-inclusive authoritative set) |
source-origins-includes-held-zones |
tests/cutover-status-source-test.lisp |
A held secondary commit reads :held and so is never answerable |
held-secondary-commit-reads-held-cutover-status |
tests/secondary-ingest-test.lisp |
| A vouched DNS module admits through the chokepoint and self-registers | vouched-dns-module-admits-and-self-registers |
tests/dns-admission-controller-test.lisp |
| A refused module leaves no partial adapter state | refused-dns-module-leaves-no-partial-state |
tests/dns-admission-controller-test.lisp |
| The controller-bound adapter resolves a query through the runciter handler | controller-binds-admitted-adapter-and-resolves-query |
tests/dns-admission-controller-test.lisp |
| A bind failure reverse-unwinds to the exact pre-attempt registry | forced-bind-failure-restores-pre-attempt-state |
tests/dns-admission-controller-test.lisp |
| An adapter declaring an unsupported contract version is refused at register time | dns-adapter-version-fail-closed |
tests/dns-adapter-registry-test.lisp |
| A decoded query resolves end to end through the registered runciter handler | dns-bind-resolves-query-through-runciter-handler |
tests/dns-adapter-registry-test.lisp |
Relocated to runciter (the nameserver's suite), cited for the auditor:
| Criterion | Home |
|---|---|
| The full RFC-2308 answer-composition decision tree (present/absent/NODATA/NXDOMAIN, CNAME chase, wildcard synthesis, not-authoritative) | runciter compose-answer parity suite (parity vs valis's prior zone-view-lookup) |
| A failed rebuild keeps the prior index answering (serve-last-good) | failed-rebuild-keeps-prior-index (runciter tests/dns-service-test.lisp) |
| A source outage after load keeps the nameserver answering | source-outage-after-load-keeps-answering (runciter tests/dns-service-test.lisp) |
The substrate and the engine: where the cutover logic lives
The primary-zone story above is valis answering names it authored. The substrate
also answers names it did not author (a legacy zone fed in from an upstream master
during a migration onto valis) and feeds its own zones out to downstream
secondaries. That secondary operation is a clean split between two owners. valis owns
the durable state (the zone rows, the provenance and lifecycle facets, the
crash-safe timer deadlines) and the operator door that flips a zone through its
lifecycle. The sibling library runciter owns the pure logic (whether a
transfer is admissible, what the served-type set is, how an SOA refresh-timer
advances, which cutover transitions are legal, and whether an entry may answer
right now) and the serving index that answers from it. valis never re-implements
any of that.
valis reaches the logic through a single late-binding seam,
src/operator-state/runciter-serving-seam.lisp, and never through a runciter symbol
directly. The seam declares the contract (accept-zone-transfer,
advance-refresh-timer, cutover-transition-legal-p, may-answer-p,
effective-status, out-of-set-records) and resolves each real runciter symbol at
call time, delegating directly. runciter is a hard dependency that delegates every
target live, so the seam carries no valis-side fallback: a pinned-but-missing symbol
fails closed (runciter-serving-unavailable) rather than serving a stale local answer.
The point of the seam is that the DNS-semantic vocabulary has exactly one home: the
secondary door and the cutover door speak to this package, so the day a newer
runciter ships a sharper policy the substrate picks it up with no caller change and
no second copy of the rule to drift.
The general naming database: a zone is the first record class
:names is not a DNS table that happens to be called a naming database: it is a
general naming database (a Plan 9 ndb analog) whose first record class is a DNS
zone. The cross-cutting facets that describe any named entry (who controls it versus
who authored its data [=authority-state=, provenance-kind=], the lifecycle position
[=cutover-state=], the monotonic version, and the crash-safe SOA refresh-timer
state) live on the =ndb-entry anchor (src/operator-state/ndb-entry.lisp), keyed
by (name, kind) and agnostic to the record class. The DNS-typed dns-zone
projection is the typed face of those facets for kind "dns-zone": its origin is
the anchor's name.
This is a deliberate altitude choice, not incidental layering. Modelling provenance,
write-authority, and version at the ndb-general altitude rather than as DNS-only
columns means a later record class (an identity or federation entry) attaches to
the same anchor with no schema migration. A one-time backfill seeds every existing
zone as a primary / operator anchor, so the general model is populated from the
DNS class that motivated it without a flag day.
The cutover lifecycle: held, answering, primary (operator-gated)
A migrated zone moves through three lifecycle states (:held,
:answer-as-secondary, :primary), and valis flips it only on an explicit operator
write. The two forward transitions ride the capability-gated :names /ctl control
surface (src/namespace/names.lisp): an axis-level declare-secondary verb
bootstraps a cold held zone (creating the held shell and the master allowlist entry in
foreign-key order so the very first transfer can self-authorize), then a per-zone
answer-as-secondary verb makes the zone answer from the upstream's data, and finally
promote-to-primary makes valis the system of record for it. Each verb is an ordinary
owner-gated 9P write whose clunk is the commit, exactly like a zone import.
Legality is runciter's, not valis's. commit-cutover-transition asks
cutover-transition-legal-p through the seam whether the requested flip is a legal
forward step from the persisted state; an illegal or out-of-order flip signals
illegal-cutover-transition and persists nothing, and valis never advances a zone's
state on its own initiative. A held secondary never answers: it surfaces
:held through zone-cutover-status, and the nameserver refuses to answer it; the
gate that enforces this is structural, described next, not a convention each caller
must remember.
Answerability and expiry: the safety core
Two invariants keep a non-authoritative zone from ever answering when it should not, and under the data/answer split they straddle the seam: valis owns the facts, runciter owns the gate, and the flip door owns a write-time refusal.
The data valis owns is the cutover-status pair. zone-cutover-status surfaces a
zone's (cutover-state, expire-deadline): :held for a not-yet-answerable
secondary, :answer-as-secondary with its absolute epoch-second expire-deadline,
or :primary with a NIL deadline (a primary never expires; valis is the system of
record). The deadline is persisted as an absolute instant (the ndb-entry timer
columns, written through persist-refresh-timer-state), never an uptime offset, and
the read is fail-closed: an unstored origin, and a secondary with no recorded
deadline, surface no answerable deadline. The nameserver gates on this (runciter's
may-answer-p treats a :held zone, an expired secondary, or one with no deadline
as not answering, and an expired owning zone as not owning the name at all), so a
:held or expired secondary cannot answer regardless of which path loaded it. These
choices are load-bearing: a restart must not silently extend a secondary's lifetime
past the expiry it inherited from its last transfer, and a master gone unreachable
must eventually cause the secondary to stop answering rather than serve stale data
forever. The valis side of this guarantee is verified in
tests/cutover-status-source-test.lisp (the :held, absolute-deadline, primary-NIL,
and fail-closed cases); the gate itself is runciter's may-answer-p parity.
The flip door adds a write-time refusal: a flip to :answer-as-secondary is
refused (secondary-timer-not-ready) unless a populated, non-expired deadline
already exists, checked against the live clock (*zone-view-clock*, the surviving
serve-time absolute clock in src/operator-state/zone-view.lisp, Unix
epoch-seconds), so a secondary can never even be made answerable with a stale or
missing timer.
The outbound zone-change feed: one signal off the shared commit
valis is also a source of zones for downstream secondaries, and that outbound side
is pure in-process data. src/operator-state/zone-feed.lisp holds an in-process
listener registry (*zone-change-listeners*, mirroring the DNS-adapter registry with
the same register-time fail-closed contract-version check) that fires off the shared
zone-commit point: commit-zone-records invokes its post-commit hook after a durable
commit, and the feed installs fire-zone-change onto it. Because both the operator
text-import path and the secondary-ingest path converge on that one door, a committed
change on either path emits a single change signal with no per-door wiring; a cutover
flip emits the same signal from its own commit point. A downstream secondary-peer
allowlist (zone-feed-peers, the secondary_peer table) gates the data side, so the
feed never offers a zone to an un-allowlisted peer.
The registry is fault-isolated by construction. fire-zone-change runs each listener
under its own handler so a listener that signals is warned and skipped, its siblings
still fire, and the error never unwinds out of the post-commit hook: an out-of-tree
consumer throwing cannot turn a durably committed transfer into an apparent failure.
This is strictly the data side: whether a given change warrants an actual wire NOTIFY,
and the binary AXFR-out that carries the zone, are the sister DNS engine's concern,
not the substrate's.
Boundary hygiene: what this substrate is not
The secondary substrate holds no wire and no secret. There is no port-53 serving loop,
no binary AXFR codec, and (by construction) no transport-secret or TSIG column
anywhere in its schema: every reference it stores is a public master or peer name or
address, and the transport secrets that authenticate a real transfer are the
factotum's domain (mercer), never :names's. The record boundary is the same one the
primary path observes: the typed envelope (owner, TTL, class, type) is valis's, the
per-type rdata grammar is runciter's, and only the rdata text crosses between them. A
record whose stored rdata is malformed fails closed at the node read boundary:
%render-zone maps runciter's typed rdata-parse error to a clean 9P protocol error
scoped to that one zone, rather than letting an arbitrary Lisp error escape past the
read into the serve path.
Validation coverage: secondary operation and cutover
Each guarantee of the secondary substrate is anchored to a named suite. Together they pin the authorization boundary, the held-never-serves invariant, serve-time expiry, the outbound feed's fault isolation, restart-safe deadlines, and the read-path typed guard.
| Suite | Property it pins |
|---|---|
tests/secondary-ingest-test.lisp |
A transfer is authorized on the master allowlist fail-closed (never the session); an out-of-set type is refused and the offender named; a held secondary commits and reads :held cutover-status (so the nameserver never answers it). |
tests/zone-serial-test.lisp |
RFC 1982 serial monotonicity on both the text and the secondary path: an advancing serial is honoured, a stale one refused as a regression, an equal one auto-bumps. |
tests/operator-state-fence-test.lisp |
A secondary ingest under a superseded generation writes nothing: the split-brain fence stops the write. |
tests/operator-state-zone-audit-test.lisp |
The ndb-general facet columns pass the confused-deputy audit and no operator-state table carries a secret column; the existing zone backfills as a primary anchor. |
tests/cutover-lifecycle-test.lisp |
The held -> answering -> primary flips are operator-gated and observable; an illegal or unknown verb is refused; a non-owner cannot flip; declare bootstraps a cold transfer. |
tests/secondary-timer-test.lisp |
Timer deadlines persist as absolute instants so a restart does not extend expiry; a held secondary never answers; a missing anchor signals rather than dropping the deadline. |
tests/cutover-status-source-test.lisp |
The cutover facets the nameserver gates on: a held zone reads :held, a secondary surfaces its absolute expire-deadline, a primary forces NIL, an unstored origin is fail-closed. (The serve-time gate over these [held/expired/no-deadline never answers] is runciter's may-answer-p parity.) |
tests/zone-feed-test.lisp |
The change signal fires from both ingest paths; a throwing listener is isolated from its siblings and the durable commit; only an allowlisted peer is fed. |
tests/names-import-test.lisp |
An operator zone imports, lands rows, reads back, and re-imports/refuses under the shared commit door; a fenced write commits nothing. |
tests/names-axis-test.lisp |
Writes are owner-gated while reads are open; a malformed master file and a corrupt stored record each return a clean Rerror at the node boundary. |
Privilege: unprivileged valis, external steering
The eBPF program load, the sockmap population, and the netns attach are privileged operations performed by a host agent external to valis. valis itself runs unprivileged and simply serves on the listening socket it is given. Holding no privileged kernel state is the same property that makes valis migratable: there is nothing host-specific for it to carry.
The one host-coupled obligation is a contract: a valis unit declares the designated ports it answers, and the host's privileged agent undertakes that those ports reach the unit's socket. It does not undertake to steer only those. The steer is a catchall and carries every port at the unit's address, so the narrowing is the agent's firewall rather than the declaration (the public surface). That contract is what "valis-compatible infrastructure" concretely means, and on migration it is the destination host's agent that re-establishes both halves so the arriving unit answers its ports again.
The IETF edge and the sovereign core
A core promise of the doctrine is that a person's valis answers the existing internet protocols on their designated ports (SMTP on 25, HTTP on 80 and 443, DNS on 53, IMAP on 143) and services them as a faithful, standards-compliant server. Those callers are the anonymous, hostile internet; they know nothing of valis's namespaces, capabilities, or 9P. This sits in natural tension with valis's evolution toward its own identity-first, capability-scoped, 9P-native services. valis resolves the tension by carrying the split inside one migratable unit, as two tiers that meet at the namespace:
- The IETF edge is port-addressed. A connection's recovered destination
port selects the protocol module that answers it: this is exactly what the
registry (
src/registry.lisp) does. Edge modules face hostile input and are anonymous-first: a foreign peer with no proven identity is the anonymous principal, reaching a deliberately narrow capability-scoped namespace (an inbound mail spool, a public file view). - The sovereign core is name-addressed. Native services are reached by 9P paths in the namespace, never by port. They are identity-first and grow as valis evolves.
The IETF protocols are first-class native modules, not a compatibility skin bolted onto a "real" inner system: keeping them first-class is how the doctrine's promise is kept. An edge module reaches the core only by carrying its connection's principal and capabilities into the namespace, exactly like any other module; the tier distinction is one of trust posture and hardening, not of mechanism.
The seam is 9P-shaped, so isolation is a transport choice
The hostile edge is the part eating untrusted input, so the long-term target is to run edge modules as sandboxed OS processes that reach the core only over 9P with their narrow capabilities: a compromised edge process then holds nothing but its own anonymous-scoped authority. The near-term implementation keeps edge and core co-located in one image with a logical boundary (fast to build), hardening the anonymous path first.
The durable commitment that makes this an evolution rather than a rewrite is that the edge↔core boundary is a 9P/capability interface from the start, never a shared-memory assumption. Co-location is then 9P over an in-process transport; isolation is the same 9P over a pipe between processes. This is the same discipline applied to the event multiplexer (epoll now, iouring later) and to module placement (local thread or remote node): design the seam at the eventual boundary, implement the cheap version first.
The edge↔core seam contract
The seam is a single module (src/edge/seam.lisp) and it is the only place an
edge module reaches the sovereign core. An adapter never touches the namespace
directly; it asks the seam for the view its connection is entitled to and walks
that. The entitlement is the connection's principal and its request-scoped
capabilities, carried across a real 9P interface: the same attach, walk, and
read a remote 9P client performs, with no privileged shortcut.
The seam holds one live attach per principal class, not per connection. Every anonymous connection shares a single cached attach to the public view, because they are all the same principal and opening a fresh 9P session per request would be ruinous on a busy port; a keyed principal, when keyed edge service arrives, gets its own attach through the identical mechanism. The cache key is the principal class rather than the principal object, because the anonymous sentinel and a keyed principal are the same Lisp type: only the class of authority distinguishes the slots.
A second, deliberately separate path serves the request-scoped phase: the
with-request-capability entry point mounts a capability surfaced inside a
request into a fresh private per-request view, runs the request body bound to
that view, and unmounts on the way out, including on a non-local exit. The
private view is not the shared cached attach, and the reason is a concrete leak
it forecloses: mounting into a shared view would make one connection's
request-scoped grant briefly visible to every other connection on the same
principal. A per-request view is structurally incapable of that cross-connection
disclosure: the mount lives and dies inside one request's dynamic extent.
The transport under the seam is a rebindable factory: *edge-channel-factory*
is a function returning the two endpoints of a channel, defaulting to an
in-process channel served against the assembler root. Rebinding it swaps the
transport with no change to any adapter: the co-located case (an in-process
channel), a TCP loopback to the fabric's own 9P listener, and a real
inter-process socket pair have all been driven through this one point, each
serving the same publication byte-for-byte. An adapter names no transport symbol
at all; it imports only view-for from the seam. That import boundary is what
makes "co-located now, sandboxed later" a binding rebind rather than a rewrite.
The anonymous-grant lifecycle and self-healing
The public edge runs on exactly one bearer grant. At fabric start
(src/fabric.lisp), start-fabric mints a single edge-wide capability name
designating the public view with read-only rights and installs it as the
anonymous grant; the seam attaches against it on first use. There is no
per-adapter grant and no per-connection minting: one name is the entire public
authority, and an auditor can find its single mint site and its single
designation.
Because that one attach is long-lived, the seam has to survive losing it, and it discriminates why the view was lost before deciding what to do. The discriminator is membership in the revocation store. If the grant's hash is in the store, the loss was a deliberate unpublish: the view goes dark and stays dark (walks fail, adapters serve absence) until the operator re-establishes the public view. The seam never silently re-mints a revoked grant. Any other loss (a closed session, a dropped channel, an expired grant) self-heals: the heal path consults the revocation store first and re-checks it on every retry (so a revocation landing mid-heal still wins), re-attaches with a bounded inline retry, and re-mints through the grant's mint hook only on genuine expiry. The re-mint can therefore never resurrect a revoked grant, because the store is consulted before the mint is reached.
Sovereignty on the wire is the absence of a name, not a policy refusal. When an anonymous caller names anything outside the public view, the walk simply fails, and the adapter surfaces that failure as the protocol's own absence response: an HTTP 404-class answer, a Gopher type-3 error item. There is no "permission denied" code path at the edge to disclose that a thing exists but is forbidden; the owner's axes have no name in the anonymous view, so the only honest answer is that they are not there.
Edge wiring and the fail-closed bind order
The edge controller (src/edge/controller.lisp) binds and unbinds the edge
ports through start-edge / stop-edge, riding the listener's already-running
event loop rather than owning a listener of its own. Dispatch is purely
port-addressed: a connection's recovered destination port selects its protocol
module through the registry, exactly as the central abstraction promises.
The bind order is fail-closed, and the gate is a real resolution, not a presence
check. Before start-edge binds a single port it asserts the fabric is up, the
anonymous grant is wired, and a real view-for for the anonymous principal can
attach and walk the public view through the seam. A grant string that decodes but
no longer resolves (revoked, or pointing at a view that has gone dark) fails
this gate, so the edge refuses to come up serving a public face that cannot
actually be reached. No namespace means no open port. Binding a port here opens it
only for dispatch, though: whether that port is reachable from off the host is a
separate, privileged decision at the host agent's firewall, not a consequence of
the edge binding it: see The public surface. Teardown runs in strict
reverse: stop-edge closes the ports before the seam resets, so the wire stops
feeding a session before that session is torn down, and a mid-start failure
unwinds whatever bound in reverse order.
The one construction point a later phase rebinds is named and isolated:
*edge-source-constructor* builds each port's event source, defaulting to a
plain bind, and is the seam where a steered file descriptor handed over by the
privileged host agent arrives. The resident boot already rebinds it
(src/main.lisp), so the edge binds a source around a passed-in fd instead of
opening its own, with nothing else in the controller changing. Alongside the wiring, the controller publishes a read-only /edge
status subtree (src/namespace/edge.lisp): a directory per bound port carrying
live connection, budget, and deadline-cut counters rendered fresh on every read,
plus an anonymous file reporting whether the public view is unwired, dark,
undetermined, or serving. Undetermined is kept distinct from dark because the
two want different repairs: dark means the grant was found revoked and is
answered by minting a fresh one, while undetermined means the node could not
establish whether the grant is revoked at all, refuses to serve on that, and
would be no better off with a fresh grant. The subtree is observable as
ordinary files, and it is named only in the
owner's canonical frame: an anonymous attach cannot walk to /edge at all, so
the diagnostic surface is itself sovereign.
The edge abuse posture
The edge faces the open internet, so abuse defense is layered, and the layering is deliberate: each defense lives at the tier that can express it most cheaply, and none of it lives in the sovereign core.
At the adapter tier, each connection carries a read deadline and a hard cap on input size: a client that connects and sends nothing, or dribbles bytes slower than the deadline, cannot hold a handler thread, and an oversized request line is refused rather than buffered. At the edge tier, each bound port carries its own concurrent-connection budget. When a port reaches its budget the controller stops accepting on that port: it deregisters the listening source, so overflow connections wait in the kernel's listen backlog and no valis thread or write is ever spent refusing an over-budget client. A connection's whole lifetime is wrapped in one place in the controller (never in adapter code), so the live count is decremented on every exit path (normal, error, or deadline cut) and the port re-arms its accept as soon as a slot frees. A flood on one protocol's port cannot starve another, because the budgets are per-port. The third tier (per-IP tracking, rate limiting, and volumetric or amplification defense) belongs to the privileged host agent that sits in front of valis, outside the process entirely.
The rationale for the split is the sovereignty invariant. Per-IP and volumetric defense need a view of the network the unprivileged, migratable valis process deliberately does not hold, and the sovereign core must carry no abuse logic at all: the core moves and mounts views, it does not reason about hostile traffic. Pushing the outermost defenses to the host agent and keeping only connection-occupancy and per-port budgets inside valis keeps the core clean while still bounding what a single open port can consume.
Connection lifetime: what the edge assumes, and what it no longer does
Nearly everything the edge serves today is request shaped: a client connects, says what it wants, is answered, and goes away. The connection exists to carry one exchange and has no reason to outlive it. For that traffic the shape described above is the right one and is deliberately unchanged. A thread per connection is cheap when the thread lives as long as one short exchange; a per-port concurrency budget is a true measure of work in flight when every live connection is work in flight; and a wall-clock deadline is a sound liveness test when silence past the deadline means a client that was never going to finish.
A protocol whose connections are long lived, mostly idle, and written to unprompted breaks all three at once, and it breaks them for one reason rather than three. The thread cost, the per-port budget and the one-shot deadline read as separate limits, but each is an expression of a single assumption nobody had written down: a connection lives exactly as long as the request that created it. The table below states that assumption and its relatives plainly and disposes of each. The disposition is rarely a removal. Most are SPLIT: the request-shaped path keeps the shape it has, and a protocol that declares itself continuous gets the other one. A protocol declares which it is; the seam never guesses.
| Assumption | Where it is made | Disposition | Reason |
|---|---|---|---|
| A connection lives exactly as long as the request that created it | src/protocol.lisp:52, where handle-connection runs to completion and the exchange ends with it |
REPLACED | A continuous session is opened by an exchange it outlives, so its lifetime belongs to the session rather than to a handler's dynamic extent. |
| Each accepted connection costs one OS thread for its whole life | src/edge/controller.lisp:312, one thread spawned per accept |
SPLIT | Kept for a request-shaped connection, where a thread that lives as long as one short exchange is the cheapest place to hold its state. Replaced for a continuous session, whose thread is idle for nearly all of its life, so the cost is paid per connection instead of per unit of work. |
| Only a listening socket is an event source | the two source classes in src/backends/epoll.lisp (:350, :456) |
REPLACED | An established connection becomes a registered source as well, which is what lets the loop drive it without a thread of its own. |
| Readiness means a connection is waiting to be accepted | source-ready in src/multiplexer.lisp:171 |
REPLACED | Readiness now also means an established connection has bytes to read or room to write, so one notification serves two kinds of source. |
| A source is monitored for readability only | the :read registration in src/backends/epoll.lisp:268 |
REPLACED | A queued write that could not complete needs writability notification to finish; without it, a slow reader can only be served by parking a thread on it. |
| Output needs no buffering because a handler writes straight to the stream | no output queue exists anywhere in the edge layer: src/protocol.lisp:52 hands the adapter the connection with nothing in between |
REPLACED | A queue is the only place a bound can be enforced, so the queue has to exist before a cap can be placed on it. Without one, a peer that reads slowly is served out of whatever the handler happens to allocate, which is bounded by nothing. |
| Liveness is a wall-clock one-shot armed by the handler | set-read-deadline in src/connection.lisp:249 |
SPLIT | Kept for a request-shaped exchange, where silence past the deadline really does mean a client that will not finish. Replaced for a continuous session, where silence is the normal state and a clock alone cannot separate a healthy idle peer from a dead one; there the exchange itself has to supply the evidence. |
| A live connection is in-flight work, so one per-port count bounds abuse | the default budget at src/edge/controller.lisp:158 |
REPLACED | A session that is idle by design holds a slot while consuming nothing, so a count of live connections stops measuring work and starts refusing legitimate peers long before any resource is scarce. |
| Bytes move through the connection's buffered stream view | connection-stream at src/connection.lisp:193, vending the accepted socket's own buffered stream |
SPLIT | Kept for a request-shaped exchange, where reading ahead is exactly what makes line and header parsing cheap. Replaced for a connection driven from the event loop: the buffered view reads ahead into a userspace buffer, so bytes already drained out of the kernel are invisible to readiness notification, and a connection driven from the loop must therefore never take that view. |
| The worker closes the connection when its handler returns | src/executor.lisp:107 and src/edge/controller.lisp:263 |
SPLIT | Kept where the handler's return really is the end of the connection. Replaced for a session that outlives the exchange which opened it: closing becomes an explicit act ordered after the last queued bytes have left, and the party that closes is whoever ends the session, not whoever ran the last handler. |
| Every adapter can be driven the same way, because one budget executor serves them all | src/edge/tls-serve.lisp:450, src/edge/mail-serve.lisp:286, src/edge/dns-serve.lisp:1005 and the controller's own ports all build the same executor |
REPLACED | An adapter is driven from the event loop only if it positively declares that it drives the raw descriptor and hands nothing above it. Anything that terminates, decodes or wraps the stream keeps the thread shape, because readiness is asked of the descriptor while the wrapper holds transformed bytes above it, and the two disagree. This is a safety property rather than a performance note: on a port where the stream is terminated, driving the raw descriptor would serve the connection in cleartext. |
| A protocol module never holds a socket or a descriptor | the module contract itself, held by every adapter and by no single file | KEPT | This is what makes a module killable and rebuildable from durable state plus a client reconnect. A module that held a descriptor could not be rebuilt without the connection dying with it, and the substrate would stop being movable. |
| Every outbound connection leaves through the one dial verb | the single egress seam | KEPT | Nothing this work adds opens a socket of its own. A continuous session's own reconnection, where it has one, is an outbound connection like any other and leaves the same way, so the egress policy that governs the process keeps governing it. |
The dispositions above leave a set of design questions that each have exactly one answer. They are recorded here so the implementation does not re-decide them one file at a time.
- Where the send-queue cap is tested
- The cap is tested against the length the
queue would have once the message is added, before the message is buffered.
The reason is arithmetic rather than taste: testing the already-queued length
bounds the queue at the cap plus one maximum message, and no maximum message
exists on this path, so that placement bounds nothing at all.
src/transport/client-secure-endpoint.lisp:148is the precedent already in the tree, rejecting an over-long declared length before any body is read. - A message larger than the whole cap
- A single message that exceeds the entire cap is refused outright rather than split across several sends. The reason: splitting does not reduce the bytes the queue holds, and the seam has no producer it can pause part way through a message, so splitting would relabel an unbounded hold rather than bound it. That refusal is reported distinctly from an ordinary over-budget refusal, because an over-budget send can succeed on a later attempt once the queue drains and this one can never succeed at all.
- Who closes on a breach
- Closing a connection that breaches the cap is the edge's act and never the protocol module's. The reason: the queue and the cap belong to the edge, and a component cannot enforce a bound it does not hold. What this leaves with the module is an obligation rather than a power. It must notice that a send was rejected, and must never assume a send lands.
- The cap is not the only backpressure
- A bulk producer can read how full the queue is and suspend itself at a fraction of the cap, which makes the close a last resort instead of the only one. The reason: a producer that can pause is far cheaper to stop than a connection is to lose.
- A registration timeout is not an idle timeout
- These are two different mechanisms and are kept apart deliberately. The registration timeout runs from the moment the connection was established and is not reset by activity, while the idle timeout runs from the last activity and is paired with a protocol-level probe that asks the peer to prove it is still there.
- A session with no liveness probe
- A session whose protocol offers no way to probe the peer gets no idle cut at all, and only the registration timeout applies to it. The reason has to be stated out loud because the opposite is the tempting default: cutting an idle session that cannot be probed reinstates the wall-clock cut on exactly the connection that is idle by design, which is the mechanism this work exists to replace. Such a session ends when its peer goes away or when the edge tears its port down, never on a clock.
- The send path copies
- What a caller hands the send path is copied at the moment it is handed over, and never retained by reference. The reason: the per-session read buffer is reused on the next read, and a short write leaves queued bytes waiting across callback boundaries, so a caller echoing straight from a read into a send would otherwise ship whatever arrived next. The cost is one allocation per enqueued message, and it is reported as a measured per-unit figure rather than left implicit.
- Every sizing constant here is provisional
- Each sizing constant this seam introduces says so in its own docstring, and each is derived from a bound measured in this tree rather than adopted from another implementation. The reason: asymptotic shape is a design-time decision while constants come from profiling this system, so a number borrowed from a daemon carrying different message sizes would be a guess wearing the authority of someone else's measurement.
What the shape costs, per unit
A design that cannot say what one unit costs cannot say how many units a machine holds, and that question was among the grounds on which the dispositions above were chosen. The figures below are what the shape actually costs. They are produced by a named probe in the suite rather than reasoned from the source, and that probe reports every reading whether or not it asserts on it.
| Unit | Measured cost |
|---|---|
| One instance at rest, a loop up and no sessions | 132,603,808 bytes resident |
| Bytes retained per idle session | 4,802, of which 4,096 is the session's own read buffer |
| The same figure gross, with the peer end held in the measuring image | 6,488 |
| OS threads per idle session | 0, against one per connection on the request-shaped path |
| The two liveness clocks a session arms | 504 bytes per session at 16 sessions, 536 at 32 |
| Allocation per queued and drained message, 256 octets of payload | 304 bytes |
At 4,802 bytes a session, ten thousand quiet peers hold about 46 MiB of session state. That is why the per-port ceiling on live sessions is a density decision rather than an abuse bound, and it is the figure a sizing pass should start from.
The per-message figure includes the copy the send path makes of the caller's bytes at the moment it accepts them. That copy is the decision recorded above, not queue overhead, and a sizing pass that read the figure as pure overhead would re-derive a question this document has already answered.
Conditions, without which none of the figures means anything. They were taken
at commit f464a62, in a freshly started image with a warm compilation cache,
nothing else running in it, one process at a time. The instrument's own noise
floor, measured first from repeated readings of the instance at rest, is 256
bytes, and every per-unit figure above stands at least thirty times clear of
it. Each figure is the median of three readings, and each session reading is a
delta against an at-rest reading taken moments before it rather than against a
baseline read at the start of the run. The figures move by tens of bytes
between runs, so a reading differing from these in its last two digits is the
same reading.
- What the probe detects, and what it does not
- It detects a per-session cost that grows worse than linearly as sessions accumulate. That is the defect invisible both to reading the code and to a green suite, and it is the one demonstrated: an injected cost growing with the number of sessions already adopted drove the cost of a doubling to 5.90 and 4.64 times against an asserted ceiling of 3, and the probe went red. It does not detect a constant that simply grew. Raising the per-session read buffer sixteenfold moved the per-session figure from 4,816 bytes to 66,288 and left every ratio at 2.00, so every assertion stayed green while the cost rose by a factor of fourteen. A grown constant shows only by comparing the figures one run reports against an earlier run's, which is why the probe prints every reading rather than only asserting on it.
- How far the probe has been falsified
- The assertions do not all carry the same weight, and the difference matters more than the figures do. Two have been observed failing: the ceiling on the cost of a doubling, and the whole thread-growth control, the latter by adopting the connections so that no worker was ever spawned, which is exactly how the instrument would stop discriminating in practice. The liveness and queued-message bands, and every structural assertion, have only ever been observed passing. Two are weaker still and are named here so that nobody credits them with more than they do. The lower bound on a doubling can fail only if a doubling costs less than 1.25 times as much, which no defect in this seam produces, so it is a rail against a broken measurement rather than a defect detector. And the liveness bound compares a figure against twice one measured moments earlier in the same test, so a defect inflating both counts equally passes it: it catches per-unit growth between 16 and 32 sessions and nothing else.
- What would falsify these figures
- A run of the same probe at the same commit, in a freshly started image with nothing else live, reporting a per-session cost outside the band the measured noise floor allows.
- What is not covered at all
- The cost of a namespace read parked on behalf of a session. That belongs to the namespace layer and was measured there, in an image rather than over a socket held open for a long period, so nothing here says what such a read costs once the connection under it has been quiet for hours.
- Every sizing constant here is still provisional
- None of them has been profiled under load, and the figures above are what a later profiling pass will move them against. The per-session read buffer of 4,096 octets, the send-queue cap at four times the carrier frame ceiling, the suspend fraction of one half, the idle, probe and registration intervals of 120, 30 and 30 seconds, and the per-port ceiling of 400 live sessions are each provisional and each says so in its own docstring. A number recorded with no statement of what it is provisional against reads as settled to the next person.
Validation coverage: the edge↔core seam
Each edge success criterion and security property is anchored to a named test an auditor can run. The acceptance suite drives plain loopback TCP clients against fully shipped wiring; the seam and controller suites exercise the seam internals and the budget machinery directly.
| Criterion | Test | File |
|---|---|---|
| Recovered destination port alone selects the protocol module | port-alone-selects-protocol-module |
tests/edge-acceptance-test.lisp |
| An anonymous caller names nothing outside the public view; absence is the wire answer | anonymous-names-nothing-outside-public-view |
tests/edge-acceptance-test.lisp |
| The same adapter serves byte-identical over a second transport (TCP loopback) | same-adapter-serves-over-second-transport |
tests/edge-acceptance-test.lisp |
| The same adapter serves byte-identical over an inter-process socket pair | same-adapter-serves-over-socketpair-transport |
tests/edge-acceptance-test.lisp |
| Adapters name no transport symbol; the transport split is seam-internal | adapter-code-unchanged-between-transports, adapters-name-no-transport-symbols |
tests/edge-acceptance-test.lisp, tests/edge-seam-test.lisp |
| A request-scoped capability never leaks across connections on the shared principal | request-capability-never-leaks-across-connections, request-capability-unmounts-on-nonlocal-exit |
tests/edge-seam-test.lisp |
| A revoked grant stays dark; loss self-heals; expiry re-mints only after the store check | revoked-grant-stays-dark, view-loss-self-heals, expired-grant-re-mints-after-store-check |
tests/edge-seam-test.lisp |
| The edge refuses to bind without a resolvable public view | start-edge-refuses-without-namespace, start-edge-refuses-dark-anonymous-view |
tests/edge-controller-test.lisp |
| A mid-start failure unwinds every bound port | mid-start-failure-unwinds-bound-ports |
tests/edge-controller-test.lisp |
| Teardown closes ports before the seam resets | stop-edge-closes-ports-before-seam |
tests/edge-controller-test.lisp |
| The connection budget stops accepting at the cap; a queued connection is served when a slot frees | budget-cap-stops-accepting, queued-connection-served-after-re-register |
tests/edge-controller-test.lisp |
| The live count is restored on every exit path and never leaks a slot | counter-decrements-on-error-exit |
tests/edge-controller-test.lisp |
The /edge status surface reports live counts and the anonymous dark state, and is unnameable anonymously |
edge-status-reports-live-counts, edge-status-tracks-anonymous-dark-state, anonymous-view-cannot-name-edge-status |
tests/edge-controller-test.lisp |
The owner-proof flow: a live signature, never a token
The edge faces the anonymous internet, but one caller on the same port is not
anonymous: the owner. The owner reaches a management view by proving possession
of the custody master key on every request, not by presenting a token. Each
request the owner makes is an HTTP Message Signature (RFC 9421): the owner signs
the canonical request material with the same Ed25519 custody key that roots all
valis authority, and the edge verifies that signature against the owner's public
key. This is the deliberate choice that separates owner authentication from the
anonymous grant. A bearer token is replayable by anyone who holds it (which is
exactly the anonymous /pub model, an unscoped grant pointed at a public view),
and pointing a bearer model at the management view would let a token-holder act
as the owner without ever holding the owner's key. A live per-request signature
cannot be lifted and reused, and it carries no server-held session state to leak
or to outlive its purpose: the proof is the key-possession itself, re-demonstrated
each time.
Verification routes through the same single seam the rest of identity admission
already uses. The authenticate generic in src/identity/authenticator.lisp is
the one named operation that turns a completed proof into a principal; the Noise
handshake is the first method on it and the anonymous resolution is the second.
The seam was built reserving exactly a third method for an HTTP-edge proof, and
this is what fills it: a signature authenticator specialises authenticate to
mint the owner principal from a valid signature, with no code above the seam
changing: the connection layer, the capability layer, and the namespace mounter
still see only a principal and never learn what proved it. The verifier itself
lives in src/identity/http-signature.lisp: it reconstructs the RFC 9421
signature base over the covered components (the request method, the request
authority, and the request path) through the same base builder the owner's
signer used, so the bytes signed and the bytes verified are identical by
construction and cannot drift apart into a canonicalization mismatch. It checks
the signature's key identifier against the owner DID and verifies the Ed25519
signature against the owner public key.
The custody boundary holds at the edge exactly as it does everywhere else: the
edge module relays only the signed request material (the method, the authority,
the path, and the two signature header values) to the seam, and never sees a key.
Verification needs only the owner public key, which is already in the image on
the factotum side; the private custody key never crosses into the edge tier, and
no ambient authority is granted to a module that eats hostile input. The replay
defence is twofold and lives with the verifier: a bounded acceptance window over
the signature's created / expires stamps rejects a signature presented too
early, too late, or long after it was minted, and a bounded, expiry-pruned nonce
cache rejects a second presentation of a nonce already seen inside the window. A
signature that is absent, malformed, signed by the wrong key, expired, or replayed
does not raise an error and does not reach management: every such failure resolves
to the anonymous principal, so the management view is reachable only by a valid,
live, first-use signature. Falling open to anonymous, never to an error and never
to elevated access, is the fail-closed posture carried to the auth path.
Identity-selected views on one URL space
Anonymous and owner walk the same URLs on the same port. What differs between
them is not the path they ask for but what is mounted under it. The anonymous
caller resolves to the published view: the /pub scope, the same public face the
anonymous grant has always served. The owner, once a valid signature has resolved
to the owner principal, resolves to the full sovereign canonical frame
(/pub /id /cap /edge /bus) rendered as a browsable read projection. The
discrimination happens at one site: view-for (src/edge/seam.lisp) hands every
non-anonymous principal to build-base-view (src/namespace/assembler.lisp),
which dispatches the owner DID to assemble-canonical-frame server-side; the seam
re-implements no identity comparison of its own. The HTTP adapter then walks an identity-dependent
root: the anonymous request walks under the /pub scope, the owner request walks
the canonical frame directly, because for the owner that frame is the root. The
principal is re-resolved for each request and never cached on the connection, so
under a persistent connection a signed request and a later unsigned request on the
same socket each resolve their own principal and their own view; a signed request
cannot infect an unsigned one that follows it.
This is sovereignty expressed as absence, carried now to the HTTP wire. An anonymous caller does not receive a permission refusal when it names a management axis: it has no name for one. The management names are simply not mounted in the anonymous view, so a walk toward them fails the way a walk toward any non-existent name fails, and the adapter surfaces that failure as the protocol's own not-found answer. There is no code path at the edge that says a thing exists but is forbidden, because such a path would itself disclose that the thing exists. The honest answer to an anonymous caller reaching for the owner's axes is that, in its view, they are not there.
The HTTP/1.1 conformance surface
The published view is served by a real HTTP/1.1 server, not a thin compatibility
skin, so an ordinary client treats valis as an ordinary origin. The surface is
deliberately bounded to the read path. A Host header is required and validated:
an HTTP/1.1 request that omits it is answered 400, per the messaging
specification. GET and HEAD are served, with HEAD returning the computed
headers and no body; any other method is answered 405 with an Allow header.
Connections are persistent, framed by Content-Length, with Date and Server
on every response and the real status taxonomy
(400 / 404 / 405 / 408 / 411 / 505 alongside 200 / 304).
Conditional requests are answered from the substrate's own version stamp: the 9P
qid.version of the addressed leaf becomes a strong ETag, and an If-None-Match
that matches it yields 304 Not Modified with the validator and no body. That
mapping is the substrate-alignment win: an ordinary HTTP cache works correctly
against the sovereign namespace because the namespace already carries the exact
per-node version a validator needs. That version is now derived from the node's
content score (the low 32 bits of its Merkle hash; see the
read-side caching tier), so the ETag is itself a
content hash rather than an opaque counter, the content-addressed norm a strong
validator is meant to be. A body mutation changes the leaf's score, so its
ETag changes with it, and a conditional GET that no longer matches gets a
fresh 200 carrying the new validator. The write-path long tail of HTTP/1.1
(chunked request bodies, =Expect=/100-continue, request pipelining) is out of scope
by design: it bites almost entirely on request bodies and writes, and this is a
read server.
The conformance surface composes cleanly with the edge abuse posture documented above rather than weakening it. A persistent connection holds exactly one of its port's concurrent-connection budget slots for its whole lifetime, while the per-request read deadline re-arms at the top of every request on that connection. The two together give the right behavior under keep-alive: a slow or dribbling client is cut by the deadline on the request it is starving, without a healthy persistent connection being dropped between requests, and the per-port budget continues to bound a flood of connections regardless of how many requests each one carries. The existing guards remain in force on the new surface as well (every response header value passes through the response-splitting guard before it is emitted, and every request path passes through the traversal-safe decode before it is walked), so the broadened method, header, and conditional-request handling adds no new injection or traversal surface.
Validation coverage: the HTTP edge
Each HTTP-edge criterion is anchored to a named test an auditor can run. The conformance cases drive plain loopback HTTP clients against fully shipped wiring; the owner-proof cases drive an in-image RFC 9421 signer over the same loopback; the signature unit verifies the cryptographic round-trip directly.
| Criterion | Test | File |
|---|---|---|
A faithful HTTP/1.1 GET returns a well-formed response over the published view |
ordinary-get-returns-wellformed-http11 |
tests/http-conformance-test.lisp |
HEAD returns the headers with no body |
head-returns-headers-no-body |
tests/http-conformance-test.lisp |
A Host-less HTTP/1.1 request is answered 400 |
missing-host-is-400 |
tests/http-conformance-test.lisp |
| Keep-alive serves two requests on one connection, each re-arming its deadline | keep-alive-two-requests-one-connection |
tests/http-conformance-test.lisp |
Conditional GET (If-None-Match against the qid.version ETag) yields 304 |
conditional-get-if-none-match-304 |
tests/http-conformance-test.lisp |
A GET after a body mutation returns 200 with a fresh, differing ETag |
conditional-get-200-after-mutation-bumps-etag |
tests/http-conformance-test.lisp |
An unsupported method is answered 405 with Allow |
post-returns-405-with-allow |
tests/http-conformance-test.lisp |
| Same port, identity-selected views: anonymous reaches the published view, a valid signature reaches management | same-port-anonymous-vs-signed-views |
tests/http-owner-proof-test.lisp |
| The anonymous byte stream names nothing of the management axes; absence is the wire answer (404, not 403) | anonymous-names-nothing-of-management |
tests/http-owner-proof-test.lisp |
| A valid owner signature reaches the management view | valid-signature-reaches-management |
tests/http-owner-proof-test.lisp |
An unsigned, invalid, expired, or replayed request resolves to anonymous; management is absent and /pub still reads |
unsigned-request-is-anonymous, invalid-signature-is-anonymous, expired-signature-is-anonymous, replayed-signature-is-anonymous |
tests/http-owner-proof-test.lisp |
| The signature base round-trips: the signer and verifier reconstruct identical bytes and a valid signature verifies | signature-base-round-trip, verify-accepts-valid-rejects-tampered |
tests/http-signature-test.lisp |
| The replay window rejects a replayed or stale signature | replay-window-rejects-replay-and-stale |
tests/http-signature-test.lisp |
The steered-fd handoff: explicit consent, not reach-in
The fail-closed bind order above names *edge-source-constructor* as the one
construction point that is rebound to a steered descriptor. That descriptor
arrives by a deliberate trust choice, and the choice is the load-bearing security
property of the whole steering arrangement: valis hands a descriptor out; the
agent never reaches in.
valis creates and owns its listening socket (it is the only party that calls
bind and listen), and the socket reaches the LISTEN state inside valis's own
unprivileged process. To let the privileged host agent steer traffic to that
socket, valis sends the agent a duplicate of that one descriptor as an
SCM_RIGHTS ancillary message over the very Unix-domain connection the 9P
control session already runs on. There is no second channel: the same
AF_UNIX transport carries both the in-band 9P frames and the out-of-band
descriptor. valis chooses exactly which descriptor crosses the boundary, and
nothing else of valis's descriptor table is exposed. This is consent expressed
as a capability: possession of the passed descriptor is the entire grant.
The rejected alternative is what makes the choice legible. The kernel also offers an agent-pull path: the privileged process could open valis's process descriptor and lift a descriptor out of its table directly. That was refused on purpose: it would grant the agent the standing authority to reach into an unprivileged process and take any descriptor, a far broader and more durable trust than "valis decided to send you this one socket." The push model grants the minimum; the pull model grants a reach-in. The seam where the inherited socket lands wraps it without re-binding (an already-LISTEN descriptor cannot be bound again), so the steered source is the same event source the bind path builds, recovering the dialed destination port exactly as the ordinary listener does (see Edge wiring and the fail-closed bind order, and the four-tuple preservation the listener relies on in the steering layer).
The descriptor crossing is also strictly sequenced against the 9P stream, not concurrent with it. valis declares its port set over 9P first; the control service advances a handoff gate to a ready state only after a valid declaration; then the standalone descriptor message crosses while no frame is in flight. The order is what keeps the zero-data ancillary message from racing the length-framed 9P bytes on the shared socket, and it is why the agent never holds a descriptor it has no port contract for.
The per-namespace steering constraint
sklookup selects a socket per network namespace. The kernel runs the steering program only for traffic in the namespace the program's link is attached to, and it will only select a socket that lives in that same namespace. valis's listening socket and the agent's link must therefore share one namespace, or the steer silently never fires: load succeeds, the socket map is populated, the link attaches, and connections are still refused, with nothing in any log to point at.
This constraint is also the unit's boundary of authority, though it bounds what the steer can offer rather than what the unit answers. The steering program is a catchall: it fans every port on the unit's IP out to valis's one socket, so the namespace and the IP it carries fix the traffic that can be offered at all. That is not the answered set. A valis unit declares the ports it intends to serve, and on a routable IP the catchall does not narrow what it offers to that declaration. What narrows the reachable set back to the ports the agent admits is a separate control, the agent's default-deny firewall inside the namespace, and not the namespace itself. On a loopback-only island the two coincided and the namespace was the whole boundary; once the namespace carries a routable address they are two mechanisms, and the surface control is the firewall. This is the subject of The public surface, including why the allowlist must stay the agent's and never a module's. The first-cut topology resolves the shared-namespace requirement the simplest correct way: the agent creates the namespace and the unit's IP, then valis is launched into that namespace so its socket is born in the right place.
Privilege drawn as a repository boundary
Every capability that needs the kernel's trust lives in the external host agent
and only there. CAP_BPF to load and attach the steering program, and
CAP_NET_ADMIN (with CAP_SYS_ADMIN) to create the namespace and assign the
unit's address, are held by the agent, a separate component in a separate
repository. The migratable valis unit holds none of them. The privilege boundary
is therefore not a runtime check that could be misconfigured; it is a boundary
drawn in the filesystem, where an auditor who wants to know what runs with
elevated privilege reads one repository and a compromised valis unit cannot reach
for authority it was never granted.
This is the same property described in
the edge abuse posture (the outermost, network-aware
defenses belong to the agent because the unprivileged unit deliberately cannot
see what they need), stated here as a privilege invariant rather than an abuse
one. The steered socket needs only LISTEN state, nothing privileged: valis binds
loopback or an ephemeral high port inside the agent's namespace and never binds
the designated public ports. That valis holds no privileged kernel state is the
very property that makes it migratable, because there is nothing host-specific for
it to carry; on migration the destination host's agent re-establishes the steer so
the arriving unit answers its ports again. The dialed port survives the steer
because the kernel preserves the original four-tuple under bpf_sk_assign, so
getsockname on the accepted socket recovers the real port the client dialed,
the same recovery the listener already proves over loopback (see
the fail-closed bind order and the four-tuple
commitment in the listener section).
Validation coverage: the steered-fd handoff
The steering data path cannot be exercised from the Lisp image, since it needs a
real kernel, a network namespace, and CAP_BPF=/=CAP_NET_ADMIN. The image-testable
glue (descriptor framing, port declaration, the seam's recovered-port wrap) carries
cold in-suite coverage. Past that there are two out-of-band gates rather than one,
and they cross different boundaries. make steer-test runs in the agent's
repository on the privileged side, signed off out of band exactly as the v9fs mount
interop test is. The proving ground's steered-delivery gate builds a node and then drives
it from a separate host, which is what puts a real off-host client on the far side
of the steer and lets it read the delivery off accept queues rather than off an
answer.
The agent repository's docs/steering.org carries the same coverage table from the
privileged side.
| Criterion | Test | File |
|---|---|---|
| The steered seam wraps an inherited LISTEN descriptor and recovers the dialed port | steered-fd-recovers-dialed-port |
tests/steered-fd-test.lisp |
| A received descriptor is validated as a LISTEN-state socket before it is steered | received-listen-fd-is-validated-and-returned, non-listening-fd-is-rejected-fail-closed |
fulcrum tests/handoff-test.lisp |
| The descriptor handoff is refused until a valid port set is declared (sequencing) | handoff-refused-before-ready-for-fd, handoff-advances-only-after-valid-port-set |
fulcrum tests/handoff-test.lisp, fulcrum tests/control-test.lisp |
| A declared port set is recorded exactly; an out-of-range port is rejected fail-closed | valid-port-set-is-recorded-exactly, out-of-range-port-is-rejected-fail-closed |
fulcrum tests/control-test.lisp |
| The full steer lands the connection, recovers the dialed port, proves valis never bound it, and confines the steer to the namespace | make steer-test (operator-run privileged gate) |
fulcrum spike/steer-test.{sh,lisp} |
| An off-host client's connection is delivered through the steer and dispatched on the port it dialed, including a port named in no configuration on either side | steered-port-delivery scenario (operator-run gate) |
proving-ground/driver/scenarios/steered-port-delivery.lisp, proving-ground/drive-steered-delivery-gate.sh |
| TCP on the served port enters the steered socket's accept queue while the inherited listener's queue stays at zero, which is what separates the two hypotheses a DNS answer cannot | steered-port-delivery scenario (operator-run gate) |
proving-ground/driver/scenarios/steered-port-delivery.lisp |
The substrate as namespace
The doctrine's eight categories are not eight flat peers; they are a dependency stack, exposed as subtrees of the user's 9P namespace that protocol modules reach through their capabilities.
Semantic objects, not protocol silos
The substrate models the semantic object (a message, a contact, a publication, a contract), and the wire protocols are projections over it. This is the mechanism behind the doctrine's claim that "messaging is one thing": SMTP, IMAP, and NNTP are not three stores but three skins over a single message substrate; a message is the same object whether it arrived by SMTP or was posted by NNTP. Publishing (web, forum, and social protocols) likewise projects one publication substrate. The substrate de-duplicates reality; protocols are views onto it. This is how the protocols are "given back": the user owns the object and chooses which protocols answer for it.
A protocol module (the edge tier) is therefore a wire-format adapter between an IETF protocol on its port and a semantic subtree of the namespace, gated by the connection's capabilities. The substrate is protocol-agnostic; the protocol is storage-agnostic; they meet at the namespace.
The dependency stack
- Foundation - crypto (1), names (2), identity (3). Every other category reaches these. Crypto is the autonegotiated, invisible-by-default layer the capability model already rests on: the factotum-shaped agent, content encryption, transport security, key custody. Names are the user's sovereign namespace of people, places, and things; identity is the user's own, canonical and transient. Identity is where the principals the connection layer authenticates come from, and names and identity are who capabilities are granted to.
- Data - data (4): the relational and reactive store over the migration backing store; the queryable substrate the semantic domains build on.
- Semantic domains - messaging (5), publishing (6), contracts (7). Each is one substrate with many protocol projections. Publishing's public face is the unencrypted, content-addressed path served from many places; contracts (services and payments) is the most forward category, capability- and crypto-adjacent.
- Composite - composite protocols (8) such as IPFS and GNU Social are assemblies of the layers below (content addressing; identity, publishing, and messaging, federated). Their place at the top confirms the stack.
Content-addressed storage (IPFS) doubles as how a migratable instance stays coherent across hosts. There is no native Common Lisp libp2p/IPFS stack; the intended path is CFFI bindings to a libp2p built as a C-ABI shared library, with an external-daemon HTTP bridge as an interim stopgap and a fully native implementation as a long-horizon ideal.
What is framed and what is open
Framed: the layering, and that the substrate models semantic objects with protocols as projections. Open, and recorded as research questions: how richly a semantic object must be modelled so a protocol projection is faithful rather than lossy (email threading, NNTP groups, social-graph semantics); what the data engine actually is in Common Lisp; whether names is one model with two faces (a sovereign address book that also answers DNS); and the shape of the contracts category. These are named so each domain is built against shared substrate interfaces rather than re-inventing storage and identity per protocol, the discipline that keeps "their computer, their data" true.
Plugin composition: protocols as content-addressed, capability-gated modules
The central abstraction promises "every protocol a plugin." This section gives
the second half of that sentence a concrete noun: a plugin is a module (a
registered, lifecycled unit of code), and a protocol is a module that fills the
edge-adapter role. The mechanism is Shinmera's modularize (a library that makes
a package first-class: its own metadata store and load/delete hooks), adopted as
valis's plugin lifecycle. The fit was validated empirically before it was
committed: a throwaway module proved, cold, that modularize composes with
valis's package-inferred discipline, and surfaced the one trap that would
otherwise read as incompatibility (the grouping form is not idempotent under
ASDF's compile-then-load and must be guarded). Composition governs what code runs
and what it may reach within the substrate; whether a module's port is reachable
from off the host is a separate, privileged decision at the surface, never
derived from the module (The public surface). Three decisions define the model.
A plugin is a module, and a module is one combined system
A protocol module is a single ASDF system that is both an
asdf:package-inferred-system (valis's one-package-per-file discipline, where
each file's defpackage declares its own dependencies and the build graph
derives from :import-from) and a modularize:virtual-module, a named,
discoverable unit with a metadata store and a teardown lifecycle. The two are not
rivals: one class is both, package-inferred derivation still fires, and the
module's :packages grouping gathers the inferred sub-packages under one name.
valis-core stays an ordinary system; only the pluggables become modules.
The core's one structural change is to stop naming concrete adapters. The edge
controller's hard-coded adapter list (%adapter-specs) becomes an
*edge-adapters* registry that modules install into on load, the same
"downstream installs the hook" idiom valis already uses for
*edge-source-constructor*, *ctl-node-factory*, and *bearer-view-decorate-fn*.
A module declares its adapter in one line of module storage (an :edge-adapter
spec); a load hook registers it; a delete hook drains then retires it.
Drain-before-delete is not optional: a live protocol has in-flight connections,
so the delete hook runs the ordered edge teardown (deregister the source, drain
in-flight work, unregister) before the package is unbound. That drain is the same
primitive evacuation uses: the session drain on the 9P listener. Two correctness
disciplines fall out of the lifecycle: registration must be idempotent (the
grouping form runs twice under compile-then-load), and a protocol's macros stay
module-local: cross a module boundary with functions, not macros, because a
reloaded producer leaves a consumer's already-expanded macro call site running
the stale expansion, silently and without error.
Module load is a capability decision, never a hash fetch
Content addressing answers which module: a module package, held in the content-addressed store or fetched over IPFS, is named by its hash, so an instance pulls exactly the bytes it asked for and verifies them on read. But a hash answers identity, not trust. Loading a module means compiling and running code, and code pulled by hash from the network is, unqualified, a supply-chain hole: running code validated only by its hash is a recipe for pain. So which modules an instance will load and compile is a capability decision, gated by the same owner-rooted authority that gates every mount: a module is admitted only when the owner, or an authority the owner delegated to, has vouched for its hash, exactly as a capability vouches for a subtree. The hash is the name; the owner's signature is the permission. This keeps the "evaporate and condense elsewhere" malleability without turning a recondensing instance into an open code-execution endpoint, the discipline that matters most before the system is opened to outside contributors.
An instance's identity is its namespace manifest plus its module manifest
A valis instance is defined by two content-addressed manifests, not one. The
namespace manifest records what is mounted: the data side, which migration
already reassembles. A module manifest records what the instance is composed
of: the protocol and plugin set it answers with. Both are durable,
content-addressed, and reassembled on recondensation: a fleeing instance does not
carry its code, it records the capability-vouched hashes of the modules it ran,
and the destination pulls, verifies, and compiles them to stand up functionally
equal to the instance that evaporated. Because the substrate already
content-addresses data, it content-addresses the modules too, and IPFS becomes the
distribution layer for protocols, not only a backend for bytes. modularize's
in-image list-modules plus per-module storage is the live projection of this
manifest; the durable module manifest is its persisted, migratable form. This
closes the migration story for code the way the namespace manifest closed it for
data: a valis instance is its mounts and its modules, both reassembled from
hashes the owner vouched for.
The module-manifest record shape
The module manifest is a durable, content-addressed record distinct from the namespace manifest, and the distinction is the whole point. The namespace manifest records mount roots: what an instance has mounted, the data side. The module manifest records the module set as code: each entry is a system name, the module's ordered source files as paths relative to the module root, and a 32-byte content score over those bytes. Names and relative paths and a score: never a key, never a secret, never the vouch itself. The record is built once, its entries sorted by system name, so the same module set serializes to the same bytes regardless of discovery order; a content-addressed record demands that determinism. Decode is fail-closed: a bad width, an unknown version, a malformed entry, or a missing referenced byte stream aborts the read rather than yielding a partial set.
The two manifests are siblings (a manifest child and a modules child)
under one durable head tree. At genesis they are written by a single combined
publish that obtains both envelopes without advancing, builds one root tree naming
both children, and advances the head exactly once. This is not a convenience: it
is the only correct sequence. Two separate publishes cannot work, because the
first advance moves the head off the genesis generation and the second's
generation-fenced compare-and-set fails closed. So the manifests are atomic
siblings by construction, and the evacuative reassembly that reads the modules
child is correct precisely because the genesis publish guaranteed that child
exists under the same root.
The record stores no vouch and no key, and that absence is deliberate. A credential frozen into a durable record would carry the authority lifetime of the moment it was written into a record that outlives it: a revoked owner key still vouching for code months later. Instead trust is decided fresh at reassembly: the owner re-vouches each module's score against live custody and live revocation/fence state at the moment of condensation. The record's job is to name which modules; the live authority decides whether to admit them. Authority lifetime is decoupled from record lifetime.
The reassembly flow
A successor instance stands its module set up from the durable record the same way the namespace assembler stands a namespace up from its manifest: reassembly, not transfer. The source instance does not hand its running code to the destination: it records the vouched hashes of what it ran, evaporates, and the destination re-derives equivalent running code from the record. This is the code-side symmetry of the data-side migration: an instance's identity is its mounts and its modules, both reassembled from hashes the owner vouched for.
The reassembly is two-pass and all-or-nothing. The first pass reads the manifest fail-closed. The second pass, over every record before any module loads, resolves the recorded relative paths against the destination's source root, pulls the bytes, re-derives the content score, and asserts it equals the recorded score: a wrong-hash record aborts the whole reassembly here, in front of the first load. The pull is transport-agnostic: a rebindable puller seam means the bytes can come from the local store today and from a content-addressed network (IPFS/libp2p) tomorrow without reshaping the record: the IPFS gate stays open. Then each vouch is located: the owner re-vouches each score against live state, and a missing vouch aborts the set. Only after the whole set passes does the third pass load, routing every module through the same capability-gated admission chokepoint that gates an interactive load, never calling the loader directly, so the gate is structurally in front of compilation. If any module is refused at the gate, the adapter registry is restored to its pre-reassembly snapshot, so an earlier-admitted module's adapter cannot leak past a later refusal. A partial module set never boots.
The resident default set (HTTP and Gopher) is handled honestly rather than forced through this path: it is recorded name-only with a zero score, is already present as part of valis itself, and is skipped by the reassembly. Only genuine non-resident modules (those carrying a real content score) run the pull-verify-vouch-admit path.
The honest scope of resident-set parity
One claim must not be overstated. The resident default adapters are recorded name-only with a zero-score sentinel, because they are part of the valis system and are already loaded: there is no separate bundle to content-address. So the migration guarantee for the resident set is naming parity: a successor's recorded module set names exactly the source's set, and the resident edge registry the successor serves through names that same set. It is not content-score parity for those entries, and the documentation does not pretend it is.
The genuine content path (re-derive the score, match it against the record, locate an owner vouch, route through the admission gate) is exercised and proven by the non-resident scratch-module reassembly, not by the resident default. A reader who wants to see the score-to-admit guarantee should look at the non-resident reassembly test, where a module with a real content score is pulled, verified, vouched, and admitted, with an unvouched control proving the successor fails closed. The resident set proves the set-naming parity; the non-resident set proves the content gate.
The reassembly security boundary
The trust boundary the reassembly defends is the one between the durable record and the live authority that admits its modules. The record is published data (anyone who can read the store can read which modules an instance named), so the record holds no secret, and the decision to run those modules is made fresh against live state. The threats and the assertions that pin them:
| Threat | STRIDE | Direct assertion |
|---|---|---|
| Hash substitution: a record names another module's bytes | Tampering | the destination re-derives module-bundle-score over the resolved bytes and aborts unless it equals the recorded score |
| Unvouched code execution: recondensing as an open code endpoint | Elevation of Privilege | every module is routed through admit-module, structurally in front of the loader; a refusal never reaches compile or load |
| Revoked or fenced module re-admitted from a stale record | Elevation of Privilege | trust is decided at condensation against live revocation and fence state (the re-mint posture), not a stored credential |
| A partial or unverified module set on a mid-reassembly failure | Tampering / DoS | two-pass reachability and vouch precheck before any load; the adapter registry is snapshot and restored on any abort |
| A key or secret leaking into the durable record | Information Disclosure | the record holds scores, names, and relative paths only; a byte-absence regression and the store-cluster import firewall pin it |
| A custodial or single-transport assumption baked into the pull | Tampering (supply chain) | the pull is a rebindable seam; a content-addressed network transport drops in without reshaping the record |
| A two-step genesis publish leaving one manifest unwritten | Tampering | both manifests are written under one root advanced once; a second advance would fail the generation fence closed |
Validation coverage: plugin composition
The record shape and the reassembly are anchored to named tests an auditor can run
and watch stand or fail. The record codec and its fail-closed decode are pinned in
tests/module-manifest-test.lisp; the reassembly chokepoint's fail-closed,
all-or-nothing contract is pinned in the same file's condensation suite; the
end-to-end migration parity (a source instance records its module set, evaporates,
and a successor reassembles it through the production path) is pinned in
tests/migration-end-to-end-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Record codec round-trips; fixed-width envelope; order-invariant; pins a generation | record-codec-round-trips, envelope-is-fixed-width, module-order-is-canonical, pinned-generation-is-independent |
tests/module-manifest-test.lisp |
| Record carries no key bytes; the manifest package imports no content symbols | record-carries-no-secret-key-bytes, store-cluster-imports-no-content-symbols |
tests/module-manifest-test.lisp, tests/migration-end-to-end-test.lisp |
| Genesis writes both manifests under one root advanced once | genesis-publishes-both-manifests-one-advance |
tests/module-manifest-test.lisp |
| Reassembly admits a vouched set; the loader fires exactly once | condensation-admits-vouched-set |
tests/module-manifest-test.lisp |
| Unvouched / wrong-hash / revoked module fails closed; no partial set | condensation-refuses-unvouched-set-no-load, condensation-refuses-wrong-hash, condensation-refuses-revoked, condensation-all-or-nothing-multi-record |
tests/module-manifest-test.lisp |
| Reassembled module set names exactly the source's set (functional parity) | resident-module-set-survives-reassembly |
tests/migration-end-to-end-test.lisp |
| Non-resident module pulled, content-verified, vouched, admitted; unvouched control fails closed | vouched-non-resident-module-condenses-unvouched-refused |
tests/migration-end-to-end-test.lisp |
What is framed and what is open
Framed: that a plugin is a combined package-inferred module; that the edge
controller iterates an installed *edge-adapters* registry rather than naming
adapters; that module load is a capability decision over a content-addressed hash;
that the active module set is migration-recorded alongside the namespace; and now
the record shape itself: names, relative source paths, and a content score,
sibling to the namespace manifest under one genesis root, carrying no stored vouch
because trust is re-decided against live authority at reassembly. The modularize
fit with the package-inferred discipline is settled and cold-verified, the
idempotency trap mapped, and the reassembly proven end to end. Open, and the work
the next arc takes up: the protocol adapters move to their own sibling repository,
c3po (the home for every wire-format module a valis consumes) extracted on the
discipline of real need rather than anticipation, with the existing HTTP and
Gopher adapters relocated so the conversion proves the seam against their
already-green tests; the production capture path that records a real non-resident
module set at publish time (the reassembly and content gate are proven, the
publish-side capture of a live non-resident set is the remaining wiring); and
whether runtime unload is exercised in production or modules are load-once per
image. These are named so the plugin layer is built against one uniform module
abstraction (discoverable, lifecycled, content-addressed, capability-gated)
rather than a bespoke registry reinvented per plugin kind.
Per-axis sovereignty: one principle, three boundaries
This chapter is the auditor's map of the data-classification model that holds across the three operator-state axes: mail, DNS, and the social graph. It states one principle and then walks the three boundaries that make it concrete, each as a threat (what leak it prevents), an enforcement point (where in the code it is guaranteed), and an audit proof (the test that keeps it honest).
The principle is reach-by-classification, never reach-by-content. Operator state that lives in PostgreSQL holds pointers and metadata (a content-hash pointer to a body, an envelope field, a DID, a lifecycle disposition) and never a message body or a header. An infrastructure or operations path reaches a datum by what class of thing it is, not by reading its content; the content of a message stays opaque on the content-addressed store and is reached only through an authorized, owner-facing view, never through an operations seam.
That invariant is enforced structurally, not by review discipline, because a
future migration is exactly where a body column could otherwise slip in. The
confused-deputy schema audit (audit-migration-schema,
src/operator-state/audit.lisp) introspects the live committed information_schema
after a migration's DDL has applied and before COMMIT: forbidden-schema-columns
rejects any column across the operator-state tables that is not in the explicit
+permitted-operator-state-columns+ allowlist or that matches a body/header shape.
The migration runner (run-pending-migrations, src/operator-state/migrate.lisp)
calls the audit in-transaction on every schema migration and rolls back
fail-closed on any finding, so a migration that would introduce a body or header
column cannot land: the runner cannot be bypassed, and asserting over the live
schema rather than the dao-class definitions catches hand-written DDL too. The
proof that the gate is live and non-vacuous is
run-pending-migrations-aborts-a-body-bearing-migration and
operator-state-schema-has-no-body-or-header-column
(tests/operator-state-sovereignty-audit-test.lisp).
The content boundary: mail bodies never reach PostgreSQL
A message body never enters PostgreSQL. It stays an opaque content-addressed block
on the store, and the mail-transport-state row names it only by a content-hash
pointer, body_ptr: an address, never invertible to content (the opaque-block
view of the mail spine is detailed at the content boundary).
Threat. A confused deputy (an operations or migration path that could name, decode, or copy a body byte through the queue or state seam) would turn a substrate-management capability into a content read. That is the leak this boundary forecloses: operations moves and meters the spine and reaches no body.
Enforcement point. mail-transport-state (src/operator-state/mail-state.lisp)
carries only body_ptr (a 64-char content-hash pointer) plus envelope and
lifecycle metadata (no body or header column), and its package imports no
body-octet reader (read-block / decode-file). The schema audit above bans any
body/header column across the operator-state tables on every migration, so the row
shape cannot regress.
Audit proof. The import-firewall sweeps
ops-queue-projection-imports-no-body-read-symbol and
ops-queue-grant-resolves-no-message-body-name
(tests/mail-sovereignty-audit-test.lisp) prove the operations view reaches no
body both by construction and over a really-queued message; the in-runner gate
a-body-bearing-mail-migration-aborts and
mail-state-migration-unit-passes-the-in-runner-audit
(tests/operator-state-sovereignty-audit-test.lisp) prove the schema audit accepts
the real mail unit and rejects a body-bearing one.
Write-authority and public serving: the DNS zone axis
DNS zone data is published by design: the records are meant to be world-readable. The protected asset is therefore not secrecy but write-authority: who may mutate a zone. valis is the authoritative ndb data owner over PostgreSQL; the nameserver reads through a narrow read seam and composes answers, holding no write authority of its own.
Threat. Unauthorised zone mutation (a session or a confused deputy committing a zone change it does not own, or smuggling its claimed authority in through the submitted bytes) is the leak this boundary forecloses.
Enforcement point. The :names write gate stamps the owning principal from the
session, never from the message bytes: node-create reads the session principal
at submission (%current-submission-did, src/namespace/names.lisp) and
node-close commits it as the zone's owner_did at the 9P clunk. owner_did is an
allowlisted operator-state column carrying the zone's write-authority owner
principal, and serving is a read-only projection over the committed data.
Audit proof. dns-schema-has-no-forbidden-column,
ndb-facet-columns-pass-confused-deputy-audit, and
existing-zone-backfilled-as-primary-on-ndb-anchor
(tests/operator-state-zone-audit-test.lisp) prove the zone schema passes the
confused-deputy audit and that the write-authority facet is carried as metadata,
not content.
Validation coverage: per-axis sovereignty
Each boundary is anchored to its enforcement point and the test that proves it. The common principle (pointers and metadata, never bodies or headers) is the schema audit the migration runner runs before every COMMIT.
| Boundary | Threat | Enforcement point (file:fn) |
Audit proof |
|---|---|---|---|
| The principle: reach by classification, not content | a future migration adds a body/header column | src/operator-state/audit.lisp → audit-migration-schema / forbidden-schema-columns, run in-transaction by src/operator-state/migrate.lisp → run-pending-migrations |
run-pending-migrations-aborts-a-body-bearing-migration (tests/operator-state-sovereignty-audit-test.lisp) |
| Mail bodies → content boundary | a confused deputy decodes or copies a body byte | src/operator-state/mail-state.lisp (body_ptr only, no body-octet import); the schema audit bans body/header columns |
ops-queue-projection-imports-no-body-read-symbol (tests/mail-sovereignty-audit-test.lisp); a-body-bearing-mail-migration-aborts (tests/operator-state-sovereignty-audit-test.lisp) |
| DNS zones → write-authority + public serving | unauthorised zone mutation | src/namespace/names.lisp → node-create / node-close stamp owner_did from %current-submission-did (the session principal, not the bytes) |
dns-schema-has-no-forbidden-column, ndb-facet-columns-pass-confused-deputy-audit (tests/operator-state-zone-audit-test.lisp) |
| Social graph → capability-scoped access | ambient / confused read of the graph | src/plugin/social-admission.lisp → query-reachable gates the raw traverse-reachable behind verify-social-vouch (/social designation + :read) |
query-reachable-is-the-only-caller-facing-entry-to-the-raw-traversal and the four bad-vouch refusals (tests/social-capability-scope-test.lisp) |
Sovereignty as an architectural invariant
Operations manages the substrate and lifecycle; it never touches the content and meaning of a user's data. The namespace fabric is where this line is drawn: operability is expressed as moving and mounting viewers, never as reading content. The capability discipline above is the enforcement: a module (or an operator's tool) holds only the authority designated into its namespace, never an ambient authority over all of a user's data. Where code could still violate the line (observability, backup, multi-tenant isolation) it must be confined the same way, by what is mounted, not left to operational good behaviour.
What this invariant does not claim
The paragraph above is a statement about operations, and a reader is owed its limits in the same place, because the gap between them is where a wrong decision gets made.
Ordering. valis is first a way for a person to hold their own data, against the capture of a network architecture that did not have to end up this way. Resistance to a well resourced adversary is a real secondary capability and it is not the headline. Where the two conflict, holding the data wins. Anyone weighing a feature that trades availability of a user's own data for resistance to seizure should know that trade has already been decided in that direction.
The ceiling, stated rather than implied. A machine that is seized and carried off yields its processor, memory and storage state to an adversary who can then work on it without time pressure. There is no defence at that scale and none is claimed. What is achievable is narrower: notice physical interference, and make the relevant data unreadable afterwards.
That last part is harder than it sounds, and the reason is structural. Overwriting does not reliably erase modern persistent storage, because wear levelling leaves the controller deciding which physical cells a write lands on. Destroying a key is the only fast and checkable way to make data unreadable. So the arrangement has to put a small secret in a position where destroying it is sufficient, with no plaintext having reached swap, hibernation, or any store that outlives the key. That is a property of how the system is laid out. It cannot be added later as a panic button.
One tension that does not dissolve. A node that stays online without a person attending it must be able to restart on its own, so its operating key has to be reachable by the machine with nobody present. Putting that key behind a person's presence defeats what the system is for. So a node that can restart unattended will surrender its operating key to whoever takes the machine. That is a consequence of the goal rather than a defect awaiting a fix, and it is why authority to reconstitute a node elsewhere is worth separating from the key the node runs on.
⚠ Sealed backup is built; splitting the secret is not. What exists
today is passphrase-sealed off-host backup (src/backup.lisp). Splitting a
secret across several holders, so that no single seized machine holds
enough to reconstitute anything, is design intent and is not implemented:
src/ contains no secret-sharing of any kind. Read any description of
reconstitution elsewhere in this document with that distinction in hand.
The public surface: what the internet reaches, and who opens it
A sovereign node answers the public on a routable address, and the question this chapter settles is the one a reader most needs before placing new behaviour: which ports the internet can reach, and who decides. Two mechanisms are involved, not one, and neither is the namespace by itself. This is worth stating plainly because an earlier framing ("the designated-ports boundary is the namespace itself") was true only while the node lived on loopback and stops being true the moment it carries a public IP.
The steer fans every port; the firewall selects them
The privileged host agent's sk_lookup program is a catchall
(fulcrum/src/steering.lisp): it stashes valis's one socket in a single-slot map
and redirects every TCP and UDP port on the unit's IP to it, transport- and
port-agnostic by construction. On a loopback-only island that was harmless:
nothing off-host could reach the namespace, so the ports the node answered were
exactly the ports anything bothered to connect to. Once the namespace carries a
routable address, the same catchall would fan every port on the public IP into
valis.
What narrows the public surface back to the ports the agent admits is a second
mechanism: a default-deny firewall the agent installs inside the namespace
(fulcrum/src/firewall.lisp). Its input chain accepts loopback, established and
related return traffic, :53 on both transports, and then, each one only when the
agent's own config carries it, the owner port and the public TLS edge port.
Everything else drops. The steer says "all ports to valis"; the firewall says
"only these enter." They are complementary and both required, and the firewall
(not the namespace) is the public-surface boundary. The distinction is not
pedantic: a namespace boundary is structural and cannot be misconfigured open,
whereas a firewall is a maintained rule that must be installed correctly, before
any traffic, and that can in principle fail open. On a public node the surface
control is therefore a security-critical, order-sensitive invariant of the same
rank as the write fence, documented here rather than treated as an afterthought.
The port allowlist is the host agent's, never a module's
Because the boundary is a policy and not a topology, who writes the policy is
the load-bearing security question, and the invariant is that the allowlist is the
privileged host agent's alone. A valis module declares the ports it intends to
serve, and that declaration reaches the agent as a contract
(fulcrum/src/control.lisp's recorded port set), but it gates the descriptor
handoff only; it opens nothing. The firewall ruleset takes no valis-supplied input
at all: it is a pure function over the agent's own configuration, fed to a
privileged nft inside the namespace. Widening it takes a privileged act at the
agent, and there are two of them rather than one. A change to the agent's source is
the obvious one. The other is a change to the agent's config store: :open-owner-port
and :edge-port (fulcrum/src/launcher.lisp, read through ubiquitous) each add an
accept line with no source change at all. An operator auditing what a live node
exposes has to read both, and the config is the one that is easy to miss.
This decoupling is what keeps a node safe when it hosts code it does not fully trust. valis is the default home for every process not delegated to a sister, including the active-module archetype's untrusted executors; if a module's declaration could open a public port, registering a hostile module would expose an arbitrary listener on a routable IP. It cannot, because reachability is granted by a privileged act at the agent, admitting a port in the agent's own ruleset, that a module can request but never perform. Registration wires dispatch; the agent grants reach; the two must never be collapsed. The pattern for adding a protocol (the constellation map) is written so they are not: a new public port is a host-agent change, deliberately.
⚠ The set the firewall admits and the set the steer carries are disjoint, and one
word doing duty for both is how a reader gets this backwards. A module's
declaration reaches the agent as its designated port, and designated-port is
precisely the key the firewall does not read: grep fulcrum/src/firewall.lisp for
the word and it is not there. What the ruleset reads is owner-port and
edge-port, both drawn from the agent's own config. A declared port is therefore
steered inward and still dropped at the public IP unless the agent's config
independently admits that same number, and that gap is the whole safety property.
Read "designated" into the firewall and you will believe it admits what the steer
carries.
DNS arrives by two paths: UDP inherited, TCP steered
The carve-out is UDP only, and it is physical rather than a preference.
sk_lookup chooses which socket receives an inbound packet but does not govern
the reply: a UDP answer is a fresh datagram that must source from the address and
port the client sent to, or the client rejects it. A TCP connection escapes that,
because once the steer selects the listener the kernel tracks the four-tuple and
sources replies correctly, whereas valis answering UDP from its generic steered
socket would source from the wrong address. The argument covers UDP replies and
nothing else. I have ruled that it must not be stretched to cover TCP: DNS over TCP
is required in a nameserver, and opening a second special case beside the UDP one
buys nothing, so the steer carrying :53 TCP is the correct path.
The kernel program matches that ruling because it never had a port or protocol test
in it. fulcrum/bpf/sk-lookup-catchall.lisp is a lookup of slot zero in a
single-entry sockmap, an sk-assign, and SK_PASS: one map, one program, no port
table, no protocol constant, and the compiled object is byte-identical everywhere it
ships. There is no build variant in which it declines a port, so "transport- and
port-agnostic by construction" above is the whole truth about it.
The privileged agent still binds the unit-IP :53 sockets on both transports and
hands the descriptors down, for the reason that covers every inherited descriptor:
binding a privileged port is the agent's to do, since valis runs unprivileged in the
namespace. What differs by transport is what becomes of those descriptors on a node
whose catchall is live. The inherited UDP socket serves. The inherited TCP socket is
bound, handed down, adopted, and then never receives a connection, because the steer
takes TCP on that port at the unit's address away from it.
What the transport split costs: one service, two independent failure modes
Read this before placing anything else on the DNS path, because it is the price the ruled design charges and nothing in the running system reports it. DNS over TCP rides the steered socket: its accept backlog, its per-port budget, its lifetime. DNS over UDP rides the inherited one. The two transports of one service share a name and share nothing else. A steered socket that saturates, that the edge budget deregisters, or that is torn down takes DNS over TCP with it while UDP keeps answering, and no counter, log line or check anywhere signals the split.
A nameserver answering UDP but not TCP is broken while presenting as healthy. A client that receives a truncated answer retries over TCP and gets nothing, so truncation fallback has no fallback. Zone transfers to a secondary fail, so a secondary serves its last good copy until the zone expires and then goes lame. Both failures surface at a resolver or at a secondary's operator, never at the node.
⚠ A green DNS answer is not evidence about which socket served it, in either
direction. The resident recovers the destination port per accepted client rather
than from the listening socket's own bind, so a hijacked connection's child carries
local port 53 and reaches the DNS handler exactly as an inherited one would. A dig
+tcp returns a perfect authoritative answer under both hypotheses. What separates
them is which listening socket's accept queue the connections enter, the one
measurement a DNS answer cannot give. That is what
proving-ground/driver/scenarios/steered-port-delivery.lisp is built to read: it
aims a burst at the inherited listener's own endpoint and attributes the resulting
queue depth to one listening socket or the other.
⚠ The UDP fall-through is REASONED and has not been measured directly. With a
live catchall the program attempts an assign for UDP too, and inherited UDP :53
keeps working because the kernel does not complete a cross-protocol assign and falls
through to its own socket lookup. That mechanism is inferred from the program's
shape and from UDP continuing to answer on a steered node. No gate observes the
kernel declining the assign, so read it as the reasoning it is rather than as a
measurement.
The owner control plane: a keyed terminus, shipped closed
The owner manages the node (mints and revokes capabilities, edits zones, enrols
keys, drives the mail spine) through the 9P namespace, as a client that mounts the
owner-proven management view. That control plane is deliberately not on the
public surface: run-daemon binds the fabric on the namespace's loopback, the
firewall drops the fabric port at the public IP and drops the owner port too unless
the agent's config deliberately opens it, and an
attach is admitted only after a NoiseXX handshake proves the owner's key
(fabric.lisp's node-auth responder, seven's Tauth=/=Tattach path, mercer's
sealed transport). Reachability and authorization are separate axes: the fabric is
never open, it is authenticated, and loopback-binding is what makes "reachable"
require first arriving on that namespace's loopback. The firewall reserves the
loopback path ("the owner's loopback fabric path") for exactly this.
Reaching that loopback from off the host is a second keyed listener, not a hole in
the first. When the resident inherits an owner-port descriptor it adopts it as the
routable owner terminus (--owner-fd, threaded into start-fabric), and that
adoption asserts the auth seam is engaged before it binds: a routable terminus that
admitted an attach without the Noise handshake would be an open owner surface, so it
fails closed exactly as the non-loopback bind assert does. *fabric-auth-enabled-p*
is what lets the fabric bind a non-loopback host at all, and it does so because the
handshake guards the attach rather than in spite of it.
The port is shipped closed. The host agent's launcher carries :open-owner-port,
defaulting to nil, so the firewall drops the owner port and no owner socket is bound
until an operator deliberately opens it. Opening it is the highest-value application
of the host-agent allowlist above, though not the first: the public TLS edge port
already carries its own accept line under the same mechanism.
⚠ A same-host proof and a cross-host proof are separate propositions, and neither stands in for the other. Driving the fence, the session listing, and a publication from a client over a real sealed socket proves that the behaviour meets a client as opposed to an in-image caller. It proves nothing about reach through the namespace and firewall boundary, which is a different path. Read any "verified live" against which of the two it actually crossed. TODO.org carries where that stands.
⚠ One detail of that path is easy to get backwards. A session admitted over the wire registers under the transport key's DID, because a Noise handshake proves the transport key and derives the principal from it, not from the signing key custody holds for the owner. The owner check therefore accepts either DID. A test that constructs principals in the image never crosses a handshake and so cannot observe this at all; only a test driving a real client does.
What :80 is for: the manual a node carries about itself
The design is that every valis serves its own documentation on :80, in the
clear, with that port enabled on every node. I rank it with the owner control
plane rather than with the protocol adapters because it is the surface a person
meets first: before a certificate exists, before a name resolves anywhere, and
before anything else on the node is working. A node whose manual you cannot read
is not one you own, so the manual travels with the node and is answered by the
node.
Plain HTTP is the reason rather than a shortcut. Point a phone, a handheld, a decade-old browser, or a text-mode client on a borrowed machine at the address you were given, and you get the documentation, because nothing in that path asks for a modern cipher suite, a trusted root, or a credential the node must first go and obtain from somebody else. A surface that comes up only after the credential machinery has succeeded cannot be the surface that tells you how to make the credential machinery succeed. That is a sovereignty property before it is a convenience: the one door that is open on a node you have just stood up is the one that explains the rest of it.
The content is the published documentation site the tree already carries
(site/, landing/), served as an ordinary publication. The descriptor for :80
is inherited from the privileged host agent for the same privilege reason :443
is, and it is the same port an ACME http-01 responder would answer a challenge on:
that use is real and it is secondary, never the reason the port is open.
⚠ This is the design, and the node is behind it. The resident parses the
inherited descriptor and hands it to no adapter, so a node that inherits :80
answers nothing on it. TODO.org carries the item that closes the gap and the shape
it copies.
Direction: unifying the boundary in the kernel
The two-mechanism split (a catchall steer plus a separate firewall) is the
current shape, not the settled end state, and this section records the intended
direction without asserting it as built. The unification is to make the sk_lookup
program itself the allowlist: a map of admitted ports, written by the agent under
privilege, that the program consults, redirecting only admitted ports to valis and
letting the rest fall through to closed. The word matters: the map holds what the
agent admits, which is not what a module designates. The boundary then lives in one place, in
the kernel, and the separate firewall becomes defence-in-depth rather than the
primary control. Two constraints survive the unification and are worth stating so
the direction is not mistaken for a cure-all: the allowlist map must remain the
agent's, never derived from a module's declaration (the invariant above is
independent of mechanism), and UDP keeps its socket-activation carve-out
regardless, since the steer cannot source a UDP reply. That second constraint
carries a consequence worth naming here rather than rediscovering later: with UDP
inherited and TCP admitted through the map, a service offering both transports
still has them arriving on separate sockets with separate fates, so unifying the
boundary does not close the transport split. Any move here changes the
privileged kernel path that decides what the internet reaches, so it is gated on a
steering test harness before the program is touched.
Deployment: condense-from-genesis on a real host
Standing up a valis node on a fresh host and condensing an evaporated unit back into existence are the same operation. Deployment is the genesis case of the reassembly path above: there is no separate installer to maintain, and deploy, evacuate (condense at a new location), and restore (condense from a durable head) unify onto one mechanism. The durable thing is the namespace, so a deploy is a reassembly whose predecessor happens to be empty: the only genesis-specific step is minting the owner Ed25519 seed rather than acquiring it from a predecessor.
The target is a stock Debian host; the resident runtime is a bare
save-lisp-and-die executable supervised by a thin Type=notify systemd unit,
isolated with ip netns, receiving its privileged wire descriptors by inheritance
or SCM_RIGHTS. No container runtime runs on the resident. The contract splits at
the point where condense can first run: adopt below it (the run account, a writable
durable-state directory the account owns, the staged serving binary, a reachable
operator-state database: ordinary host bootstrap), and reuse above it (the netns,
the descriptor handoff, the fail-closed serve loop, the generation fence: valis's own
machinery, driven, never re-implemented by a YAML control plane that would drift from
the live image).
The bring-up order and the fail-closed invariants
The resident's production entry is run-daemon (src/main.lisp), which shares
run-foreground's fail-closed bring-up: the fabric comes up and the anonymous
public view resolves before the edge binds. For the public :53 service, fulcrum
(privileged) creates the netns and moves a dedicated public NIC wholesale into it (the
launcher's mode :dedicated, with the netns default route via that interface's own
gateway), binds :53 UDP and TCP in the netns,
clears CLOEXEC, and execs the unprivileged valis (ip netns exec → setpriv → env →
valis) with the inherited descriptors; run-daemon adopts them, constructs a
pg-zone-source over the operator-state rows, and drives the sibling serve loop over
the inherited fds, binding no privileged port itself. (Both :53 descriptors are
inherited because binding a privileged port is the agent's to do, but on a node whose
catchall is live only the UDP one goes on to serve: the steer takes :53 TCP away
from the inherited listener, so DNS over TCP arrives by the same route as the steered
push-boot handoff and carries the split set out at what the transport split
costs.) The invariants a
deploy must preserve are the substrate's own: unprivileged valis dies rather than
privileged-bind; a binary without the serving codec fails closed at boot; only the
exposed ports are reachable behind fulcrum's default-deny (the
surface control, the boundary that admits them); and two instances sharing
one database are held to a single writer by the generation fence
(the fence and the sweep). The written contract is the host-deployment
contract.
Public TLS: the credential a name is answered under
A certificate is obtained through a dns-01 challenge, and the authority that answers
that challenge is the node itself. I made the :443 edge cert-gated rather than
required because of that circularity: a node must already be serving :53 before it
can ever be issued a certificate, so a certless genesis boot has to be a healthy
state and not a failure. With no usable certificate in custody for the configured
name, the public edge stays dark and the resident still comes up, and the boot output
says which case held. A silently dark port is indistinguishable from a descriptor
that never arrived, which is the confusion that costs an operator an evening.
The edge holds one credential per name it serves and resolves it per connection, so a request for one name is never answered under another name's certificate. A renewal swaps the live leaf on the same cell the connection path reads, and the listening socket is never touched.
Two constraints carry into any operation on this path. Both are irreversible.
- One certificate per registrable domain. Never SAN-pack. Certificate Transparency records the set of names on a certificate permanently and publicly. Pack several of the owner's domains into one credential and you have published, for good, the fact that one party holds them all. Nothing retracts it afterwards.
- The ACME account key shares the certificate store path. Rename or move that store and the next order registers a fresh account, spending production rate limits that do not come back.
The proving ground: the migration thesis under test
Because deployment is the genesis case of condense, the harness that exercises a
deploy is the same harness that exercises evacuate and restore: the migration-thesis
regression harness. It lives under proving-ground/ as committed, declarative
tooling: a Nix devshell pins the verification fabric (the guests it boots stay stock
Debian, and the core is built impurely from the working tree, never as a Nix
derivation);
Terraform-libvirt boots Debian guests on two distinct VLAN-tagged network locations
so a successor and predecessor sit at genuinely different locations: the topology a
single-VM harness cannot express, and the precondition for the fence-and-reconnect
path; and cloud-init provisions each guest to the adopt-below seam, staging the serving
binary rather than rebuilding it. An in-image Lisp assertion layer (valis-proving-ground,
its own ASDF system, never entering the resident binary) drives three scenarios over
one mechanism (deploy-from-genesis, evacuate-cross-location, restore-from-backup)
through three real interfaces: SSH for unit/netns/fd state, off-host dig for
authoritative :53, and a fabric-liveness probe on the owner-proven loopback path.
Each assertion is exit-coded and fail-closed; a scenario reduces to one go/no-go
verdict. The scenarios assert the effects of the shipped condense and fence path,
they do not re-implement it. See proving-ground/README.org.
The restore scenario exercises a passphrase-escrowed backup of the irreplaceable durable state, and a deployed resident reports substrate-only health so a silent failure surfaces before it becomes an outage. Both surfaces (the backup/restore and passphrase-escrow flow, and the logging/liveness/certificate-expiry observability surface) are documented for the operator in Backup and observability.
System shape (Lisp)
valis is an umbrella of cooperating Common Lisp systems, not a monolith.
The core valis system stays focused (listener, registry, protocol
contract, substrate interfaces) and depends on separate repositories for
the heavy, independently-useful pieces:
- seven: the 9P library carrying the namespace fabric and the bus.
- mercer: the factotum. Transport authentication, key custody, and the ACME certificate lifecycle.
- runciter: the authoritative-DNS answer logic the
:53serving path binds to. - c3po: the wire engines, adopted here as content-addressed edge modules.
- fulcrum: the privileged host agent. eBPF steering and the descriptor handoff.
- Whistler: the eBPF compiler and pure-CL loader fulcrum drives.
- libp2p/IPFS bindings: content addressing, and the one dependency the Common Lisp ecosystem does not yet answer. CONSTELLATION.org is the map of which repo owns what.
Within the core:
package-inferred-system: each file'sdefpackagedeclares its own dependencies; there is no central component list. Adding a file undersrc/and importing it is how you extend the system.- A new protocol plugin is, minimally: a new
src/<proto>.lispthat defines aprotocolsubclass (or instance), specialiseshandle-connection, and registers itself on its ports. - Cold-verify every change (fresh image, cleared fasls): a warm REPL
hides missing
:import-fromdeclarations.
Status
This document says what valis is and what it is meant to be. It carries no status, deliberately. A design record that also tracks what has landed goes stale every time something ships, and when a reader catches one wrong paragraph they discount the rest of the document with it.
TODO.org is the status board: where valis stands, what has landed, what is built but
not yet deployed, what remains, and the order it assembles in. It also separates what
has run on the live node from what is merely on main, which is the distinction this
section used to blur.
Bibliography
External resources this design draws on, grouped by the part of the architecture they ground.
eBPF steering
- BPF sklookup program type - the kernel mechanism that lets one socket answer every port (the steering layer).
Listener and event multiplexing
- iolib - the Common Lisp I/O library whose epoll multiplexer the listener is built on.
- Efficient IO with iouring (Jens Axboe) - the eventual multiplexer backend behind valis's own interface.
Authoritative DNS
- RFC 1035 - domain names: the message format and master-file presentation the substrate serves and the wire engine encodes.
- RFC 2308 - negative caching: the apex SOA on a negative answer and the
min(SOA.MINIMUM, SOA.TTL)negative-cache lifetime the seam contract requires. - RFC 4592 - the role of wildcards: the synthesis-and-existence rules the nameserver's lookup tree follows.
- RFC 3597 - handling of unknown RR types: the generic rdata form the record boundary round-trips through.
Plan 9 and 9P
- The Use of Name Spaces in Plan 9 - per-process namespaces and 9P, the model the substrate reifies.
- 9P specification and papers - the protocol that is both substrate and bus.
- v9fs (Linux 9P client) - lets the host kernel mount a valis namespace.
- factotum(4) - Plan 9's auth agent (a 9P file system holding keys); the model for valis's auth agent.
- c9 - a small 9P client and server; reference for a Common Lisp implementation.
- Interim - a Lisp environment where "everything is a file is a symbol"; closest prior art.
- awesome-plan9 - curated index of Plan 9 / 9P software.
Storage, consistency, and migration
- Venti: a new approach to archival storage - content-addressed, write-once block storage (the immutable-bulk model).
- Fossil, an Archival File Server - the mutable file server over Venti (the single-writer mutable head).
- CephFS Dynamic Subtree Partitioning - the per-subtree-authority write-scaling growth path.
- Conflict-free replicated data types - leaderless merge model, considered and rejected for the authoritative namespace.
- Git internals: objects - the canonical immutable-objects + mutable-refs pattern.
- CRIU live migration - checkpoint/restore including established TCP connections (TCPREPAIR); the optional in-flight-continuity enhancement.
- Tahoe-LAFS architecture - a least-authority store where the capability is the name (read/write/verify caps).
Identity, capabilities, and cryptography
- UCAN specification - delegable, attenuatable, public-key/DID-rooted capabilities; the primary token model.
- Macaroons - bearer capabilities with contextual caveats (the attenuation/confinement idea).
- Good Practices for Capability URLs (W3C) - the unguessable-URL-as-capability pattern for anonymous publishing.
- Capability Myths Demolished - ambient authority and the confused deputy; the no-ambient-authority discipline.
- The Noise Protocol Framework - connection-time mutual key authentication (also libp2p's secure channel).
- SPKI/SDSI (RFC 2693) - the certificate-capability ancestry of UCAN.
- Decentralized Identifiers (DIDs) (W3C) - the principal identifiers UCAN uses.
Content addressing
- IPFS - content-addressed storage for bulk and published data.
- libp2p - the networking stack a future Common Lisp CFFI binding would expose.
- cl-ipfs-api2 - an existing CL HTTP-API client to an external IPFS daemon (the interim bridge).
🄯 Brian O'Reilly <fade@deepsky.com>, 2026