valis / Start here

valis - Building Your Own System on valis

For the operator who wants to build their own system on valis rather than contribute to valis itself. valis is a substrate: it owns the steering edge, the 9P namespace, identity, capabilities, and enforcement-by-mounting, and it offers two surfaces you extend. This guide is the map of those surfaces: what to subclass, what to call, and which invariants you must respect so your system inherits valis's sovereignty guarantees instead of undermining them.

For what valis is, read the README; for how it is shaped, the Architecture; for the workflow and conventions of editing the code, the Working with the code guide; for every exported symbol with its signature and docstring, the API reference.

Before placing a project that is larger than one adapter, read Evaluation and authority. It records why this is a federation of cooperating instances rather than one image serving many parties, and its section Direction for work built on valis places a new project by shape. It answers the question this guide does not: what to do when your system needs to run code on behalf of somebody who is not you.

The two extension surfaces

There are exactly two places a builder plugs in, and they answer different needs:

  1. A wire-protocol adapter: you speak a network protocol (HTTP, Gopher, your own) on a port, and project a person's data through the namespace to anyone who dials it. This is the common case: a new edge onto an established subtree.
  2. A keyed, authenticated service: you stand up a 9P responder that a remote peer reaches by proving a key, so each session carries a resolved principal and its traffic is sealed end to end. This is for inter-node and owner-proven surfaces, not anonymous public reads.

Both surfaces sit on the same authority model, described last. Read it before you ship, because it is the part that is easy to get subtly wrong.

Surface one: a wire-protocol adapter

A new protocol is a new file src/<proto>.lisp (or your own system depending on valis) that does four things. The worked references are the c3po-http and c3po-gopher modules from the c3po protocol sibling: the HTTP and Gopher engines, adopted as content-addressed edge-adapter modules rather than carried in-tree; model a new one on c3po-http, and read c3po-gopher for the single-request shape. Both register through the edge-adapter registry (valis/src/plugin/edge-adapter); the four-step contract below is unchanged.

Subclass the protocol contract

protocol (valis/src/protocol) carries the per-port contract: a name, the ports it claims, and a description. Subclass it and provide a constructor that fixes those:

(defclass echo-protocol (valis/src/protocol:protocol) ())

(defun make-echo-protocol ()
  (make-instance 'echo-protocol
                 :name :echo
                 :ports (list 7007)
                 :description "Line echo over the recovered port."))

Specialise handle-connection

handle-connection (valis/src/protocol, a generic of (protocol connection)) is your per-connection entry point. By the time it runs, the recovered destination port has already selected your module and the connection arrives with its principal resolved. Read and write over connection-stream; the connection also exposes connection-principal and connection-destination-port. You never parse the wire to learn who is calling: identity is resolved for you.

(defmethod valis/src/protocol:handle-connection
    ((protocol echo-protocol) connection)
  (let ((stream (valis/src/connection:connection-stream connection)))
    (loop for line = (read-line stream nil)
          while line do (write-line line stream) (force-output stream))))

Register the protocol against its port

register-protocol (valis/src/registry) claims every port in the protocol's protocol-ports; the listener then routes matching connections to your handle-connection through dispatch-connection. Register once at boot:

(valis/src/registry:register-protocol (make-echo-protocol))

A port may be claimed once: a second claim signals port-already-claimed, and a connection for an unclaimed port signals no-protocol-for-port. The process-wide *registry* is the default; pass :registry to use your own.

Project over the namespace, never around it

The point of an adapter is to project a person's data, not to own it. Obtain a view for the connection's principal and walk it, rather than reaching for files directly. view-for (valis/src/edge/seam) returns a root handle and a 9P session for the resolved principal; and, when the view is per-request (a keyed principal, or a capability mount), a teardown thunk you must call when the request ends:

(multiple-value-bind (root session teardown)
    (valis/src/edge/seam:view-for connection principal nil)
  (unwind-protect
       (;; walk ROOT with seven's client (walk / read-bytes) and render
        )
    (when teardown (funcall teardown))))

An anonymous caller (a nil principal) resolves to the read-only published /pub floor; an owner-proven caller reaches the fuller frame. To mount a specific capability for the span of one request, use with-request-capability (valis/src/edge/seam), which mounts the designated grant, runs the body, and unmounts under unwind-protect. The HTTP adapter does exactly this to serve its two identity-selected views over one URL space.

Surface two: a keyed, authenticated service

When a remote peer must prove who it is before it reaches anything (an inter-node link, an owner-only management surface), valis serves a keyed 9P responder. You do not assemble the crypto; you bring up the fabric and let the attach path resolve principals and seal sessions for you.

Bring up the fabric as a keyed responder

start-fabric (valis/src/fabric) stands up the namespace tree, the identity seals, the capability service, and the 9P listener:

(valis/src/fabric:start-fabric 0
  :host "127.0.0.1"
  :keyfile "/path/to/owner.key")   ; the owner's Ed25519 seed

It binds HOST:PORT (pass 0 for an ephemeral port and read the bound port back with listener-port). The default host is loopback, the shape a developer uses to bring a keyed responder up locally. A production owner terminus does not self-bind a public address this way; it adopts an already-listening descriptor the host agent bound for it (see below). Either way, serving a non-loopback surface requires *fabric-auth-enabled-p* to stay true, so every remote peer must complete the NoiseXX handshake: valis refuses to serve a public network surface in the clear. Tear down with stop-fabric.

The routable owner terminus

In production valis reaches the network on a port it is too unprivileged to bind itself. A co-located privileged host agent binds the routable owner port, then hands valis the already-listening descriptor, the same inherited-descriptor handoff valis already uses for :53 and :443. The resident boot receives that descriptor as --owner-fd <N> and threads it into start-fabric, exactly as it threads --dns-tcp-fd and --edge-tcp-fd. start-fabric adopts it by rebinding the *keyed-socket-source* seam (valis/src/transport/keyed-acceptor) to wrap the inherited fd as a keyed accept loop's passive socket. There is no second bind, so an already-listening fd cannot collide, and the adoption dups the descriptor: valis closes only its own dup on teardown and never the borrowed fd.

The owner terminus is a second keyed listener on the same namespace root: the loopback fabric (start-fabric 0) stays the local management path, and the owner terminus is the routable addition, present only when --owner-fd is supplied. Absent it, no owner terminus comes up (the default-closed posture) and every other resident shape is unchanged. Its reachability is owned by the host agent's descriptor, not by valis self-binding a public address. The registered 9P port (564) is the default valis records; the number the host agent actually opens is authoritative, and valis serves whatever descriptor it is handed.

Opening the owner port: the gated go-live sequence

The owner terminus carries the full owner surface to any peer that completes the handshake, so who answers matters as much as who dials. The anti-spoof guarantee is inherently client-side: the owner's client (ubik) must pin the deployed node's transport did:key and verify the peer's recovered static key against that pin, refusing a peer that does not match. valis cannot enforce this from the server: a server cannot stop a client from dialing an impostor.

The honest server-side lever is therefore exposure, not a code flag. The host agent's firewall keeps the owner port closed by default; it carries no valis switch that claims to prevent a client-side impersonation, because none could. Opening the port is a deliberate operator step, and the sequence is fixed:

  1. Verify the deployed node's transport did:key is pinned in the owner's client, so a first off-host attach validates the responder rather than trusting whatever peer answers at the address.
  2. Only then open the owner port in the host agent's allowlist.

Reverse that order and the first off-host use is spoofable. Until the pin is in place, the owner manages the node over the existing local path; the routable terminus stays firewall-dark even though the adoption code is wired.

What the attach path does for you

A peer dials the listener and runs the standard 9P Tauth=/=Tattach exchange carrying a NoiseXX handshake. On a completed handshake, valis maps the peer's static key to a did:key principal and binds that principal to the session; from the first reply onward, every record on that connection is sealed. A peer that proves no key resolves to anonymous. You write none of this: it happens on seven's auth seam, with the cryptography owned by the mercer sibling (the floor in valis.asd is 0.6.0). valis holds only an opaque session handle and performs zero crypto itself.

The resolved principal then sees its own view. The factotum subtree exposes a read-only /id/owner node whose contents are the session principal's own did:key. A peer that reads it over the sealed session confirms it reached its authenticated view, not a shared or anonymous one.

The client side is verification-scoped today

valis is the responder. A reusable client that drives the keyed handshake and holds a sealing session lives under tests/support/ as test support: it proves the responder against a standard-conformant transcript and is not a shipped dial surface. If your system needs to initiate keyed sessions in production, that client is the seam to promote; a shippable dial client is a deliberate future addition, not part of the proven surface. Until then, model an initiator on the test-support sealing client and seven's client.lisp session.

The authority model you must respect

Both surfaces inherit valis's central invariant, and a builder who breaks it breaks the sovereignty guarantee for everyone mounted alongside them.

  • Authority is what is mounted, not what is checked. A caller can name only what was mounted into its view. There is no per-access guard and no ambient authority. Do not add your own "is this allowed?" check around a resource: if the caller should not have it, it must be absent from the mount, and an ungranted name simply misses. Absence, not refusal: a caller names nothing of what it lacks (a miss, never a 403).
  • Two phases resolve authority. A connect-time principal sets the base namespace; a per-request capability may mount further within it. The published /pub floor is the anonymous base; /mail, /names, /proto, /id, /cap, /edge, /bus, /acme, and /backup are credential-gated and absent from an anonymous view. Mount a narrower capability for a single request with with-request-capability; never widen a base view by hand.
  • Project read-only where you only read. A view designated read-only is enforced by projection, not by you remembering not to write. Take the view you are given and respect it.
  • Never reach around the namespace to durable state. A module is an ephemeral viewer onto a person's namespace, never the owner of irreplaceable data. Touch the owner's data only through the mounted view, so a revoked grant or an evacuated namespace takes effect without your cooperation.

Running your unit

The process lifecycle lives in src/main.lisp. The main entry point dispatches the CLI:

# Foreground: listener + 9P fabric + edge ports.
sbcl --eval '(asdf:load-system :valis)' --eval '(valis/src/main:main)'

make build                          # standard driver → asdf:make :valis/delivery → bin/valis
make dist                           # stage the versioned delivery tarball (valis-serving-<version>.tar.gz)
./scripts/dev-boot.sh               # dev image with a Slynk listener

The binary takes --resident for the steered production boot, --dev for a Slynk listener (SLYNK_PORT / SLYNK_HOST, loopback only), --backup and --restore for the backup-critical set, and the zone, obtain, publish and apply verbs that drive a running resident owner-keyed over its loopback fabric. Two more verbs act on the host rather than on a resident: dist registers the private dependency distribution as this node's dependency source, and assess reads the host and reports whether it is fit to run a node, changing nothing. valis --help prints the grammar. start-fabric takes the keyfile, revocation path, the published- and durable-store data directories, and the set of locally-delivered mail domains (local-domains, empty by default, so the mail router fails closed and relays for no one until the operator names the domains valis terminates); a durable store with no keyfile fails closed with owner-key-required rather than serving unprotected.

Real eBPF steering, making one IP answer on every port, is provided by the privileged fulcrum sibling, the only component holding CAP_BPF=/=CAP_NET_ADMIN. valis mints the steered listening socket itself and pushes that descriptor to fulcrum over SCM_RIGHTS; the descriptors for the privileged ports fulcrum binds (:53, :443, :80) travel the other way, by exec inheritance. valis itself runs unprivileged and binds only loopback; in development you reach a protocol on its plain port (for example c3po-http/src/handler:*http-port*, default 7080) without steering.

The :valis/delivery system composes the additive DNS-serving stack (runciter's serve-exchange and c3po's :53 codec) into the delivery binary, so the resident can answer public :53; the plain :valis core cannot, and a binary lacking the serving codec fails closed at boot. make build is the standard command-line driver for that composition (make dist stages it into the versioned delivery tarball); it supersedes the older build.sh entry point. Standing the unit up on a real host (the fulcrum-privileged → valis-unprivileged bring-up, provisioning, and the fail-closed invariants) is the domain of the host-deployment contract (deploy = condense-from-genesis), and =proving-ground/ is the harness that validates that bring-up against a two-location Debian topology.

Where to go next

  • Architecture: the design of record behind both surfaces: the edge↔core seam your adapter sits behind, the NoiseXX handshake and factotum, the capability and enforcement model, and plugin composition as content-addressed modules.
  • API reference: every exported symbol named above, with its full signature and docstring.
  • Working with the code: the REPL-driven loop and the conventions, if your system lives inside the valis tree.