valis - Working with the Code

valis is grown against a live image, not a cold edit-compile-run cycle. You keep an SBCL image loaded with the system and its state, prototype a form at the REPL, persist it to its file, re-evaluate to confirm, then run the tests, all without restarting. The editor and the running image see the same world.

EXPLORE -> EXPERIMENT -> PERSIST -> VERIFY
   ^                              |
   +----------- REFINE -----------+

The one discipline this trades for: warm images hide missing dependency declarations. valis is a package-inferred-system where each file's defpackage declares its own :import-from, with no central component list. A form that resolves in your warm REPL because some other file already loaded the symbol will fail a cold build. So the authoritative gate is always a cold run: make test in a fresh SBCL (see Testing).

• SBCL 2.x and ASDF 3.x.
• The seven sibling 9P library, version 1.0.0 or later. The floor that carries the close/EOF wake and the node-qid writer valis depends on.
• The mercer sibling, version 0.6.0 or later. The factotum: the Noise_XX transport authenticator, the post-attach session cipher, and the ACME certificate custody the :443 edge draws on.
• The runciter sibling, version 0.8.0 or later. The authoritative-DNS answer logic the :53 serving path binds to.
• A PostgreSQL instance for the operator-state tests. Without VALIS_PG_DSN the suite stands those tests down and tells you so; scripts/pg-dev.sh brings a development one up.
• The remaining dependencies, resolvable through $LISP_WORKSPACE/ or Quicklisp.

valis resolves dependencies in a strict order (see Conventions): the project's own code first, then a local fork under $LISP_WORKSPACE/, then Quicklisp as a last resort. Point $LISP_WORKSPACE at your local-projects tree (default ~/SourceCode/lisp/) so the build picks up seven and the other local checkouts.

Bring up a live image with a Slynk listener and attach your editor (Sly, SLIME, or any Swank/Slynk client) to it:

./scripts/dev-boot.sh               # foreground
./scripts/dev-boot.sh --background  # detach; logs to /tmp/valis-dev.log

The listener honours SLYNK_PORT (default 20159 from this script; the built binary's own --dev defaults to 4005), SLYNK_HOST (default 127.0.0.1), LISP_WORKSPACE, and QUICKLISP_SETUP. The boot loads the valis system, so by the time you connect the world is ready.

From a bare SBCL the equivalent is:

(asdf:load-system :valis)

The minimal cycle is prototype → persist → re-load → test:

1. Prototype a form by evaluating it at the REPL (in the right package: each file has its own, e.g. valis/src/listener, valis/src/fabric).
2. Persist it: edit the form in its source file. Prefer structure-aware editing (your editor's "compile defun / send form" command) so indentation and surrounding comments are preserved.
3. Re-load to pick up file changes. Edited files do not auto-reload; the reliable way is (asdf:load-system :valis) (it recompiles only what changed). Compiling a single form from the editor is fine for a tight loop, but reload the system before you trust a result.
4. Verify with the test suite (see below).

• The package-per-file shape means symbols are namespaced. When a symbol is not visible, either the system is not loaded, or you are in the wrong package: qualify it (valis/src/fabric::symbol) or switch packages.
• Once the system is loaded, the editor's "find definition", "who-calls", and "describe symbol" commands are the fastest way to move; they read the live image, so they are precise.
• For a filesystem-level search that does not need the system loaded, grep the src/ tree.

Tests are written against zebra (define-test, true / false / fail / is), which each test package =:use=s. The authoritative gate is the cold suite:

make test          # the whole suite in a fresh SBCL
make test-one TEST=valis/tests/<name>   # one suite, for the inner loop
make interop-test  # drive a real Linux v9fs mount against the namespace
                   # (requires sudo / CAP_SYS_ADMIN)

make test reports its own counts as it runs, and it distinguishes three outcomes that a single exit code would flatten: tests ran and some failed, a named test package was never interned so nothing was judged at all, or the image died. Read what it prints rather than quoting a count from here, because the count moves daily and a number written down in a document is wrong the moment it is written.

⚠ Without VALIS_PG_DSN the operator-state tests stand down. That is a success, not a failure: they say why they stood down, and going red because Postgres is not running would put a false entry beside the real failures and make those easy to skim past. The run prints a banner loud enough to survive a scrollback. Bring a database up with eval "$(scripts/pg-dev.sh dsn)" before you trust a green on anything they cover.

During the inner loop you can run the test system from the REPL for fast feedback:

(asdf:test-system :valis/tests)

…but a green warm REPL is not sufficient. Re-run make test cold before you consider a change done, so a missing :import-from cannot hide.

make interop-test stands up an in-process fabric, publishes a sample item under /pub, performs an anonymous v9fs attach (-t 9p ... access=any), and asserts the reconciled model: the mount names /pub and traverses the item while the credential-gated base tree is absent, the same published floor an unauthenticated HTTP GET sees. Run it in the foreground: its readiness fifo read blocks if the server child dies, so a backgrounded run can hang.

The real-NIC steering acceptance gate is not in this repo: it lives in fulcrum (the privileged sibling) as make steer-test, a netns target proving a client dialing :80 lands on valis's single steered socket with the original port recovered.

The extension surface valis is built for. A new wire protocol is a new file src/<proto>.lisp that:

1. Subclasses protocol (src/protocol.lisp), the class that carries the per-port contract.
2. Specialises handle-connection, the per-connection entry point. The recovered destination port has already selected your module; you receive the connection with its principal and capabilities resolved.
3. Registers the protocol against the port it claims, so dispatch-connection (src/registry.lisp) routes matching connections to it.
4. Projects over a semantic subtree through the capability-mounted namespace, rather than owning state directly. Read through the published /pub view for an anonymous caller and the fuller frame for a proven owner; never reach around the namespace to touch irreplaceable state.

Registering a protocol wires dispatch: it routes the connections that arrive on your port to your module, and step 2's "the recovered destination port has already selected your module" assumes the traffic reached valis. It does not open the port to the public. On a deployed host the privileged agent's catchall steer fans every port to valis, and a separate, privileged firewall selects which are reachable from off-host, so a new public port is a host-agent change, not only a registration, and that allowlist is the agent's alone (a module's declaration can never open a port). A UDP protocol needs one step more: the steer cannot carry a UDP reply, so the agent binds the port and hands its descriptor down, as :53 already does. The public surface chapter is the full treatment.

The HTTP adapter is the worked reference: a conformant-but-bounded HTTP/1.1 server that serves two identity-selected views over one URL space. It is no longer in this tree: the HTTP and Gopher engines are the c3po-http and c3po-gopher modules from the c3po protocol sibling, adopted as content-addressed edge-adapter modules that self-register onto the edge-adapter registry (valis/src/plugin/edge-adapter) and bind their ports through the edge controller. Read them there and model a new adapter on them. The Architecture document explains the edge↔core seam your adapter sits behind, and the API reference lists the exported entry points you specialise.

A wire-protocol adapter is one of two extension surfaces; the other is a keyed, authenticated service: a 9P responder a remote peer reaches by proving a key, its session sealed end to end. Both, and the mount-not-guard authority model they share, are written up for the operator building their own system in Building on valis. This section is the in-tree adapter walk-through; that guide is the from-outside one.

make build          # the shipped binary at bin/valis
make dist           # stage the versioned delivery tarball for a host

make build drives asdf:make :valis/delivery, which composes the additive DNS serving stack into the image so the resident can answer public :53. A binary built from the plain :valis system cannot, and fails closed at boot saying so. The older build.sh still works and is superseded.

Process lifecycle lives in src/main.lisp. The resident steered boot (--resident, run-daemon) adopts the inherited :53 descriptors from fulcrum and serves over them without a privileged bind. Beyond that the binary carries --dev for a Slynk listener, --backup and --restore for the backup-critical set, and the zone, obtain, publish and apply verbs, which drive a running resident owner-keyed over its loopback fabric rather than editing anything on the host. The dist and assess verbs are the ones that do address the host: dist registers the private dependency distribution as this node's dependency source, and assess reads the host and reports whether it is fit to run a node, changing nothing. valis --help prints the grammar; --daemon survives as a no-op.

Standing a node up on a real Debian host (the bring-up order, provisioning, and fail-closed invariants) is written up in the host-deployment contract (deploy = condense-from-genesis). The =proving-ground/ tree is its regression harness: a Nix devshell fabric, a Terraform-libvirt two-location Debian topology, cloud-init provisioning, and an in-image Lisp/SSH assertion layer (valis-proving-ground, its own ASDF system) that drives the deploy / evacuate / restore scenarios. Load and test the harness like any system: (asdf:test-system :valis-proving-ground/tests).

These are not suggestions; they are how the substrate stays coherent.

• Dependency hierarchy. When you need a capability, look in strict order: (1) the project's own code: search before reaching for a library; (2) a local fork under $LISP_WORKSPACE/: fork the upstream, keep it current, add it as a local ASDF dependency so it stays patchable; (3) Quicklisp, only for stable deps with no local checkout. Prefer com.inuoe.jzon over yason for JSON. Adding a dependency is a durable decision. When in doubt, ask.
• System shape. valis.asd is a package-inferred-system: each file's defpackage declares its own dependencies; there is no central component list. Always cold-verify: a warm REPL hides a missing :import-from.
• Licensing. Every source file opens with an SPDX header (;;;; SPDX-License-Identifier: AGPL-3.0-or-later).
• Naming. Name a symbol (function, variable, class, macro, test) for the behaviour it implements. Never embed planning or ticket indices in a symbol.
• Comments. Write the what and the why. The code is the canonical how; prose narration of the implementation rots silently when the code changes.
• Handling failure. Define a condition, bind a handler that can discriminate causes, offer a named restart. A handler that cannot tell two causes apart converts a defect into a plausible value. See Condition handling. It is enforced, not merely advised.

scripts/lint-lisp.sh runs the mallet linter over the tree; --staged limits it to staged files, which is how the pre-commit hook invokes it. Configuration is .mallet.lisp at the repo root.

scripts/lint-lisp.sh              # whole tree
scripts/lint-lisp.sh --staged     # staged files only
scripts/lint-lisp.sh src/mail     # a subtree

The linter lives at $LISP_WORKSPACE/mallet. Beware a name collision: a mallet on the system PATH is an unrelated topic-modeling toolkit, so the runner resolves the path explicitly rather than by PATH: resolving by PATH would lint nothing and report success. Override with MALLET_BIN if your checkout is elsewhere.

To gate commits locally, have .git/hooks/pre-commit call scripts/lint-lisp.sh --staged. The hook is skipped when the linter is not installed, so a missing checkout cannot block a commit, which also means the gate is advisory until you have the workspace.

What it enforces, and why each is set where it is:

A genuinely ignorable failure is suppressed at the site with a written reason (; mallet:suppress no-ignore-errors -- why). The reason is the point: the linter reports suppressions that have gone stale, so they do not accumulate silently.

Treat a rule's count as a floor until you have checked what it matches. error-without-custom-condition fires only when the message string contains no colon (format arguments are irrelevant), so (error "prefix: message") goes unreported. It sees 13 sites in src/ where (error " appears 77 times, because 65 of them carry a colon. The rule is blinded by good message hygiene, which is a defect in the linter rather than a design choice.

Before trusting any rule's number, write a file containing the shapes you believe it catches and confirm which report, and if the tool is one of ours, read its source rather than probing it. Do not strip colons from error messages to make this rule's count move: the prefixes are worth more than the metric, and the defect is the linter's. That predicate is six lines; black-box probing of it produced three different confidently-held theories before anyone opened the file.

Two shell-level versions of the same trap, both encountered in practice: a mallet resolved from PATH lints nothing and exits 0, and a file list built with mapfile under zsh lints an unintended set while still printing a confident number. Assert your file count if you glob it.

• "Symbol not found": the system is not loaded ((asdf:load-system :valis)), or you are in the wrong package (qualify with pkg::symbol).
• Green warm, red cold: a missing :import-from in some file's defpackage. The warm image had the symbol from another file; the cold build does not. Add the declaration to the file that uses the symbol.
• make interop-test hangs: it was backgrounded, or the server child died while the harness waited on the readiness fifo. Run it in the foreground; ensure passwordless sudo and the 9p kernel modules are present.