Store - API reference

A content-addressed, write-once block store over a block-device. write-block returns a SHA-256 content address; read-block verifies on read and refuses a corrupted block; has-block answers presence. Identical writes dedup idempotently.

Signalled when read-block re-hashes a block's on-disk bytes and the digest does not equal the requested score — on-disk bit-rot or substitution. The corrupted block is never served; the caller gets this condition, not bytes.

Signalled when read-block is asked for a score that is not present. An absent score is answered without touching disk whenever the seek-avoidance gate can prove absence.

(block-corrupt-score condition)

Undocumented: this exported symbol needs a docstring.

(block-not-found-score condition)

Undocumented: this exported symbol needs a docstring.

(block-store-cache object)

Undocumented: this exported symbol needs a docstring.

(block-store-device object)

Undocumented: this exported symbol needs a docstring.

(has-block store score)

Return true if a block named SCORE is present in STORE. The bloom gate answers a definite absence with no disk IO; a bloom maybe falls through to the in-memory index, which is authoritative for presence (verify-on-read confirms integrity on the actual read).

(make-block-store &key device data-dir cache-budget-bytes cache-slot-cap)

Construct a BLOCK-STORE. DEVICE, when given, is used directly; otherwise a posix-block-device is built over DATA-DIR (defaulting to block-data-dir). The in-memory index and bloom are rebuilt from the device's on-disk blocks so a fresh store over an existing data directory finds every prior block. The per-store immutable-block cache is installed with a loader that closes over this store's %read-block-verified path, so admission verifies bytes against their score before they enter the cache (verify-then-admit). CACHE-BUDGET-BYTES and CACHE-SLOT-CAP override the cache's byte budget and slot cap (defaulting to the block-cache knobs) so a test can construct a small-budget store.

(pin-block cache score)

Pin the block for SCORE in CACHE and return its octets, making the slot eviction-immune until unpinned (venti ref++). When SCORE is not yet resident, load-and-admit it first (through block-cache-get, which verifies before admit), so a caller can pin a block straight from the store without a separate get — a pin is itself a hold that implies admission. The refcount bump runs under the lock; the load (if any) happens outside it via block-cache-get.

(read-block store score)

Return the exact octets stored under SCORE in STORE, verified on read. The bloom/index gate short-circuits an absent score (no disk seek on a proven miss), signalling block-not-found. On a present score, the per-store cache is consulted FIRST: a hit serves the admitted bytes with no re-hash and no device touch (the score is the validator). On a miss the cache loads through %read-block-verified — device read, then the equalp digest check — and admits ONLY the verified bytes (verify-then-admit), so a corrupt on-disk block signals block-corrupt and never enters the cache.

(rebuild-index store)

Rebuild STORE's in-memory index and bloom from the device's on-disk blocks (block-device-scan is the source of truth). A fresh bloom is sized to the scanned block count, then every scanned name is recorded in both the index and the bloom, so every previously written block is found again after in-memory state is dropped. Both structures are rebuildable caches, never an authority that can disagree with disk — verify-on-read backstops any divergence.

(store-block-digest octets)

Return the content address of OCTETS as a 32-byte vector. This is the single swap point for the block store's hash algorithm: SHA-256 today (mirroring the in-tree digest idiom in src/namespace/assembler.lisp), a future BLAKE3 by changing only this body and the on-disk name width. Called unqualified so the single-call-site discipline is grep-checkable.

(unpin-block cache score)

Decrement the pin refcount on the slot for SCORE in CACHE (venti –ref), flooring at zero so an over-unpin cannot drive the count negative. When the count reaches zero the slot becomes evictable again; broadcast the unpin condition so a writer waiting because every candidate was pinned can retry. A no-op when SCORE is not resident (the slot was already evicted while unpinned). Under the lock.

(write-block store octets)

Store OCTETS in STORE and return its content address (a 32-byte score). The score is store-block-digest of the bytes actually stored — never a caller-supplied name. Writing identical content twice is an idempotent no-op (dedup by hash): the second call returns the same score and stores nothing. The score is returned only after block-device-write completes its durable flush. The check-then-write and the index/bloom mutation run under the store lock.

(with-pinned-block (cache score) &body body)

Pin the block named by SCORE in CACHE for the extent of BODY, and release the pin on EVERY exit path — normal return, error unwind, or a non-local (deadline) transfer of control — via unwind-protect, with NO finalizer. Mirrors the %fsync-directory unwind idiom: the unpin is the cleanup clause, so a read-deadline abort or a nine-p-error unwind through BODY still releases the pin and the slot becomes evictable again. pin-block returns the pinned octets if BODY wants them; a caller that needs the bytes can wrap a (let ((bytes (pin-block …))) …) directly, but the common pin-for-the-hold shape needs only the scope, so the binding list is (cache score).

A per-store immutable-block cache decorating a content-addressed block store. Keys resident blocks by their hex score name (the score is the validator, so a hit serves admitted bytes with no re-hash and is immune to later on-disk corruption). TABLE maps a name to its cache-slot; ORDER is the CLOCK ring swept for eviction; RESIDENT tracks the current byte total against BYTE-BUDGET, with SLOT-CAP bounding the descriptor count independently. LOCK guards all mutation; UNPIN-CV wakes a writer that was waiting because every eviction candidate was pinned. Constructed via make-block-cache, never a process-global.

(block-cache-get cache score)

Return the octets for SCORE from CACHE. On a HIT, set the reference bit and return the cached octets with NO re-hash — the score is the validator, so a hit is immune to later on-disk corruption of the same name. On a MISS, load through the cache's loader (which verifies the bytes re-hash to SCORE, signalling block-corrupt on a substituted block and block-not-found when absent), admit the verified bytes, and return them (verify-then-admit).

(block-cache-pinned-count cache)

Return the number of resident slots currently pinned (refcount > 0) in CACHE. Returns to baseline after every with-pinned-block exit (the unwind-protect release proof).

(block-cache-put cache score octets)

Admit OCTETS (already verified to hash to SCORE) into CACHE under SCORE, returning the resident octets. This is the low-level admit primitive the store decorator calls AFTER its verify step (verify-then-admit): only verified bytes ever reach here. Byte-accounted and CLOCK-evicting under the lock.

(block-cache-resident-bytes cache)

Return the total resident byte count of CACHE — never above the byte budget after a put (CLOCK keeps it bounded).

(block-cache-resident-count cache)

Return the number of resident slots (descriptors) in CACHE. Bounded by the slot cap: CLOCK evicts before the count would exceed the cap, so this never rises above the configured slot cap after a put.

(make-block-cache store &key byte-budget slot-cap loader)

Construct an immutable-block cache decorating STORE. BYTE-BUDGET and SLOT-CAP default to the default-block-cache-… knobs. LOADER, when supplied, is a one-argument function (score -> verified octets) the cache calls on a miss; it MUST verify-then-return (signal rather than yield corrupt bytes). When omitted, a bare device-read-plus-digest-verify loader over STORE is used so a cache built directly (the unit suite) is self-sufficient and coherent.

(pin-block cache score)

Pin the block for SCORE in CACHE and return its octets, making the slot eviction-immune until unpinned (venti ref++). When SCORE is not yet resident, load-and-admit it first (through block-cache-get, which verifies before admit), so a caller can pin a block straight from the store without a separate get — a pin is itself a hold that implies admission. The refcount bump runs under the lock; the load (if any) happens outside it via block-cache-get.

(unpin-block cache score)

Decrement the pin refcount on the slot for SCORE in CACHE (venti –ref), flooring at zero so an over-unpin cannot drive the count negative. When the count reaches zero the slot becomes evictable again; broadcast the unpin condition so a writer waiting because every candidate was pinned can retry. A no-op when SCORE is not resident (the slot was already evicted while unpinned). Under the lock.

(with-pinned-block (cache score) &body body)

Pin the block named by SCORE in CACHE for the extent of BODY, and release the pin on EVERY exit path — normal return, error unwind, or a non-local (deadline) transfer of control — via unwind-protect, with NO finalizer. Mirrors the %fsync-directory unwind idiom: the unpin is the cleanup clause, so a read-deadline abort or a nine-p-error unwind through BODY still releases the pin and the slot becomes evictable again. pin-block returns the pinned octets if BODY wants them; a caller that needs the bytes can wrap a (let ((bytes (pin-block …))) …) directly, but the common pin-for-the-hold shape needs only the scope, so the binding list is (cache score).

Default resident byte budget for the immutable-block cache (~128 MiB). Resident bytes never exceed this; CLOCK evicts unpinned slots to make room.

Default descriptor/slot cap: the maximum number of resident cache entries. A second bound alongside the byte budget so a flood of tiny blocks cannot exhaust descriptors while staying well under the byte budget (venti's dual bound, lumpcache.c:22-23).

Block device backed by ordinary files on a normal filesystem. Each block is one file under a git-style two-char hex fanout; payload IO is stream IO (with-open-file + read-sequence/write-sequence — positional pread/pwrite are unavailable in this SBCL and file-per-block needs neither). The durable-write discipline is the fs-store path, lifted verbatim. The slot-in point for a future arena or mmap backend: implement the same four block-device GFs.

Signalled when a block write cannot be made durable: a temp-file flush, an atomic rename, or a directory fsync failed. Such a failure is fatal to the write and is never retried — the original syscall condition rides along in CAUSE for diagnosis.

(block-device-error-cause condition)

Undocumented: this exported symbol needs a docstring.

(block-device-error-name condition)

Undocumented: this exported symbol needs a docstring.

(block-device-exists-p device name)

Return true if a block named NAME is present on DEVICE, NIL otherwise.

(block-device-read device name)

Return the octet vector stored under NAME on DEVICE, or NIL when no block of that name is present.

(block-device-scan device)

Return a list of every block NAME (full-width hex) present on DEVICE, reconstructed from the on-disk fanout. The disk is the source of truth; this feeds the startup rebuild of the in-memory index and bloom filter (both rebuildable caches, never an authority that can disagree with disk).

(block-device-write device name octets)

Durably store OCTETS under NAME (a full hex score) on DEVICE. Atomic and crash-consistent: temp -> fdatasync -> rename -> fsync dir. A flush/rename/dir fsync failure signals BLOCK-DEVICE-ERROR and is never retried. Writing the same NAME with identical bytes again is harmless (the rename of byte-identical content is idempotent). Returns NAME.

(posix-block-device-data-dir object)

Undocumented: this exported symbol needs a docstring.

(durable-rename-write final-path octets &key tmp-path (name nil))

Durably write OCTETS to FINAL-PATH via the single fsyncgate-sensitive sequence: write to the caller-supplied TMP-PATH, fdatasync the data, atomically sb-posix:rename into place, then fsync the directory holding FINAL-PATH. This is the ONE implementation of that sequence; both the content-addressed block-device-write and the fixed-path head sit on it, so the most safety-critical few lines in the store live in exactly one place.

TMP-PATH MUST be on the same filesystem as FINAL-PATH (the caller controls placement so the rename is intra-device — a cross-device sb-posix:rename fails). NAME labels a BLOCK-DEVICE-ERROR. A flush/rename/fsync failure signals BLOCK-DEVICE-ERROR and is NEVER retried (the Postgres fsyncgate trap). Returns FINAL-PATH. The directory holding FINAL-PATH must already exist; creating a fanout subdir and fsync'ing its parent is the caller's job (block-device-write).

(make-posix-block-device data-dir)

Construct a POSIX-BLOCK-DEVICE rooted at DATA-DIR.

Pathname for the block store's data directory. A fresh seam variable, deliberately NOT a reuse of fs-store's pub-data-dir: the block store is independent of the publication store and stays import-clean. Nil before the store is up; tests bind it (or pass an explicit data-dir to the device).

When true (the default), block writes are flushed to stable storage before reporting success: the temp file is fdatasync'd before the rename, and the containing directory is fsync'd after, so a write that has returned survives a host crash or power loss. Bind to NIL only where durability is deliberately traded for speed (bulk imports, benchmarks). Mirrors fs-store's flag so the durability test surface is identical.

The in-image handle for a store's mutable HEAD pointer. LOCK serializes the read-compare-write CAS critical section in this image. It is NOT the authority — the on-disk HEAD generation is; the lock only makes this image's advances mutually exclusive. A second OS process is fenced by the on-disk generation compare at the rename, not by this lock.

Signalled when the head record is structurally invalid or its integrity digest does not match its bytes (media rot — the atomic rename guarantees no torn head ever, so a digest mismatch means corruption, not a partial write). The head is never trusted on a mismatch: the caller gets this condition rather than a stale or partial generation. A hard fault, not auto-repair. This is the ONLY condition the head signals; an expected stale CAS rejection returns the sentinel (values nil :stale-generation current-generation).

(head-corrupt-reason condition)

Undocumented: this exported symbol needs a docstring.

(advance-head head expected-generation new-root-entry)

Compare-and-swap the head. Under the in-image head lock: read the current on-disk generation; if it does not equal EXPECTED-GENERATION, REJECT (write nothing) and return (values nil :stale-generation current-generation); else allocate (1+ current), pack a record carrying NEW-ROOT-ENTRY and the prior root entry as prev, durable-rename-write it over HEAD, and return (values new-generation nil). EXPECTED-GENERATION is NIL for the genesis create, which succeeds only if no HEAD exists yet (the git 40-zeros create guard); a NIL- expected create against an existing head returns the stale sentinel. The compare and the rename are one atomic critical section. A stale rejection is an EXPECTED outcome — the sentinel, never a condition; only a corrupt head signals.

The fault-injection seam fires BEFORE the lock is acquired: an injected stop-the-world pause models the writer arriving LATE at the resource (Kleppmann), so a concurrent writer can advance the head meanwhile and this writer's now- superseded generation is rejected when it finally acquires the lock and re-reads the current generation. Firing the hook inside the lock would deadlock a concurrent advancer and make the in-image race unobservable.

(current-head store)

Read and verify the current on-disk head. Returns (values generation root-entry prev-entry), or NIL when no HEAD exists yet (genesis state). A present-but-corrupt head signals head-corrupt — never silently treated as genesis.

(current-head-generation store)

Read and verify the on-disk head, returning ONLY its generation (or NIL at genesis). The cheap revalidation primitive the head decode cache keys on: it reads and integrity-verifies the 124-byte HEAD record exactly as current-head does (a present-but-corrupt head signals head-corrupt, never silently genesis), but discards the root and prev entries so it triggers no tree decode — a revalidation that hits the cache pays no read-block / decode-directory.

(make-head store)

Construct the in-image HEAD handle over STORE.

(pack-head generation root-entry prev-entry)

Return a fresh head-record-size octet vector: the canonical head record. PREV-ENTRY may be NIL (genesis) -> an all-zero prev slot. The SHA-256 integrity digest is computed over bytes [0 .. digest-offset) and written last, then an exact-width assert proves the encoding has no slack.

(unpack-head vec)

Decode a head record. Returns (values generation root-entry prev-entry). Validates width, then verifies the integrity digest BEFORE trusting any field (verify-on-read), then validates the version, then unpacks generation + root + prev. PREV-ENTRY is NIL when the prev slot is all-zero (genesis). Any failure fails closed with head-corrupt — never a stale or partial value.

A test/fault-injection seam. When non-NIL, advance-head funcalls it with (head expected-generation) BEFORE advance-head acquires the head lock (and thus before the compare and the durable rename). A test rebinds it to block writer A (on a condition variable) so writer B can acquire the lock and advance the head meanwhile; A then resumes, acquires the lock, and its CAS is rejected against the now-superseded generation. Firing the hook outside the lock is what keeps the in-image two-writer race deadlock-free and observable. The pause may be arbitrarily long, proving the fence is clock-free (no timeout governs the rejection). Defaults to NIL — zero cost, no behavior change in production.

The generation a genesis head lands at; the first advance lands 1.

Total fixed head record width: version[2] + generation[8] + root-entry[41] + prev-entry[41] + integrity-digest[32] = 124.

Own head record version (mirrors the tree codec's version-first discipline).

A decoded namespace manifest: PINNED-GENERATION (the captured state's generation) and MOUNTS, the ordered (name . mount-target) list.

Signalled fail-closed when a manifest record is structurally invalid: a bad envelope width, an unknown version, a malformed mount table, or a referenced subtree root that is missing (block-not-found) or corrupt (block-corrupt) at read time. A hard fault, never auto-repair, never a partial/empty namespace. Byte integrity is provided by the store's content addressing (read-block re-hashes on every read); this condition covers STRUCTURAL failure only — there is no manifest-level integrity digest.

(manifest-corrupt-reason condition)

Undocumented: this exported symbol needs a docstring.

(axis-name->designator name)

Map an axis NAME string to its built-in designator keyword. The default axes are named exactly by their designators, so the keyword is recoverable from the name on decode — no designator byte is stored for a built-in target.

(decode-manifest store envelope-entry &key (resolve-subtrees t))

Invert encode-manifest: read the envelope named by ENVELOPE-ENTRY, unpack it, recover the name->target directory and the tag stream, and reconstruct the ordered (name . mount-target) list with the pinned generation. Any block read failure or structural mismatch fails closed as manifest-corrupt — never a partial namespace. Returns the in-memory manifest value.

RESOLVE-SUBTREES (default T) controls whether each subtree-variant mount's root block is fetched to confirm it resolves. Pass nil for a structural-only decode that leaves reachability to a downstream check (the namespace assembler verifies every referenced root once, so re-reading the blocks here would be wasted work).

(encode-manifest store manifest)

Encode MANIFEST into STORE and return the manifest envelope's content-addressed ENTRY. Builds the name->target directory (a builtin target's slot is a zero entry; a subtree target's slot is its real entry), the one-byte-per-child tag stream, and the reserved (empty) key-location stream, packs the envelope, and write-blocks each — durable-before-returned (write-block flushes before returning the score).

(make-builtin-target designator)

Construct a built-in-axis-designator mount target (variant a).

(make-default-manifest &key (pinned-generation 0))

Build the default manifest: each default-axis-names entry as a built-in designator (variant a), zero subtree entries, zero key locations, pinning PINNED-GENERATION (default 0 — a literal, NOT the head's genesis constant, since the default exists before any head). Descriptive only — wires no live node (the assembler does that).

(make-manifest pinned-generation mounts)

Construct a manifest value pinning PINNED-GENERATION over the ordered MOUNTS list of (name . mount-target) pairs.

(make-subtree-target entry)

Construct a content-addressed subtree-root mount target (variant b).

(manifest-mounts instance)

Undocumented: this exported symbol needs a docstring.

(manifest-pinned-generation instance)

Undocumented: this exported symbol needs a docstring.

(manifest-reachable-scores store head-root-entry)

Return a list of every block score the namespace manifest under HEAD-ROOT-ENTRY references, or NIL when the head tree carries no "manifest" child. A plain tree walk stops at the manifest envelope's own file blocks — the envelope's bytes NAME further blocks (its name->target directory frame, the tag stream, the reserved key-location stream, and every subtree-mount root) that only this codec knows how to reach. Builtin mounts are inline zero-entries naming no block and are skipped; each subtree mount's whole tree is collected in full. This is the manifest half of the store's reachable-set walk, complementing collect-tree-scores.

(manifest-to-octets store manifest)

Serialize MANIFEST into STORE and return the concatenation of the envelope and EVERY block it references — the full serialized surface. This walks the envelope, the name->target directory (its root block AND both internal stream blocks), the per-child tag stream, the reserved key-location stream, and every subtree-root block, so a future keyed mount cannot hide key bytes in an unwalked stream past a byte-absence assertion.

(mount-target-builtin-p mount-target)

True when MOUNT-TARGET is a built-in axis designator (variant a).

(mount-target-designator instance)

Undocumented: this exported symbol needs a docstring.

(mount-target-subtree instance)

Undocumented: this exported symbol needs a docstring.

(mount-target-tag instance)

Undocumented: this exported symbol needs a docstring.

(pack-manifest-envelope version pinned-generation dir-entry tag-entry keyloc-entry)

Return a fresh manifest-envelope-size octet vector: the canonical manifest envelope. Version-first, then the pinned generation, then the three stream-naming entries (dir name->target, tag bytes, reserved key-location). The exact-width assert proves no slack. NO integrity digest — the envelope block's content address authenticates the bytes.

(publish-manifest head store manifest &key expected-generation)

Encode MANIFEST into STORE, place it in a tree naming a "manifest" child, and advance-head HEAD to that tree. Returns advance-head's (values new-generation reason). The pinned generation inside MANIFEST is INDEPENDENT of the generation advance-head mints: the head advances (1+ cur-gen) on every commit; the manifest's pinned value records the captured state's generation. Everything the manifest references is durable before the head names it (write-block flushes before the advance).

(read-manifest store root-entry &key (resolve-subtrees t))

Walk ROOT-ENTRY (a tree naming a "manifest" child) to the manifest envelope entry and decode-manifest it. A missing "manifest" child, or a missing/corrupt referenced root reached during decode, fails closed as manifest-corrupt — never a partial namespace.

RESOLVE-SUBTREES (default T) is forwarded to decode-manifest: pass nil for a structural-only read that defers subtree-root reachability to a downstream check.

(store-manifest store manifest)

Encode MANIFEST into STORE and return the manifest envelope's content-addressed ENTRY, WITHOUT advancing any head — the caller controls the tree the manifest is placed in. Durable-before-returned (write-block flushes before returning).

(unpack-manifest-envelope vec)

Decode a manifest envelope. Returns (values version pinned-generation dir-entry tag-entry keyloc-entry). Validates width FIRST, then version, BEFORE indexing any field (verify-on-read). No integrity digest check. Fails closed with manifest-corrupt.

The live code's canonical axis set, in code order (the axis loop in src/namespace/assembler.lisp). Each is a built-in axis designator in the default manifest. /ctl is NOT here — it is a live fabric-synthesized control door, not a durable axis. These strings are DATA: holding the axis names needs no namespace import, which is what keeps the manifest import-clean while reproducing the axes.

Total fixed envelope width: version[2] + pinned-generation[8] + dir-entry[41] + tag-entry[41] + keyloc-entry[41] = 133. The dir-entry names the two-stream name->target directory; the tag-entry names the per-child variant tag stream; the keyloc-entry names the reserved (all-empty) key-location stream. NO integrity digest field.

Own manifest record version (version-first, mirroring the tree/head codec).

Mount-target variant (a): a built-in axis designator.

Mount-target variant (b): a content-addressed subtree-root entry.

A decoded module manifest: PINNED-GENERATION and RECORDS, the canonically ordered (system-name . module-record) list. PINNED-GENERATION is the generation captured when the manifest was published — descriptive provenance that is round-tripped faithfully but is NOT consulted as a fence or freshness gate by any consumer: condense-modules re-derives trust freshly against the LIVE revocation/fence and ignores this field. It records the captured generation, it enforces nothing — do not read it as a pin.

A decoded module record: the module's ASDF SYSTEM-NAME, its ordered RELATIVE-FILES list of source-path strings, and its 32-byte content SCORE (opaque — this codec records it, the publisher computes it).

Signalled fail-closed when a module-manifest record is structurally invalid: a bad envelope width, an unknown version, a malformed module-record, or a referenced detail-stream block that is missing (block-not-found) or corrupt (block-corrupt) at read time. A hard fault, never auto-repair, never a partial/empty module set. Byte integrity is provided by the store's content addressing (read-block re-hashes on every read); this condition covers STRUCTURAL failure only — there is no manifest-level integrity digest.

(module-manifest-corrupt-reason condition)

Undocumented: this exported symbol needs a docstring.

(decode-module-manifest store envelope-entry)

Invert encode-module-manifest: read the envelope named by ENVELOPE-ENTRY, unpack it, recover the system-name -> detail directory, and reconstruct the ordered (system-name . module-record) list with the pinned generation. Any block read failure or structural mismatch fails closed as module-manifest-corrupt — never a partial module set. Returns the in-memory module-manifest value.

(encode-module-manifest store manifest)

Encode MANIFEST into STORE and return the envelope's content-addressed ENTRY. Serializes each module-record's detail stream, builds the system-name -> detail directory, packs the envelope, and write-blocks each — durable-before-returned (write-block flushes before returning the score).

(make-default-module-manifest &key (pinned-generation 0))

Build the default module manifest: each default-module-systems name as a record with an empty relative-files list and an all-zero score sentinel, pinning PINNED-GENERATION (default 0 — a literal, NOT a head constant, since the default exists before any head). Descriptive only — names the resident set, loads no module.

(make-module-manifest pinned-generation records)

Construct a module-manifest recording PINNED-GENERATION over RECORDS, a list of (system-name . module-record) pairs. The records are canonically sorted by system-name (string<) so input order does not change the serialized bytes — the determinism a content-addressed record requires.

(make-module-record system-name relative-files score)

Construct a module-record over SYSTEM-NAME (string), RELATIVE-FILES (a list of relative-path strings), and SCORE (a 32-byte content score, stored opaque).

(module-manifest-pinned-generation instance)

Undocumented: this exported symbol needs a docstring.

(module-manifest-reachable-scores store head-root-entry)

Return a list of every block score the module manifest under HEAD-ROOT-ENTRY references, or NIL when the head tree carries no "modules" child. A plain tree walk stops at the module envelope's own file blocks — the envelope names a further system-name->detail directory whose every per-module detail stream only this codec knows to reach. This is the module half of the store's reachable-set walk, complementing collect-tree-scores. A module record's opaque bundle SCORE is NOT walked here: bundles are resolved by the module puller (a local-store or future network transport), not guaranteed to be local blocks, so they lie outside the store's block-reachability closure.

(module-manifest-records instance)

Undocumented: this exported symbol needs a docstring.

(module-manifest-to-octets store manifest)

Serialize MANIFEST into STORE and return the concatenation of the envelope and EVERY block it references — the full serialized surface. This walks the envelope, the directory root block AND both internal stream blocks it names, and every per-module detail stream, so a byte-absence assertion cannot be evaded by an unwalked stream (mirrors manifest-to-octets exactly).

(module-record-relative-files instance)

Undocumented: this exported symbol needs a docstring.

(module-record-score instance)

Undocumented: this exported symbol needs a docstring.

(module-record-system-name instance)

Undocumented: this exported symbol needs a docstring.

(pack-module-manifest-envelope version pinned-generation dir-entry)

Return a fresh module-manifest-envelope-size octet vector: the canonical module-manifest envelope. Version-first, then the pinned generation, then the directory-naming entry (system-name -> detail stream). The exact-width assert proves no slack. NO integrity digest — the envelope block's content address authenticates the bytes.

(publish-genesis-manifests head store namespace-manifest module-manifest &key expected-generation)

Publish BOTH the namespace manifest and the module manifest under ONE durable root tree advanced ONCE. Obtains the namespace envelope entry via store-manifest (encode without advancing), the module envelope via encode-module-manifest, builds ONE root tree naming "manifest" and "modules", and advance-heads ONCE. Returns advance-head's (values new-generation reason). This is the ONLY genesis-correct sibling path — the two-advance sequence would CAS-fail the second advance closed. Everything both manifests reference is durable before the single advance names the root (write-block flushes before the advance).

(publish-module-manifest head store manifest &key expected-generation)

Encode MANIFEST into STORE, place it in a tree naming a "modules" child, and advance-head HEAD to that tree. Returns advance-head's (values new-generation reason). This is the SINGLE-child publish — used when "modules" is the only thing changing (e.g. a module-set update over an already-genesis'd store). At genesis, use publish-genesis-manifests instead (both children, one advance). Everything the manifest references is durable before the head names it (write-block flushes before the advance).

(read-module-manifest store root-entry)

Walk ROOT-ENTRY (a tree naming a "modules" child) to the module-manifest envelope entry and decode-module-manifest it. A missing "modules" child, or a missing/corrupt referenced detail stream reached during decode, fails closed as module-manifest-corrupt — never a partial module set.

(unpack-module-manifest-envelope vec)

Decode a module-manifest envelope. Returns (values version pinned-generation dir-entry). Validates width FIRST, then version, BEFORE indexing any field (verify-on-read). No integrity digest check. Fails closed with module-manifest-corrupt.

The resident self-registering adapter set, in code order — the names the resident adapters pass to register-edge-adapter (the adopted c3po-http and c3po-gopher modules). These strings are DATA: holding the names needs no plugin/edge import, which is what keeps the codec import-clean while reproducing the resident set.

Total fixed envelope width: version[2] + pinned-generation[8] + dir-entry[41] = 51. The dir-entry names the content-addressed directory mapping system-name -> the module-record's detail stream; the module count is recoverable from the directory length. Derived from entry-size, never a literal. NO integrity digest field — the envelope block's content address authenticates the bytes.

Own module-manifest record version (version-first, mirroring the tree/head/ manifest codec).

(collect-reachable-scores store root-entry)

Return a deduplicated list of every block score reachable from ROOT-ENTRY, the directory entry a store HEAD names. This turns a 124-byte HEAD pointer into the concrete block set a restored node must have on disk to stand up: the plain tree (the root frame, the publication tree, any mail directory structure) UNION the namespace-manifest closure UNION the module-manifest closure. A NIL ROOT-ENTRY — a genesis store with no HEAD — yields the empty set.

Coverage boundary: the walk reaches every block the store-layer codecs name — the head tree, both manifests with their subtrees and per-module detail streams, and publication attribute/body blocks. It deliberately does NOT open a mail transport-state envelope's detail/body streams (mail is a higher layer than the store, and a queued message body is transient operator state, not part of the critical stand-up set), nor a module record's opaque bundle score (resolved by the module puller, not guaranteed to be a local block). A mail directory's envelope blocks are still carried as ordinary file blocks by the plain tree walk; only the streams their bytes name are outside the closure.

A decoded node entry: its self-describing TYPE-DEPTH byte, logical SIZE, and the 32-byte SCORE of its top block.

Signalled when decoding stored node bytes that are structurally invalid: a truncated record, an unknown type+depth base, an out-of-range depth, a bad MetaBlock magic, or an index/offset out of bounds. The decode validates every bound BEFORE indexing, so a malformed node fails closed with this condition rather than crashing or reading out of bounds.

(tree-decode-error-reason condition)

Undocumented: this exported symbol needs a docstring.

(collect-tree-scores store entry)

Walk the plain tree rooted at ENTRY and return a deduplicated list of every reachable block score. A type-data (file) entry contributes its whole pointer-tree; a type-root (directory) entry contributes its frame (root, entry-array, MetaBlock) and then recurses into every child entry. A NIL ENTRY yields the empty list (a genesis store with no HEAD names no tree).

Every child slot is dereferenced, so ENTRY's directories must bind their children to real block-naming entries — as the head's root tree, a publication tree, and a module manifest's system->detail directory all do. A directory whose children are inline sentinel zero-entries (a namespace manifest's name->target table) must use directory-frame-scores instead, so the walk never dereferences a zero score.

(decode-directory store root-entry)

Read the directory root named by ROOT-ENTRY, recover both streams, and return the ordered vector of (name . child-entry) pairs. Validates the root base and that the two streams agree in child count.

(decode-file store file-entry)

Read FILE-ENTRY's pointer-tree through read-block and reconstruct the byte-identical octets, trimmed to the entry's logical size. Validates the type+depth byte against type-data and the size-derived depth before descending.

(decode-tree store entry)

Alias for resolve-tree: reconstruct the in-memory tree from its root ENTRY.

(directory-frame-scores store dir-root-entry)

Return the three block scores that FRAME a directory — its root block, its VtDirType entry-array block, and its MetaBlock — WITHOUT descending into the directory's children. DIR-ROOT-ENTRY must be a type-root directory entry. This is the shallow frame a reachability walk collects for a directory whose children are decoded by a higher codec: a namespace manifest's name->target table binds its builtin mounts to inline zero-entries that name no block, so the child slots must never be dereferenced as scores.

(encode-directory store children)

Encode CHILDREN — a sequence of (name . child-entry) — as a venti/vac two-stream directory: a VtDirType entry-array (structure) + a MetaBlock (names), named together by a root block. Returns the directory's root ENTRY (typed type-root, size = the root byte count).

(encode-file store octets)

Encode OCTETS as a depth-typed venti pointer-tree over block-size leaves and return a file ENTRY naming the top score and the logical size. The depth is DERIVED from the byte count (size->depth), never stored arbitrarily.

(encode-tree store node)

Encode NODE recursively and return its root ENTRY. NODE is (:file OCTETS) or (:dir (NAME . CHILD-NODE) …). Each file's bytes become a file entry, each sub-directory a directory root, then the directory naming them is encoded. Every node threads through write-block, so a one-leaf change rewrites only the root-path blocks and an unchanged subtree dedups to a no-op.

(entry-score instance)

Undocumented: this exported symbol needs a docstring.

(entry-size instance)

Undocumented: this exported symbol needs a docstring.

(entry-type-depth instance)

Undocumented: this exported symbol needs a docstring.

(pack-entry type-depth size score)

Return a fresh entry-size octet vector: the canonical fixed-width entry. SCORE is a 32-byte vector, SIZE the logical byte count, TYPE-DEPTH the self-describing byte. Zeroed pad, no slack.

(resolve-tree store entry)

Invert encode-tree: reconstruct the in-memory NODE from its root ENTRY. A type-data entry decodes to (:file OCTETS); a type-root entry decodes to (:dir (NAME . CHILD-NODE) …).

(unpack-entry vec off)

Decode one entry from VEC at OFF. Returns (values entry new-offset). Validates remaining width and the type+depth byte before indexing.

Fixed width in octets of an on-disk node entry: the type+depth byte (1) plus the logical SIZE (size-bytes = 8) plus the SCORE (score-size = 32) = 41. This layout is pinned — it is the determinism guarantee, so changing it changes every root hash.

Width in octets of a score — the content-addressed digest that names a block. Pinned at 32; it sizes every entry's score field and the pointer-block fan-out, so changing it changes every root hash and breaks read-back of existing trees.