varde/docs/milestone-2.md

Milestone 2 — Store

iroh-blobs 0.35 persistent fs store wired in under <store_dir>/blobs. Add, Pin/Unpin (local intent only — fetch comes with milestone 3), Materialize, Status {hash}, List, Gc all work end to end.

Decisions

• Pins and formats live in meta.json (spec allows JSON for MVP), written atomically via temp file + rename. It records pin policies, trusted peers (used from milestone 4), and the format of hashes we've seen (raw vs hash_seq) — the bytes of a hash alone don't say whether it's a directory root, and materialize needs to know. Unknown hashes default to raw.
• Directories are iroh Collections (HashSeq with a metadata child), imported by walking the tree in sorted order for deterministic roots. File contents only; names are relative paths with / separators. Symlinked directories are rejected (cycle risk), symlinked files are followed. Collection entry names are sanitized on export (no .., no absolute components) so a hostile collection can't escape the target directory.
• Materialize is our own export loop, not iroh's export(), for one reason: honest reflink reporting. iroh's fs export does reflink internally but doesn't say whether it happened, and its "copy" mode reflinks too, which would make our mode: copy a lie. We reflink (reflink-copy crate → FICLONE) straight from the store's data/<hash>.data file when the blob is complete and file-backed (small blobs are inlined in redb and are always streamed), and fall back to a chunked streaming copy. Verified against real XFS in the test suite: reflink_when_supported runs under CARGO_TARGET_TMPDIR (same filesystem as the repo) and degrades to verifying the copy fallback on filesystems without reflink support.
• Store reads run on a LocalPool. iroh-blobs entry readers yield non-Send futures; the daemon uses the same local-pool pattern iroh-blobs itself uses for its provider and GC tasks, wrapped in one on_pool helper.
• GC is an explicit one-shot mark and sweep (the spec's Gc {} op), not iroh's periodic gc_run loop, which only offers a recurring timer. Roots are the pins (HashSeq roots expand to their children), plus any in-flight temp tags and database tags, so a concurrent Add can't be swept mid-import. Add without Pin leaves content GC-able by design — add-then-pin is the intended sequence, and GC never runs implicitly.
• Property test (32 cases, 0–64 KiB, spanning the ~16 KiB inline threshold) drives the real daemon binary over the socket, not the library: add → materialize → bit-identical.

Deviations from the spec

• The spec suggested reflink via the reflink-copy crate; we do use it, but from the store's data file rather than wrapping iroh's export — see above. Never hardlinks, store files stay immutable, and reflink still requires store and destination on one filesystem (documented).
• btrfs verification specifically: CI/dev runs use whatever filesystem hosts the repo (XFS here, reflink exercised for real); a loopback btrfs mount would need root, which integration tests shouldn't assume.