Commit Graph

12 Commits

Author SHA1 Message Date
bl 5d29bdc4bd Pre-0.1 cleanup: fmt, docs, licenses, AUR-ready packaging
ci / quality (push) Failing after 6s
- cargo fmt across the workspace (the CI gate the last push tripped).
- varde-daemon: drop the now-unused bytes dependency.
- varde-ctl(1): document push; gc reflects the continuous-sweep
  reality; subscribe mentions push events.
- config.toml example: gc_interval_secs, and an honest note that
  max_download_bytes_per_sec is currently unenforced.
- LICENSE-MIT + LICENSE-APACHE at the root (the declared license now
  ships as files), bundled into release tarballs and installed by the
  PKGBUILD.
- PKGBUILD: real maintainer, AUR pre-flight note (updpkgsums), license
  installation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 15:04:22 +02:00
bl c52c3b1238 Release pipeline: cargo-release config + Gitea Actions workflows
ci / quality (push) Failing after 1m43s
- release.toml: one shared workspace version, single vX.Y.Z tag,
  full test suite as the pre-release gate, no crates.io publishing.
- .gitea/workflows/ci.yml: the Woodpecker quality bar (fmt, clippy
  -D warnings, tests) for repos served by act_runner.
- .gitea/workflows/release.yml: on a version tag, build stripped
  release binaries for x86_64 and aarch64 Linux, bundle them with the
  systemd units, rendered man pages and example config, and attach the
  tarballs (+ sha256) to a Gitea release via the API.
- dist/PKGBUILD: point at the repo's new home on project.uhhm.no.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 14:54:17 +02:00
bl 4033c0ffab Protocol v2: trusted-peer push, truthful partials, split downloads
Three capabilities the iroh-blobs 0.103 line makes possible:

- Push (new API surface, protocol v2): `Push { hash, node_id }` hands
  fully-present content to a trusted peer, unprompted. Consent is
  mutual — the sender pushes only to peers it trusts, and the receiver
  (which now accepts connections unconditionally and gates per request)
  admits pushes only from peers *it* trusts, deferring with RateLimited
  while metered. An accepted push is pinned by the receiver (default
  policy, format inferred from the request ranges) once its transfer
  completes, and surfaces as a PushReceived event. Because QUIC writes
  are fire-and-forget, the sender confirms delivery by observing the
  receiver's bitfields (root + last hashseq child) before replying —
  Pushed { bytes } means verified received, not merely sent. New
  varde-ctl `push` command.

- Truthful partial presence: Status/List report bytes actually present
  and verified for partial blobs, from the store's bitfields via
  observe, instead of the old "0 until complete".

- Split downloads: multi-provider fetches stripe one request across
  providers (SplitStrategy::Split) instead of trying them serially.

Trusted peers may observe bitfields even when serving is disabled or
metered — bitfields are metadata, and push confirmation rides on them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 14:52:27 +02:00
bl ad69f7d884 Migrate to iroh 1.0 and iroh-blobs 0.103
The 0.35 pin's rationale ("the post-0.35 rewrite is not yet production
quality") expired when iroh hit 1.0 in June 2026: the rewrite line
(0.103) is now the production line and the only one receiving fixes.

The rewrite replaced wrappable store traits with an irpc-based API, so
the seams moved:

- shaped.rs: the 440-line ShapedStore trait wrapper becomes a provider
  event handler. Trust gating (untrusted peers see only openly-served
  hashes) now intercepts requests before any bytes move; upload rate
  limiting rides the provider's Throttle hook; upload progress events
  come from per-request update streams. The TokenBucket is unchanged.
- store.rs: FsStore's API handle replaces the store traits, and the
  LocalPool machinery for non-Send futures is gone. GC is now the
  store's built-in periodic mark-and-sweep, fed by a pin-roots snapshot
  via the protect callback (new config knob gc_interval_secs, default
  300); the on-demand Gc request answers Unimplemented, and the gc
  conformance test polls the sweep instead.
- transfer.rs: BlobsProtocol + Router replace handle_connection, the
  new multi-provider Downloader replaces the old queue, and mdns
  discovery moved to the iroh-mdns-address-lookup crate (it left iroh
  core in 1.0). Ticket-embedded provider addresses feed a MemoryLookup
  address book. Endpoint presets: Minimal (LAN-only default) or N0
  (wan_upload), preserving the old relay posture.
- Node* became Endpoint* throughout; announcement signatures use iroh's
  own Signature type (ed25519-dalek dep dropped); iroh-io dropped.

Known regression: max_download_bytes_per_sec is currently not enforced
— download shaping rode the old store's batch writer and 0.103's
downloader has no equivalent seam yet.

Announcement wire format note: EndpointAddr serializes differently than
NodeAddr, so pre- and post-migration daemons won't parse each other's
LAN announcements. Announcements are live-only, nothing stored breaks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 14:03:15 +02:00
bl a2d150225a Point workspace repository at the new uhhm home
varde moved from bl/varde on klingenbergbygg to the uhhm org on
project.uhhm.no.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 12:44:47 +02:00
bl 8c98c00549 socket-api feature: make the daemon core embeddable in-process
Gate the unix-socket server, systemd activation, and the socket-serving
run() behind a new default socket-api feature, with required-features on
the binary. The core modules (daemon, store, transfer, discovery, meta)
stay feature-free, so platforms without a daemon model (iOS — redoal
ADR-0012) can embed Daemon directly via
cargo check -p varde-daemon --no-default-features.

Also cfg-gate the abstract-socket branch of sd_notify to Linux: abstract
socket names don't exist elsewhere, and this was the one spot keeping
varde-daemon from compiling on macOS at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 12:41:01 +02:00
bl 258fa072aa Milestone 6: DiscoveryProvider seam, signed announcements, redoal sketch
The discovery module defines the seam: DiscoveryProvider (subscribe/
announce over 32-byte TopicKeys) and a signed Announcement carrying
root hash, ed25519 author, metadata and provider addresses. Signatures
cover a deterministic postcard encoding including the topic (no cross-
channel replay) and the provider identities (addresses stay refreshable
hints). LanDiscovery conforms to the trait — mdns sightings become
locally-authored announcements, one per pinned root — and the daemon's
auto-sync now runs entirely through it: verify, index unconditionally,
fetch only for trusted authors from allowlisted providers on already-
pinned incomplete roots. The milestone-4 real-mdns sync test passes
unchanged through the new path. Verification unit tests cover round
trip, tampered root, wrong topic, forged author, mismatched signing
key, and serde survival. docs/redoal-integration.md sketches the
gesture-topic gossip provider against this contract.

Also: fix a flaky hang in the socket-activation test (dup2(3,3) leaves
CLOEXEC set when the listener already sits on fd 3; parent's listener
copy masked daemon death), and give the test client a read-timeout hang
guard. Add a top-level README.

Dependencies: ed25519-dalek (Signature type; same implementation iroh
keys use), postcard (deterministic signed encoding, iroh's canonical
compact codec).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:51:57 +02:00
bl 871280553e Milestone 5: metered awareness, DSCP, systemd, man pages, packaging
Metered detection polls NetworkManager's Metered property over D-Bus
(feature "metered", default on; builds without D-Bus via
no-default-features). While metered the daemon closes incoming blob
connections and defers every fetch; VARDE_FORCE_METERED=true forces the
state as a kill switch and test hook. Endpoint UDP sockets get DSCP CS1
best-effort by matching bound ports to /proc/net/udp inodes (iroh hides
its fds). systemd socket activation adopts LISTEN_FDS fd 3, readiness
is a hand-rolled sd_notify READY=1 (abstract + path sockets), and
standalone binding still works unchanged. dist/ ships hardened system
and user units (DynamicUser, ProtectSystem=strict, StateDirectory,
RestrictAddressFamilies), a commented config example, scdoc man pages
validated with scdoc, and an untested PKGBUILD skeleton.

Tests: activation-socket round trip via a real fd-3 handoff, READY=1
received on a NOTIFY_SOCKET, metered daemons neither serve nor fetch.

Dependencies: zbus (optional, feature-gated D-Bus client for the
NetworkManager metered flag).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:07:24 +02:00
bl 20bdf56668 Milestone 4: LAN discovery, trust gate, auto-sync, rate limits, events
mDNS-style LAN discovery (iroh MdnsDiscovery, discovery flag, default
on) feeds a presence tracker; trusted peers that appear trigger fetches
of every incomplete pin, and Pin itself now fetches from present
trusted peers. Serving is our own ProtocolHandler: trusted NodeIds get
the full store, everyone else a filtered view limited to open_lan pins
and ticket-exported hashes (plus hashseq children) that answers "not
found" for the rest. Ticket export records standing serve-consent for
that hash; trust changes take effect on new connections. ShapedStore
implements the full iroh-blobs Store trait to charge provider reads to
an upload token bucket and downloader writes to a download bucket;
upload cap 0 closes incoming connections at accept. Subscribe now
streams transfer_progress both ways, peer_joined, and pin_complete.

Tests: forged-ticket trust gating (denied untrusted, served after
trust), 256 KiB/s upload cap enforced by wall clock, event stream
during a transfer, and real-mdns auto-sync between two daemons
(skips where multicast is unavailable).

Dependencies: n0-future, async-channel, futures-lite, bytes — all
already in the tree via iroh; needed directly to name types in
iroh-blobs trait signatures and channels. iroh feature
discovery-local-network for MdnsDiscovery.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 22:27:58 +02:00
bl 61171a5ea8 Milestone 3: iroh endpoint, tickets, pin-triggered fetch
The daemon binds an iroh endpoint (ed25519 identity at secret.key,
0600), serves its store via Blobs/Router on the standard ALPN, and
fetches with the iroh-blobs Downloader rather than the rpc client to
keep quic-rpc out of the tree. Relay is disabled unless wan_upload is
set: LAN-only, zero WAN upload by default. TicketImport pins first
(the pin is the GC root protecting in-flight data), registers the
ticket's NodeAddr with the endpoint (the downloader dials by NodeId
alone), then fetches in the background; completion emits pin_complete.

Tests: two-daemon localhost transfers (blob + directory collection,
byte-identical), and kill -9 mid-transfer followed by restart on the
same store resuming to completion.

Dependencies: iroh 0.35 (endpoint/router, pairs with iroh-blobs 0.35),
rand 0.8 (secret key generation, same version iroh uses).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 21:18:25 +02:00
bl e256f73439 Milestone 2: persistent blob store, add/materialize/gc
iroh-blobs 0.35 fs store under <store_dir>/blobs. Add imports files or
whole directory trees (as Collections, deterministic order), Materialize
exports them with a real FICLONE reflink attempt and streaming-copy
fallback, Gc is an explicit mark-and-sweep rooted at the pins. Pins,
trusted peers and hash formats persist in meta.json (atomic writes).
Store reads run on a LocalPool because iroh-blobs entry readers are not
Send. Integration tests drive the real daemon binary: directory round
trip byte-identical, gc keeps pinned/drops unpinned, reflink verified on
the repo's own filesystem (XFS), plus a 32-case proptest round trip.

Dependencies: iroh-blobs =0.35.0 (the store itself; pinned per spec),
iroh-io (AsyncSliceReader traits to read store entries), reflink-copy
(FICLONE with copy fallback, per spec), proptest (dev-only, round-trip
property test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:47:11 +02:00
bl 4e1a95613b Milestone 1: workspace skeleton, wire protocol, socket round-trip
Three-crate workspace per SPECS.md. varde-proto defines the full JSON
Lines protocol (requests, envelopes, structured errors, events) with
string-typed hashes so the crate carries no iroh dependency. The daemon
binds its unix socket, loads config with flags > env > file > defaults
precedence, and answers status/list; everything else returns a
structured "unimplemented" error. varde-ctl maps subcommands 1:1 onto
requests and round-trips status against a real daemon in the tests.

Dependencies: serde/serde_json (wire format), tokio (async runtime and
unix sockets), tracing/tracing-subscriber (structured logging), toml
(config file), anyhow (binary-edge errors), thiserror (reserved for
library errors), clap (ctl flag parsing, per spec), tempfile (dev-only,
ephemeral test dirs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:10:29 +02:00