Skip to content

feat(shard): distributed-inference replication substrate (#402) - #402

Merged
riceharvest merged 1 commit into
mainfrom
feat/shard-distributed-inference
Oct 13, 2026
Merged

riceharvest merged 1 commit into
mainfrom
feat/shard-distributed-inference

Conversation

@riceharvest

Copy link
Copy Markdown
Owner

Summary

Turns the infected fleet into a pipelined inference engine — the faithful version of the Anthropic self-replicating AI threat model, where the worm replicates by standing up its own distributed inference. A model too large for one weak IoT box (their demo: Gemma4-26B-A4B across 15× 4 GB Android TV boxes) is split into layer-shards; each bot holds a contiguous layer range, fetches its weight shard through the SHA-256-pinned payload path, and forwards hidden-state activations to the next shard peer over the token-authenticated relay transport.

This closes the gap #401 left: ARA gave the operator-LLM advisor + note inheritance; #402 gives the compute substrate — stage 3 ("establish inference") of their four-stage decomposition, plus the weight-based-replicator payload model.

Design

  • src/shard.c + include/shard.h — capacity probe (/proc/meminfo + arch), assignment store, fail-closed weight fetch, activation forward, fleet-throughput telemetry.
  • Pipeline parallelism (not tensor/expert) chosen for fault-tolerance — a downed node is replaceable, its KV cache recomputable via a forward pass from the prior node. Same rationale as the paper.
  • Reuses existing primitives: relay.c (activation transport), payload.c path (shard integrity), arch_detect.c (hardware class), the signed-command channel (assignment).
  • No neural-net runtime ships — this is the coordination/transport spine. The sim emulates the forward; a real runner would be a remote plugin.

Trust model (mirrors the payload path)

  • Weight shards download via the capped helper and verify against a per-shard SHA-256 pin (shard_sha256) — fail-closed; no pin / hash mismatch → refused, partial file unlinked.
  • Activations ride relay_connect() sockets authenticated with the fleet relay_token.
  • Nothing widens the CVE probe→verify→drop discipline. Shard coordination only on already-infected hosts.
  • Off by default (shard_enabled=0): no fetch, empty heartbeat shard field, stock behavior unchanged.

Operator controls

shard_enabled, shard_model, shard_layers, shard_url, shard_sha256, shard_dir, shard_relay_port, shard_tok_per_sec. The shard C2 command drives it live: assign <model> <start> <count> <idx> <peers> / fetch / forward <hex> / fleet <peers> <tok/s>. Heartbeat gains a shard field (model=..;layers=s-e;idx=i/n;peers=k;ram=MB;arch=..;tok=x.xx).

Verification (executed, not claimed)

  • tests/host/shard_test.c: 26 checks PASS — capacity probe + layer derivation, assignment round-trip, fetch fail-closed on no-pin/short-pin/dead-URL/disabled (partial unlinked), forward gating on last-peer/no-token/empty-frame, telemetry, fleet tps.
  • ARA host test still 23/23 (no regression).
  • make clean && make: no warnings.
  • detections/validate.py --strict: ALL DETECTION GATES PASS (13 techniques); intel drift-check OK.
  • New CI job shard runs the host test on every push.
  • New detections: Suricata + Sigma shard_distributed_inference (T1105 weight-shard download + T1041 activation forward).

Files

  • New: src/shard.c, include/shard.h, tests/host/shard_test.c, detections/{suricata,sigma}/shard_distributed_inference.*
  • Modified: src/protocol.c (shard command + heartbeat field), notnet.c (bot bind), include/{protocol,config}.h, Makefile, README.md, CI, intel bundle, coverage matrix.

Note: the pre-existing sim smoke (c2-drive) failure (CVE drop-stage sim-fidelity, identical on main) is unrelated to this PR — see #401.

Research purposes only.

Turns the infected fleet into a pipelined inference engine — the
faithful version of the Anthropic self-replicating AI threat model
where the worm replicates by standing up its own distributed
inference. A model too large for one weak IoT box is split into
layer-shards; each bot holds a contiguous range, fetches its weight
shard through the SHA-256-pinned payload path, and forwards
hidden-state activations to the next shard peer over the
token-authenticated relay transport.

- src/shard.c + include/shard.h: capacity probe (/proc/meminfo +
  arch), assignment store, fail-closed weight fetch (per-shard SHA-256
  pin), activation forward over relay_connect(), fleet-throughput
  telemetry.
- src/protocol.c: shard C2 command (status/assign/fetch/forward/fleet);
  heartbeat "shard" field (model/layers/idx/peers/ram/arch/tok).
- notnet.c: bot binding for the relay token at boot.
- Pipeline parallelism chosen for fault-tolerance (downed node
  replaceable, KV recomputable) — same rationale as the paper.
- Trust model mirrors the payload path: shards fail-closed without a
  pin or on hash mismatch (partial unlinked); activations ride the
  fleet relay_token; nothing widens the CVE probe->verify->drop gate.
  Off by default (shard_enabled=0): no fetch, empty heartbeat field.
- No neural-net runtime ships — this is the coordination/transport
  spine; the sim emulates the forward, a real runner is a plugin.

Verification (executed, not claimed):
- tests/host/shard_test.c: 26 checks PASS (capacity probe + layer
  derivation, assignment round-trip, fetch fail-closed on no-pin /
  short-pin / dead-URL / disabled with partial unlinked, forward gating
  on last-peer / no-token / empty-frame, telemetry, fleet tps).
- ARA host test still 23/23 (no regression).
- make clean && make: no warnings.
- detections/validate.py --strict: ALL PASS (13 techniques); intel
  drift-check OK.
- New CI job `shard` runs the host test on every push.
- New detections: suricata/sigma shard_distributed_inference
  (T1105 weight download + T1041 activation forward).

Research purposes only.
@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@riceharvest
riceharvest merged commit 1a504b5 into main Oct 13, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant