Skip to content

Latest commit

 

History

History
668 lines (531 loc) · 123 KB

searchspace-matrix.md

File metadata and controls

668 lines (531 loc) · 123 KB

InferRSI SearchSpace Matrix

Branch: opus/meta-skill-bootstrap | Integration date: 2026-10-16 | Comparison anchor: d96ff8d3

Current skill links resolve to the complete categorized skill library. Hardware-specific procedures live under their hardware owner; this relocation does not change the source-inspection or maturity claims below.

Top-level rows are hardware resource layers.
Second-level optimization targets (Kernel, Operator, KV Cache, Attention, Runtime, Distributed, Serving, VAE/Codec) live inside the row that owns their bottleneck.

Maturity key: 🟢 covered (candidate skill + source evidence — candidate intelligence with source support, not proven gains) · 🟡 partial (source read, no dedicated skill) · 🔴 gap (no source, no skill)

Per decision D05 (2026-10-15): approval, extraction, coverage, and empirical validation are distinct. Green here means extracted candidate intelligence with source support; no cell claims a measured improvement. Provenance and inspected scope belong in the Reference Matrix, covering repositories, papers, official documentation, PRs and expert material.


1 · Graphics Card / Accelerator

The primary compute and HBM resource. Most current work lives here.

Lifecycle step What it covers Candidate Skills Cases / Experience Evidence Maturity
Diagnose / Configure / Verify CUDA eager dense-FFN Attention/FFN rank namespaces and per-stage/per-peer hidden-state ownership audit-afd-p2p-exchanges A=3,F=2 control sender differs from data peer; matching total tokens does not preserve peer rows afd-plugin @ df8d6a0c403c3ca1b4ed9cf1687e7204908bebec; source ledger, proposed cases 🟢 (source-derived candidate; native/model/efficacy not_run; no Network qualification)
Diagnose / Configure / Optimize / Verify Chip-qualified task fusion, normalized-versus-fused controls, ICB hazards/scratch and GPU-resident accepted-prefix state lithos-metal, compile-fusion, generation-state Layer mixer and MLP regions differ; CPU/synthetic ICB checks cannot establish full-model/DSpark numerics or E2E rates Lithos Metal @ 6653b7df5f2c4392ec7b038960f2796b1499923f; bounded source readset Source-derived candidate; efficacy not_run
Observe / Diagnose / Verify MiniMax-H3 joint audio/video AdaLN schedule/row-map validity, Metal BF16 rounding and actual int8 dispatch versus approximate controls audit-h3-runtime-residency Fifty layers/reuse 1 does not disable M5 int8 or BF16 head; test-source parity is not full media quality h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; precision contracts 🟢 (source-derived candidate; native/quality/performance not_run)
Observe / Diagnose / Verify FastH3 effective parameters/ambient override, semantic checkpoint transformations, partial/prefix VSA membership and sparse eligibility/fallback audit-h3-metal-fasth3-semantics Shape agreement is not conversion; sparse-disabled remains dense fine+coarse; parameter normalization does not erase ambient token enable h3-metal @ 4ba47a3eabdbebf801ac6664b31f8ec168bc57a8; semantic counterchecks 🟢 (source-derived candidate; native/sparse/schedule/media/performance not_run)
Diagnose / Configure / Verify Workflow model-weight residency, partial unload versus wrapper removal, patch identity and static/dynamic backend dispatch comfy-ui CPU dynamic construction reroutes to the static patcher; wrapper/free-byte bookkeeping is not physical allocator release ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; model ownership 🟢 (source-derived audit; device/native allocator unverified)
Observe / Diagnose / Verify Native diffusion managed budget versus device headroom, graph-segment weight protection and optional prefetch completion audit-sd-cpp-residency CPU bypasses accelerator capacity and segmentation gates; pin release is not physical free; backend capability does not prove overlap stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; phase residency 🟢 (source-derived audit; native/device/model unverified)
Observe / Profile System-level GPU/NPU timeline, per-kernel stats, SOL%, AI Core utilization ratios, step-trace four-tuple ascend-performance-analysis, dense-performance-evidence (L2) CANN CSV pipeline, MindIE-SD profiling-analyze lessons MindIE-SD @ 8481d39b; TRT-LLM nsys/ncu @ f33e0e5f 🟢
Observe / Profile Single-operator benchmark (msprof op, 8 CSV metrics, per-core occupancy) ascend-operator-profiling ops-profiling loop awesome-ascend-skills @ 75ec89ce 🟢
Observe / Profile Trace-to-source attribution: representative-step selection, kernel→source-module ownership via code_ref, compact vs full detail views cann-attribution-overlap Selection reason + shapes + streams + code_ref retained per kernel owner cann/oam-tools @ 8440935f (pack-mediated, wave 6) 🟢
Diagnose Bottleneck classification: compute-bound vs memory-bound vs AI-CPU fallback vs comm-exposed ascend-performance-analysis (5-layer pipeline), performance-ceiling-analysis (L1–L3), cann-attribution-overlap (attribution-aware) SOL% / mte2-ratio / mac-ratio heuristic MindIE-SD; TRT-LLM roofline @ f33e0e5f; LLM-Viewer @ 1893e4b5 🟢
Diagnose Roofline model, arithmetic intensity, hardware table entry for new accelerator performance-ceiling-analysis (L1) FLOPs formulas per layer type LLM-Viewer roofline_model.py, hardware_params.py 🟢
Diagnose / Configure / Verify TorchAir configuration-to-capture path: backend selection, graph keys/recapture, persistent eager fallback, input staging and output lifetime ascend-graph-mode Separate registered-backend documentation from CompilerConfig; same shape is not address stability, output retention or replay coverage; source map Official TorchAir @ e1118e8b0fc1b95190ab53b732c0872989550e20; historical awesome-ascend-skills import retained separately Candidate source procedure; TorchNPU/CANN native graph, model and service validation not_run
Configure Parallelism strategy (USP/CP/TP/PP), topology-aware card-group selection ascend-distributed-parallel USP activation requirement, masking upper-bound formula, AlltoAllV workaround MindIE-SD dit-parallel-opt @ 8481d39b (9 reference files) 🟢
Configure Model adapter: FP8 dequantization, KV-head replication, multimodal feature validation order ascend-model-adaptation vllm-ascend FP8/multimodal lessons; separate runtime-wrapper route vllm-ascend @ bc8115ed 🟢
Optimize Kernel-level: Triton/CuTe/TileIR writing, kernel tuning, fusion scope analysis kernel-writing-guide Framework routing (Triton vs CuTe vs AscendC), 6 universal correctness rules, verify-before-benchmark workflow TRT-LLM kernel-triton-writing + kernel-cute-writing @ f33e0e5f 🟢
Diagnose / Configure / Verify Existing causal depthwise convolution: API/state ownership, dtype/layout/packed boundaries, gathered rows, circular cursor and build/binding evidence kernel-writing-guide Five forward/update decision cases; invalid layout/state and unproved dispatch stop qualification Dao-AILab/causal-conv1d @ cd81f0413cad2fc1e6f17e785ac39f59aae690cd; contract/readset; source and test intent only Candidate; native/build/binary compatibility, downstream readiness, numerics, performance and efficacy not evaluated; no new measured coverage
Diagnose / Configure / Verify HF Qwen3.5 conv/GDN binding lifetime, independent fallback, dense history/recurrent state and cache-free classification-prefill limits kernel-writing-guide Two-release contract and five source cases; resolve actual method/callable and prior state before reuse Transformers v5.10.2 @ 0dad7b822255a0ae261ec45ae937371e859ffd1a and v5.17.0 @ 856157a2f3e9594954310df18fdccc31ffddebe9; selected HF source only Candidate; native/model/numerical/device/performance not_run, efficacy not_evaluated; no new measured coverage
Diagnose / Configure / Verify AKG static CUDA Affine loop tails: effective target/option gates, full point-loop bounds and protected input/output accesses kernel-writing-guide Tail-guard reference: separation precedence, non-divisible fixture and diagnostic boundary AKG @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff; source + FileCheck expectations only Candidate; compiler/IR command, CUDA runtime and agent efficacy not run; no new measured coverage
Optimize Operator-level: staged accuracy-gated tuning campaign (code-level → scheduling → instruction-level), metric→action routing; AscendC tiling design per operator class (reduction/broadcast/elewise/matmul/conversion) ascend-operator-tuning, ascendc-tiling-design PyPTO frontend/swimlane/incore playbooks; fullload-vs-split, four-element tiling template, chip-generation (DAV_2201/3510) routing awesome-ascend-skills pypto-op-perf-tune + ascendc-tiling-design @ 75ec89c 🟢
Optimize Casebook of past NPU optimization decisions: AI-CPU fallback, memory-bound, comm-exposed, graph-compilation, scheduling idle gaps ascend-optimization-casebook, inference-optimization-casebook Five bottleneck classes with symptom→diagnosis→action→evidence schema MindIE-SD profiling-analyze + dit-perf-opt + perf-gate + dit-parallel-opt @ 8481d39b 🟢
Optimize Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity optimization-search-policy Sarathi chunk-budget case: decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. Infera provenance adds a seed-only comparator and evaluated-trial ledger example; its trace-ranking check is proposed. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate Sarathi case and bounded Infera Git readset in Reference Matrix 🟢 (source-derived candidate procedure; empirical not_run)
Optimize Bounded multistream overlap: dependency-DAG constrained (data/state/event/collective_order edges; shared input ≠ dependency), minimal-first, execution-mode sync contract (eager streams vs TorchAir) cann-attribution-overlap Metric separation (wall / union-of-busy / summed durations / summed duration+wait; repeated-instance product = estimate); device-memory domain qualification; formula conflicts carried unresolved cann/oam-tools @ 8440935f (pack-mediated, wave 6) 🟢
Verify Evidence taxonomy (L0 correctness → L5 production), claim-evidence matching dense-performance-evidence, rsi-evidence, claim-contract InferRSI PASS/FAIL/INCONCLUSIVE protocol InferRSI @ 4c8e1285 🟢
Diagnose / Verify Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline ascend-performance-analysis Stage-evidence rule: workflow instructions versus registered marker/keyword checks; completion is not numerical validation canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage
Verify Same-window A/B protocol, noise floor, acceptance-state discipline; correctness-first overlap acceptance (fallback works, no KV/state reorder, actual overlap, no outside-module regression) ascend-optimization-casebook (evidence-required section per case), dense-performance-evidence, rsi-evidence, cann-attribution-overlap MindIE-SD perf-gate protocol: exploration vs acceptance state, three-proof requirement MindIE-SD perf-gate @ 8481d39b; cann/oam-tools @ 8440935f 🟢
Observe / Configure / Optimize DALI CPU/GPU queue overlap and operator buffer sizing under explicit executor-mode gates training-input-pipeline Separated queues require async+pipelined and non-dynamic mode; output synchronization retained Reference B1, DALI @ 6fe47c48d3b0 🟢 (Python API evidence, not C++/device validation)
Diagnose / Configure / Optimize Training capacity versus throughput, parameter/optimizer tier budgets, recomputation/offload choices; DCP exposed staging pause training-resource-budget, dcp-async-save-performance DeepSpeed stage gates; expert MFU/HFU denominator discipline; historical MindSpeed selection/allocator/rescue source boundaries Reference CR1, CR6, CR5, C1 async-only 🟢 (MindSpeed historical source candidate; current support unqualified)
Observe / Diagnose / Configure / Optimize / Verify vLLM V1 token/input/active-request budgets, allocation/preemption gates and async-KV admission; cap changes are conditional hypotheses diagnose-vllm-admission-pressure Queue length and KV usage do not establish allocation pressure; stats are gated and incomplete; verification proposed only vLLM @ 7f730b4cea9a, Dot bundle; pinned scheduler/connector source ledger 🟢 (candidate diagnosis; no runtime or gain validation)
Diagnose / Verify Context partial-ubatch progress, topology/input graph reuse and backend copy/event readiness audit-llama-runtime-state; owner map Retained completed ubatches differ from whole-batch rollback; frontend reuse differs from backend captured graphs llama.cpp @ 707616feb5f379e99ff7442680a55b4a1704c7c9; runtime/source notes and native test assertions; static inspection 2026-10-21 carried 🟢 (source-derived candidate; native/backend execution not run)
Observe / Diagnose / Verify KV receive/save completion, delayed-free ownership, in-flight-write fences and failure reporting audit-vllm-kv-lifetimes Event ledger is proposed, not a telemetry exporter; abstract obligations require concrete connector/caller evidence vLLM @ 7f730b4cea9a, Dot bundle 🟢 (contract intelligence; concrete backend unverified)
Diagnose / Configure / Optimize / Verify Runtime ACLGraph dispatch, stable input addresses, capture-resource limits and branch-specific replay/update ordering ascend-aclgraph-capture-replay Caller owns persistent buffers; DEBUG checks positional tensors only; reduced capture sets trade coverage for resources; version/responsibility routing vllm-ascend @ 8a8e1e05ecb6, Dot bundle; wrapper/guide/tests inspected by supplier 🟢 (conditional proposals; tests not run)
Diagnose / Configure / Optimize / Verify Device-HBM stream-use lifetime, fragmentation/counter interpretation and gated allocator controls torchnpu-device-memory-lifecycle Lifetime recording does not insert data dependencies; expandable segments change counters/control legality; GC synchronization/comment conflict retained TorchNPU @ 67511c53cd4c, Dot bundle; device allocator ledger 🟢 (source-supported procedure; NPU execution not run)
Diagnose / Verify Serving model: operator/context dependencies, calibration/load domain, fidelity provenance and joint-SLO prediction uncertainty calibrate-serving-simulation, performance-ceiling-analysis Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound. Infera source reference separates scalar tuner scoring from DES percentiles and retains missing-SLO/stale-document limits; no service guarantee Vidur v2 and bounded Infera Git readset; Reference Matrix systems sources 🟢 (prediction guidance; calibration not run)
Diagnose / Verify Reusable-prefix compute work versus host/network KV loading cost performance-ceiling-analysis, Mooncake condition Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit)
Diagnose / Verify Offline offload feasibility across GPU, CPU RAM and disk performance-ceiling-analysis, FlexGen condition Joint weights/KV/activation placement; RAM resident plus staging/working peaks, SSD state capacity and disk↔CPU cost, effective batch versus GPU batch, exposed CPU/transfer cost and block-latency objective FlexGen official ICML 2023, §§3–4.3 and selected A.3; separate public source, foundation archive still unavailable Source-only candidate; empirical not_run, fitted/perfect-overlap model and relaxed OOM limits; no blanket RAM/SSD, Network/OS or current-engine qualification
Diagnose / Configure / Optimize / Verify Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups assess-serving-preemption, evaluate-elastic-serving-placement Hypothetical pressure/migration cases; cohort/fairness and destination-reservation costs Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate 🟢 (conditional paper-derived plans)
Diagnose / Optimize / Verify Training activation lifetime, rematerialization and reserved-address packing plan-activation-residency Dependency repair and capacity feasibility checked separately; no recovery-checkpoint coverage Checkmate v3, OLLA v2; C readset 🟢 (planning, not solver/runtime validation)
Diagnose / Configure / Optimize / Verify Training stage/mesh/sharding feasibility and split F/B/W execution under total-HBM and optimizer-semantics constraints choose-parallel-training-plan, assess-split-backward-schedule Alpa conditional plans; Zero Bubble activation caps versus total memory; rollback versus stale updates Alpa v3, Zero Bubble v1 🟢 (author evidence, no current stack validation)
Diagnose / Optimize / Verify Attention IO, operation-class ceilings, output ownership, work supply, async dependencies and FP8 layout/accuracy diagnose-attention-bottlenecks, kernel-writing-guide Author-reported attention cases; compiler/register and precision counter-signals FlashAttention v2 / FA2 v1 / FA3 v2 🟢 (Accelerator only; no Ascend implementation claim)
Diagnose / Verify Diffusion stage/component-use readiness, retention and mutable request-scheduler aliases; client timeout/deletion versus worker/device completion audit-sglang-diffusion-lifetimes Default ParallelExecutor and concrete rank/strategy must be traced; closing a use does not imply eviction or device cancellation SGLang @ df13d5dce31a; bounded multimodal_gen source readset, separate from LLM/HiCache 🟢 (candidate diagnosis; native/quality/performance not run)
Diagnose / Verify Synchronous Ouro recurrent KV finalization: copying skipped deeper planes versus marking validity, complete-prefix attention reads, request-event ordering and retirement before page reuse audit-recurrent-kv-finalization Copy invocation and written-position bookkeeping are distinct; consumers and reclaimers follow the recorded-event wait contract vLLM-RLT @ 4a350344027960a48384626aa646f1765fda6d4f; caller/consumer and retirement ledger Source-derived candidate; CPU/CUDA/model/quality/performance/efficacy not_run; no host-capacity, Storage, Network or OS qualification

Second-level targets in this row: Kernel, Operator, KV Cache, Attention, Diffusion denoising steps, VAE/Codec, Graph compilation, Serving/Runtime admission, device-HBM allocator lifecycle, Training activation residency and pipeline scheduling


vLLM version and responsibility routing

For the vLLM admission/KV rows above, the existing responsibility map separates broad engine/configuration/extension guidance at 2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5 from focused scheduler branches and connector obligations at 7f730b4cea9a66520a9200265928ebe016b48a39. Original inspection dates and candidate/unverified limits remain unchanged. Concrete caller/backend conformance needs its own evidence; this navigation adds no RAM/Host, SSD, Network, OS or runtime/performance qualification.

2 · RAM / Host Memory

Host-side memory: CPU DRAM, pinned memory, page tables, PyTorch tensor lifecycle.

Lifecycle step What it covers Candidate Skills Cases / Experience Evidence Maturity
Diagnose / Configure / Verify Program/pipeline cache, GPU allocation adoption, exact host-prefix snapshots and pending-request idle release lithos-metal serving-cache Allocation reuse is separate from sequence validity; host snapshots and OS file cache have distinct owners Lithos Metal pinned serving/Session/PrefixCache/lifecycle paths Source-derived candidate; full-model residency and continuation unverified
Observe / Diagnose / Verify MiniMax-H3 TE/vision/input-output VAE phase lifetime, prepared DiT/decoder retention, shared/mapped weight and host media/output residency audit-h3-runtime-residency Preview/cache changes overlap; output tiling retains full RGB; checkpoint bytes and tracked tensors do not establish total RAM fit h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; phase ledger 🟢 (source-derived candidate; physical/compressed/swap peaks unmeasured)
Diagnose / Configure / Verify Workflow-output signatures and object identity, ancestor/fingerprint validity versus retention, lazy consumer references and host/pinned state comfy-ui A fixed JSON filename is not an external-state fingerprint; NOT_IDEMPOTENT node identity does not force every-run invalidation ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; cache ledger 🟢 (source-derived procedure; executor cache suite not run)
Observe / Diagnose / Verify Native diffusion parameter storage, compute staging and retained runtime buffers across conditioning/denoise/VAE phases; loader read/conversion buffers and shared mapping owners audit-sd-cpp-residency Device fit does not establish sufficient host RAM; initial host preparation may precede async transfer; releasing one owner is not final unmap; track host/device peaks separately stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; validation, loader/storage 🟢 (source-derived ownership audit; host peaks unmeasured)
Observe / Profile Host overhead in nsys: GPU idle ratio (>0.30 aggregate/>0.15 generation), host-prep exposed ratio (M3b>0.05), NVTX breakdown, inter-kernel gaps; cgroup memory pressure (PSI) and event counters host-overhead-profiling, cgroup-resource-controls (pressure/events observation) GPU idle + host-prep verdict metrics; NVTX root-cause breakdown; regression comparison across two traces TRT-LLM perf-host-analysis @ f33e0e5f; kernel cgroup-v2 docs (pack-mediated) 🟢
Diagnose YES/NO bottleneck verdict; multi-rank straggler detection; NVTX-to-source function mapping host-overhead-profiling M3b>0.05 AND M3c>0.50 together confirm host-prep bottleneck TRT-LLM perf-host-analysis @ f33e0e5f 🟢
Configure / Verify Copy-stream ordering, pinned CPU mutation/completion and host/device allocation lifetimes; version-aware GIL detection; conditional NUMA preflight; versioned Ascend native descriptor-pool ownership host-memory-management Separate producer/consumer ordering, application reuse and allocator cache reuse; no universal tensor-size or pinned-memory performance threshold. Ascend normal head-event query gates descriptor reuse; waiting alone does not recycle PyTorch v2.8.0 @ ba56102387ef21a3b04b357e5b183d48f0afefc7 (readset); CPython v3.13.0 documentation; legacy NUMA guidance; vllm-ascend @ 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf Candidate source procedure; CUDA/NPU performance, NUMA and free-threaded stack compatibility not_run
Configure KV host-tier sizing and multi-L1 composition; cgroup memory protection and limits (memory.high/low/max) kv-tier-management, cgroup-resource-controls Retain the respective tier-capacity and OS memory-control contracts; these settings do not establish transfer completion LMCache @ c3b875a; kernel cgroup-v2 docs (pack-mediated) Existing candidate guidance; no new source read or validation
Optimize line_profiler loop targeting _prepare_tp_inputs; NVTX regression triage to alternative targets (_fetch_new_requests +643%, broadcast_requests, _update_requests +75%); runtime KV tier management (store/retain/prefetch/evict policies, concurrency-vs-RAM admission dial) host-overhead-optimization, kv-tier-management Iterative profile→identify→optimize→verify loop; RAM-release lever ranking TRT-LLM perf-host-optimization @ f33e0e5f; LMCache MP mode @ c3b875a 🟢
Verify Memory-event verification: matched baseline/outcome windows, PSI + event deltas, configuration provenance, rollback contract dense-performance-evidence (evidence levels apply), cgroup-resource-controls (verification contract) Application outcomes + sibling pressure tracked together; raw counter deltas retained kernel cgroup-v2 docs (pack-mediated, wave 6) 🟢 (proposed procedure, unvalidated)
Observe / Profile Per-host checkpoint snapshot/pinned-memory residency, copy and staging-worker pressure dcp-async-save-performance Sum local-rank state plus cache and input/offload overhead; no global-rank-to-single-host RAM assumption PyTorch DCP @ 7d239835e773 🟢 (source-supported estimate/measurement plan)
Configure / Optimize / Verify CPU snapshot lifetime: source mutation after staging, cached storage reuse after upload, caller-owned cleanup dcp-async-save-performance D2H and same-host IPC remain host resources, not network cells DCP DefaultStager, StateDictStager @ 7d239835e773 🟢 (unvalidated candidate procedure)
Observe / Diagnose / Configure / Optimize Input queue/cached-shard working-set budgets; offload CPU/pinned-memory costs and locality training-input-pipeline, training-resource-budget Per-worker samples differ from bytes; host budget includes input/checkpoint/offload together Reference B1/B2/B3, CR1/CR6/CR5 🟢 (candidate decisions, no measurements)
Configure / Verify Shared-memory snapshot readiness, positive-step agreement, replica/staging lifetime checkpoint-recovery-consistency Worker survival differs from host survival; state completeness remains separate Reference C3/C4 🟢 (source contracts; recovery untested)
Diagnose / Configure / Optimize / Verify TorchNPU pinned-host ownership, pending-copy reuse, host-specific byte accounting and default/expandable cache controls torchnpu-pinned-host-memory-lifecycle External registration ownership and ATen internals unresolved; per-block threshold is not a total-cache cap; version-warning conflict retained TorchNPU @ 67511c53cd4c, Dot bundle; host allocator/copy/config ledger 🟢 (conditional procedure; no runtime validation)
Diagnose / Configure / Optimize / Verify Suspended KV host residency, predicted recovery and migration staging under actual host capacity/bandwidth assess-serving-preemption, evaluate-elastic-serving-placement Running-batch bytes exclude other unfinished state; device space is also required for restoration FastServe v3, Llumnix v1; hypothetical cases, not observed failures 🟢 (conditional host mechanism)
Diagnose / Optimize / Verify CPU optimizer state/compute ownership and tiled update/return cost training-resource-budget ZeRO-Offload delta; staleness requires a separate semantic decision ZeRO-Offload v1, distinct from the existing DeepSpeed config pin 🟢 (author evidence, unmeasured locally)
Diagnose / Optimize / Verify Host feature materialization, invariant reuse and assembly lifetime host-overhead-optimization Cross-layer procedure; local gain requires full-stack SLO verification FlashVector v1, same source identity as Reference Matrix §2 🟢 (scoped host mechanism, not service gain)
Diagnose / Verify Host-side request/job records, scheduler-object sharing and output-finalizer task lifetime audit-sglang-diffusion-lifetimes Req field is not proof of object isolation; socket close and store deletion do not establish worker termination; device HBM excluded SGLang @ df13d5dce31a; same bounded source identity 🟢 (source-derived candidate, no measured memory saving)
Diagnose / Verify Host-owned reader/task/result and slot prompt-token lifecycle under cancellation audit-llama-runtime-state Waiter removal, queued-work cleanup and slot release are distinct; stateful prompts may remain; no host byte-saving claim llama.cpp @ 707616feb5f3; server queue/context source and cancellation test assertion 🟢 (candidate lifecycle diagnosis; native server case not run)

Second-level targets: PyTorch tensor lifecycle, request scheduling, KV cache CPU offload, TorchNPU pinned-host allocator lifecycle, CPU optimizer ownership and host feature assembly


3 · SSD / Storage

Storage I/O: model weight loading, checkpoint save/restore, profiling artifact I/O, dataset prefetch.

Lifecycle step What it covers Candidate Skills Cases / Experience Evidence Maturity
Diagnose / Verify Native diffusion catalog/source identity → non-ZIP safetensors byte reads → host staging or retained mapping → immediate ModelManager completion/cleanup audit-sd-cpp-residency Stamps/internal revisions are not content hashes; valid metadata or a destination callback is not read completion; failure cleanup covers this load's newly created blocks stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; loader/storage boundaries 🟢 (bounded source-derived candidate; other formats, native I/O and performance unqualified)
Diagnose / Configure / Verify Prepared-pack structural identity and exact-prefix disk blobs, hashes, quota and bounded flush lithos-metal serving-cache Weight payload size/mtime is not byte authentication; queued writes are not persistence; corrupt/stale entries prefill Lithos Metal pinned pack/PrefixStore paths Source-derived candidate; durability/performance unverified
Diagnose / Configure / Verify Raw tensor-schema compatibility, file-backed DiT versus two-slot BF16 SSD streaming, Qwen read-worker/ring-depth distinction and submit/join completion audit-h3-runtime-residency SSD mode disables runtime int8; compare matching BF16 paths. F_NOCACHE request and zero join wait do not prove bypass or free I/O h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; I/O tradeoffs 🟢 (source-derived candidate; native I/O/overlap/performance not_run)
Observe / Profile Workload-matched block-device measurement (fio methodology: signature recording, requested-vs-achieved depth, cache-state ledger, run-shape classification), per-device attribution (iostat), per-process attribution (pidstat) storage-io-measurement, weight-loading-optimization (§7 phase timers) Bounded observation templates; unit/latency-provenance schema; logical-vs-physical I/O distinction fio @ acb7a302 + sysstat @ efd0c445 (pack-mediated, wave 6); fastsafetensors perf.py @ 219e256; mindie-agent docs @ 6a81b41 🟢 (methodology only — running benchmarks or manipulating caches requires separate authorization)
Diagnose Slow startup attribution: distinguish graph compilation/capture from weight I/O using phase traces ascend-graph-mode, ascend-model-adaptation (dummy-to-real sequence) Graph source identifies capture/compile work but does not rule out concurrent or preceding storage bottlenecks TorchAir @ e1118e8b0fc1b95190ab53b732c0872989550e20; historical MindIE-SD/vllm-ascend guidance Incidental diagnosis only; no added storage implementation or measured startup benefit
Diagnose Device saturation and attribution: workload matching, achieved-depth validation, %util/%iowait saturation guards, per-process I/O vs CPU-scheduling-pressure separation storage-io-measurement No universal percent threshold; corroboration required before naming a storage bottleneck fio/sysstat pack (wave 6) 🟢
Diagnose Load-path judgment (source/format/placement/duplication) and startup fixed-overhead accounting (bytes+counts lines, hit-criteria warmup validation) weight-loading-optimization Four-dimension load-path judgment; fixed-vs-per-request split; 7 accounting errors MindIE-SD host-opt + host-overhead-account @ 8481d39b 🟢
Configure Checkpoint format for fast loading (FP8 paired tensors, shard loading order) ascend-model-adaptation FP8 paired-tensor loading pattern vllm-ascend @ bc8115ed 🟡
Optimize Serving-level load path: device-side weight prefetch (switch-gated, 50%-of-theoretical start), CPU-tier offload ordering by change surface, duplication removal weight-loading-optimization npu_prefetch decision rules; OOM fallback tiers; construction vs inference OOM cannbot model-infer-prefetch + diffusers memory-optimization @ 75ec89c; MindIE-SD resource-fallback-tiers @ 8481d39b 🟢 (serving level)
Optimize Disk-level loading mechanics: mmap/mlock/dio/lazy loading modes, pinned bounce-buffer parallel reads, GDS direct-to-device, byte-range/shard-parallel loading, incremental mlock growth weight-loading-optimization (§6-7); llama.cpp owner map Mode trade-off table; NUMA-disables-prefetch coupling; block-size formula; readiness overlap; static fit planner ggml-org/llama.cpp @ 3e390af18de1131dda27913c00511725b4274c8a (MIT; static inspection 2026-10-22 carried); foundation-model-stack/fastsafetensors @ 219e256 (Apache-2.0) 🟢 (wave 5; CUDA/ROCm/POSIX-bound — no Ascend path in these sources)
Optimize Runtime KV-to-disk tier: L2 local adapters (fs/nixl/raw_block/dax), persist/recover, per-adapter serde capacity lever kv-tier-management Adapter selection table; persist/recover semantics LMCache MP mode @ c3b875a (Apache-2.0) 🟢
Optimize Checkpoint save/restore: synchronous sharded writes, per-file atomic publish, opt-in fsync, reload via existing loader; whole-set atomicity / async / distributed / crash durability explicitly unsupported weight-loading-optimization (§8) Supported-vs-unknown table; mixed-set rename-failure risk as caller obligation fastsafetensors saver.py @ 219e256 🟡 (fastsafetensors-specific semantics extracted; its save-performance optimization remains unsourced — no perf harness or documented tuning levers)
Verify Load-path measurement protocol (phase timers, fadvise cold-cache, baseline comparison); storage-side verification: unit/latency-provenance-aware schema, two-layer evidence gate (device metric + application outcome) weight-loading-optimization (§7), storage-io-measurement (prime rule, §5) Author-reported numbers labeled as such; device metrics never merged with application outcomes fastsafetensors perf.py @ 219e256; fio/sysstat pack (wave 6) 🟢 (methodology; no empirical validation)
Observe / Profile Checkpoint caller pause, staging elapsed, upload elapsed and exposed training waits; first-use versus cached-buffer steady state dcp-async-save-performance Distinct timing boundaries; asynchronous return is not write completion DCP async recipe + PyTorch @ 7d239835e773 (wave 7) 🟢 (proposed measurement, not executed)
Configure Async save mode, one outstanding save policy, staging cache/writer ownership and completion gates dcp-async-save-performance Tutorial/API reconciliation; explicit caller-owned cleanup PyTorch DCP @ 7d239835e773 🟢 (DCP-only; backend qualification outstanding)
Optimize Training-visible pause reduction via asynchronous staging and cached pinned buffers; CPU/RAM/I/O contention limits dcp-async-save-performance Wait for staging before source mutation; wait for upload before cached snapshot reuse PyTorch DCP @ 7d239835e773; live recipe inspected 2026-10-15 🟢 (source-supported intelligence, no gains measured)
Verify Observe final upload errors and step-time impact separately from backend durability and distributed recovery validity dcp-async-save-performance Async completion is not crash-durability evidence PyTorch DCP save/stager/thread-writer paths @ 7d239835e773 🟢 (verification procedure only)
Diagnose / Configure / Optimize Dataset shard residency, compressed/raw admission, cache churn, FFCV mmap/process cache and traversal order training-input-pipeline Mosaic minimum-cache implementation gate; distributed QUASI_RANDOM excluded Reference B2/B3 🟢 (sample correctness and throughput untested)
Configure / Optimize DeepSpeed NVMe parameter/optimizer buffer pools and tile read/compute/write overlap intent training-resource-budget Parameter and optimizer budgets distinct; buffer_size unit/dispatch not verified Reference CR1 🟢 (configuration evidence only)
Configure / Verify Complete-generation selection; NVIDIA dirty-to-final publication; DLRover queued/shard/done/tracker boundaries checkpoint-recovery-consistency DLRover common commit counts done files; payload serialization and POSIX synchronization require separate durability evidence. Rank coverage does not prove state completeness Reference C3/C4; DLRover persistence 🟢 (source-derived recovery procedure; recovery/durability untested)

Second-level targets: Model weight sharding, checkpoint I/O, profiling output archival


4 · Communication Card / Network

Inter-device and inter-node communication: HCCL (Ascend), NCCL (CUDA), InfiniBand, RoCE.

Lifecycle step What it covers Candidate Skills Cases / Experience Evidence Maturity
Observe / Profile Communication time in step-trace four-tuple, HCCL per-collective timing, communication.json ascend-performance-analysis (Layer 4) HCCL single-collective latency varies ~2× across devices MindIE-SD profiling-analyze @ 8481d39b 🟢
Diagnose Exposed communication: effective_comm = Communication - Overlapped; communication-bottleneck class; wait-aware diagnosis (wait pollution, transport/rank metrics, device imbalance) ascend-performance-analysis (L1 step-trace derivation), cann-attribution-overlap (wait-aware) Overlapping vs exposed comm in step_trace_time.csv; zero/missing bandwidth = unavailable evidence, single-rank data cannot establish cross-rank balance MindIE-SD; cann/oam-tools @ 8440935f (pack-mediated) 🟢
Diagnose Topology-aware card group validation: intra-domain (UB/HCCS) vs inter-domain (PCIe/SYS) ascend-distributed-parallel, ascend-device-health npu-smi topo output; legal card-group rules; Health=OK not sufficient MindIE-SD dit-parallel-opt @ 8481d39b 🟢
Configure Collective-in-graph capture (HCCL graph mode), comm-stream masking; masking formula 1-1/n when F≥C ascend-distributed-parallel Masking upper bound derivation; AlltoAllV defect workaround MindIE-SD @ 8481d39b 🟢
Optimize USP activation requirement (ParallelConfig wrapping), strategy selection, AlltoAllV workaround; evidence-gated compute/communication overlap of independently-ready work ascend-distributed-parallel, cann-attribution-overlap USP must set ParallelConfig; verify allToAll appears not just allGather; overlap preserves ordering and joins, no comm recommendation without comm evidence MindIE-SD dit-parallel-opt @ 8481d39b (9 ref files); cann/oam-tools @ 8440935f 🟢
Verify Masking ratio measurement, before/after comm-fraction comparison; strategy activation attribution ascend-distributed-parallel (attribution method), ascend-performance-analysis, dense-performance-evidence, cann-attribution-overlap (overlap acceptance) Step-trace four-tuple as ground truth; scope-effectiveness-check.md MindIE-SD perf-gate + dit-parallel-opt; cann/oam-tools @ 8440935f 🟢
Observe / Diagnose / Verify Native HCCL-test versus PyTorch/HCCL wrapper selection; per-operation tensor-byte meaning, completed-call timing, rank-local reports and actual oracle coverage ascend-distributed-parallel (test acceptance) Unchecked operations can print PASS; subgroup tags and P2P peer schedules need separate references; async timing does not prove overlap awesome-ascend-skills @ 75ec89ce9d401be8ac95980e1c3f42f8d4315057, wrapper code and source docs 🟢 (candidate evidence procedure; native collective/transport, backend and agent efficacy not_run; host-KV deferred, no RAM increment)
Diagnose / Configure / Verify One native multi-rank OPBASE AICPU mesh AllReduce path: actual algorithm/engine, rank-scaled scratch chunks, peer writes, joins and local reduction ascend-distributed-parallel Native source map: cost-selector versus size-heuristic routes; final selection after fallback; simulator semantics versus completed native numerical checks Official cann/hccl @ 96fab0a2acb761e7f8ab0f94aeb5138f200a345b, selected implementation/test definitions Candidate source procedure only; HCOMM/ACL transport, other devices/engines/collectives, native numerical and performance validation not_run
Configure / Optimize Remote dataset shard download look-ahead versus cache churn training-input-pipeline Network bytes/readiness are distinct from local-cache footprint Reference B2 🟢 (candidate mechanism; no network measurements)
Configure / Optimize / Verify File-owner whole-tensor broadcast versus TP scatter; adjacent whole-tensor broadcast coalescing weight-loading-optimization (§9) Group-rank order, coalescing eligibility, source readiness, run-storage retention and early close; local clones/H2D excluded fastsafetensors @ 219e256, full-pin delivery readset 🟢 (source-derived candidate procedure; no runtime, transport, NPU or gain qualification)
Diagnose / Configure / Verify Saver/loader agreement, complete-iteration coverage and DLRover full/sharded Gloo byte/metadata transport checkpoint-recovery-consistency Same-host shared memory is excluded; replica group/placement and restore consumer need payload checks. CPU/Gloo source is not HCCL/NPU qualification Reference C3/C4; DLRover replica boundaries 🟢 (source-derived transport procedure; distributed restore untested)
Observe / Diagnose / Verify Collective kind/payload/ranks, algbw versus busbw, rank-side GC stalls versus fabric stalls training-resource-budget (attributed expert note) No network-script execution or universal bandwidth/gain claim Reference CR6 🟢 (expert methodology)
Diagnose / Configure / Optimize / Verify Cross-node request-state pre-copy/catch-up, migration versus ongoing sequence-attention traffic evaluate-elastic-serving-placement Total bytes, duration and final downtime differ; charge congestion and protected-class regressions Llumnix v1, LoongServe v2; actual inter-node placement required 🟢 (paper decision contract; transport unvalidated)
Diagnose / Configure / Optimize / Verify Cross-mesh shard intersections, required slow-link sends and local allgather replacement cost choose-parallel-training-plan Separate inter-node traffic from local accelerator collectives; include staging and per-rank memory Alpa v3 AL2; no current compiler or transport qualification 🟢 (conditional communication mechanism)

Host/device transfers and same-host IPC are excluded from this row. Network coverage above comes only from remote downloads, distributed agreement/replicas and collective-specific evidence.

Second-level targets: AllReduce, AllGather, AllToAll, P2P ring; TP/SP/PP/EP collective patterns


5 · OS / System

System-level: scheduler, DVFS, process isolation, environment variables, driver state.

Lifecycle step What it covers Candidate Skills Cases / Experience Evidence Maturity
Observe / Profile DVFS effect on first-run timing, process-level NPU ownership (proc-mem via npu-smi); cgroup CPU bandwidth counters (cpu.stat vs cpu.stat.local for ancestor throttling), hierarchy state ascend-operator-profiling (--warm-up rationale), ascend-device-health (proc-mem check), cgroup-resource-controls DVFS: always warm up before measuring; proc-mem occupancy check awesome-ascend-skills ops-profiling; MindIE-SD dit-parallel-opt @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) 🟢
Diagnose Shared-machine contamination: neighbor load, inter-window drift, temperature variation ascend-performance-analysis (measurement discipline section) Same-window A/B required; cross-window delta ≈ real gain magnitude MindIE-SD perf-gate @ 8481d39b 🟢
Diagnose Quota versus contention: cpu.stat vs cpu.stat.local (ancestor-caused throttling), memory PSI + event deltas, requested-versus-effective cpuset cgroup-resource-controls Allocation-is-not-efficiency prime rule; pressure alone is not causal proof kernel cgroup-v2 docs (live 2026-10-15 + v6.12, pack-mediated, wave 6) 🟢 (proposed procedure, unvalidated)
Diagnose SIGKILL residual state: per-step time uniformly elevated by ~1 order of magnitude ascend-device-health npu-smi Health=OK does not prove card is clean; 4-step smoke required MindIE-SD dit-parallel-opt @ 8481d39b 🟢
Configure Card-group selection, env variables (HCCL_OP_EXPANSION_MODE=AIV), proc-mem occupancy check; cgroup quota/weight/cpuset configuration with hierarchy/delegation/permission preflight ascend-device-health, cgroup-resource-controls Card-group change = contract change; all baselines must be re-measured on new group; identify the authorized owner before proposing changes MindIE-SD dit-parallel-opt + perf-gate @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) 🟢
Configure Env-preflight: verify device availability, driver health, Python env before measurement performance-ceiling-analysis (env-preflight references TRT-LLM exec-env-check) TRT-LLM exec-env-check pattern TRT-LLM @ f33e0e5f 🟡
Optimize CPU affinity check-then-pin (accelerator-local NUMA node, cross-round consistency), NUMA placement mode semantics (distribute/isolate/numactl) with page-cache-drop and numa_balancing rules, generation-vs-batch thread/affinity splits, thread-count env discipline, device-busy preflight with graceful-to-force PID escalation, structured ERROR/WARNING env checks, container flags and ulimits; cgroup resource reallocation as capacity spend (quota/weight/headroom reviews gated on observed stalls) os-process-placement, cgroup-resource-controls; llama.cpp owner map OMP_NUM_THREADS/PROC_BIND per scenario; CPU_AFFINITY_CONF; npu-smi usages preflight; ASCEND_HOME_PATH named-var correctness; rollback manifest before any change TRT-LLM perf-host-optimization @ f33e0e5f; awesome-ascend-skills @ 75ec89c; ggml-org/llama.cpp @ 3e390af18de1131dda27913c00511725b4274c8a (MIT; static inspection 2026-10-22 carried); kernel cgroup-v2 docs (pack-mediated); historical vllm-ascend Dockerfile/thread-env shipping defaults @ bc8115ed (Reference Matrix); ACLGraph @ 8a8e1e05ecb6 adds no OS-control evidence 🟢 (isolcpus and scheduler priorities nice/ionice/SCHED_FIFO/RR remain unsourced)
Verify Card health check sequence: npu-smi Health, 4-step smoke verify, proc-mem occupancy; cgroup change verification (matched windows, application metrics, counter deltas, provenance, rollback) ascend-device-health, cgroup-resource-controls Health=OK ≠ card usable; SIGKILL residual state only detectable via smoke run MindIE-SD @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) 🟢
Diagnose / Configure CPU optimizer share, NUMA locality and per-rank garbage-collection stalls; training/helper lifetime in recovery training-resource-budget, checkpoint-recovery-consistency Gate CPUAdam cores to stage-3 super_offload; fallback saver lifetime matters; historical MindSpeed 910A/910B binding requires actual device/NUMA/CPU mapping beyond tool presence Reference CR1/CR6, C3; CR5 pinned affinity source 🟢 (source guidance; no scheduling or privileged changes)
Configure / Optimize FFCV OS-managed versus process-managed dataset caching training-input-pipeline Cache ownership is independent of nice/ionice/realtime scheduling Reference B3 🟢 (configuration decision, not runtime result)
Diagnose / Configure / Verify Scoped RTK learning transition: manager discovery versus consumer PATH resolution; unknown-path and binary-identity limits; platform-specific repair, diagnostic effects and process-restart boundary repository-evolution-analysis Manager discovery is distinct from consumer command resolution; repair and process-environment refresh have separate boundaries Magpie @ 9848a9062a65c49e074c09770dc7de73dd012bea, parent 4b5d6d725fae3d8a263f6e43c707400e3765b84f; RTK transition Source-derived conditional lesson only; upstream tests, consumer execution and platform repair not_run. No scheduler/cgroup/performance or skill-efficacy coverage

Second-level targets: NPU driver, CANN runtime, container/process isolation


Gap Summary

Gap Row Priority Next action
Further native hardware-source coverage Accelerator / Communication Later source expansion Native NCCL remains unmined. Existing CUTLASS/DeepGEMM/FlashInfer slices do not cover broad dispatch. Investigate UCX when a multi-node workload requires it, and ROCm when an AMD target exists; none is newly covered by this delivery.
Earlier six-paper foundation archive unavailable Uncredited original-archive mechanisms; original synthesis and standalone paper candidates Input blocked for original-archive import Supply the approved archive/readsets before importing its ZeRO-Infinity, FlexGen, Sarathi-Serve, DistServe, PagedAttention or Mooncake candidates; do not substitute config/discovery evidence. Separately, the fresh official-edition DistServe phase-placement reference is retained under the existing search-policy owner as candidate / not_run source knowledge; it does not recover that archive or qualify runtime, device or service behavior. The separately sourced Mooncake analytical condition above does not recover the archive; novelty against the missing synthesis remains UNKNOWN.
Reviewed A–F qualification Scoped Accelerator / Host / Network rows above Separate future task Candidates and author evidence do not establish runtime compatibility, measured gains or NPU support; no reproduction authorized
Save-path performance qualification (checkpoint) SSD / Optimize + Verify Low DCP asynchronous staging, pinned-buffer reuse and completion gates are now sourced in dcp-async-save-performance; no local gains, NPU compatibility or backend durability verified. fastsafetensors-specific performance remains unsourced; no repeat mining authorized
Wave-7 input pipeline qualification Accelerator / RAM / SSD / Network Low Mode-specific DALI behavior, FFCV current toolchain/distributed limits, Mosaic cache-churn and sample correctness remain unmeasured; no hardware run approved
Distributed recovery qualification RAM / SSD / Network / Verify Low C3/C4 contracts extracted; replica transport, whole-state completeness, world-size changes and actual failure recovery remain unvalidated
Cross-resource applicability and historical evidence Accelerator / RAM / OS Low DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed historical pin/default license and selected allocator/wrapper/affinity paths resolved, while full search/cache behavior, current compatibility and measured benefit remain open; CR6 is expert methodology, not measured gains
OS/System isolcpus and scheduler priorities (nice / ionice / SCHED_FIFO / RR) OS / Optimize Low cgroup v2 controls are now sourced (cgroup-resource-controls); the scheduler-priority class remains without a source and deferred
vLLM-RLT recurrent-depth KV finalization Accelerator / KV correctness / Verify Source-guidance candidate; native/efficacy not_run One operating candidate traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. Intake retained
AFD P2pNccl completion, graph and transport qualification Accelerator / Communication / Verify Candidate audit delivered; native qualification gap remains audit-afd-p2p-exchanges covers rank and peer/stage ownership at its pin. The afd_plugin runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical intake retained; no new Network coverage

Resolved gaps since initial matrix (all high/medium priority cleared):

  • Host overhead profiling + optimization skills → host-overhead-profiling, host-overhead-optimization ✅
  • Parallelism strategy skill → ascend-distributed-parallel ✅
  • NPU optimization casebook → ascend-optimization-casebook ✅
  • SIGKILL / card-health recovery → ascend-device-health ✅
  • Same-window A/B acceptance skill → covered in ascend-optimization-casebook (per-case evidence schema) ✅
  • Kernel writing skill (Triton-Ascend / AscendC TileIR / CuTe) → kernel-writing-guide ✅
  • Host memory lifecycle (pinned memory / GIL / H2D async) → host-memory-management ✅
  • Operator-level tuning (pypto-op-perf-tune, AscendC tiling) → ascend-operator-tuning, ascendc-tiling-design ✅
  • Serving-level weight loading / startup overhead → weight-loading-optimization ✅
  • OS/System Optimize (CPU affinity, NUMA process placement, env discipline) → os-process-placement ✅ (cgroups/scheduler priorities remain open, see gap table)
  • Disk-level loading mechanics (mmap/mlock/dio/lazy, pinned staging, parallel reads) → weight-loading-optimization §6-7 ✅ (wave 5: llama.cpp, fastsafetensors)
  • Runtime KV-to-disk tiering → kv-tier-management ✅ (wave 5: LMCache)
  • Load-path measurement protocol (SSD Verify) → weight-loading-optimization §7 ✅ (wave 5: fastsafetensors)
  • Checkpoint save/restore semantics → weight-loading-optimization §8 ✅ (wave 6: fastsafetensors saver, pack-approved scope)
  • Standalone block-device measurement methodology (fio-class) → storage-io-measurement ✅ (wave 6: fio + sysstat pack)
  • DCP save-path performance mechanisms → dcp-async-save-performance ✅ (wave 7; staging/pause/reuse/CPU-RAM-I/O trade-offs sourced, no measured gains)
  • cgroup v2 resource controls → cgroup-resource-controls ✅ (wave 6: kernel docs pack)

PR coverage table

Use this table in the PR description for opus/meta-skill-bootstrap.

Search Space Observe / Profile Diagnose Configure Optimize Verify
Accelerator 🟢 ascend-performance-analysis, ascend-operator-profiling, dense-performance-evidence, cann-attribution-overlap (code_ref attribution), diagnose-vllm-admission-pressure, audit-vllm-kv-lifetimes (event-ledger requirement) 🟢 ascend-performance-analysis, performance-ceiling-analysis, cann-attribution-overlap, diagnose-vllm-admission-pressure, audit-vllm-kv-lifetimes, ascend-aclgraph-capture-replay, torchnpu-device-memory-lifecycle, audit-llama-runtime-state (source candidate; native not run), audit-sglang-diffusion-lifetimes (source candidate; native not run) 🟢 ascend-graph-mode, ascend-model-adaptation, ascend-distributed-parallel, diagnose-vllm-admission-pressure, ascend-aclgraph-capture-replay, torchnpu-device-memory-lifecycle (conditional proposals) 🟢 ascend-optimization-casebook, optimization-search-policy, kernel-writing-guide, ascend-operator-tuning, ascendc-tiling-design, cann-attribution-overlap (bounded overlap), diagnose-vllm-admission-pressure, ascend-aclgraph-capture-replay, torchnpu-device-memory-lifecycle (unmeasured proposals) 🟢 dense-performance-evidence, rsi-evidence, claim-contract, cann-attribution-overlap (overlap acceptance), diagnose-vllm-admission-pressure, audit-vllm-kv-lifetimes, ascend-aclgraph-capture-replay, torchnpu-device-memory-lifecycle (proposed checks), audit-llama-runtime-state (source candidate; native not run), audit-sglang-diffusion-lifetimes (source candidate; native not run)
RAM / Host 🟢 host-overhead-profiling, cgroup-resource-controls (PSI/events) 🟢 host-overhead-profiling, torchnpu-pinned-host-memory-lifecycle, audit-llama-runtime-state (source candidate; native not run), audit-sglang-diffusion-lifetimes (source candidate; native not run) 🟢 host-memory-management, kv-tier-management, cgroup-resource-controls (memory limits), torchnpu-pinned-host-memory-lifecycle (version/path-gated) 🟢 host-overhead-optimization, kv-tier-management, torchnpu-pinned-host-memory-lifecycle (unmeasured proposals) 🟢 cgroup-resource-controls (verification contract), dense-performance-evidence, torchnpu-pinned-host-memory-lifecycle (proposed checks), audit-llama-runtime-state (source candidate; native not run), audit-sglang-diffusion-lifetimes (source candidate; native not run)
SSD / Storage 🟢 storage-io-measurement, dcp-async-save-performance (phase boundaries) 🟢 weight-loading-optimization, storage-io-measurement, training-input-pipeline; 🟡 startup incidental 🟢 training-input-pipeline, training-resource-budget, dcp-async-save-performance, checkpoint-recovery-consistency; 🟡 FP8 loading 🟢 weight-loading-optimization, kv-tier-management, training-input-pipeline, training-resource-budget, dcp-async-save-performance (gains unmeasured) 🟢 weight-loading-optimization, storage-io-measurement, dcp-async-save-performance, checkpoint-recovery-consistency (procedures only)
Communication 🟢 ascend-performance-analysis 🟢 ascend-performance-analysis, ascend-distributed-parallel, cann-attribution-overlap (wait-aware) 🟢 ascend-distributed-parallel 🟢 ascend-distributed-parallel, cann-attribution-overlap (evidence-gated overlap) 🟢 ascend-distributed-parallel, ascend-performance-analysis, cann-attribution-overlap
OS / System 🟢 ascend-operator-profiling (DVFS), ascend-device-health (proc-mem), cgroup-resource-controls (cpu.stat/local) 🟢 ascend-device-health, ascend-performance-analysis, cgroup-resource-controls (quota-vs-contention) 🟢 ascend-device-health, cgroup-resource-controls (quota/weight/cpuset) 🟢 os-process-placement (affinity/NUMA/threads/env), cgroup-resource-controls (capacity-spend reallocation) 🟢 ascend-device-health, cgroup-resource-controls (verification contract)

Wave-7 additions to the coverage summary above:

Search Space Observe / Profile Diagnose Configure Optimize Verify
Accelerator training-input-pipeline training-resource-budget, dcp-async-save-performance training-input-pipeline, training-resource-budget training-input-pipeline, training-resource-budget No new device validation
RAM / Host training-input-pipeline, dcp-async-save-performance training-input-pipeline, training-resource-budget All four wave-7 skills Input/budget/DCP candidates DCP lifetime + recovery contracts only
Communication / Network training-resource-budget expert note Expert collective diagnosis + recovery agreement Remote input downloads + recovery replicas Remote input predownload only Expert measurement + recovery agreement procedures
OS / System No new observation tool training-resource-budget, checkpoint-recovery-consistency Budget/locality/lifetime + FFCV cache ownership FFCV cache ownership only No new OS runtime validation

Reviewed-paper additions: the individual rows in §§1, 2 and 4 bind each mechanism to its named target and stage. A–F add no SSD or OS mechanism here. E/F are methodology references only, and Magpie coordination remains pending; none fills a Network cell. The summary tables immediately above retain the earlier coverage as a baseline; the detailed rows in §§1, 2 and 4 include the paper additions.

Skill-consumption integration: Reference A1 → existing CLI metadata catalog, explicit named instructions and on-demand resource-path resolution. This is a cross-cutting consumption connection, not a new hardware performance cell or a third authoritative matrix. Narrow offline checks do not promote skills.

Optimization targets added in this branch:

  • Kernel profiling (CANN CSV pipeline, 5-layer analysis, operator-level msprof loop)

  • Graph compilation (npugraph_ex capture-replay, FX passes, memory reuse)

  • Model adapter (FP8, KV replication, multimodal)

  • Distributed parallelism (USP/CP strategy, topology rules, masking formula, attribution)

  • Host overhead (GPU idle / host-prep verdict, NVTX root-cause, line_profiler loop)

  • Host memory (pinned memory, async H2D, GIL avoidance, NUMA allocation)

  • Device health (topology validation, SIGKILL detection, 4-step smoke)

  • NPU optimization casebook (5 bottleneck classes: AI-CPU fallback, memory-bound, comm-exposed, graph-compile, scheduling)

  • Kernel writing (Triton/CuTe/AscendC routing, 6 correctness rules, verify-before-benchmark workflow)

  • Search policy (phase-locked round, RRSI constraints, CRISP critical-step)

  • Skill evolution (Trace2Skill pool consolidation, SkillOpt bounded edit)

  • Benchmark design (goodput, three-regime, InferRSI evidence trail)

  • Repository intelligence (LTC temporal split, Magpie patterns, Change2Task, Chronos)

  • Operator tuning campaign (accuracy gate, staged escalation, metric contract, routing tables)

  • AscendC tiling design (four-element template, fullload-vs-split, per-class routing, chip-generation tables)

  • Weight load path and startup overhead (fixed-overhead accounting, sharded verification, prefetch rules, offload tiers)

  • OS-level process placement (affinity pinning, NUMA mode semantics, thread env discipline, device-busy preflight, env checks)

  • Loading modes and high-throughput loading (mmap/mlock/dio/lazy, bounce-buffer readers, GDS direct reads, readiness overlap, static fit planner, phase-timer measurement)

  • KV cache tiering (L1/L2 hierarchy, store/retain/prefetch/evict policies, concurrency-vs-RAM admission, local adapters, serde)

  • Checkpoint save/restore semantics (sync sharded writes, per-file atomic publish, opt-in fsync, explicit unsupported list)

  • Storage I/O measurement (workload matching, requested-vs-achieved depth, cache-state ledger, device/process attribution, saturation guards, two-layer evidence gate)

  • Cgroup v2 resource controls (quota-vs-contention, requested/effective cpuset, memory pressure+events, io caps, delegation preflight, rollback contract)

  • CANN attribution and overlap (code_ref ownership, dependency DAG, bounded multistream, metric separation, lineage discipline)

  • Training input pipelines (executor-gated queue/buffer sizing, bounded sample look-ahead, shard cache churn, OS/process cache ownership)

  • Checkpoint recovery consistency (readiness/step gates, complete iteration selection, helper/replica failure domains)

  • Cross-resource training budgets (ZeRO offload constraints, attributed compute/collective/NUMA/GC diagnosis, historical NPU adaptive-memory case)

  • DCP asynchronous save performance (caller pause, staging/upload distinction, cache reuse, bounded in-flight saves and contention)

  • vLLM admission and KV lifetime diagnosis (independent budgets, async readiness, completion/retention/failure ownership; concrete backend unverified)

  • vLLM-Ascend ACLGraph runtime (effective dispatch, caller-owned replay buffers, branch-specific ordering and capture-resource proposals)

  • TorchNPU device/host allocator lifecycles (separate memory domains/counters, stream-use versus ordering, feature gates and unresolved conflicts)

Skill count: 36 branch-added candidate skills + 5 inherited skills = 41 skill directories. All branch-added skills remain status: candidate; inherited skills retain their original metadata. None promoted.

The five wave-8 additions are Dot bundle-mediated code/docs intelligence, not paper extraction or measured gains. Their source pins are independent; no combined stack compatibility is established. No new SSD, Network or OS-priority coverage is claimed; device HBM remains separate from host RAM.

RFC 17: existing main knowledge mapped to resources

These are locally inspected existing imported records, not new skills or new measured effectiveness. The Reference Matrix reconciliation links their exact IDs/pins/readsets and the ownership contract. The five resource rows above remain authoritative; no new coverage is credited for translation, filename, package installation or an unexamined trace.

Resource Stage / named subtargets Existing canonical support Boundary
Accelerator Diagnose/Configure/Verify: attention-generation dispatch, paged-decode planning, compiler guards, KV strategies, fusion resource limits Dependency records; FlashAttention, FlashInfer, PyTorch, Transformers, Triton Source guidance only; selected backend, pointer lifetime, precision and resource envelope require target validation
Accelerator Configure/Optimize/Verify: residual reuse, KV compression, sequence partition and pipeline freshness Peer records; Cache-DiT, LMDeploy, xDiT Approximation/capacity gains do not establish latency or quality gains; preserved LMDeploy regression
RAM / Host Diagnose/Configure: group offload staging, CPU expert work, host prefix residency Diffusers dependency card; Strata tuning; SGLang HiCache peer card Host budget and transfer/recompute cost remain distinct from device HBM
SSD / Storage Configure/Verify: prefix L3 sharing, prefetch/layout/eviction and hit-state comparisons SGLang HiCache Storage backend qualification pending; larger private host cache is not cross-instance sharing
Communication / Network Diagnose/Configure/Verify: disaggregated KV transfer, topology-dependent sequence parallelism TRT-LLM disaggregation; xDiT Interconnect/layout/queue cost, backpressure and recovery; H2D/IPC are not counted here
OS / System No new OS-control mechanism credited Existing OS-specific rows retained Process/stage ownership and installed access are not scheduler-priority or cgroup evidence
Accelerator / Host Observe/Verify: actual active batch, admission/drain scope and state lifetime Shared internal case; main audio guidance Recorded observation versus interpretation; frozen evidence preserved, no new service gain

The existing casebook/search-policy path now also exposes rejected directions for Qwen B300, Qwen Ascend and Drama3. They inform Observe/Diagnose/Verify under their recorded Accelerator/Host scope; no new green coverage, Network/OS capability or measured gain is credited. Current applicability and a changed condition must be established before retry.

The 15 contribution abstracts for Strata, vLLM, SGLang, SGLang-Omni and vLLM-Omni supply scoped architecture, tuning and validation navigation, including their English procedures. They do not qualify every backend listed by their upstream projects. Main-only contribution, dependency and peer material remains at its original owners rather than duplicated under local skills.

PyTorch compiler and checkpoint scope selection

Use the existing-owner scope map to distinguish CUDA compiler guards/capture/output lifetime from the DCP Accelerator staging-pause, RAM/Host snapshot and SSD/Storage save entries. The shared diagnostic questions are actual mutation/liveness and named timing boundaries; graph replay is not staging/upload completion, and DCP completion does not establish replay eligibility. Existing source-derived maturity and resource coverage remain unchanged; no combined protocol is introduced.

TensorRT-LLM shared assumptions

Use the methodology and disaggregation map for the TensorRT-LLM Accelerator, RAM/Host, SSD/Storage, Communication/Network and OS/System entries. Its evidence row fixes the comparison contract; the five resource rows assess host exposure, startup/request units, transport/layout and queue costs, kernel correctness and placement controls against the existing peer card. The retained methodology and PD pins are separate source scopes; this adds no measured/green cell, new runtime mechanism or compatible-stack claim.

Issue #27: Drama3 NPU readiness (guidance only, no new coverage)

The Drama3 NPU readiness supplement is a scoped task artifact: a stage/state execution map, a state-correctness contract (with a CPU-only reference fixture) and a measurement/quality contract reusing the pinned GPU benchmark.py and only the load_runtime record/summary checks of the frozen evidence/drama3-b300-scale150 validator; a target plan/verdict remains a gap. It adds no 🟢 cell and no measured NPU coverage; every Drama3 NPU status is unverified, and target qualification is pending the owner-resolved hardware/software/model/service/evaluation/authority tuple. The five resource rows above remain authoritative; the two-matrix contract is unchanged.

MLX framework guidance (source-derived)

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator Observe / Diagnose / Verify: submission versus completion, fence/event dependency ordering and selected backend dispatch lazy-evaluation-stream-lifetimes Pinned CPU/Metal/CUDA source paths; no native device execution or measured gain
RAM / Host Observe / Diagnose / Verify: graph-reference detachment versus in-flight Data retention and result availability execution/lifetime contract Backend-specific retention mechanisms; allocator eviction and physical-memory release remain outside the readset

No new measured or green coverage is credited. Stream dependency guidance does not add a Communication/Network capability; no SSD or OS control is introduced. common indexing categorizes framework guidance and certifies no backend.

vLLM-Omni incremental abort guidance

Route general stage/model/output decisions through the engine source and responsibility map. For receipt, wait-only timeout or retained transfer resources, follow abort validation and its three completion boundaries. The map keeps the engine and incremental abort source snapshots separate.

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator / RAM / Host Diagnose / Verify: abort submission, logical acknowledgment, partial AR output and separately queued transfer reclamation Abort acknowledgment reference under the existing validation skill Source control-flow and queue ownership, not native device synchronization or measured release completion
RAM / Host Observe / Diagnose / Verify: correlated waiter lifetime, blocking submission, wait-only timeout and late result routing Same canonical reference, source 18c3bd5d2d4323bc7adca1ac3c465c4a53bbfb6e Queue timeout does not certify engine cancellation; actual API/backend/next-request validation not_run

These extend source-derived guidance without a new measured/green cell. No new SSD, Network or OS-control coverage is credited by local RPC or release-task code.

MLX-LM application guidance (source-derived)

Resource Stage / named mechanism Canonical owner Evidence boundary
RAM / Host Observe / Diagnose / Configure / Verify: request UID/stage/cache-row alignment, prefix identity/copy/trim/eviction and cooperative stop generation-cache-lifecycle Logical cache state and allocated storage differ; native model/row/stop behavior unrun
Accelerator Observe / Diagnose / Verify: prefill cache evaluation and asynchronous decode submission/read boundary generation/batch contract Application calls MLX primitives; CPU checks cannot certify Metal/CUDA dispatch, quality or serving performance

No measured or green coverage is credited. Application batching is not a new Network/OS capability; no SSD mechanism is added. common indexes guidance and does not certify backend compatibility.

Diffusers invocation guidance (source-derived)

Resource Stage / named mechanism Canonical owner Evidence boundary
RAM / Host Observe / Diagnose / Configure / Verify: wrapper versus reused component identity, per-invocation scheduler schedule/index ownership diffusers-invocation-state Configuration means selecting the existing invocation/scheduler owner; no new manager. Real caller admission and runtime overlap are unverified.
Accelerator Observe / Diagnose / Verify: post-step callback, next-iteration interrupt observation and normal output/cleanup tail invocation ownership path Cooperative loop skipping does not revoke submitted kernels; model/device/quality, exception cleanup and performance remain unrun.

For group/leaf offload staging and host/device transfer tradeoffs, select the separate CUDA dependency offload record. The two-pin invocation/offload boundary adds routing only: the common invocation candidate does not qualify that CUDA recipe, an NPU path or a combined stack. Existing coverage/maturity stays unchanged.

No measured or green coverage is credited. This adds no SSD, Network or OS capability and does not retest the existing offload mechanism. common indexes framework guidance without certifying backend compatibility.

DeepInfer bounded source guidance

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator Diagnose / Configure / Verify: configured-width DFlash drafts versus budgeted verification rows; lagged CPU totals and GPU ZOS per-request metadata correction budget-speculative-verification Source constraints and cost objective, not sampling-distribution proof, dispatch qualification or measured gain
Accelerator / RAM / Host Diagnose / Configure / Verify: retained query dependency closure, separately preserved full-layout KV, SWA valid-state bounds and nested ring-sizing gates bound-windowed-attention-replay Model-specific boundary/rollback/cache contract; native equivalence and resource measurements unrun
RAM / Host Observe / Diagnose / Verify: host work overlap, one-step retained state, repair and drain obligations Existing host-overhead owner case Source-level schedule, no reproduced host-gap reduction or complete-service latency result

These rows add source-derived procedure guidance; native execution and measured gains remain unverified. They do not add measured resource coverage: local host scheduling and source references to collectives do not establish new SSD, Network or OS-control coverage. Backend and production-service scope remain separate from the public release's inspected code paths.

Task-specific evaluation coverage (issue #60)

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator / Host / service boundary Observe / Verify: select task quality and reference parity; freeze workload, timing, SLOs, units and failure accounting; distinguish agent replay from live task success inference-benchmark-design task reference and commands Cross-task measurement guidance, not new measured hardware coverage. A text HTTP smoke cannot validate TTS, duplex, image/video quality, repeated A/B or production capacity. No runtime adapter or automatic evaluator is added.
Accelerator / Host / decoded-audio boundary Observe / Verify: derive cache-aware ASR startup/steady mel geometry and decoded frontier; distinguish slot reset/reuse, EOU, EOF and full stream reset; retain optional-model test coverage inference-benchmark-design ASR reference NeMo-Speech.cpp 20999cc921d2b14aa58708f61756a8b2e787ba20; selected source and test definitions, author ASR benchmark only. Candidate / empirical not_run; no executed model, device, long-stream/load, ASR quality or service gain and no new runtime adapter.

SGLang scope selection

Use the versioned entry comparison to distinguish LLM engine/Radix ownership, the separate CUDA HiCache peer's prefix-KV host/storage policy, and multimodal_gen diffusion stage/instance, component-use and timeout/job-finalizer lifetimes. This adds routing only. Common indexing is not CUDA/NPU qualification; process transport/IPC is not new Network coverage. Existing resource cells and maturity boundaries remain unchanged.

Hopper kernel reasoning (source-derived)

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator Observe / Diagnose / Configure / Verify: TMA full/empty stage ownership and parity, WGMMA group completion and SS/RS layouts, register/SMEM/epilogue budgets, software persistent scheduling and FP8 scale/promotion contracts reason-hopper-kernels, with systematic source selection Candidate mechanisms qualified by selected pinned code and versioned documentation; remaining source bodies unmined, Blackwell-only recipes excluded. No GPU compilation, accuracy, profiling or performance run.

This adds conditional Hopper guidance without new measured or green coverage. Registers, SMEM and HBM remain device resources; local CTA/cluster coordination and tile scheduling add no RAM/Host, SSD, Network or OS-control coverage.

DwarfStar source-derived execution/state graph

Resource Stage / named mechanism Canonical owner Evidence boundary
RAM / Accelerator / SSD Diagnose / Configure: routed-expert cache versus non-routed weights, context, shared prefill workspace and per-session recurrent/KV state DwarfStar runtime-memory Bounded source-derived budgeting guidance; native GPU/model SSD I/O and actual peak memory are not validated.
Host / Accelerator / service boundary Diagnose / Verify: token and image prefix identity, disk payload compatibility, tool replay, speculation rollback and actual batching/fallback path DwarfStar serving-state and qualification Model-free CPU bookkeeping and source contracts are distinct from accelerated correctness, physical TCP/RDMA or service performance. This adds no measured Network coverage.
RAM / Accelerator / SSD Diagnose / Configure / Verify: Root routing: bind model/backend/cache-state identity, then select runtime-memory, serving-state or qualification ownership dwarfstar Navigation to the existing focused rows; no additional mechanism or native/model/SSD-I/O/Network/performance qualification.

Splash source-derived planning and state

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator / RAM / SSD Diagnose / Configure: mandatory memory floors, expert/PLE stores, dynamic KV/state, disk spill and precision boundaries runtime-planning Source and CPU syntax checks, not measured device/process peak memory or actual spill IO.
Host / Accelerator / service boundary Diagnose / Verify: salted prompt identity versus global image reuse, composite-state commit/reclaim, persistent namespace and process-local Responses history, adaptive DFlash2 row budgeting serving-cache and qualification Python codec/LRU and descriptive statistics do not qualify native Metal/ANE/model state, service SLOs or efficacy. No measured Network coverage is added.
Accelerator / RAM / SSD Diagnose / Configure / Verify: Root routing: bind representation, memory/cache-state and actual-path identity before runtime-planning, serving-cache or qualification splash Navigation to focused rows; required Metal/ANE/model/disk-state evidence remains blocked, with no new performance, service or Network qualification.

Lily exact-model source contracts

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator / RAM Diagnose / Configure / Verify: exact affine-Q4 plus role-Q8, shared buffer views, state/scratch/sequence capacity and completion ownership model-runtime Source/Python selected-rule checks, not Rust loading, actual memory fit, Metal synchronization or model quality.
Host / Accelerator / service boundary Diagnose / Verify: strict API, extra lookahead token, strict token prefix before key preference, actually-fed state retention and version/metric qualification serving-cache and qualification Compiler/native/model stages not run; CPU references and ignored/cfg-omitted tests are not GPU evidence. No measured service or Network coverage is added.
Accelerator / RAM Diagnose / Configure / Verify: Root routing: exact model/affine payload, token frontier and extra lookahead-slot identity before model-runtime, serving-cache and qualification checks lily Navigation to existing focused owners; no added native memory-fit, Metal state, model-quality, service or Network qualification.

Husky research questions, not measured coverage

Resource Stage Owner Evidence boundary
Unqualified / conditional host-device questions Diagnose / propose a falsifiable comparison and required artifacts Husky document research Author-reported unversioned page and original audit questions only. No inspected implementation, backend validation, measured service improvement or expanded resource coverage.

LoopX Goal governance (cross-cutting workflow)

Scope / stage Named decision mechanism Canonical owner Evidence boundary
Cross-cutting Observe / Diagnose / Verify Current Todo authority versus Markdown/index/audit projections; missing active Goal state, complete terminal evidence and ordered capability/workspace/inbox/repair decisions rsi-evidence Goal governance Pinned LoopX source/doc/test assertions, f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9; source-derived candidate, runtime and agent evaluation not_run
Cross-cutting Diagnose / Verify, consumed by experiment selection Local accounting effect/request identity, index CAS, commit repair versus replay lookup, exact Goal instance and external-effect crash-window limits Same owner; optimization-search-policy links it Local receipt idempotency does not prove external exactly-once or a supplier-side hard cost cap; a decision/receipt is not a campaign verdict

This adds workflow knowledge, with no new green or measured resource cell. Agent coordination/accounting is not Accelerator, Communication/Network or OS-control qualification; local journal files do not establish SSD performance. The existing five resource rows and all prior source/evidence boundaries remain unchanged.

Project Maya model-specific source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM / SSD Diagnose / plan: family/representation guards, physical-pool ownership, expert tiers, private state and scratch overlap project-maya: model-runtime Source plus bounded host arithmetic/simulation; not actual accelerator allocation, pressure, tensor payload or residency performance.
Host / Accelerator / service boundary Diagnose / verify: API snapshots versus native slots, parallel state, completion, speculative retained state, calibration controls and quality evidence project-maya: serving-state and qualification Required CUDA/HIP/model/parallel restore/cancellation evidence remains unavailable. HTTP procedure adds no measured Network/OS or service-gain coverage. Qwen deep state/kernels remain unmined.

vLLM Metal selected source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM / SSD Diagnose / configure: lazy KV capacity versus resident commitment, one physical unified pool, host/disk ownership, transfer/sleep/reuse and identity vllm-metal: cache-lifecycle Selected substantive source/test assertions plus dense-uniform arithmetic only; real transfers/pressure/sleep/model/backend evidence blocked. No measured memory-fit or IO gain.
Accelerator / host control boundary Diagnose / verify: native ABI/backend/runner pairing, GDN state and deferred output ownership, executed dispatch versus selector, serving evidence vllm-metal: platform-config and qualification Installed pure API and selected parser/default tests do not qualify Apple/native/model/quality/service. No new Network/OS control or green resource coverage.

mistral.rs Metal paged-attention source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM Diagnose / configure: HND/packed cache/scale/GQA contracts, instantaneous scratch versus capacity and prefill output/metadata reservation mistral-rs: decode-contract and prefill-contract Source-derived byte/metadata support only; no actual storage/allocator/pressure/parity or physical fit. No measured resource coverage.
Accelerator / host control boundary Diagnose / verify: absolute positions/masks/ranges and actual-model warmup/pipeline cache versus executed native test evidence mistral-rs: qualification Compiler and native backend not_run; shape inventory is not GPU simulation or dispatch. No new SSD/Network/OS control, quality/service or performance qualification.

MLX Swift LM ownership and recurrent Metal source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Host control / accelerator lifetime Diagnose / verify: exclusive async access, Sendable transfer, lookahead/cancellation/finalization/default-stream sync versus task join mlx-swift-lm: generation-lifecycle Caller evidence declarations only; no actual task/device execution, native cleanup, backgrounding or safe shared-cache proof. No measured resource coverage.
Accelerator / RAM Diagnose / configure: recurrent FP32 state/layout, Dk tails, public kernel/ops gate, source mask discrepancy and dispatch/parity evidence mlx-swift-lm: gated-delta-contract Static source/declared bytes only; no actual provider stride/allocator/dispatch/numerical or physical-fit qualification. No new SSD/Network/OS control, quality/service or performance result.

MLC LLM artifact, target and runtime source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM Diagnose / configure: parameter-only resource declarations versus runtime KV/workspace/limit origins mlc-llm: artifact-contract and runtime-kv-limits Source/hash/declared minima only; no authenticated weights/allocator/free-memory/cache or physical-fit evidence. No measured resource coverage.
Accelerator / host completion boundary Diagnose / verify: SDK/export/prefix, runnable versus serializable module, private/staging buffers, copy/flush/wait/default-stream contracts mlc-llm: metal-build-runtime Native compile/load/dispatch/sync/parity not_run. No new SSD/Network/OS control or quality/service/performance qualification.

Flare preparation and complete-session source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator Diagnose / Verify: complete KV page chain plus continuation row; publish/use/retire fences, shared inbound ticket and five admission outcomes audit-flare-session-lifecycle Fixed bce032c014bae34502ba22d3304fa084d5b40afe CODE/DOC/TEST-SOURCE; NVIDIA SM120/SM121 research prototype; no kernel/model/device/performance qualification
RAM / Host Diagnose / Verify: registered/staging storage lifetime through accounted transfer completion/drain; waiter cancellation does not free ticket destinations audit-flare-session-lifecycle, lifetime detail Source lifetime contract; no physical-RAM peak, bandwidth or pressure result
SSD Diagnose / Verify: record namespace/layout/row/chain completeness, restart/corruption versus unreadable-file handling; conditional transfer-versus-additional-prefill cost audit-flare-session-lifecycle, diagnostic detail No GDS implementation or measured storage gain; actual device/server tests not_run
SSD / RAM / Accelerator Startup Diagnose / Verify: Python preparation/native export ownership, nonportable live handles, execution ABI 4 versus prepared contract 2, separate phase accounting Existing weight-loading owner, preparation detail Source-derived candidate; no preparation/runtime execution or Python-to-Rust performance-causation claim

There is no new measured/green resource cell, Network or OS coverage. Multi-rank source branches and tests do not qualify multi-host transport or multi-GPU behavior. The source-derived candidates remain empirical not_run; the bounded evaluation plan is not evaluated.

vLLM GDN operator qualification

Resource Stages and mechanism Owner and evidence Qualification boundary
Accelerator Diagnose / Configure / Verify: concrete Qwen3-Next GDN caller, requested/effective prefill recurrence, separate in-tree conv fwd/update, mixed/speculative metadata, state-row and progress contract kernel-writing-guide reference, four source cases; vLLM @ 537a79f0fc5ba564d2bef43d54915b2f80f9f77d Source-derived candidate; native not_run, agent efficacy not_evaluated. HF caller binding, external implementation equivalence, physical allocation/recycling, target hardware and performance remain unqualified.

SGLang GDN operator qualification

Resource Scoped stages and question Canonical owner / fixed source Evidence boundary
Accelerator Diagnose / Configure / Verify: Qwen3.5/Qwen3-Next linear-layer caller, per-runner GDN prefill/decode and architecture predicates, separate conv stride/alias selection, packed metadata, state rows/write-back and conditional classification-prefill transfer kernel-writing-guide reference, four source cases; SGLang @ 16176278d0180f1a1a578e53e6eef432577876c2 Source-derived candidate/not_run; agent efficacy not_evaluated. No HF binding, installed/native backend, sm89 compatibility, numerical/model-quality or performance claim. No new RAM, Storage, Network or OS coverage.

Local Dream Android diffusion source contracts

Resource Stage / mechanism Canonical owner Evidence boundary
Host control / accelerator lifetime Diagnose / verify: desired versus serving full configuration, startup-only preferences, grace/restart, process-spawn versus health and actual shape local-dream, Android lifecycle Static source and exact declaration comparison, not Android lifecycle execution, native readiness, tensor shape or safe reuse proof.
Accelerator / RAM / persistent-cache boundary Diagnose / configure: MNN/QNN/Anima/DiT dispatch, shared generation/upscale exclusion, callback cancellation, mode/layout versus model-revision cache identity local-dream native contracts No actual allocations/cache parity/model/image quality/fit/thermal/performance result; not a measured SSD gain or universal cache-safety claim.
Native packaging / remote exposure boundary Diagnose / verify: core executable, CPU DiT library versus DSP skels, ABI and SDK provenance; unauthenticated LAN control versus readiness local-dream packaging, host limits Required native evidence BLOCKED_REQUIRED_BACKEND / not_run; no authorization to expose ports, Network transport optimization, OS control, native compatibility or service gain.

This source-derived candidate adds no measured/green resource coverage. Matching configuration/ABI/SDK declarations are not native execution; qualification remains INCONCLUSIVE. Upstream CC BY-NC 4.0 rights are retained separately from API NOASSERTION and InferRSI's repository license.

Video diffusion campaign guidance (unvalidated)

Resource Stage / named mechanism Canonical owner Evidence boundary
Accelerator (guidance only) Observe / Profile / Diagnose / Optimize / Verify: Ascend specialization of the shared workflow: contiguous packed-layout conversion, TransData attribution and CANN stream/lifetime checks ascend-video-diffusion-optimization First-hand candidate guidance; field notes are untraceable/non-reproducible. No native, numerical or measured-gain qualification.
Accelerator (workflow guidance only) Observe / Profile / Diagnose / Configure / Optimize / Verify: Video-stage planning and acceptance: residency/quantization trade-offs, dependency-and-lifetime gated overlap, attention eligibility and memory/quality-gated VAE chunking video-diffusion-optimization First-hand candidate workflow, not independently validated; no model/backend/numerical/performance qualification or additional Host/Storage/Network/OS mechanism.

JustGoal controlled-evaluation guidance (source-derived)

Resource context Stage / scoped question Canonical owner Evidence boundary
Accelerator (methodology only) Verify: same-model/budget/scope controls, independent complete-output correctness, released versus repaired counts and workspace/stream/cache timing contracts inference-benchmark-design reference and source record; JustGoal repo + paired paper Source-derived candidate support for benchmark verification only. Author-reported results; native/device/performance not_run, efficacy not_evaluated. No new hardware implementation, measured resource coverage, backend compatibility or RAM/Storage/Network/OS coverage; no full campaign runner.

AutoTuneBench trial-validity guidance (source-derived)

Resource context Stage / scoped question Canonical owner Evidence boundary
Accelerator (methodology only) Verify: trace treatment to the eligible consumer, apply frozen arm validity before interpreting a null, and separate declared protocol, reached checks and persisted gate evidence inference-benchmark-design reference and source record; AutoTuneBench repo + paired paper Source-derived candidate / empirical not_run; paper measurements remain author-reported, efficacy not_evaluated. Contextual source bars are not InferRSI defaults. No new implementation, measured hardware/resource coverage, backend compatibility, or RAM/Storage/Network/OS coverage.

Xe-Forge trial-evidence guidance (source-derived)

Resource Stage and mechanism Source and owner Evidence boundary
Accelerator (methodology only) Verify: bind saved candidate and comparator identities, distinguish tool origin from resolved performance, inspect host gate/ratio persistence, and follow manual versus measured-only delivery and baseline-oracle boundaries inference-benchmark-design reference and source record; Xe-Forge repo + paired paper Source-derived candidate / empirical not_run; paper/README results remain author-reported, efficacy not_evaluated. No new implementation, measured hardware/resource coverage, backend compatibility, whole-model/service promotion or RAM/Storage/Network/OS coverage; draft extensions remain proposals.

NInfer state and completion validation (source-derived)

Resource context Stage / scoped question Canonical owner Evidence boundary
Accelerator / Host / service boundary (methodology only) Observe / Verify: freeze coupled checkpoint and same-block state oracle; distinguish zero accepted drafts from zero adopted inputs, logical commit from device retirement, and selected steady intervals from complete-cohort/termination accounting inference-benchmark-design reference and source record; NInfer 1882c6adbc98043fcf89f3589aa5addb3742777a Source-derived candidate / empirical not_run; unexecuted test definitions and historical author results with a separate execution revision. No new implementation, measured hardware/resource coverage, backend compatibility, native state/model/quality/performance result or service acceptance; efficacy not_evaluated.

TorchNPU framework source procedures

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM / Host Diagnose / configure / verify: PrivateUse1 registration versus native readiness, operator schema/fallback/Meta boundaries, stream ordering versus allocation lifetime, pinned copies, graph storage and AMP state torch-npu operating graph, memory route, graph route Selected source spans and test definitions, not actual allocations, dispatch, numerical parity, compiler conformance, physical fit or measured resource benefit. Earlier allocator skill pins remain independent.
Communication / Network / Accelerator Diagnose / verify: rank/device mapping, matching collective sequence, HCCL work handles, communication-stream ownership and host versus consumer completion torch-npu: HCCL route, source and dependency boundaries HCCL process-group source and test assertions do not qualify transport, multi-rank correctness, overlap or throughput. Required Linux/Ascend/CANN/HCCL execution remains BLOCKED_REQUIRED_BACKEND / not_run; no new measured/green resource cell or OS control.

PyTorch framework source procedures

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / RAM / Host Diagnose / configure / verify: tensor/alias/gradient state, training and data ownership, compiler guards and export contracts, allocator accounting, stream dependencies and graph input/output lifetimes pytorch operating graph, memory/graph route, compiler/export route Source version 2.16.0a0 and observed CPU wheel 2.15.0+cpu are distinct. CPU observations do not qualify pinned-source execution, CUDA/ROCm/MPS/XPU, native custom operators, memory fit or model quality.
SSD / Storage / RAM Diagnose / verify: serialization trust and state identity, checkpoint ownership, distributed restore and capture/staging/upload completion boundaries pytorch: distributed/checkpoint route; existing DCP async-save owner retains its separate source pin No source-build checkpoint run, backend durability, distributed recovery or measured storage benefit is established. A state-dict round trip in a separate installed wheel does not qualify those properties.
Communication / Network / Accelerator Diagnose / verify: process/rank/device mapping, backend and collective schedule, DDP/FSDP2/DTensor state and completion pytorch: distributed/checkpoint route, source and runtime boundary Selected source and CPU controls do not establish NCCL or other accelerator transport, multi-device/multi-node correctness, overlap or throughput. Required backend checks remain BLOCKED_REQUIRED_BACKEND; no new measured/green resource cell, OS control or compatible latest PyTorch/TorchNPU stack is asserted.

vLLM Ascend v0.27.1rc1 serving diagnosis

Resource Stage / mechanism Canonical owner Evidence boundary
Accelerator / Host Diagnose engine-process plugin discovery and MiniMax-M3 mixed KV storage from hardware capability through per-layer resolution vllm-ascend, cache path Entry-point metadata and cache specs can be inspected separately from device dispatch and numerics.
Communication / Accelerator Verify DSpark draft budget, effective P/D ranks and target capture tiers vllm-ascend, DSpark PD TP4 acceptance-length fixture and two-node 950DT deployment have separate evidence requirements.
Accelerator / RAM / Host Diagnose fused preprocessing, grouped GEMM/activation/quant, residual specialization, KV index reuse and reverted optimizations vllm-ascend, fusion, model performance Whole-history discovery and selected semantic review remain separate from native performance validation.

DeepSeek streamed expert-cache verification

Resource Stage / scoped question Canonical owner Evidence boundary
SSD / Storage + RAM / Host Diagnose / Verify: independent tensor-byte identity after aligned/coalesced reads, resident versus pending slot lifetime, ordered consumer accumulation and actual oracle callback coverage weight-loading-optimization (expert-cache oracle reference) and source record; DeepSeek source at d0402056d0da4bf7edfd2d4977de479a0db9d201 Source-derived candidate / empirical not_run; byte oracle shares tensor metadata, resident snapshot covers w1, and tiny PyTorch/batched checks are source definitions. No measured resource gain, full-model quality, native/device execution or agent efficacy is established.

Faster CosyVoice evaluation source context

The existing inference-benchmark-design uses dependency-aware TTS playout and its source record as methodology/source context for server PCM credit, client playback observations, chunk dependencies and partial cost denominators. This adds no numbered resource-layer mapping, lifecycle-stage mapping, supplementary hardware coverage or hardware prerequisite. The owner's existing N/A reason remains unchanged; source use of GPU, TensorRT, Flow or HiFT does not alter that boundary. Source-derived candidate / empirical not_run / efficacy not_evaluated; author reports are not local quality or service results.

Embodied.cpp benchmark methodology

The inference-benchmark-design method uses Embodied.cpp at 156bed15 and arXiv:2607.02501v3 as source context for observation-to-action timing, generated versus replayed actions, precision identity and per-task closed-loop quality. Its operating reference and readset retain the later author's measurement and source-version limits.

These are comparison methods. Experiment hardware is source context rather than a device prerequisite or a resource mechanism for this method.