docs(ane-profiler): explain why convolutions map to the ANE and attention doesn't - #887
docs(ane-profiler): explain why convolutions map to the ANE and attention doesn't#887Alex-Wengg wants to merge 1 commit into
Conversation
…esn't The profiler doc records the ANE/GPU/CPU splits and lists the gotchas, but never explains the architectural cause. Adds a closing section tying the symptoms together: the ANE is a weight-stationary convolution engine, and attention is activation-x-activation matmuls + transposes + softmax + dynamic KV state — each mapped to the specific gotcha and repo example it produces (Kokoro vocoder 99% ANE, PocketTTS flowlm scatter rejection, Parakeet v3 encoder shipping on GPU). Also notes the mirror-image on NVIDIA tensor cores and the practical rule: ANE for transformer graphs is a power/residency play earned by graph surgery; for conv graphs it's free performance.
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m32s • 09/03/2026, 12:01 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 41.7s diarization time • Test runtime: 3m 23s • 09/03/2026, 12:02 PM EST |
PocketTTS Smoke Test ✅
Runtime: 0m22s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon. |
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 6m 19s • 2026-09-03T16:05:35.451Z |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
Supertonic3 Smoke Test ✅
Runtime: 1m14s Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf. |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 127.7s processing • Test runtime: 2m 20s • 09/03/2026, 12:14 PM EST |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 15m26s • 09/03/2026, 12:23 PM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
1 participant
The ANE Profiler doc records the measured ANE/GPU/CPU splits and lists the gotchas that knock work off the ANE, but never explains why the splits fall the way they do.
Adds a closing section, "Why convolutions love the ANE and attention doesn't", tying the symptoms to the architectural cause:
Kᵀtransposes as real memory-bound ops, softmax stalling the pipeline, S×S matrices that don't tile, and dynamic KV-cache state.Docs-only change; no code.
🤖 Generated with Claude Code