Pocket 06: named voices and delayed loop decisions — native C++ / Max for Live: beatbox kick, snare and hats in one pass, capture two bars with your original timing and dynamics, then overdub or undo. Auto uses a compiled causal classifier and optional personal voice examples. It runs offline; Python stays in research. See the Pocket guide for playing, tests and the limits of the current voice model.
A voice-driven arrangement agent for Ableton Live. Make a sound with your voice, get it into your session — as drums, bass, lead, or an ambient drone.
Status: working 4-voice umbrella — a shared voice→MIDI→Ableton engine with a device registry. Drums are the most mature voice (regression oracle + per-user classifier); bass has been verified on real takes in Live; lead and drone work but are less exercised. Pitched output gets confidence-gated note correction and bar-fit/looping. Known weaknesses are catalogued honestly — reviewers should start there.
Reading order for reviewers:
docs/ARCHITECTURE.md— current architecture + per-voice maturitydocs/KNOWN-LIMITATIONS.md— ranked, honest gap listdocs/handover.md— living status / next workeval/README.md— quality methodology + actual numbers
See also docs/spec.md (historical v0 vision spec — predates
most of the code), docs/corpus.md (labelling convention),
and docs/mac-handoff.md (first-run setup on the Mac).
Repo codename:
uhntiss. Project namemouthflowis a placeholder — rename before the first public release if something better shows up.
Prereqs: Python 3.11, uv, Ableton Live 11+,
ableton-mcp installed as a Live
Remote Script, and an ANTHROPIC_API_KEY in your environment.
Forked-bridge features:
transcribe-clip(transcribe the clip selected in Live) and drone macro automation need two extra Remote Script commands that are not in stock ableton-mcp — splice them in perbridge/README.md. Everything else runs on a stock install.
uv sync
uv run pytest # 75 tests, all offline
uv run mouthflow doctor # preflight: API key, Live socket, browser
uv run mouthflow record --device bass # 15s capture → pipeline → applied to Live
uv run mouthflow record-stream --device lead # open-ended take; 'stop' on stdin ends it
uv run mouthflow transcribe-clip --device bass # transcribe the clip selected in Live (needs fork)
uv run mouthflow run clip.wav --device drone # skip the capture step
uv run mouthflow dry-run clip.wav --device lead --json # pipeline, print Plan, don't touch Live--device is one of drums | bass | lead | drone (default drums), or
auto to route by ear. Pitched voices also take --bars auto|off|4|8|16
(fit + loop), --correct/--no-correct, --key, --scale (note correction).
Other commands: input-devices, list-kits. See
docs/handover.md for the current state of each voice.
The Max-for-Live device panels live in m4l/ —
python m4l/generate.py --install regenerates them and syncs them into Live's
User Library.
See docs/ARCHITECTURE.md for the module map and data
flow. Quality methodology and current numbers: eval/README.md.
MIT. See LICENSE and CONTRIBUTING.md.
The shipped cross-performer drum model (mouthflow/drum_global.json) is
trained on the Amateur Vocal Percussion (AVP) dataset — Delgado, McDonald,
Xu, Saitis, Sandler, A New Dataset for Amateur Vocal Percussion Analysis
(AM'19), https://zenodo.org/records/3250230, CC BY 4.0. The dataset itself is
not redistributed here; eval/avp.py explains how to fetch it.