Skip to content

Make the ONNX runtime path torch-free - #99

Open
li-lizhe wants to merge 1 commit into
OpenMOSS:mainfrom
li-lizhe:torch-free-onnx-runtime
Open

li-lizhe wants to merge 1 commit into
OpenMOSS:mainfrom
li-lizhe:torch-free-onnx-runtime

Conversation

@li-lizhe

Copy link
Copy Markdown

Problem

The README advertises:

No PyTorch dependency during inference: it runs directly on ONNX Runtime CPU.

However, onnx_tts_runtime.py does import torch / import torchaudio at module level (lines 12-13), so app_onnx.py and infer_onnx.py fail with ImportError on any machine where PyTorch is not installed. This defeats the purpose of the lightweight ONNX deployment path on resource-constrained devices (ARM64 boards, proot environments — see #73).

Root cause

The only actual torch/torchaudio usage in the ONNX runtime is _load_reference_audio(): audio loading, float32 conversion, resampling, and channel conversion — all only needed for reference-audio voice cloning.

Fix

Replace them with NumPy + soundfile (soundfile is already in requirements.txt):

  • torchaudio.load() → soundfile.read(dtype="float32", always_2d=True) — same (channels, time) layout and the same [-1, 1] float normalization convention
  • torchaudio.functional.resample() → a pure-NumPy polyphase windowed-sinc resampler (Hann-windowed, 32 taps per side — the same interpolation family torchaudio uses), added as _resample_waveform()
  • torch.Tensor.repeat / .mean → np.repeat / np.mean
  • soundfile is now an optional import: if voice cloning is requested without it, a clear ImportError with install instructions is raised instead of a crash at module import

Verification

Verified on aarch64 (Kunpeng ARM, openEuler) in an environment with no torch/torchaudio installed:

  1. import onnx_tts_runtime now succeeds without PyTorch (previously ImportError).
  2. _resample_waveform matches scipy.signal.resample_poly (the same windowed-sinc family torchaudio uses) on band-limited test signals for 44100→24000, 48000→16000, 16000→24000, 22050→16000 — output lengths identical, central-region max abs error < 8e-4 (relative < 8e-4).
  3. _load_reference_audio end-to-end on real WAV files: mono 24 kHz passthrough (bit-exact max amplitude), stereo→mono downmix, and 44.1 kHz→24 kHz resampling all produce the expected (1, 1, N) float32 arrays.
  4. Identity case (same sample rate) returns the input unchanged.

Fixes #73

The README advertises "No PyTorch dependency during inference: it runs
directly on ONNX Runtime CPU", but `onnx_tts_runtime.py` imports torch and
torchaudio at module level, so `app_onnx.py` / `infer_onnx.py` fail with
ImportError on any machine without PyTorch installed (e.g. ARM64 boards,
proot environments).

The only actual torch/torchaudio usage was `_load_reference_audio()`:
audio loading, float32 conversion, resampling, and channel conversion —
all of which are only needed for reference-audio voice cloning.

Replace them with NumPy + soundfile (already in requirements.txt):

- `torchaudio.load` -> `soundfile.read(dtype="float32", always_2d=True)`
  with the same (channels, time) layout and [-1, 1] normalization
- `torchaudio.functional.resample` -> a pure-NumPy polyphase
  windowed-sinc resampler (Hann-windowed, 32 taps per side — the same
  interpolation family torchaudio uses), added as `_resample_waveform()`
- `torch.Tensor.repeat/mean` -> `np.repeat` / `np.mean`
- `soundfile` becomes an optional import with a clear error message if
  voice cloning is attempted without it

Fixes OpenMOSS#73
@li-lizhe

Copy link
Copy Markdown
Author

Gentle ping on this one — it's been quiet for 27 days since the last activity, so I wanted to check whether there's anything else you'd like me to adjust or add before this can move forward. Happy to make changes if so. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ONNX runtime still imports torch/torchaudio in onnx_tts_runtime.py

1 participant