Conversation
The README advertises "No PyTorch dependency during inference: it runs directly on ONNX Runtime CPU", but `onnx_tts_runtime.py` imports torch and torchaudio at module level, so `app_onnx.py` / `infer_onnx.py` fail with ImportError on any machine without PyTorch installed (e.g. ARM64 boards, proot environments). The only actual torch/torchaudio usage was `_load_reference_audio()`: audio loading, float32 conversion, resampling, and channel conversion — all of which are only needed for reference-audio voice cloning. Replace them with NumPy + soundfile (already in requirements.txt): - `torchaudio.load` -> `soundfile.read(dtype="float32", always_2d=True)` with the same (channels, time) layout and [-1, 1] normalization - `torchaudio.functional.resample` -> a pure-NumPy polyphase windowed-sinc resampler (Hann-windowed, 32 taps per side — the same interpolation family torchaudio uses), added as `_resample_waveform()` - `torch.Tensor.repeat/mean` -> `np.repeat` / `np.mean` - `soundfile` becomes an optional import with a clear error message if voice cloning is attempted without it Fixes OpenMOSS#73
Author
|
Gentle ping on this one — it's been quiet for 27 days since the last activity, so I wanted to check whether there's anything else you'd like me to adjust or add before this can move forward. Happy to make changes if so. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The README advertises:
However,
onnx_tts_runtime.pydoesimport torch/import torchaudioat module level (lines 12-13), soapp_onnx.pyandinfer_onnx.pyfail withImportErroron any machine where PyTorch is not installed. This defeats the purpose of the lightweight ONNX deployment path on resource-constrained devices (ARM64 boards, proot environments — see #73).Root cause
The only actual torch/torchaudio usage in the ONNX runtime is
_load_reference_audio(): audio loading, float32 conversion, resampling, and channel conversion — all only needed for reference-audio voice cloning.Fix
Replace them with NumPy + soundfile (soundfile is already in
requirements.txt):torchaudio.load()→soundfile.read(dtype="float32", always_2d=True)— same(channels, time)layout and the same[-1, 1]float normalization conventiontorchaudio.functional.resample()→ a pure-NumPy polyphase windowed-sinc resampler (Hann-windowed, 32 taps per side — the same interpolation family torchaudio uses), added as_resample_waveform()torch.Tensor.repeat/.mean→np.repeat/np.meansoundfileis now an optional import: if voice cloning is requested without it, a clearImportErrorwith install instructions is raised instead of a crash at module importVerification
Verified on aarch64 (Kunpeng ARM, openEuler) in an environment with no torch/torchaudio installed:
import onnx_tts_runtimenow succeeds without PyTorch (previouslyImportError)._resample_waveformmatchesscipy.signal.resample_poly(the same windowed-sinc family torchaudio uses) on band-limited test signals for 44100→24000, 48000→16000, 16000→24000, 22050→16000 — output lengths identical, central-region max abs error < 8e-4 (relative < 8e-4)._load_reference_audioend-to-end on real WAV files: mono 24 kHz passthrough (bit-exact max amplitude), stereo→mono downmix, and 44.1 kHz→24 kHz resampling all produce the expected(1, 1, N)float32 arrays.Fixes #73