7 modelsAll on one Radeon RX 9070 XT
How much memory does a speech model need?
The most popular open speech-to-text and text-to-speech models from Hugging Face, run on one desktop graphics card: what they take in memory, how fast they go and how accurate they are.
Speech to text
Memory at its peak while transcribing 200 LibriSpeech clips one at a time, the way dictation sends them. Speed is seconds of audio per second of work; error is the share of words it got wrong on test-clean.
ModelGPU memoryRAMSpeedError
- Whisper smallOpenAI · 242M · 722 MB in allGPU memory540 MBRAM182 MBSpeed69.5×Error2.5%
- Whisper large-v3 turboOpenAI · 809M · 1.1 GB in allGPU memory908 MBRAM173 MBSpeed48.3×Error1.4%
- Nemotron 3.5 ASRNVIDIA · 638M · 1.8 GB in allGPU memory1.4 GBRAM426 MBSpeed97.9×Error3.6%
- Parakeet TDT 0.6B v3NVIDIA · 627M · 3.6 GB in allGPU memory2.0 GBRAM1.6 GBSpeed173×Error1.5%
Text to speech
Memory at its peak while speaking a paragraph and twenty test sentences. Speed is seconds of speech per second of work; error is the share of words Whisper couldn't hear back.
How it's measured
The same tests for every model on the same machine, so the numbers line up. Your computer will give different numbers, but the order should hold.
The machine
- AMD Radeon RX 9070 XT, 16 GB, Vulkan through RADV (Mesa 26.2.2)
- AMD Ryzen 5 5600, 32 GB of RAM
- Linux 7.2 (Omarchy)
Memory
- RAM is the process's peak resident memory
- GPU memory is what the graphics driver charges to the process at its peak
- Both include the runtime, not just the weights; most models use 8-bit weights
Speed
- Speech to text: one clip at a time, two warm-up clips first
- Text to speech: five runs after two warm-ups, averaged
- Every model on the GPU through Vulkan: whisper.cpp, audio.cpp, or Phonon's own runtime
Accuracy
- Speech to text: 100 clips each from LibriSpeech test-clean and test-other
- Text to speech: Whisper large-v3 turbo listens to 20 tricky sentences
- Both sides normalized, so “5” and “five” match