Vocalbench
All speech to text modelsSpeech to textOpenAI · 2024

Whisper large-v3 turbo

Whisper large-v3 with its decoder cut from 32 layers to 4, for speed.

openai/whisper-large-v3-turbo on Hugging Face
908 MBof GPU memory at its peak
173 MBof RAM at its peak
48.3×faster than real time, one clip at a time
1.4%of words wrong on LibriSpeech test-clean

Memory

The most it held at once, from loading to the last test. Next to every other speech to text model here.

GPU memoryShorter is lighter. #2 of 4.
RAMShorter is lighter. Best of 4.

Speed

Clips go in one at a time, the way a dictation app sends them, so this is the speed you'd feel.

Seconds of audio per second of workLonger is faster. #4 of 4.
155 msfor a typical clip (the median; clips run 2 to 24 s)
32 sto transcribe all 26 minutes of the test set
0.3 sto load the model

Accuracy

Word error rate on 100 clips each of LibriSpeech test-clean (clear reading) and test-other (harder speakers). Below, what it heard on six of them.

Word error rate, test-cleanShorter is more accurate. Best of 4.
Word error rate, test-otherShorter is more accurate. #2 of 4.
  1. 01

    He cried in high dudgeon just as if he owned the whole of the peppers and could dispose of them all to suit his fancy

    “he cried in high dudgeon, just as if he owned the whole of the peppers, and could dispose of them all to suit his fancy.”· test-clean

    0:07
  2. 02

    She screamed and shouted too with a terrific volume of sound which doubtless caused the hearts of the fugitives to quake within them

    “she screamed and shouted too with a terrific volume of sound which doubtless caused the hearts of the fugitives to quake within them”· test-clean

    0:08
  3. 03

    If you are not then knocked on the head your being a non composser will protect you and you'll then have a good reason to expect to die in your bed

    “If you are not then knocked on the head, your being a non-composer will protect you, and you'll then have a good reason to expect to die in your bed.”· test-clean

    0:08
  4. 04

    So far it has been used mostly for garnishment of other dishes and it is only recently that the hof brau has been making a specialty of them

    “So far it has been used mostly for garnishment of other dishes, and it is only recently that the Hofbrau has been making a specialty of them.”· test-other

    0:07
  5. 05

    By allah an thou fetch him not to me i will assuredly rouse the chamberlain on thee and he shall beat thee and cast thee out

    “by allah and thou fetch him not to me i will assuredly rouse the chamberlain on thee and he shall beat thee and cast thee out”· test-other

    0:10
  6. 06

    I have missed four fits and had but five and have recovered so much strength as made me venture to meet your letter on wednesday a mile from home

    “i have missed four fits and have had but five and have recovered so much strength as made me venture to meet your letter on wednesday a mile from home”· test-other

    0:11

How it ran

809Mparameters
0.3 sto load onto the GPU
MITlicense
99 languageslanguages

Ran with whisper.cpp on Vulkan, q5_0 weights, greedy decoding on an AMD Radeon RX 9070 XT (16 GB, Vulkan through RADV (Mesa 26.2.2)).