Vocalbench
All speech to text modelsSpeech to textOpenAI · 2022

Whisper small

The small multilingual Whisper: the usual pick when large-v3 is too heavy.

openai/whisper-small on Hugging Face
540 MBof GPU memory at its peak
182 MBof RAM at its peak
69.5×faster than real time, one clip at a time
2.5%of words wrong on LibriSpeech test-clean

Memory

The most it held at once, from loading to the last test. Next to every other speech to text model here.

GPU memoryShorter is lighter. Best of 4.
RAMShorter is lighter. #2 of 4.

Speed

Clips go in one at a time, the way a dictation app sends them, so this is the speed you'd feel.

Seconds of audio per second of workLonger is faster. #3 of 4.
102 msfor a typical clip (the median; clips run 2 to 24 s)
22 sto transcribe all 26 minutes of the test set
0.1 sto load the model

Accuracy

Word error rate on 100 clips each of LibriSpeech test-clean (clear reading) and test-other (harder speakers). Below, what it heard on six of them.

Word error rate, test-cleanShorter is more accurate. #3 of 4.
Word error rate, test-otherShorter is more accurate. #3 of 4.
  1. 01

    He cried in high dudgeon just as if he owned the whole of the peppers and could dispose of them all to suit his fancy

    “He cried and hid dudgeon, just as if he owned the whole of the peppers, and could dispose of them all to suit his fancy.”· test-clean

    0:07
  2. 02

    She screamed and shouted too with a terrific volume of sound which doubtless caused the hearts of the fugitives to quake within them

    “She screamed and shouted, too, with a terrific volume of sound, which doubtless caused the hearts of the fugitives to quake within them.”· test-clean

    0:08
  3. 03

    If you are not then knocked on the head your being a non composser will protect you and you'll then have a good reason to expect to die in your bed

    “If you are not then knocked on the head, you're being a non-composer will protect you, and you'll then have a good reason to expect to die in your bed.”· test-clean

    0:08
  4. 04

    So far it has been used mostly for garnishment of other dishes and it is only recently that the hof brau has been making a specialty of them

    “So far it has been used mostly for garnishment of other dishes, and it is only recently that the Holfbrow has been making a specialty of them.”· test-other

    0:07
  5. 05

    By allah an thou fetch him not to me i will assuredly rouse the chamberlain on thee and he shall beat thee and cast thee out

    “By Allah, and thou fetch him not to me, I will assuredly rouse the chamberlain on thee, and he shall beat thee and cast thee out.”· test-other

    0:10
  6. 06

    I have missed four fits and had but five and have recovered so much strength as made me venture to meet your letter on wednesday a mile from home

    “I have missed four fits, and I've had but five, and have recovered so much strength, as made me venture to meet your letter on Wednesday, a mile from home.”· test-other

    0:11

How it ran

242Mparameters
0.1 sto load onto the GPU
Apache 2.0license
99 languageslanguages

Ran with whisper.cpp on Vulkan, q5_1 weights, greedy decoding on an AMD Radeon RX 9070 XT (16 GB, Vulkan through RADV (Mesa 26.2.2)).