R
RunPod2mo ago
svennova

Faster-Whisper worker template is not fully up-to-date

Hi, We're using the Faster-Whisper worker (https://github.com/runpod-workers/worker-faster_whisper) on Serverless. I saw that Faster-Whisper itself is currently on version 1.0.2, whereas the Runpod template is still on 0.10.0. There are a few changes that have been introduced in Faster-Whisper (now using CUDA 12) since, that we would like to benefit from, especially the language_detection_threshold setting, since it seems like most of our transcriptions done by people with British accent are being transcribed into Welsh (with a language detection confidence of around 0.51 to 0.55) - which could be circumvented by increasing the threshold. Are there any plans for this? It's kind of a blocker for us at the moment.
8 Replies
digigoblin
digigoblin2mo ago
I suggest forking the repo and changing it yourself.
svennova
svennova2mo ago
Easier said than done 😉 We did actually make some attempts, but are not able to get it working. Seeing that this is a template provided by a paid service I thought it wouldn't hurt to ask.
digigoblin
digigoblin2mo ago
RunPod sunset the managed endpoints, so they probably aren't maintaining the repos anymore. https://blog.runpod.io/refocusing-core-strengths-shift-managed-ai-apis-serverless-flexibility/
RunPod Blog
Refocusing on Core Strengths: The Shift from Managed AI APIs to Ser...
RunPod is transitioning from Managed AI APIs to focusing on Serverless solutions, offering users more control and customization with comprehensive guidance.
svennova
svennova2mo ago
Maybe, still it's a Quick Deploy template on Serverless. As mentioned in the article you've shared:
Simplified Guide to Leveraging Serverless AI Functionalities [...] 2. Deploy with Ease: Use our Serverless platform to deploy your AI models without the complexity of managing infrastructure. Explore our easy-to-use quick deploy option.
I would like to "deploy with ease" and said template is part of the section called "Quick Deploy" in the console 😅
digigoblin
digigoblin2mo ago
Yeah you can quick deploy it, but to customise it thats your job.
svennova
svennova2mo ago
Perhaps I'll be better of with Customer Support, thanks anyways.
digigoblin
digigoblin2mo ago
Yeah log a ticket on website.
svennova
svennova2mo ago
We did another attempt and it actually turned out to be a non-issue. We simply didn't make enough memory available to Docker in order to build the image.