I save a lot of podcasts, conference talks from YT. I want to transcribe them to grep some keywords. My assumption has been that the only realistic options are to a cloud service or running Whisper on a decent GPU, which I don't have. What's your setup? Can I get away with CPU?
I have some Google Collab notebooks where I download/upload the videos, load whisper there and use the cloud GPUs to process them!
Here is an example: https://github.com/apresmoi/jianglens/blob/main/ops/notebook...
The repo could also be helpful.. it was a research project I had on how to convert a Youtube corpus into a static browsable docs
In order to do something like this, you'd have to run ASR (Automatic Speech Recognition) on the audio you're wanting to transcribe. This can get difficult, to do quickly on CPU-only devices. I've been working on a product that uses a feature like this for some of it's functionality, and have found pretty good results with some of the smaller models like Parakeet-TDT 0.6B. This model can run faster than real time on a Samsung S20 FE.
You can get pretty far even using the CPU. Just did a bit of testing with a 20-minute Family Guy episode. Took 14 minutes using Whisper Small (500 MB model) on an older Intel i5-9600T CPU. Obviously, that might be fine for the occasional transcription and a total show-stopper for high volume transcription. For comparison the same video took just two minutes using Whisper Large v3 Turbo on my 4080 (tested in TrueScribe on Linux).