video to text transcription

The workflow I use for these 2 steps is pretty easy:
1) extract the Audio using ffmeg (I am on Windows).
Powershell script contents:
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

$inputFolder = "D:\Subtitle"
$outputFolder = "D:\Subtitle"

# Create folder if it does not exist
if (!(Test-Path $outputFolder)) {
New-Item -ItemType Directory -Path $outputFolder | Out-Null
}
#Get-ChildItem $inputFolder -Include *.mp4, *.ts -File | ForEach-Object {
# $input = $_.FullName
# $output = Join-Path $outputFolder ($_.BaseName + ".wav")

# ffmpeg -i "$input" -vn -ac 1 -ar 16000 -acodec pcm_s16le "$output"
#}

Get-ChildItem $inputFolder -Filter *.mp4 | ForEach-Object {
$input = $_.FullName
$output = Join-Path $outputFolder ($_.BaseName + ".wav")

ffmpeg -i "$input" -vn -ac 1 -ar 16000 -acodec pcm_s16le "$output"

2) For WAV to text you can use the AI model Whisper
https://github.com/openai/whisper
Model medium/large goes in the same folder as the whisper-cli.exe
https://huggingface.co/ggerganov/whisper.cpp/tree/main
Powershell Script to run the audio to text job:

# Folder with extracted WAV files (from MP4 Video)
$inputFolder = "E:\Subtitle"

# Whisper-CLI Pfad
$whisper = "C:\whisper\whisper-cli.exe"

# Whisper Modell (Large v3)
#$model = "C:\whisper\ggml-large-v3.bin"
# Whisper Modell Medium - less loops
$model = "C:\whisper\ggml-medium.bin"

# Output folder for SRT subtitle files
$outputFolder = "D:\Subtitle"

# Force original language (e.g.: es = Spanish)
$origLang = "es"

# Create folder if it does not exist
if (!(Test-Path $outputFolder)) {
New-Item -ItemType Directory -Path $outputFolder | Out-Null
}

# Process all available WAV files
Get-ChildItem $inputFolder -Filter *.wav | ForEach-Object {
$input = $_.FullName
$basename = $_.BaseName

# -----------------------------
# 1) Original Language → SRT
# -----------------------------
Write-Host "original language: $basename"

& $whisper `
-m $model `
-f "$input" `
--output-srt `
--threads 24 `
--language $origLang `
--temperature 0 `
--beam-size 1 `
--best-of 1 `
--split-on-word `
--entropy-thold 2.4 `
--logprob-thold -1.0 `
--output-file "$outputFolder\$basename.original"

Write-Host "Completed: $basename"
}

Write-Host "All files processed."


Above I am using --output-srt to get also the timestamps.
Whisper also supports an nvida including rtx 50 models

 

Thank you, very fast!!!!

 

Did it work out for you?
The next level would be/ is text to voice incl sort of voice cloning which also is not too bad. Updated some videos so I can finally understand them and not having to read all the subtitles.

 

Yes worked fine, now I'm involved on text/to voice with translation, IA is very easy and nice to use.
I have just look some of voice available, you can also choose the tone, sex and a lot of other stuff....

 

Nice happy to hear that. Yes there are some ways to clone/mimic the voice of the original speaker.