Overview
The AI Video Dubbing Pipeline turns any English video, from a YouTube link or an MP4 file of any length, into a dubbed video in another language (Spanish by default), plus subtitles in the target language. It runs entirely on free, open-source tools on a local NVIDIA GPU, orchestrated by a self-hosted n8n instance.
Challenge
English talks, lessons, tutorials and hour-long recordings reach far more people in other languages, but manual dubbing is slow and paid dubbing services charge per minute, which gets expensive for long videos. The pipeline had to use only free, open-source tools with no paid recurring APIs, run the heavy processing on a local Windows workstation with an NVIDIA GPU, be orchestrated by an existing n8n instance on a separate machine, and extend to any target language. Lip-sync was out of scope.
Solution
n8n only coordinates: it validates requests from a web form, webhook or Telegram bot, submits each job over the local network, checks its status every two minutes and sends updates. All heavy work runs on the GPU workstation, where a FastAPI worker with an SQLite job queue processes one job at a time in five resumable stages, each saving a checkpoint so a failed job resumes where it stopped. Making each stage reliable on a real Windows machine was most of the work: Whisper now runs in its own process so its GPU memory is freed before translation, repeated phrases are collapsed and the LLM replies in numbered lines to stop it looping, long translations are shortened first and only then sped up by at most 1.25×, and the output carries only the dubbed audio track so players don’t fall back to English.
Results
The worker self-test passed on the GPU workstation, covering the GPU, Whisper large-v3 on CUDA, Qwen2.5 translation, Edge-TTS voice and FFmpeg. The first end-to-end runs produced a dubbed MP4 with correct Spanish subtitles, and the whole pipeline runs on free software with no recurring API cost.
- LUFS measured output loudness
- -16.1
- Resumable worker stages
- 5
- Recurring API cost
- $0
Key features
- Three ways to submit: n8n web form, HTTP webhook, or a Telegram message with a YouTube link
- Any target language — Spanish by default, others by adding a word to the message
- Natural spoken translation, plus a custom glossary for names and key terms
- Timing fit without lip-sync: lines placed at their original timestamps
- Original background audio mixed quietly under the dub
- A translated .srt subtitle file with every video
- Job queue that survives reboots and can redo a single stage
- Per-job logs, with failures reported to n8n and Telegram
- One-command PowerShell install with a GPU, Whisper, LLM and voice self-test
Technologies
- n8n
- Python
- FastAPI
- SQLite
- faster-whisper
- Ollama
- Qwen2.5
- Edge-TTS
- FFmpeg
- yt-dlp
- Telegram Bot
- PowerShell
Development process
-
Step 01
Download
yt-dlp fetches the YouTube video, with Deno as the JavaScript runtime YouTube now requires; MP4 uploads skip this step.
-
Step 02
Transcribe
faster-whisper (Whisper large-v3, CUDA float16) produces timestamped English text on the local GPU.
-
Step 03
Translate
Ollama with Qwen2.5 7B translates each line for natural spoken voice-over, using an optional glossary.
-
Step 04
Voice
Edge-TTS generates neural voices in the target language, with Piper as an offline fallback.
-
Step 05
Mix
FFmpeg fits timing, mixes in the background audio, normalizes to -16 LUFS (EBU R128) and writes the MP4 without re-encoding the video.
