OpenSource Clipping

Ultimate AI Auto-Clipper & Teaser Generator
An open-source content factory that transforms long-form videos into cinematic short-form highlights with hook teasers, karaoke subtitles, and auto-thumbnails.

🇮🇩 Baca dalam Bahasa Indonesia · Report Bug · Request Feature

🟢 Actively Maintained: contributions, bug reports, and feature PRs are welcome.

✨ Features

🎬 NEW: Story Clip Mode (--story-mode)
Need to assemble a narrative from multiple specific video sources (like a brand campaign)? We've just introduced the Multi-Source Story Clip Mode!
👉 Read the full Story Clip Documentation

📋 Prerequisites

• Python 3.10+
• FFmpeg installed and available in PATH
• CUDA GPU recommended (for Whisper; CPU fallback available)
• Google Gemini API Key (get one here)
• Pexels API Key (optional, for B-roll — get one here)
• HuggingFace Token (optional, for split-screen / camera-switch — get one here, requires accepting Pyannote model agreement)

☁️ Running on Google Colab (Recommended)

If you don't have a local GPU, the easiest way to run this pipeline is via Google Colab. Open a new Google Colab notebook, set the Runtime to T4 GPU, and create the following cells:

Cell 1: Setup & Clone

!rm -rf ./* ./.*
!git clone https://github.com/your-username/opensource-clipping.git .
!pip install -r requirements.txt

Cell 2: Setup API Keys

import os
from pathlib import Path
from google.colab import userdata

# Store your keys in Colab Secrets first!
GOOGLE_API_KEY = userdata.get("GOOGLE_API_KEY")

env_text = f"GOOGLE_API_KEY={GOOGLE_API_KEY}\n"
Path(".env").write_text(env_text, encoding="utf-8")

Cell 3: Execute (Example including Kaggle fallback for float32)

URL_YOUTUBE = "https://www.youtube.com/watch?v=Dc4_aBFAYWE&pp=0gcJCdkKAYcqIYzv"
JUMLAH_CLIP = 10
RASIO = "9:16"
FONT_STYLE = "DEFAULT"
GEMINI_MODEL = "gemini-3-flash-preview"
# Use 'float32' for Kaggle CPU/T4 limitations, or 'float16' for standard Colab T4 GPUs
WHISPER_COMPUTE_TYPE = "float32"

!python main.py \
  --url "{URL_YOUTUBE}" \
  --clips {JUMLAH_CLIP} \
  --ratio "{RASIO}" \
  --font-style "{FONT_STYLE}" \
  --hook-duration 3 \
  --words-per-sub 5 \
  --gemini-model "{GEMINI_MODEL}" \
  --whisper-compute-type "{WHISPER_COMPUTE_TYPE}" \
  --no-bgm

💡 New to this project? Use the Quick Start Notebook — a beginner-friendly notebook with minimal configuration, step-by-step English instructions, and only 3 settings to change (YouTube URL, total clips, and Whisper model).

We also include notebooks/Lib_OpenSource_Clipping.ipynb as a full-featured template.

🎬 Web Studio (GitHub Pages + Remote GPU)

The Clipping Studio is a browser-based dashboard hosted for free on GitHub Pages that connects to a Kaggle/Colab notebook as its backend — giving you a full GUI to control the AI clipping pipeline without any local setup.

🔗 Open Studio: naufalrizqullah.github.io/opensource-clipping/studio/

How It Works

┌─────────────────────┐     HTTPS (ngrok)     ┌──────────────────────────┐
│   GitHub Pages      │ ◄──────────────────►   │   Kaggle / Colab         │
│   (Static Frontend) │                        │   (FastAPI + GPU)        │
│                     │   POST /api/jobs       │                          │
│   studio/index.html │ ────────────────────►  │   web/api/app.py         │
│   studio/new-job    │   GET  /api/jobs/:id   │   clipping pipeline      │
│   studio/settings   │ ◄────────────────────  │   Whisper + Gemini       │
└─────────────────────┘                        └──────────────────────────┘
        FREE                                           FREE (GPU)

Quick Start

1. Start the backend — Open notebooks/kaggle-studio-server.ipynb in Kaggle (or Colab), enable Internet and a GPU, add your API keys to Secrets, and run all cells. Setup uses an isolated Python environment; the Public URL appears only after the backend passes its dependency and renderer readiness checks. See the setup and troubleshooting guide.

2. Open the Studio — Visit the Studio page in your browser.

3. Connect — Click the Connect button in the sidebar, paste the tunnel URL, and click Test & Connect.

4. Create a job — Go to New Job, enter a YouTube URL, configure your clip settings, and hit Start Clipping. Monitor progress in real-time from the Dashboard.

Note: The tunnel URL changes each time the notebook restarts. The Studio saves your last URL in localStorage for convenience, but you'll need to update it after each new session.

🚀 Local Quick Start

# 1. Clone the repo
git clone https://github.com/your-username/opensource-clipping.git
cd opensource-clipping

# 2. Install dependencies (pick one)
pip install -r requirements.txt          # pip / Colab
# uv sync                               # or use uv (reads pyproject.toml)

# 3. Set up API keys
cp .env.sample .env
# Edit .env and add your GOOGLE_API_KEY

# 4. Run (Must include --url)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID"
# 5. Examples of Execution

# Standard run (Default options with 5 clips)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" --clips 5 --ratio 16:9

# Using YouTube Cookies (To avoid 403 Forbidden / bot detection errors)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" --yt-cookies chrome
# or using an exported cookies.txt file
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" --yt-cookies youtube_cookies.txt

# Batch Processing (Multi-URL) & Cleanup Source
# Process multiple videos sequentially and auto-delete original videos after rendering
python main.py --url "https://youtube.com/watch?v=ID1" "https://youtube.com/watch?v=ID2" --clips 3 --ratio 9:16 --cleanup-source

# Prefer highest available source quality (default behavior)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" --source-height max

# Cap source download to 1440p (2K)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" --source-height 1440

# Sharper output tuning (works for normal and dynamic-split modes)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" \
  --source-height 2160 \
  --video-cq 19 \
  --video-crf 17 \
  --video-preset slow \
  --video-scale-algo lanczos

# Advanced run (Using YOLOv8 GPU Face Tracking & Custom Fonts)
python main.py --url "https://youtube.com/watch?v=VIDEO_ID" \
  --clips 7 \
  --face-detector yolo \
  --yolo-size 8m \
  --font-style STORYTELLER

# Podcast Split-Screen (2 speakers, 9:16)
python main.py --url "https://youtube.com/watch?v=PODCAST_ID" \
  --clips 3 \
  --ratio "9:16" \
  --split-screen

# Podcast Camera Switch (auto-switches to active speaker, blurred pillarbox on overlap)
python main.py --url "https://youtube.com/watch?v=PODCAST_ID" \
  --clips 3 \
  --ratio "9:16" \
  --camera-switch \
  --switch-hold-duration 2.0 \
  --switch-blend-duration 0.0

# Multi-Speaker Podcast (3 speakers across 2 scenes)
python main.py --url "https://youtube.com/watch?v=PODCAST_ID" \
  --clips 3 \
  --ratio "9:16" \
  --camera-switch \
  --diarization-speakers 3

# Manual Custom Hook (using external .mp4 clip)
python main.py --url "VIDEO_URL" --hook-source "DRIVE_URL_OR_PATH" --hook-source-start 5.0 --hook-duration 4

# Ultra-HD 2K Rendering (Fetch 1440p and render at native 1440p vertical resolution with sharpening)
python main.py --url "VIDEO_URL" --source-height 1440 --render-height source --video-sharpen

# Use NVIDIA NIM (DeepSeek-V3) instead of Gemini
python main.py --url "VIDEO_URL" --ai-provider nvidia --nvidia-model "deepseek-ai/deepseek-v4-pro"

# Square output for Instagram Feed (1:1)
python main.py --url "VIDEO_URL" --ratio "1:1" --clips 5

# Instagram/Facebook portrait (4:5)
python main.py --url "VIDEO_URL" --ratio "4:5" --clips 5

# Classic portrait (3:4)
python main.py --url "VIDEO_URL" --ratio "3:4" --clips 5

# TikTok source
python main.py --url "https://www.tiktok.com/@username/video/1234567890" --source tiktok --clips 3

# Instagram source
python main.py --url "https://www.instagram.com/reel/123456789/" --source instagram --clips 3

# Google Drive source
python main.py --url "https://drive.google.com/file/d/1234567890/view" --source gdrive --clips 3

⚙️ CLI Options

python main.py --help

📐 Aspect Ratios

OpenSource Clipping supports 5 output aspect ratios. All vertical/square ratios include face-tracking by default to keep the subject centered.

[!NOTE] When using 16:9 output with a non-16:9 source (e.g., vertical video), the system applies letterboxing (black bars) to preserve the original proportions instead of stretching.

🎙️ Podcast Modes

When processing podcast videos, you can choose between several intelligent rendering modes. These modes support 3+ speakers across multiple scenes.

1. --split-screen (Split Layout)

Divides the screen into panels to show multiple speakers simultaneously.

• Default: Permanent Top-Bottom layout (supports 3+ speakers via panel-swapping).
• --dynamic-split: Automatically switches between Full 9:16 (when 1 person is talking/visible) and Split (when 2+ are active).
• Trigger Modes (--split-trigger):
  • diarization (Default): Uses audio to know who is talking. Requires HF_TOKEN. Dimming effect on inactive speaker.
  • face: Uses visual face count. No token required. No dimming effect.
• Optimization Features:
  • Smart Separation Zoom (--split-auto-zoom): Dynamically adjusts the zoom level of each panel to keep the framing tight on the speaker while excluding other detected faces. Ensures no "overlap" even when subjects are sitting close together.
  • Vertical Tracking: Automatically follows face height, keeping the subject centered vertically (adjustable via --split-v-align).
• Best for: Educational podcasts or when reaction shots are important.

2. --camera-switch (Cinematic Switching)

Mimics professional editing by focusing only on the active speaker in full screen.

• View: Full 9:16 that cuts between speakers.
• Scene-Aware: Automatically uses Blurred Pillarbox if two people in the same wide-shot are talking; otherwise stays in clean full-crop.
• Best for: Storytelling, interviews, or high-energy clips.

Comparison Table

[!TIP] Use --split-screen --dynamic-split --split-trigger face for the fastest rendering without needing any special API tokens or Diarization models.

🚀 Quick Start Examples

# 1. Standard AI Clipping (7 clips, 9:16)
python main.py --url "VIDEO_URL"

# 2. Dynamic Split-Screen (Visual-based, NO TOKEN REQUIRED)
python main.py --url "VIDEO_URL" --split-screen --dynamic-split --split-trigger face

# 3. Dynamic Split-Screen (Audio-based, Highlight active speaker, needs HF_TOKEN)
python main.py --url "VIDEO_URL" --split-screen --dynamic-split --split-trigger diarization

# 4. Cinematic Camera Switch (Needs HF_TOKEN)
python main.py --url "VIDEO_URL" --camera-switch

# 5. Smart Separation Split-Screen (Auto-Zoom & Vertical Track)
python main.py --url "VIDEO_URL" --split-screen --dynamic-split --split-trigger face --split-auto-zoom --split-v-align 0.4

# 6. Square output (1:1) with Split-Screen
python main.py --url "VIDEO_URL" --ratio "1:1" --split-screen --dynamic-split --split-trigger face

# 7. Hook V2 + Segment Trimming (default)
python main.py --url "VIDEO_URL" --hook-v2

# 8. Hook V2 + Aggressive Silence Trimming
python main.py --url "VIDEO_URL" --hook-v2 --silence-trim

# 9. Hook V2 without Segment Trimming (full render)
python main.py --url "VIDEO_URL" --hook-v2 --no-segment-trim

# 10. Hook V2 Custom: 4 micro-hooks with glitch style
python main.py --url "VIDEO_URL" --hook-v2 --hook-v2-items 4 --hook-v2-style "glitch_fast"

[!IMPORTANT] Audio-based features (diarization) require a HuggingFace Token (HF_TOKEN) in your .env file and acceptance of the Pyannote model agreement on HuggingFace.

🗣️ AI Voice-Over Commentary

Prevent YouTube copyright/reused content strikes by turning raw clips into original commentary/reaction videos automatically.

When you pass the --voiceover flag, the pipeline will:

1. Generate a sharp, opinionated analysis script using Gemini AI based on the clip's specific transcript.
2. Synthesize the script into natural-sounding speech using edge-tts (free, no GPU required).
3. Duck the original video's audio down to 15% volume and overlay the AI voice-over at 100% volume.
4. Override the burned-in karaoke subtitles so they display the AI narrator's words instead of the original video transcript.
5. Visual Enhancements: Displays a relaxed audio spectrum visualizer and a premium, slow-moving ambient edge glow around the freeze-frame intro.

Example Usage:

# Basic voice-over (Uses default en-US-AvaNeural and English language)
python main.py --url "VIDEO_URL" --voiceover

# Voice-over in Indonesian with reaction style
python main.py --url "VIDEO_URL" --voiceover --voiceover-lang id --voiceover-voice id-ID-ArdiNeural --voiceover-style reaction

Config Options:

• --voiceover-voice: Choose the TTS voice (e.g. en-US-AvaNeural, id-ID-GadisNeural).
• --voiceover-lang: Script language (en or id).
• --voiceover-style: analysis (default), reaction, lesson, or summary.
• --voiceover-length: Length of the script (short [default: 5-15s], normal [20-40s], long [40-60s]).
• --voiceover-volume: Volume of narrator (default 1.0).
• --original-volume: Volume of ducked video audio (default 0.15).
• --edge-glow: Apply the ambient edge glow effect to the entire video, instead of just the voice-over intro.
• --edge-glow-mode: Edge glow strategy (default, smooth [default], or full). smooth mathematically adjusts speed to remove stuttering in loops.

🎵 BGM (Background Music) Settings

• --no-bgm : Disable the BGM feature entirely.
• --bgm-mode ducking (default) : Sidechain Ducking mode — BGM volume automatically lowers when the speaker is talking, then rises during pauses. Provides a professional effect like premium podcasts/YouTube videos.
• --bgm-mode background : Constant Background mode — BGM plays at a stable, low volume throughout the video without dynamic adjustments. Best for content with few pauses.

📁 BGM Setup: Place your royalty-free .mp3 files into the assets/bgm/<mood>/ folder (e.g., assets/bgm/chill/, assets/bgm/epic/, assets/bgm/sad/, assets/bgm/upbeat/, assets/bgm/suspense/). The script will randomly select a song from the mood folder requested by the AI. If the folder is empty, BGM will be skipped automatically.

BGM files shorter than the video will be auto-looped (-stream_loop -1).

See assets/bgm/README.md for recommended sources to download free BGM.

🎬 Understanding Hook V2 & Segment Trimming

Final Video Structure

[Hook V2 Intro] → [MAIN CLIP] → done
   ↑                    ↑
   Rapid micro-hooks    This part is affected by Segment Trimming
   (0.5-2s × 3-4)

Hook V2 and Segment Trimming are two independent features that operate on different parts of the video.

Hook V2 (Multi-Hook Intro)

Hook V2 creates a rapid-fire intro at the beginning of the video — 3-4 short clips (0.5-2 seconds) taken from the most punchy/controversial moments within the clip. Each piece is separated by a white flash or glitch transition. The goal: stop the viewer from scrolling within the first 3-5 seconds.

Example Hook V2:
  [Clip 1: "NOBODY DARES" (1s)] → ⚡flash → [Clip 2: "THEY'RE ALL WRONG" (0.8s)] → ⚡flash → [Clip 3: "HERE'S THE TRUTH" (1.2s)] → [MAIN CLIP]

Segment Trimming

Segment Trimming only applies to the main clip (after the hook). AI analyzes the main clip and removes boring sections — they're not sped up, they're cut out entirely, and the good parts are stitched together seamlessly.

Example:
  Main clip: second 30 - 90 (60 seconds total)
  
  AI finds:
    ✅ Second 30-55 : strong content, engaging
    ❌ Second 55-58 : speaker pauses/filler (removed)
    ✅ Second 58-90 : strong punchline

  Result: segment 1 + segment 2 joined directly
  Final duration: 57 seconds (3 seconds of filler removed)

Flag Comparison

[!NOTE]

• Hook V2 is not affected by any of the above flags. Hook V2 always picks its rapid-fire clips as chosen by AI.
• --no-hook only disables Hook V1 (the 3-second glitch teaser). Hook V2 (--hook-v2) works independently even when --no-hook is active.
• Segment Trimming and Silence Trimming work without Hook V2 — just omit the --hook-v2 flag.
• If AI determines the entire clip is already tight and engaging, keep_segments will contain a single segment spanning the full duration (same effect as --no-segment-trim).

🐍 Recommended Configurations (Notebook/Colab)

These are verified configurations for optimal results in different scenarios.

1. Standard Mode (Standard Clipping)

Best for general videos where you want the best accuracy and focus.

# Constants for Standard Clipping
URL_YOUTUBE = "https://www.youtube.com/watch?v=UXhdIF8kvCI"
JUMLAH_CLIP = 7
RASIO = "9:16"
FONT_STYLE = "DEFAULT"
GEMINI_MODEL = "gemini-2.0-flash"

!python main.py \
  --url "{URL_YOUTUBE}" \
  --clips {JUMLAH_CLIP} \
  --ratio "{RASIO}" \
  --font-style "{FONT_STYLE}" \
  --hook-duration 3 \
  --words-per-sub 5 \
  --face-detector yolo \
  --gemini-model "{GEMINI_MODEL}" \
  --no-bgm \
  --no-subs \
  --no-broll \
  --use-dlp-subs

2. Split-Screen Mode (Podcasts)

Optimized for podcasts with 2+ speakers using stable YOLO detection.

# Constants for Split-Screen
URL_YOUTUBE = "https://www.youtube.com/watch?v=UXhdIF8kvCI"
JUMLAH_CLIP = 3
RASIO = "9:16"
FONT_STYLE = "DEFAULT"
GEMINI_MODEL = "gemini-2.0-flash"

!python main.py \
  --url "{URL_YOUTUBE}" \
  --clips {JUMLAH_CLIP} \
  --ratio "{RASIO}" \
  --font-style "{FONT_STYLE}" \
  --hook-duration 3 \
  --words-per-sub 5 \
  --gemini-model "{GEMINI_MODEL}" \
  --no-bgm \
  --no-subs \
  --no-broll \
  --split-screen \
  --dynamic-split \
  --split-trigger face \
  --face-detector yolo \
  --use-dlp-subs

📂 Project Structure

opensource-clipping/
├── main.py                  # CLI entry point
├── run_upload.py            # YouTube auto-uploader CLI
├── run_fb_upload.py         # Facebook Pages Reels uploader CLI
├── pyproject.toml           # Dependencies & metadata
├── .env.sample              # API key template
├── .gitignore
├── README.md                # English docs
├── README_ID.md             # Indonesian docs
├── clipping/
│   ├── config.py            # Master configuration & argparse
│   ├── engine.py            # Download → Transcribe → Gemini AI
│   ├── diarization.py       # Pyannote speaker diarization
│   ├── metadata.py          # QA metadata normalization
│   ├── runner.py            # Pipeline orchestrator
│   ├── story/               # Story mode modules
│   └── studio/              # Video render engine modules
├── facebook_uploader/       # Facebook Pages Reels upload & scheduling
├── web/                     # Web API and React Dashboard
├── youtube_tracker/         # YouTube Tracker Web App
└── youtube_uploader/        # YouTube upload & scheduling logic

📊 Results

1 video 9:16	1 video split from youtube shorts link
Watch Example (Standard 9:16 Auto-Framing)	Watch Example (Split-Screen Podcast Mode)

🔄 Pipeline Flow

graph LR
    A[Video URL] --> B[Download Video]
    B --> C[Whisper Transcription]
    C --> D[Gemini AI Analysis]
    D --> E[Metadata QA]
    E --> F[Render Loop]
    F --> G[Face-Track Crop]
    F --> H[B-Roll + BGM]
    F --> I[ASS Subtitles]
    F --> J[Hook + Glitch]
    G & H & I & J --> K[Final MP4 + Thumbnail]

📤 Output

For each clip, the pipeline creates an outputs/ directory and generates:

🎵 Font Styles

📺 Auto-Upload to YouTube

The project now includes a standalone YouTube auto-uploader with scheduling support!

1. Place your configured youtube_token.json file inside the .credentials/ directory.
2. After the rendering process finishes, the script will automatically read from the generated outputs/ directory (e.g., outputs/render_manifest.json and the final videos). Simply run the uploader:

   # Basic run (uses default 8-hour interval and auto timezone)
   python run_upload.py
   
   # Or run with custom arguments (example):
   python run_upload.py --interval-hours 12 --tz-name "Asia/Jakarta"

3. To run a test with only the first video, use python run_upload.py --test-mode. Run python run_upload.py --help to see all scheduling and timezone options.

📘 Auto-Upload to Facebook Pages (Reels)

The project also includes a standalone Facebook Pages Reels uploader with native scheduling support via the Meta Graph API!

Prerequisites:

• A Facebook Page Access Token (long-lived) with pages_manage_posts and pages_read_engagement permissions.
• Set META_PAGE_ID, META_PAGE_ACCESS_TOKEN, and META_GRAPH_VERSION in your .env file.

How it works:

1. Validates your Page Access Token against the Graph API.
2. Reads existing scheduled posts to determine the next available time slot.
3. Uploads each clip as a Reel: create session → upload binary → poll processing → publish/schedule.
4. First clip publishes immediately if no queue exists; subsequent clips are scheduled at 5-hour intervals (configurable).
5. If any step fails, the batch stops immediately (no fallback to instant publish).

# Basic run (reads .env for META_PAGE_ID & META_PAGE_ACCESS_TOKEN)
python run_fb_upload.py

# Test mode — only upload the first clip
python run_fb_upload.py --test-mode

# Custom interval (3 hours between videos)
python run_fb_upload.py --interval-hours 3

# See all options
python run_fb_upload.py --help

🧹 Disk Cleanup

Since the pipeline downloads full source videos and creates intermediate files (wav, chunks, transcripts), the outputs/ and uploads/ directories can grow very large over time.

We provide a simple bash script to safely clean up all temporary files while preserving your final generated clips and job history (jobs.json):

bash cleanup.sh

❤️ Support & Contributing

Feel free for contributing, support, fork, likes, etc. Any feedback is greatly appreciated to keep this open-source project growing!

• Saweria: https://saweria.co/NaufalRizqullah17
• Ko-fi: https://ko-fi.com/naufalrizqullah

📄 License

Open source. Feel free to use, modify, and distribute.