{
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3",
      "language": "python"
    },
    "language_info": {
      "name": "python",
      "version": "3.12.13",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "accelerator": "GPU"
  },
  "nbformat_minor": 4,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": "# \ud83c\udfac OpenSource Clipping Studio \u2014 Backend Server\n\nRun the Web Studio backend on a Kaggle or Colab GPU and connect from the browser dashboard.\n\n**Runtime:** Linux x86_64, Python 3.10\u20133.12 (prefer 3.12), Internet enabled.\n\nSetup uses an isolated virtual environment and starts FastAPI in a fresh process. The public URL appears only after actual renderer imports and FFmpeg pass readiness checks. Keep the final cell running while using Studio.\n\nSee `docs/KAGGLE_STUDIO.md` in the repository for troubleshooting and dependency details.",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## 1. Setup Project\n\nClone into a named directory without deleting existing files. Existing checkouts are preserved, not automatically updated. Ensure your checkout contains this notebook's matching `scripts/kaggle_studio.py` and requirements files before running setup.",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "# Clone repository without deleting the notebook working directory.\nimport os\nfrom pathlib import Path\nimport subprocess\nimport sys\n\nREPO_URL = \"https://github.com/NaufalRizqullah/opensource-clipping.git\"\nworkspace = Path(\"/kaggle/working\") if Path(\"/kaggle/working\").is_dir() else Path(\"/content\") if Path(\"/content\").is_dir() else Path.cwd()\nPROJECT_DIR = Path.cwd() if (Path.cwd() / \"scripts/kaggle_studio.py\").is_file() else workspace / \"opensource-clipping\"\nif not PROJECT_DIR.exists():\n    subprocess.run([\"git\", \"clone\", REPO_URL, str(PROJECT_DIR)], check=True)\nelif not (PROJECT_DIR / \".git\").is_dir():\n    raise RuntimeError(\"Project directory already exists and is not a checkout. Select a different PROJECT_DIR.\")\nif not (PROJECT_DIR / \"scripts/kaggle_studio.py\").is_file():\n    raise RuntimeError(\"Update this checkout to the revision containing the Kaggle Studio fixes, then rerun setup.\")\nos.chdir(PROJECT_DIR)\nprint(\"Project checkout ready. Existing outputs are retained.\")",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "source": "# Fail fast and keep installer output visible. Do not pip-install into this kernel.\nSTUDIO_ENV = PROJECT_DIR / \".cache\" / \"kaggle-studio\" / \"venv\"\nSTUDIO_PYTHON = STUDIO_ENV / \"bin\" / \"python\"\nsubprocess.run(\n    [sys.executable, str(PROJECT_DIR / \"scripts/kaggle_studio.py\"), \"setup\", \"--venv\", str(STUDIO_ENV)],\n    cwd=PROJECT_DIR, check=True,\n)",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "source": "# Install FFmpeg only if missing; failed system commands stop this cell.\nimport shutil\nif shutil.which(\"ffmpeg\") is None:\n    subprocess.run([\"apt-get\", \"update\", \"-qq\"], check=True)\n    subprocess.run([\"apt-get\", \"install\", \"-y\", \"ffmpeg\"], check=True)\nsubprocess.run([\"ffmpeg\", \"-version\"], check=True, stdout=subprocess.DEVNULL)\nprint(\"FFmpeg ready.\")",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "source": "## 2. Configure API Keys\n\nIn Kaggle **Add-ons \u2192 Secrets**, create the exact label `NGROK_AUTHTOKEN` and enable/attach it to this notebook. A secret saved in your account but not attached to the running notebook cannot be read. In Colab, grant notebook access to the matching userdata secret.\n\nConfigure `GOOGLE_API_KEY` for Gemini or `NVIDIA_API_KEY` for NVIDIA; `PEXELS_API_KEY` and `HF_TOKEN` support optional features. After adding or changing a secret, rerun this cell before starting the server. Setting a Python variable alone does not export it to subprocesses.\n\nKeys are passed through the environment only. Their values are never printed or written to `.env`. The managed server ignores existing `.env` files.",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "import os\n\nsecret_readers = []\ntry:\n    from kaggle_secrets import UserSecretsClient\n    secret_readers.append(UserSecretsClient().get_secret)\nexcept ImportError:\n    pass\nexcept Exception as exc:\n    print(f\"Kaggle Secrets unavailable ({type(exc).__name__}); check notebook access.\")\ntry:\n    from google.colab import userdata\n    secret_readers.append(userdata.get)\nexcept ImportError:\n    pass\n\nfor name in (\"GOOGLE_API_KEY\", \"NVIDIA_API_KEY\", \"PEXELS_API_KEY\", \"HF_TOKEN\", \"NGROK_AUTHTOKEN\"):\n    value = os.environ.get(name, \"\").strip()\n    for read_secret in secret_readers:\n        if value:\n            break\n        try:\n            value = (read_secret(name) or \"\").strip()\n        except Exception as exc:\n            if name == \"NGROK_AUTHTOKEN\":\n                print(f\"NGROK_AUTHTOKEN could not be read ({type(exc).__name__}); check its exact label and notebook attachment.\")\n    if value:\n        os.environ[name] = value\n    print(f\"{name}: {'configured' if value else 'not configured'}\")\ndel value\n\nif not os.environ.get(\"NGROK_AUTHTOKEN\", \"\").strip():\n    raise RuntimeError(\"NGROK_AUTHTOKEN is not available. Attach this exact secret label to the notebook, then rerun this cell before starting the server.\")\n",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n",
        "# YouTube Cookies Setup (Optional) - Solves yt-dlp 403 Forbidden errors\n",
        "# \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n",
        "import os\n",
        "import shutil\n",
        "\n",
        "COOKIES_PATH = os.path.abspath(\"youtube_cookies.txt\")\n",
        "cookie_candidates = [\n",
        "    \"/kaggle/input/datasets/muhammadnaufal/tiktok-secret-cookies/youtube_cookies.txt\",\n",
        "    \"/content/youtube_cookies.txt\",\n",
        "    COOKIES_PATH,\n",
        "]\n",
        "\n",
        "source_cookie = next((p for p in cookie_candidates if os.path.isfile(p)), None)\n",
        "\n",
        "if source_cookie:\n",
        "    if os.path.realpath(source_cookie) != os.path.realpath(COOKIES_PATH):\n",
        "        shutil.copy(source_cookie, COOKIES_PATH)\n",
        "    print(f\"\u2705 YouTube cookies aktif: {COOKIES_PATH}\")\n",
        "else:\n",
        "    print(\"\u2139\ufe0f youtube_cookies.txt tidak ditemukan. Download akan dicoba tanpa cookies.\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": "## 3. Start Backend Server + Tunnel\n\nThis cell launches a fresh Python process from the isolated environment. Startup verifies actual renderer imports before opening ngrok; allow several minutes for the first check.\n\nCopy the **Public URL** only after **BACKEND SERVER READY** appears, then paste it into Studio \u2192 Connect. Keep this cell running. Stop the cell or use Studio's Stop Server button to close the backend and tunnel.\n\nIf setup or readiness fails, fix that error and rerun; a healthy HTTP server alone is not sufficient.\n\nFlow diagnostics are saved automatically to `logs/kaggle-studio.jsonl` (setup/supervisor) and `logs/studio-api.jsonl` (API/worker). If startup or a job fails, download these files and any `.1`/`.2`/`.3` backups from Kaggle Files/Output. They contain stage names, correlation IDs and safe traceback locations, without credential values or raw provider responses.",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "from scripts.kaggle_studio import runtime_environment, stop_process\n\n# The supervisor and Uvicorn both use the isolated interpreter, never this kernel.\nsupervisor = subprocess.Popen(\n    [str(STUDIO_PYTHON), str(PROJECT_DIR / \"scripts/kaggle_studio.py\"), \"start\", \"--venv\", str(STUDIO_ENV)],\n    cwd=PROJECT_DIR, env=runtime_environment(STUDIO_ENV), start_new_session=True,\n)\ntry:\n    return_code = supervisor.wait()\n    if return_code not in (0, 130):\n        raise RuntimeError(f\"Studio server exited with code {return_code}. Check the startup diagnostics above.\")\nexcept KeyboardInterrupt:\n    print(\"Stopping Studio backend and tunnel...\")\nfinally:\n    stop_process(supervisor)\n",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "source": "# Run after stopping the server to locate saved diagnostic files.\nlog_dir = Path(os.environ.get(\"OSC_LOG_DIR\", str(PROJECT_DIR / \"logs\")))\nlog_files = sorted(log_dir.glob(\"*.jsonl*\")) if log_dir.is_dir() else []\nif not log_files:\n    print(\"No diagnostic files yet. Run setup/start with the updated checkout first.\")\nfor log_file in log_files:\n    print(f\"{log_file} ({log_file.stat().st_size:,} bytes)\")\nprint(\"Download these files from Kaggle Files/Output; include rotated backups if present.\")\n",
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "execution_count": null
    }
  ]
}