{ "cells": [ { "cell_type": "markdown", "id": "733fdf58", "metadata": {}, "source": [ "# Deploy VSS 3.3.0 (Video Search & Summarization)\n", "\n", "This notebook deploys the NVIDIA VSS 3.3.0 Blueprint on GPU-equipped cloud instances.\n", "\n", "**What it does:**\n", "1. Validates GPU hardware and Docker prerequisites\n", "2. Installs and configures the NGC CLI\n", "3. Configures Docker storage for large image pulls\n", "4. Gets the deployment code (from a local path or GitHub)\n", "5. Detects network configuration (internal + external IPs)\n", "6. Generates the profile's environment file and deploys with `docker compose`\n", "7. Verifies all services are healthy\n", "\n", "**Supported profiles:** `base`, `search`, `alerts`, `lvs` \n", "**Supported hardware profiles:** `H100`, `L40S`, `RTXPRO4500BW`, `RTXPRO6000BW`, `DGX-SPARK`\n", "\n", "On `DGX-SPARK` the profiles are `base`, `alerts` and `search`, and `search` there requires a remote LLM **and** a remote VLM — the board's single GPU is fully taken by the perception pipeline.\n", "\n", "---\n", "\n", "## Prerequisites\n", "\n", "- Linux instance with a supported hardware profile (`H100`, `L40S`, `RTXPRO4500BW`, `RTXPRO6000BW`, or `DGX-SPARK`)\n", "- Ubuntu on x86 hosts. 24.04 is the validated release; other Ubuntu releases run with a warning from Section 2 onward but are untested. Non-Ubuntu distributions are not supported — Sections 4 and 4.1 configure Docker through Ubuntu apt packages.\n", "- NVIDIA driver with CUDA 12.0+ runtime support. Minimum: Brev x86 → `595.91.07`, DGX Spark → `580.173.02`.\n", "- On `DGX-SPARK`, the `sys-cache-cleaner.sh` prerequisite. Section 8 installs and starts it, and re-checks on every deploy because a reboot stops it.\n", "- Docker Engine 28.3.3+ and <29.5.0 with Docker Compose v2.39.1+. If your launchable image starts with a newer Docker Engine, Section 4.1 pins it before deployment.\n", "- NGC API key from [ngc.nvidia.com](https://ngc.nvidia.com)\n", "- **500GB+ disk space** for Docker images and models. Most GPU cloud instances have a small root disk (~200-250GB) plus a large ephemeral NVMe. Section 4 will auto-detect this and move Docker/containerd storage to the NVMe." ] }, { "cell_type": "markdown", "id": "65ad0a67", "metadata": {}, "source": [ "## 1. Configuration\n", "\n", "Set your NGC API key, deployment profile, and hardware below. These variables are used by all subsequent cells." ] }, { "cell_type": "code", "execution_count": null, "id": "24d4a05e", "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "# ============================================================\n", "# REQUIRED: Set these before running anything else\n", "# ============================================================\n", "\n", "NGC_CLI_API_KEY = \"\" # Your NGC API key — get one at https://ngc.nvidia.com\n", "NGC_CLI_ORG = \"nvidia\" # change NGC org as needed based on your model source\n", "\n", "PROFILE = \"base\" # Deployment profile: base, search, alerts, lvs\n", "\n", "HARDWARE_PROFILE = \"RTXPRO6000BW\" # Hardware: RTXPRO6000BW, RTXPRO4500BW, H100, L40S, DGX-SPARK\n", "\n", "# ============================================================\n", "# OPTIONAL: Override defaults if needed\n", "# ============================================================\n", "\n", "# Deployment source — set ONE of these:\n", "# DEPLOY_SOURCE_PATH: Path to the repo cloned during launchable bringup.\n", "# Must contain deploy/docker/ with compose.yml and containers.env.\n", "# If the default launchable path is missing, falls back to GitHub clone.\n", "_DEFAULT_DEPLOY_SOURCE_PATH = os.path.expanduser(\"~/video-search-and-summarization\")\n", "DEPLOY_SOURCE_PATH = _DEFAULT_DEPLOY_SOURCE_PATH if os.path.isdir(_DEFAULT_DEPLOY_SOURCE_PATH) else \"\"\n", "\n", "GIT_BRANCH = \"release/3.3.0\" # Git branch or tag (only used when cloning from GitHub)\n", "\n", "ALERTS_MODE = \"verification\" # Only used when PROFILE=alerts: verification or real-time\n", "\n", "USE_REMOTE_LLM = False # Set True to use a remote LLM endpoint instead of local\n", "REMOTE_LLM_ENDPOINT_URL = \"\" # Required when USE_REMOTE_LLM=True. Base URL without /v1, e.g. http://global.stg.ga.launchpad.nvidia.com:11571\n", "\n", "USE_REMOTE_VLM = False # Set True to use a remote VLM endpoint instead of local\n", "REMOTE_VLM_ENDPOINT_URL = \"\" # Required when USE_REMOTE_VLM=True. Base URL without /v1, e.g. http://global.stg.ga.launchpad.nvidia.com:11432\n", "\n", "# Network overrides (auto-detected in Section 7 if left empty)\n", "HOST_IP_OVERRIDE = \"\" # Internal IP — leave empty for auto-detect\n", "EXTERNAL_IP_OVERRIDE = \"\" # External IP — leave empty for auto-detect" ] }, { "cell_type": "code", "execution_count": null, "id": "758413a8", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "# ---- Validate configuration ----\n", "import os, sys\n", "\n", "assert NGC_CLI_API_KEY, \"NGC_CLI_API_KEY is required. Get one at https://ngc.nvidia.com\"\n", "assert PROFILE in (\"base\", \"search\", \"alerts\", \"lvs\"), f\"Invalid PROFILE: {PROFILE}\"\n", "assert HARDWARE_PROFILE in (\"H100\", \"L40S\", \"RTXPRO4500BW\", \"RTXPRO6000BW\", \"DGX-SPARK\"), \\\n", " f\"Invalid HARDWARE_PROFILE: {HARDWARE_PROFILE}\"\n", "\n", "if PROFILE == \"alerts\":\n", " assert ALERTS_MODE in (\"verification\", \"real-time\"), f\"Invalid ALERTS_MODE: {ALERTS_MODE}\"\n", "\n", "# DGX-SPARK has a single GPU. base and alerts run locally on it; search's\n", "# perception pipeline owns the GPU outright, so both models must be remote.\n", "if HARDWARE_PROFILE == \"DGX-SPARK\":\n", " assert PROFILE in (\"base\", \"alerts\", \"search\"), \\\n", " f\"{HARDWARE_PROFILE} only supports base, alerts and search profiles, not {PROFILE}\"\n", " if PROFILE == \"search\":\n", " assert USE_REMOTE_LLM and REMOTE_LLM_ENDPOINT_URL, \\\n", " \"Search on DGX-SPARK cannot host a local LLM: set USE_REMOTE_LLM=True with REMOTE_LLM_ENDPOINT_URL\"\n", " assert USE_REMOTE_VLM and REMOTE_VLM_ENDPOINT_URL, \\\n", " \"Search on DGX-SPARK cannot host a local VLM: set USE_REMOTE_VLM=True with REMOTE_VLM_ENDPOINT_URL\"\n", "\n", "if DEPLOY_SOURCE_PATH:\n", " assert os.path.isdir(DEPLOY_SOURCE_PATH), f\"DEPLOY_SOURCE_PATH does not exist: {DEPLOY_SOURCE_PATH}\"\n", "\n", "if USE_REMOTE_LLM:\n", " assert REMOTE_LLM_ENDPOINT_URL, \"REMOTE_LLM_ENDPOINT_URL is required when USE_REMOTE_LLM=True\"\n", "if USE_REMOTE_VLM:\n", " assert REMOTE_VLM_ENDPOINT_URL, \"REMOTE_VLM_ENDPOINT_URL is required when USE_REMOTE_VLM=True\"\n", "\n", "# Export NGC key to environment for shell cells and the deployment helpers\n", "os.environ[\"NGC_CLI_API_KEY\"] = NGC_CLI_API_KEY\n", "if USE_REMOTE_LLM:\n", " os.environ[\"LLM_ENDPOINT_URL\"] = REMOTE_LLM_ENDPOINT_URL\n", "if USE_REMOTE_VLM:\n", " os.environ[\"VLM_ENDPOINT_URL\"] = REMOTE_VLM_ENDPOINT_URL\n", "\n", "print(\"Configuration valid.\")\n", "print(f\" Profile: {PROFILE}\")\n", "print(f\" Hardware: {HARDWARE_PROFILE}\")\n", "print(f\" Source: {DEPLOY_SOURCE_PATH or f'GitHub (branch: {GIT_BRANCH})'}\")\n", "print(f\" NGC org: {NGC_CLI_ORG or 'account default'}\")\n", "print(f\" LLM: {'remote' if USE_REMOTE_LLM else 'local'}\")\n", "if USE_REMOTE_LLM:\n", " print(f\" endpoint: {REMOTE_LLM_ENDPOINT_URL}\")\n", "print(f\" VLM: {'remote' if USE_REMOTE_VLM else 'local'}\")\n", "if USE_REMOTE_VLM:\n", " print(f\" endpoint: {REMOTE_VLM_ENDPOINT_URL}\")\n", "if PROFILE == \"alerts\":\n", " print(f\" Alerts: {ALERTS_MODE}\")\n", "print(f\" NGC key: {NGC_CLI_API_KEY[:4]}...{NGC_CLI_API_KEY[-4:]}\")" ] }, { "cell_type": "markdown", "id": "d3157060", "metadata": {}, "source": [ "## 1.1 Test Remote LLM / VLM Endpoints (optional)\n", "\n", "If you enabled `USE_REMOTE_LLM` and/or `USE_REMOTE_VLM` in Section 1, run this cell to verify each endpoint **before** deploying. It probes `{ENDPOINT}/v1/models` — the same call Section 8 makes to auto-detect the model name.\n", "\n", "An endpoint passes when:\n", "- `GET /v1/models` returns HTTP 200 with an OpenAI-compatible `data[].id` list, and\n", "- it advertises **exactly one** model (so the model name can be auto-detected).\n", "\n", "Skipped automatically for local (non-remote) LLM/VLM." ] }, { "cell_type": "code", "execution_count": null, "id": "ae19d9b6", "metadata": {}, "outputs": [], "source": [ "import json, urllib.request, urllib.error\n", "\n", "def probe_remote_endpoint(name, base_url):\n", " \"\"\"Probe {base_url}/v1/models — the same call Section 8 uses to auto-detect the model.\"\"\"\n", " # Normalize: strip trailing slash and any /v1 or /v1/models the user may have added.\n", " url_base = base_url.rstrip(\"/\")\n", " for suffix in (\"/v1/models\", \"/v1\"):\n", " if url_base.endswith(suffix):\n", " url_base = url_base[: -len(suffix)]\n", " url = f\"{url_base}/v1/models\"\n", "\n", " try:\n", " with urllib.request.urlopen(url, timeout=15) as resp:\n", " body = json.loads(resp.read().decode())\n", " except urllib.error.HTTPError as e:\n", " raise RuntimeError(f\"{name}: HTTP {e.code} from {url}\") from None\n", " except Exception as e:\n", " raise RuntimeError(f\"{name}: could not reach {url}: {e}\") from None\n", "\n", " models = [m.get(\"id\") for m in body.get(\"data\", []) if m.get(\"id\")]\n", " if not models:\n", " raise RuntimeError(f\"{name}: no models advertised at {url}\")\n", "\n", " print(f\"{name}: OK — {len(models)} model(s) at {url_base}\")\n", " for m in models:\n", " print(f\" - {m}\")\n", " if len(models) > 1:\n", " print(f\" WARNING: {name} advertises multiple models. Auto-select is unsafe;\")\n", " print(f\" use an endpoint that serves a single model, or a model-specific URL.\")\n", " return models\n", "\n", "if USE_REMOTE_LLM:\n", " probe_remote_endpoint(\"LLM\", REMOTE_LLM_ENDPOINT_URL)\n", "else:\n", " print(\"LLM: local — skipping remote probe (verify after deploy in Section 9).\")\n", "\n", "if USE_REMOTE_VLM:\n", " probe_remote_endpoint(\"VLM\", REMOTE_VLM_ENDPOINT_URL)\n", "else:\n", " print(\"VLM: local — skipping remote probe (verify after deploy in Section 9).\")\n", "\n", "print(\"\\nEndpoint check complete.\")" ] }, { "cell_type": "markdown", "id": "06dd9db8", "metadata": {}, "source": [ "## 2. Prerequisites Check\n", "\n", "Validate that the NVIDIA driver, CUDA, Docker, Docker Compose, and NVIDIA container runtime meet the deployment prerequisites. Docker versions newer than the supported range are reported here, then pinned in Section 4.1 before deployment. An Ubuntu release other than 24.04 on x86 is reported as a warning rather than a hard failure, so the remaining sections still run on an unvalidated host." ] }, { "cell_type": "code", "execution_count": null, "id": "cbe43d5d", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "%%bash\n", "set -euo pipefail\n", "\n", "set_min_driver_version() {\n", " # Documented minimum NVIDIA driver by platform:\n", " # Brev x86 Ubuntu 24.04 -> 595.91.07\n", " # DGX Spark -> 580.173.02\n", " local os_id=\"\" os_version_id=\"\" product_name=\"\" gpu_name=\"\" platform_info=\"\"\n", "\n", " if [ -r /etc/os-release ]; then\n", " os_id=$(. /etc/os-release; echo \"${ID:-}\")\n", " os_version_id=$(. /etc/os-release; echo \"${VERSION_ID:-}\")\n", " fi\n", "\n", " for f in /sys/devices/virtual/dmi/id/product_name /proc/device-tree/model; do\n", " if [ -r \"$f\" ]; then\n", " product_name=$(tr -d '\\0' < \"$f\" 2>/dev/null)\n", " [ -n \"$product_name\" ] && break\n", " fi\n", " done\n", "\n", " if command -v nvidia-smi >/dev/null 2>&1; then\n", " gpu_name=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | head -n1)\n", " fi\n", " platform_info=\"$product_name $gpu_name\"\n", "\n", " if echo \"$platform_info\" | grep -qiE 'spark'; then\n", " MIN_DRIVER_VERSION=\"580.173.02\"\n", " echo \"DGX Spark detected (Ubuntu ${os_version_id:-unknown}): OK\"\n", " elif [ \"$os_version_id\" = \"24.04\" ]; then\n", " MIN_DRIVER_VERSION=\"595.91.07\"\n", " echo \"Ubuntu 24.04 on x86: OK\"\n", " elif [ \"$os_id\" != \"ubuntu\" ]; then\n", " # Sections 4 and 4.1 reconfigure Docker storage and pin Ubuntu apt\n", " # packages, so a non-Ubuntu host must stop before anything is changed.\n", " echo \"ERROR: x86 VSS hosts require Ubuntu; detected ${os_id:-unknown} ${os_version_id:-}\" >&2\n", " return 1\n", " else\n", " MIN_DRIVER_VERSION=\"595.91.07\"\n", " echo \"WARNING: x86 VSS hosts are validated on Ubuntu 24.04; detected Ubuntu ${os_version_id:-unknown}.\"\n", " echo \" Continuing with the x86 driver requirement ($MIN_DRIVER_VERSION). This release is\"\n", " echo \" untested for VSS, so package pinning and deployment steps may behave differently.\"\n", " fi\n", "}\n", "echo \"=== Operating System ===\"\n", "set_min_driver_version\n", "echo \"\"\n", "\n", "MIN_CUDA_VERSION=\"12.0\"\n", "MIN_DOCKER_VERSION=\"28.3.3\"\n", "MAX_DOCKER_VERSION=\"29.5.0\"\n", "MIN_COMPOSE_VERSION=\"2.39.1\"\n", "\n", "fail() {\n", " echo \"ERROR: $*\" >&2\n", " exit 1\n", "}\n", "\n", "require_command() {\n", " command -v \"$1\" >/dev/null 2>&1 || fail \"$1 is not installed or not on PATH\"\n", "}\n", "\n", "version_ge() {\n", " [ \"$(printf '%s\\n%s\\n' \"$2\" \"$1\" | sort -V | head -n1)\" = \"$2\" ]\n", "}\n", "\n", "version_lt() {\n", " [ \"$1\" != \"$2\" ] && [ \"$(printf '%s\\n%s\\n' \"$1\" \"$2\" | sort -V | head -n1)\" = \"$1\" ]\n", "}\n", "\n", "echo \"=== NVIDIA Driver & GPU ===\"\n", "require_command nvidia-smi\n", "nvidia-smi --query-gpu=index,name,driver_version,memory.total --format=csv,noheader\n", "\n", "GPU_COUNT=$(nvidia-smi --query-gpu=index --format=csv,noheader | wc -l | tr -d ' ')\n", "[ \"$GPU_COUNT\" -gt 0 ] || fail \"No NVIDIA GPUs detected\"\n", "echo \"Detected $GPU_COUNT GPU(s)\"\n", "\n", "DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader | head -n1 | tr -d ' ')\n", "version_ge \"$DRIVER_VERSION\" \"$MIN_DRIVER_VERSION\" \\\n", " || fail \"NVIDIA driver $DRIVER_VERSION is older than required $MIN_DRIVER_VERSION\"\n", "echo \"NVIDIA driver: OK ($DRIVER_VERSION >= $MIN_DRIVER_VERSION)\"\n", "\n", "CUDA_VERSION=$(nvidia-smi | awk -F'CUDA Version: ' '/CUDA Version/ {split($2,a,\" \"); version=a[1]} END {print version}')\n", "[ -n \"$CUDA_VERSION\" ] || fail \"Could not read CUDA version from nvidia-smi\"\n", "version_ge \"$CUDA_VERSION\" \"$MIN_CUDA_VERSION\" \\\n", " || fail \"CUDA version $CUDA_VERSION is older than required $MIN_CUDA_VERSION\"\n", "echo \"CUDA version: OK ($CUDA_VERSION >= $MIN_CUDA_VERSION)\"\n", "echo \"\"\n", "\n", "echo \"=== Docker ===\"\n", "require_command docker\n", "docker ps >/dev/null 2>&1 \\\n", " || fail \"Docker daemon is not reachable by this user. Add the user to the docker group or start Docker.\"\n", "\n", "DOCKER_VERSION=$(docker version --format '{{.Server.Version}}')\n", "version_ge \"$DOCKER_VERSION\" \"$MIN_DOCKER_VERSION\" \\\n", " || fail \"Docker Engine $DOCKER_VERSION is older than required $MIN_DOCKER_VERSION\"\n", "if version_lt \"$DOCKER_VERSION\" \"$MAX_DOCKER_VERSION\"; then\n", " echo \"Docker Engine: OK ($DOCKER_VERSION >= $MIN_DOCKER_VERSION and < $MAX_DOCKER_VERSION)\"\n", "else\n", " echo \"WARNING: Docker Engine $DOCKER_VERSION is newer than the supported range (< $MAX_DOCKER_VERSION).\"\n", " echo \"Section 4.1 will pin Docker before deployment image pulls. Continue through the notebook in order.\"\n", "fi\n", "\n", "COMPOSE_VERSION=$(docker compose version --short 2>/dev/null | sed 's/^v//')\n", "[ -n \"$COMPOSE_VERSION\" ] || fail \"Docker Compose plugin is not installed\"\n", "version_ge \"$COMPOSE_VERSION\" \"$MIN_COMPOSE_VERSION\" \\\n", " || fail \"Docker Compose $COMPOSE_VERSION is older than required $MIN_COMPOSE_VERSION\"\n", "echo \"Docker Compose: OK ($COMPOSE_VERSION >= $MIN_COMPOSE_VERSION)\"\n", "echo \"\"\n", "\n", "echo \"=== NVIDIA Container Toolkit ===\"\n", "docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi >/dev/null 2>&1 \\\n", " || fail \"NVIDIA Container Toolkit is not functional. Install or repair it: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html\"\n", "echo \"NVIDIA Container Toolkit: OK\"\n", "echo \"\"\n", "\n", "echo \"=== Disk Space ===\"\n", "df -h / | tail -1 | awk '{print \"Root:\", $4, \"available of\", $2}'\n", "echo \"\"\n", "echo \"Prerequisites check passed.\"" ] }, { "cell_type": "markdown", "id": "8dfab3ce", "metadata": {}, "source": [ "## 3. Install NGC CLI\n", "\n", "The NGC CLI is required to download models during deployment. This cell installs it if not already present, then configures it with your API key." ] }, { "cell_type": "code", "execution_count": null, "id": "2da46f45", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, os, shutil\n", "\n", "def run(cmd, **kwargs):\n", " \"\"\"Run a shell command, raise on failure with output.\"\"\"\n", " r = subprocess.run(cmd, shell=True, capture_output=True, text=True, **kwargs)\n", " if r.returncode != 0:\n", " raise RuntimeError(f\"Command failed: {cmd}\\n{r.stderr}\\n{r.stdout}\")\n", " return r.stdout.strip()\n", "\n", "# Check if NGC CLI is already installed\n", "ngc_path = shutil.which(\"ngc\")\n", "if ngc_path:\n", " ver = run(\"ngc --version 2>&1 | head -1\")\n", " print(f\"NGC CLI already installed: {ver}\")\n", "else:\n", " import platform\n", " arch = platform.machine()\n", " if arch in (\"aarch64\", \"arm64\"):\n", " filename = \"ngccli_linux_arm64.zip\"\n", " else:\n", " filename = \"ngccli_linux.zip\"\n", "\n", " # Use version-pinned URL (update this if a newer version is needed)\n", " NGC_CLI_VERSION = \"4.13.0\"\n", " url = f\"https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/{NGC_CLI_VERSION}/files/{filename}\"\n", "\n", " print(f\"Installing NGC CLI {NGC_CLI_VERSION} ...\")\n", " run(f\"cd /tmp && wget -q --content-disposition '{url}' -O ngc_cli.zip\")\n", "\n", " # Verify download is not empty\n", " size = os.path.getsize(\"/tmp/ngc_cli.zip\")\n", " if size < 1000:\n", " raise RuntimeError(f\"NGC CLI download failed — file is only {size} bytes. Check the version URL.\")\n", " print(f\" Downloaded {size / 1024 / 1024:.1f} MB\")\n", "\n", " run(\"cd /tmp && unzip -o ngc_cli.zip\")\n", " # NGC bundles its own Python — copy the entire directory\n", " run(\"sudo cp -r /tmp/ngc-cli/* /usr/local/bin/\")\n", " run(\"rm -rf /tmp/ngc_cli.zip /tmp/ngc-cli\")\n", "\n", " ver = run(\"ngc --version 2>&1 | head -1\")\n", " print(f\" Installed: {ver}\")\n", "\n", "# Configure NGC CLI with API key. Leave org unset unless the user supplied one.\n", "print(\"Configuring NGC CLI...\")\n", "ngc_dir = os.path.expanduser(\"~/.ngc\")\n", "os.makedirs(ngc_dir, exist_ok=True)\n", "\n", "config_lines = [\n", " \";WARNING - This is a machine generated file. Do not edit manually.\\n\",\n", " \";WARNING - To update local config settings, see 'ngc config set -h'.\\n\",\n", " \"\\n\",\n", " \"[CURRENT]\\n\",\n", " f\"apikey = {NGC_CLI_API_KEY}\\n\",\n", " \"format_type = ascii\\n\",\n", "]\n", "if NGC_CLI_ORG:\n", " config_lines.append(f\"org = {NGC_CLI_ORG}\\n\")\n", "\n", "with open(os.path.join(ngc_dir, \"config\"), \"w\") as f:\n", " f.writelines(config_lines)\n", "\n", "print(\"NGC CLI configured.\")\n", "print(run(\"ngc config current\"))" ] }, { "cell_type": "markdown", "id": "01ce3dd3", "metadata": {}, "source": [ "## 4. Docker & Containerd Storage\n", "\n", "Docker images and containerd layers for VSS require **~250GB** (NIM models, DeepStream, ELK, etc.). Most GPU cloud instances ship with a small root disk (200-250GB) that **will run out of space** during deployment.\n", "\n", "This cell auto-detects whether your root disk is too small and moves Docker and containerd storage to a larger mount. Docker **volumes** (Elasticsearch indices, uploaded videos, Kafka data) are kept on the root disk so your data persists even if the instance is stopped and the ephemeral NVMe is wiped. Images and layers are re-pulled automatically on next deploy.\n", "\n", "**Common NVMe mount points** (auto-detected):\n", "- AWS DLAMI: `/opt/dlami/nvme`\n", "- Brev/Crusoe: `/ephemeral`\n", "- Custom RAID: `/data`\n", "\n", "To override auto-detection, set `STORAGE_ROOT` below." ] }, { "cell_type": "code", "execution_count": null, "id": "ee76da6b", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, json, os, shutil, tempfile\n", "\n", "STORAGE_ROOT = \"\" # Override: set to a mount path (e.g. \"/mnt/data\") to skip auto-detection\n", "\n", "MIN_ROOT_FREE_GB = 350 # If root has less than this free, move storage\n", "\n", "# --- Auto-detect large mount ---\n", "\n", "def get_disk_free_gb(path):\n", " \"\"\"Return free space in GB for the filesystem containing path.\"\"\"\n", " st = os.statvfs(path)\n", " return (st.f_bavail * st.f_frsize) / (1024 ** 3)\n", "\n", "def get_disk_total_gb(path):\n", " st = os.statvfs(path)\n", " return (st.f_blocks * st.f_frsize) / (1024 ** 3)\n", "\n", "def find_large_mount():\n", " \"\"\"Look for a large non-root mount suitable for Docker storage.\"\"\"\n", " candidates = [\"/opt/dlami/nvme\", \"/ephemeral\", \"/data\"]\n", " for path in candidates:\n", " if os.path.isdir(path) and os.path.ismount(path):\n", " free = get_disk_free_gb(path)\n", " if free > 200:\n", " return path, free\n", " return None, 0\n", "\n", "def find_mount_unit(mount_path):\n", " \"\"\"Convert a mount path to a systemd mount unit name (e.g. /opt/dlami/nvme -> opt-dlami-nvme.mount).\"\"\"\n", " # Strip leading slash, replace remaining slashes with dashes\n", " unit = mount_path.strip(\"/\").replace(\"/\", \"-\") + \".mount\"\n", " # Verify this unit exists on the system\n", " r = subprocess.run([\"systemctl\", \"cat\", unit], capture_output=True, text=True)\n", " if r.returncode == 0:\n", " return unit\n", " return None\n", "\n", "def get_docker_engine_version():\n", " \"\"\"Return the Docker Engine server version, or an empty string if unavailable.\"\"\"\n", " r = subprocess.run([\"docker\", \"version\", \"--format\", \"{{.Server.Version}}\"],\n", " capture_output=True, text=True)\n", " if r.returncode == 0:\n", " return r.stdout.strip()\n", "\n", " print(\"Unable to determine Docker Engine server version; skipping Docker 29.5.x/NVCR workaround.\")\n", " return \"\"\n", "\n", "def read_daemon_config(daemon_json):\n", " try:\n", " with open(daemon_json) as f:\n", " return json.load(f)\n", " except (FileNotFoundError, json.JSONDecodeError):\n", " return {}\n", "\n", "def write_daemon_config(daemon_json, config):\n", " with tempfile.NamedTemporaryFile(mode=\"w\", suffix=\".json\", delete=False) as tmp:\n", " json.dump(config, tmp, indent=2)\n", " tmp.write(\"\\n\")\n", " tmp_path = tmp.name\n", " try:\n", " subprocess.run([\"sudo\", \"cp\", tmp_path, daemon_json], check=True)\n", " finally:\n", " os.unlink(tmp_path)\n", "\n", "def prepare_docker_295_nvcr_workaround(config, docker_version):\n", " \"\"\"Disable Docker's containerd image store for Docker 29.5.x NVCR pulls.\"\"\"\n", " if not docker_version.startswith(\"29.5.\"):\n", " return False\n", "\n", " print(f\"\\nApplying temporary Docker 29.5.x/NVCR workaround for Docker {docker_version}.\")\n", " print(\" Setting daemon feature containerd-snapshotter=false to avoid NVCR Incorrect Repository Format pulls.\")\n", "\n", " features = config.get(\"features\")\n", " if not isinstance(features, dict):\n", " features = {}\n", " config[\"features\"] = features\n", "\n", " if features.get(\"containerd-snapshotter\") is False:\n", " print(\" Docker 29.5.x/NVCR workaround already configured; no Docker restart needed for this patch.\")\n", " return False\n", "\n", " # Temporary Docker 29.5.x/NVCR workaround: Docker 29.5.x with the\n", " # containerd image store can fail some nvcr.io pulls with\n", " # \"Incorrect Repository Format\". Remove this once Docker/NVCR resolves it.\n", " features[\"containerd-snapshotter\"] = False\n", " return True\n", "\n", "daemon_json = \"/etc/docker/daemon.json\"\n", "daemon_config = read_daemon_config(daemon_json)\n", "docker_engine_version = get_docker_engine_version()\n", "need_docker_295_daemon_json = prepare_docker_295_nvcr_workaround(daemon_config, docker_engine_version)\n", "\n", "root_free = get_disk_free_gb(\"/\")\n", "root_total = get_disk_total_gb(\"/\")\n", "\n", "print(f\"Root disk: {root_free:.0f} GB free / {root_total:.0f} GB total\")\n", "\n", "if STORAGE_ROOT:\n", " large_mount = STORAGE_ROOT\n", " mount_free = get_disk_free_gb(STORAGE_ROOT)\n", " print(f\"Using override: {STORAGE_ROOT} ({mount_free:.0f} GB free)\")\n", " need_move = True\n", "else:\n", " large_mount, mount_free = find_large_mount()\n", " need_move = root_free < MIN_ROOT_FREE_GB and large_mount is not None\n", "\n", " if large_mount:\n", " print(f\"Large mount: {large_mount} ({mount_free:.0f} GB free)\")\n", " else:\n", " print(\"No large ephemeral mount detected.\")\n", "\n", " if root_free >= MIN_ROOT_FREE_GB:\n", " print(f\"\\nRoot disk has enough space ({root_free:.0f} GB free). No storage move needed.\")\n", " elif not large_mount:\n", " print(f\"\\nWARNING: Root disk only has {root_free:.0f} GB free and no large mount was found.\")\n", " print(\"Deployment may fail due to disk space. Consider attaching a larger volume.\")\n", "\n", "if need_move:\n", " DOCKER_DATA_ROOT = os.path.join(large_mount, \"docker\")\n", " CONTAINERD_ROOT = os.path.join(large_mount, \"containerd\")\n", " VOLUMES_DIR = \"/var/lib/docker/volumes\" # Keep volumes on persistent root disk\n", "\n", " print(f\"\\nMoving Docker and containerd storage to {large_mount}\")\n", " print(f\" Docker images/layers: {DOCKER_DATA_ROOT}\")\n", " print(f\" Containerd: {CONTAINERD_ROOT}\")\n", " print(f\" Docker volumes: {VOLUMES_DIR} (stays on root for persistence)\")\n", "\n", " # --- Check what needs changing ---\n", " config = daemon_config\n", "\n", " need_data_root_daemon_json = config.get(\"data-root\") != DOCKER_DATA_ROOT\n", " need_daemon_json = need_data_root_daemon_json or need_docker_295_daemon_json\n", "\n", " subprocess.run([\"sudo\", \"mkdir\", \"-p\", DOCKER_DATA_ROOT], check=True)\n", " subprocess.run([\"sudo\", \"mkdir\", \"-p\", VOLUMES_DIR], check=True)\n", "\n", " volumes_link = os.path.join(DOCKER_DATA_ROOT, \"volumes\")\n", " need_volumes_symlink = not (os.path.islink(volumes_link) and os.readlink(volumes_link) == VOLUMES_DIR)\n", "\n", " containerd_link = \"/var/lib/containerd\"\n", " need_containerd = not (os.path.islink(containerd_link) and os.readlink(containerd_link) == CONTAINERD_ROOT)\n", "\n", " # Even if symlinks are correct, ensure NVMe target dirs actually exist\n", " # (they get wiped when ephemeral NVMe is reset on instance stop/start)\n", " need_target_dirs = not os.path.isdir(DOCKER_DATA_ROOT) or not os.path.isdir(CONTAINERD_ROOT)\n", " if need_target_dirs:\n", " print(f\"\\n NVMe target dir(s) missing — recreating...\")\n", " subprocess.run([\"sudo\", \"mkdir\", \"-p\", DOCKER_DATA_ROOT, CONTAINERD_ROOT], check=True)\n", "\n", " if not need_daemon_json and not need_volumes_symlink and not need_containerd:\n", " print(f\"\\n Docker data-root already set to {DOCKER_DATA_ROOT}\")\n", " print(f\" Volumes symlink already correct: {volumes_link} -> {VOLUMES_DIR}\")\n", " print(f\" Containerd already symlinked: {containerd_link} -> {CONTAINERD_ROOT}\")\n", "\n", " # Always ensure the boot-time restore service is up to date\n", " # (handles the case where service exists but is missing mount dependencies)\n", " _update_restore_service = True\n", " _need_restart = need_target_dirs # Restart Docker/containerd if we had to recreate dirs\n", " else:\n", " _update_restore_service = True\n", " _need_restart = True\n", "\n", " # Stop Docker AND docker.socket (socket can reactivate Docker and recreate dirs)\n", " print(\"\\n Stopping Docker and containerd for storage reconfiguration...\")\n", " subprocess.run([\"sudo\", \"systemctl\", \"stop\", \"docker.socket\"], check=False)\n", " subprocess.run([\"sudo\", \"systemctl\", \"stop\", \"docker\"], check=True)\n", " subprocess.run([\"sudo\", \"systemctl\", \"stop\", \"containerd\"], check=True)\n", "\n", " # --- Docker daemon.json ---\n", " if need_daemon_json:\n", " if need_data_root_daemon_json:\n", " config[\"data-root\"] = DOCKER_DATA_ROOT\n", " write_daemon_config(daemon_json, config)\n", " if need_docker_295_daemon_json:\n", " print(\" Docker 29.5.x/NVCR workaround written to daemon.json\")\n", " if need_data_root_daemon_json:\n", " print(f\" Docker data-root set to {DOCKER_DATA_ROOT}\")\n", " else:\n", " print(f\" Docker data-root already set to {DOCKER_DATA_ROOT}\")\n", " else:\n", " print(f\" Docker data-root already set to {DOCKER_DATA_ROOT}\")\n", "\n", " # --- Volumes symlink (use ln -sfn for idempotency) ---\n", " if need_volumes_symlink:\n", " # ln -sfn: force, no-dereference (replaces existing dir/symlink atomically)\n", " subprocess.run([\"sudo\", \"rm\", \"-rf\", volumes_link], check=True)\n", " subprocess.run([\"sudo\", \"ln\", \"-sfn\", VOLUMES_DIR, volumes_link], check=True)\n", " print(f\" Created symlink: {volumes_link} -> {VOLUMES_DIR}\")\n", " else:\n", " print(f\" Volumes symlink already correct: {volumes_link} -> {VOLUMES_DIR}\")\n", "\n", " # --- Containerd ---\n", " if need_containerd:\n", " subprocess.run([\"sudo\", \"mkdir\", \"-p\", CONTAINERD_ROOT], check=True)\n", " if os.path.isdir(containerd_link) and not os.path.islink(containerd_link):\n", " # Move existing containerd data\n", " subprocess.run(f\"sudo mv {containerd_link}/* {CONTAINERD_ROOT}/ 2>/dev/null; true\",\n", " shell=True, check=False)\n", " subprocess.run([\"sudo\", \"rm\", \"-rf\", containerd_link], check=True)\n", " print(f\" Containerd data moved to {CONTAINERD_ROOT}\")\n", " elif os.path.lexists(containerd_link):\n", " subprocess.run([\"sudo\", \"rm\", \"-f\", containerd_link], check=True)\n", " subprocess.run([\"sudo\", \"ln\", \"-sfn\", CONTAINERD_ROOT, containerd_link], check=True)\n", " print(f\" Containerd symlinked: {containerd_link} -> {CONTAINERD_ROOT}\")\n", " else:\n", " print(f\" Containerd already symlinked: {containerd_link} -> {CONTAINERD_ROOT}\")\n", "\n", " # --- Install/update boot-time restore service ---\n", " # Ephemeral NVMe is wiped on instance stop/start. This systemd service\n", " # recreates the directories before Docker/containerd start so they don't crash-loop.\n", " # It also re-creates the volumes symlink: without it Docker would populate a\n", " # fresh volumes directory on the NVMe and stop seeing the data kept on root.\n", " # We use RequiresMountsFor= so the service waits for the NVMe to actually be mounted.\n", " if _update_restore_service:\n", " unit_name = \"docker-nvme-restore.service\"\n", " unit_path = f\"/etc/systemd/system/{unit_name}\"\n", "\n", " # Build After= line — include the mount unit if systemd knows about it\n", " after_targets = \"local-fs.target\"\n", " mount_unit = find_mount_unit(large_mount)\n", " if mount_unit:\n", " after_targets += f\" {mount_unit}\"\n", "\n", " unit_content = f\"\"\"[Unit]\n", "Description=Restore Docker/containerd dirs on ephemeral NVMe\n", "Before=containerd.service docker.service\n", "After={after_targets}\n", "RequiresMountsFor={large_mount}\n", "\n", "[Service]\n", "Type=oneshot\n", "ExecStart=/bin/bash -c 'mkdir -p {DOCKER_DATA_ROOT} {CONTAINERD_ROOT} {VOLUMES_DIR} && if [ ! -e {DOCKER_DATA_ROOT}/volumes ] || [ -L {DOCKER_DATA_ROOT}/volumes ]; then ln -sfn {VOLUMES_DIR} {DOCKER_DATA_ROOT}/volumes; fi'\n", "\n", "[Install]\n", "WantedBy=multi-user.target\n", "\"\"\"\n", " import tempfile\n", " with tempfile.NamedTemporaryFile(mode='w', suffix='.service', delete=False) as tmp:\n", " tmp.write(unit_content)\n", " tmp_path = tmp.name\n", " subprocess.run([\"sudo\", \"cp\", tmp_path, unit_path], check=True)\n", " os.unlink(tmp_path)\n", " subprocess.run([\"sudo\", \"systemctl\", \"daemon-reload\"], check=True)\n", " subprocess.run([\"sudo\", \"systemctl\", \"enable\", unit_name], check=True, capture_output=True)\n", " print(f\" Installed {unit_name} (restores NVMe dirs on boot, waits for mount)\")\n", "\n", " # --- Restart if needed ---\n", " if _need_restart:\n", " print(\"\\n Starting containerd and Docker...\")\n", " subprocess.run([\"sudo\", \"systemctl\", \"start\", \"containerd\"], check=True)\n", " subprocess.run([\"sudo\", \"systemctl\", \"start\", \"docker.socket\"], check=True)\n", " subprocess.run([\"sudo\", \"systemctl\", \"start\", \"docker\"], check=True)\n", "\n", " r = subprocess.run([\"docker\", \"info\", \"--format\", \"{{.DockerRootDir}}\"],\n", " capture_output=True, text=True)\n", " print(f\"\\n Docker data-root: {r.stdout.strip()}\")\n", " target = os.readlink(containerd_link) if os.path.islink(containerd_link) else containerd_link\n", " print(f\" Containerd root: {target}\")\n", " print(f\"\\n Storage configuration complete.\")\n", "else:\n", " if need_docker_295_daemon_json:\n", " write_daemon_config(daemon_json, daemon_config)\n", " print(\" Docker 29.5.x/NVCR workaround written to daemon.json\")\n", " print(\"\\n Restarting Docker to activate daemon.json change...\")\n", " subprocess.run([\"sudo\", \"systemctl\", \"restart\", \"docker\"], check=True)\n", " if not STORAGE_ROOT and root_free >= MIN_ROOT_FREE_GB:\n", " print(\"Skipping storage move.\")" ] }, { "cell_type": "markdown", "id": "737a4711-8616-4d8e-81f1-dad62c9301b0", "metadata": {}, "source": [ "### 4.1 Pin Docker version\n", "\n", "Pin Docker CE + plugins + containerd.io to a known-good combination (CE **29.4.3**, buildx **0.33.0**, compose **5.1.3**, containerd **2.2.3**). Some Brev launchables ship newer versions than the VSS deploy profiles are tested against; pin explicitly so compose/buildx incompatibilities don't surface in later sections. `apt-mark hold` prevents unattended-upgrades or later cells from drifting the box back.\n", "\n", "The cell first reads the installed Docker Engine version: if it already falls in the tested range **[28.3.3, 29.5.0)** (the same range Section 3 validates) the version downgrade is **skipped** — re-pinning to an exact version the platform's apt repo may not carry (e.g. DGX Spark / DGX-OS on arm64) would fail with *version not found* for no benefit. The packages are still `apt-mark hold`-ed at their current versions so the box can't drift past the tested range mid-run. Safe to re-run.\n", "\n", "Runs *after* the section 4 storage relocation so the APT download lands on the relocated volume and the dockerd restart triggered by the downgrade picks up the new data-root." ] }, { "cell_type": "code", "execution_count": null, "id": "bbb3b970-81f7-4afa-941b-b2c85dbf25df", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Pin Docker CE + plugins + containerd.io to a known-good combination, but\n", "# ONLY when the host's Docker is outside the tested range. Some Brev\n", "# launchables ship a newer Docker than the VSS deploy profiles are tested\n", "# against; pin explicitly so compose/buildx incompatibilities don't surface\n", "# mid-deployment.\n", "#\n", "# When the installed Docker already falls in [28.3.3, 29.5.0) the version\n", "# downgrade is skipped: re-pinning to an exact epoch-versioned package that\n", "# the platform's apt repo may not carry (e.g. DGX Spark / DGX-OS on arm64)\n", "# fails with \"version not found\" for no benefit. The in-range packages are\n", "# still held so the box can't drift past the tested range mid-notebook.\n", "# Idempotent -- safe to re-run.\n", "\n", "set -euo pipefail\n", "\n", "# Tested Docker Engine range -- keep in sync with the VSS launchable prereq check.\n", "MIN_DOCKER_VERSION=\"28.3.3\"\n", "MAX_DOCKER_VERSION=\"29.5.0\"\n", "\n", "# Packages frozen with `apt-mark hold` so unattended-upgrades / later\n", "# `apt-get install` calls can't drift the box before the notebook finishes.\n", "HOLD_PKGS=\"docker-ce docker-ce-cli docker-buildx-plugin docker-compose-plugin containerd.io\"\n", "\n", "version_ge() { [ \"$(printf '%s\\n%s\\n' \"$2\" \"$1\" | sort -V | head -n1)\" = \"$2\" ]; }\n", "version_lt() { [ \"$1\" != \"$2\" ] && [ \"$(printf '%s\\n%s\\n' \"$1\" \"$2\" | sort -V | head -n1)\" = \"$1\" ]; }\n", "\n", "DOCKER_VERSION=\"$(docker version --format '{{.Server.Version}}' 2>/dev/null || true)\"\n", "if [ -n \"$DOCKER_VERSION\" ] \\\n", " && version_ge \"$DOCKER_VERSION\" \"$MIN_DOCKER_VERSION\" \\\n", " && version_lt \"$DOCKER_VERSION\" \"$MAX_DOCKER_VERSION\"; then\n", " echo \"Docker $DOCKER_VERSION is within the tested range [$MIN_DOCKER_VERSION, $MAX_DOCKER_VERSION); skipping the Docker version pin.\"\n", " # No downgrade needed, but still hold the in-range packages at their\n", " # current versions so unattended-upgrades / later apt-get calls can't drift\n", " # the box past the tested range for the remainder of the notebook.\n", " sudo apt-mark hold $HOLD_PKGS\n", " exit 0\n", "fi\n", "\n", "if [ -n \"$DOCKER_VERSION\" ]; then\n", " echo \"Docker $DOCKER_VERSION is outside the tested range [$MIN_DOCKER_VERSION, $MAX_DOCKER_VERSION); pinning to known-good versions.\"\n", "else\n", " echo \"Could not read the installed Docker version; pinning to known-good versions.\"\n", "fi\n", "\n", "# Read distro info from /etc/os-release (always present on Ubuntu; minimal\n", "# images don't ship `lsb_release`).\n", ". /etc/os-release\n", "DISTRO=\"${VERSION_ID}\"\n", "CODENAME=\"${UBUNTU_CODENAME:-${VERSION_CODENAME}}\"\n", "\n", "# Versions hard-coded to what shipped alongside docker-ce 29.4.3 on the\n", "# Docker apt repo (verified against download.docker.com + upstream GitHub\n", "# release timestamps). When bumping DOCKER_CE_VER, bump these four together.\n", "DOCKER_CE_VER=\"5:29.4.3-1~ubuntu.${DISTRO}~${CODENAME}\"\n", "BUILDX_VER=\"0.33.0-1~ubuntu.${DISTRO}~${CODENAME}\"\n", "COMPOSE_VER=\"5.1.3-1~ubuntu.${DISTRO}~${CODENAME}\"\n", "CONTAINERD_VER=\"2.2.3-1~ubuntu.${DISTRO}~${CODENAME}\"\n", "\n", "# Refresh the APT cache first -- without this, the specific epoch-versioned\n", "# package may not be in the local index and the install would fail with\n", "# version-not-found before any pinning takes effect.\n", "sudo apt-get update -qq\n", "\n", "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \\\n", " --allow-downgrades \\\n", " -o Dpkg::Options::=--force-confdef \\\n", " -o Dpkg::Options::=--force-confold \\\n", " docker-ce=\"$DOCKER_CE_VER\" \\\n", " docker-ce-cli=\"$DOCKER_CE_VER\" \\\n", " docker-buildx-plugin=\"$BUILDX_VER\" \\\n", " docker-compose-plugin=\"$COMPOSE_VER\" \\\n", " containerd.io=\"$CONTAINERD_VER\"\n", "\n", "# Hold so unattended-upgrades / later `apt-get install` calls don't drift\n", "# the box back to newer versions before the rest of the notebook runs.\n", "sudo apt-mark hold $HOLD_PKGS" ] }, { "cell_type": "markdown", "id": "06c8b733", "metadata": {}, "source": [ "## 5. Docker Login\n", "\n", "Authenticate with the NVIDIA Container Registry (`nvcr.io`) to pull deployment images." ] }, { "cell_type": "code", "execution_count": null, "id": "6d0987a1", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess\n", "\n", "result = subprocess.run(\n", " [\"docker\", \"login\", \"nvcr.io\",\n", " \"--username\", \"$oauthtoken\",\n", " \"--password\", NGC_CLI_API_KEY],\n", " capture_output=True, text=True\n", ")\n", "if result.returncode == 0:\n", " print(\"Docker login to nvcr.io: OK\")\n", "else:\n", " print(f\"Docker login FAILED:\\n{result.stderr}\")\n", " raise RuntimeError(\"Docker login to nvcr.io failed\")" ] }, { "cell_type": "markdown", "id": "95db2ee0", "metadata": {}, "source": [ "## 6. Get Deployment Code\n", "\n", "This cell locates the deployment code (scripts, compose files, configs). There are two ways to get it onto the server:\n", "\n", "### Option 1 — Brev Launchable (automatic)\n", "\n", "When configured as a Brev Launchable, the git repository is cloned onto the instance automatically. Set `DEPLOY_SOURCE_PATH` in Section 1 to the path where Brev placed it (typically `~/video-search-and-summarization`).\n", "\n", "### Option 2 — Manual tarball\n", "\n", "If the code isn't already on the server, create a tarball from your local checkout and copy it over:\n", "\n", "```bash\n", "# On your local machine:\n", "cd /path/to/video-search-and-summarization\n", "tar czf ~/vss-deploy.tar.gz --exclude='.git' .\n", "scp ~/vss-deploy.tar.gz @:~/\n", "\n", "# On the server:\n", "mkdir -p ~/video-search-and-summarization\n", "tar xzf ~/vss-deploy.tar.gz -C ~/video-search-and-summarization\n", "```\n", "\n", "Then set in **Section 1**: `DEPLOY_SOURCE_PATH = \"/home//video-search-and-summarization\"`\n", "\n", "---\n", "\n", "If `DEPLOY_SOURCE_PATH` is set, uses the code at that path directly. Otherwise, attempts to clone from GitHub (requires the repo to be accessible)." ] }, { "cell_type": "code", "execution_count": null, "id": "482ee2dd", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, os\n", "\n", "if DEPLOY_SOURCE_PATH:\n", " # --- Use pre-extracted local repo ---\n", " REPO_DIR = DEPLOY_SOURCE_PATH\n", " print(f\"Using local deployment source: {REPO_DIR}\")\n", "else:\n", " # --- Clone from GitHub ---\n", " DEPLOY_DIR = os.path.expanduser(\"~/deployments\")\n", " REPO_DIR = os.path.join(DEPLOY_DIR, \"video-search-and-summarization\")\n", " GITHUB_REPO = \"https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git\"\n", "\n", " os.makedirs(DEPLOY_DIR, exist_ok=True)\n", "\n", " if os.path.isdir(REPO_DIR):\n", " print(f\"Repo already exists at {REPO_DIR}\")\n", " print(f\"Fetching latest and checking out {GIT_BRANCH}...\")\n", " subprocess.run([\"git\", \"fetch\", \"--all\", \"--prune\"], cwd=REPO_DIR, check=True,\n", " capture_output=True)\n", " result = subprocess.run([\"git\", \"checkout\", GIT_BRANCH], cwd=REPO_DIR,\n", " capture_output=True, text=True)\n", " if result.returncode != 0:\n", " raise RuntimeError(f\"Failed to checkout {GIT_BRANCH}:\\n{result.stderr}\")\n", " subprocess.run([\"git\", \"pull\", \"--ff-only\"], cwd=REPO_DIR, check=False,\n", " capture_output=True)\n", " else:\n", " print(f\"Cloning from GitHub (branch: {GIT_BRANCH})...\")\n", " result = subprocess.run(\n", " [\"git\", \"clone\", \"--branch\", GIT_BRANCH, \"--single-branch\", GITHUB_REPO, REPO_DIR],\n", " capture_output=True, text=True\n", " )\n", " if result.returncode != 0:\n", " raise RuntimeError(f\"Clone failed:\\n{result.stderr}\")\n", " print(\"Clone complete.\")\n", "\n", "# Validate repo structure\n", "SCRIPT_DIR = os.path.join(REPO_DIR, \"deploy/docker/scripts\")\n", "assert os.path.isdir(os.path.join(REPO_DIR, \"deploy/docker\")), \\\n", " f\"deploy/docker/ not found in {REPO_DIR}\"\n", "for _required in (\"compose.yml\", \"containers.env\", \"developer-profiles\"):\n", " assert os.path.exists(os.path.join(REPO_DIR, \"deploy/docker\", _required)), \\\n", " f\"{_required} not found in {REPO_DIR}/deploy/docker\"\n", "\n", "# Show commit info (if it's a git repo)\n", "commit = \"(not a git repo)\"\n", "branch = \"\"\n", "if os.path.isdir(os.path.join(REPO_DIR, \".git\")):\n", " commit = subprocess.run(\n", " [\"git\", \"log\", \"--oneline\", \"-1\"],\n", " cwd=REPO_DIR, capture_output=True, text=True\n", " ).stdout.strip()\n", " branch = subprocess.run(\n", " [\"git\", \"branch\", \"--show-current\"],\n", " cwd=REPO_DIR, capture_output=True, text=True\n", " ).stdout.strip()\n", "\n", "print(f\"\\nRepo: {REPO_DIR}\")\n", "if branch:\n", " print(f\"Branch: {branch}\")\n", "print(f\"Commit: {commit}\")\n", "print(f\"Scripts: {SCRIPT_DIR}\")\n", "print(f\"\\nContents of deploy/docker/:\")\n", "for entry in sorted(os.listdir(os.path.join(REPO_DIR, \"deploy/docker\"))):\n", " print(f\" {entry}\")" ] }, { "cell_type": "markdown", "id": "80986362", "metadata": {}, "source": [ "## 7. Detect Network Configuration\n", "\n", "Auto-detects internal (`HOST_IP`) and external (`EXTERNAL_IP`) addresses. On NAT'd cloud instances (Brev, AWS), these are different — the internal IP is used for inter-container communication while the external IP is used for browser access.\n", "\n", "If auto-detection fails or gives the wrong result, set the overrides in Section 1." ] }, { "cell_type": "code", "execution_count": null, "id": "c610f5aa", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import json, os, subprocess\n", "\n", "def detect_internal_ip():\n", " \"\"\"Detect internal IP via ip route.\"\"\"\n", " try:\n", " out = subprocess.run(\n", " [\"bash\", \"-c\", \"ip route get 1.1.1.1 | awk '/src/ {for (i=1;i<=NF;i++) if ($i==\\\"src\\\") print $(i+1)}'\"],\n", " capture_output=True, text=True, timeout=5\n", " )\n", " return out.stdout.strip()\n", " except Exception:\n", " return \"\"\n", "\n", "def detect_external_ip():\n", " \"\"\"Detect external IP via public service.\"\"\"\n", " for cmd in [\"curl -s --max-time 5 ifconfig.me\", \"curl -s --max-time 5 icanhazip.com\"]:\n", " try:\n", " out = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=10)\n", " ip = out.stdout.strip()\n", " if ip:\n", " return ip\n", " except Exception:\n", " continue\n", " return \"\"\n", "\n", "def read_etc_environment():\n", " \"\"\"Read key=value pairs from /etc/environment (Brev sets BREV_ENV_ID there).\"\"\"\n", " env = {}\n", " try:\n", " with open(\"/etc/environment\") as f:\n", " for line in f:\n", " line = line.strip()\n", " if \"=\" in line and not line.startswith(\"#\"):\n", " key, _, value = line.partition(\"=\")\n", " env[key.strip()] = value.strip().strip('\"')\n", " except FileNotFoundError:\n", " pass\n", " return env\n", "\n", "def brev_link_for_port(port):\n", " \"\"\"Resolve the secure link Brev publishes for a destination port.\n", "\n", " Returns (fqdn, public_port), or (None, None) when the port has no link.\n", "\n", " Brev publishes an environment context file listing one entry per exposed\n", " port. Read the fqdn from it -- never build one from a pattern. The domain\n", " varies per instance (gobrev.dev, brevlab.com, apps.run.brev.nvidia.com, ...)\n", " and a constructed hostname that happens to be wrong is indistinguishable\n", " from a missing link: Brev's edge answers `404 route_not_found` for both,\n", " and the same wrong hostname lands in VSS_PUBLIC_HOST, so HAProxy's\n", " known_host ACL then rejects the *correct* URL too.\n", "\n", " Key names are matched loosely so a schema tweak degrades to \"no link found\"\n", " (which is reported) rather than a wrong hostname (which is not).\n", " \"\"\"\n", " global BREV_CONTEXT_PROBLEM\n", " path = os.environ.get(\"BREV_ENVIRONMENT_CONTEXT_PATH\", \"/etc/brev/environment-context.json\")\n", " BREV_CONTEXT_PROBLEM = None\n", " try:\n", " with open(path, encoding=\"utf-8\") as f:\n", " data = json.load(f)\n", " except PermissionError:\n", " # Some Brev images ship this file readable only by root, and Jupyter runs\n", " # as the instance user. Passwordless sudo is the norm on Brev, so retry\n", " # through it rather than reporting a link that is actually published.\n", " proc = subprocess.run([\"sudo\", \"-n\", \"cat\", path], capture_output=True, text=True)\n", " if proc.returncode != 0:\n", " BREV_CONTEXT_PROBLEM = (f\"{path} is not readable by {os.environ.get('USER', 'this user')} \"\n", " \"and `sudo -n cat` failed\")\n", " return None, None\n", " try:\n", " data = json.loads(proc.stdout)\n", " except ValueError as exc:\n", " BREV_CONTEXT_PROBLEM = f\"{path} is not valid JSON ({exc})\"\n", " return None, None\n", " except FileNotFoundError:\n", " BREV_CONTEXT_PROBLEM = f\"{path} does not exist\"\n", " return None, None\n", " except (OSError, ValueError) as exc:\n", " BREV_CONTEXT_PROBLEM = f\"{path} could not be read ({exc})\"\n", " return None, None\n", "\n", " entries = data.get(\"ports\") if isinstance(data, dict) else data\n", " if not isinstance(entries, list):\n", " return None, None\n", "\n", " def _get(entry, *names):\n", " for name in names:\n", " if isinstance(entry, dict) and entry.get(name) not in (None, \"\"):\n", " return entry[name]\n", " return None\n", "\n", " for entry in entries:\n", " dest = _get(entry, \"destination_port\", \"destinationPort\", \"target_port\", \"port\")\n", " try:\n", " if dest is None or int(dest) != int(port):\n", " continue\n", " except (TypeError, ValueError):\n", " continue\n", " fqdn = _get(entry, \"fqdn\", \"hostname\", \"host\", \"url\")\n", " if not fqdn:\n", " continue\n", " fqdn = str(fqdn).split(\"://\", 1)[-1].split(\"/\", 1)[0]\n", " public = _get(entry, \"public_port\", \"publicPort\") or 443\n", " return fqdn, str(public)\n", " return None, None\n", "\n", "\n", "def require_brev_link_for_port(port, *, host_override=\"\", port_override=\"\"):\n", " \"\"\"Resolve the secure link for *port*, or raise with the reason and the fix.\n", "\n", " *host_override* is the escape hatch: the full hostname copied from the Brev\n", " console. It wins over the context file: an operator who exports a\n", " replacement host is working around a published link that does not route,\n", " so reading the file first would silently ignore the fix. It has to be the\n", " whole host, not just the domain -- the secure-link prefix is a NAME the\n", " creator chooses, not the port number (Brev's docs: \"Specify its name and\n", " port\"; port 8888 is commonly published as \"jupyter-.\"). So\n", " \"-.\" is a convention, not a rule, and constructing it\n", " is how the wrong hostname gets baked in.\n", " \"\"\"\n", " if host_override.strip():\n", " return (host_override.strip(),\n", " port_override.strip() or \"443\",\n", " \"BREV_PUBLIC_HOST override\")\n", "\n", " fqdn, public_port = brev_link_for_port(port)\n", " if fqdn:\n", " return fqdn, public_port, \"environment-context.json\"\n", "\n", " ctx = os.environ.get(\"BREV_ENVIRONMENT_CONTEXT_PATH\", \"/etc/brev/environment-context.json\")\n", " if BREV_CONTEXT_PROBLEM:\n", " why = BREV_CONTEXT_PROBLEM\n", " fix = f\" sudo chmod a+r {ctx}\\nand re-run this cell.\"\n", " else:\n", " why = f\"no entry for destination port {port} in {ctx}\"\n", " fix = (\"Create the link in the Brev console: Secure Links -> + HTTP port \"\n", " f\"-> destination port {port}, then re-run this cell.\")\n", " raise RuntimeError(\n", " f\"Could not resolve the Brev secure link for port {port}: {why}.\\n{fix}\\n\"\n", " \"Or copy the endpoint from the Secure Links page and export it verbatim, e.g.\\n\"\n", " \" export BREV_PUBLIC_HOST=7777-abcd1234.gobrev.dev\\n\"\n", " \"Do not assemble the hostname from a pattern: the link name is user-chosen \"\n", " \"and the domain varies per instance, and a wrong value is baked into \"\n", " \"VSS_PUBLIC_HOST, after which HAProxy rejects the correct URL too.\"\n", " )\n", "\n", "HOST_IP = HOST_IP_OVERRIDE or detect_internal_ip()\n", "EXTERNAL_IP = EXTERNAL_IP_OVERRIDE or detect_external_ip()\n", "\n", "print(f\"Internal IP (HOST_IP): {HOST_IP}\")\n", "print(f\"External IP: {EXTERNAL_IP}\")\n", "\n", "if HOST_IP == EXTERNAL_IP:\n", " print(\"\\nInternal == External (direct connection, no NAT)\")\n", "else:\n", " print(\"\\nNAT detected — internal and external IPs differ.\")\n", " print(\"Section 8 will set EXTERNAL_IP automatically.\")\n", "\n", "if not HOST_IP:\n", " print(\"\\nWARNING: Could not detect internal IP. Set HOST_IP_OVERRIDE in Section 1.\")\n", "if not EXTERNAL_IP:\n", " print(\"\\nWARNING: Could not detect external IP. Set EXTERNAL_IP_OVERRIDE in Section 1.\")\n", "\n", "# --- Brev Secure Links ---\n", "# On Brev, all browser-facing traffic routes through the HAProxy ingress\n", "# on a single port (default 7777). This avoids cross-origin issues when each\n", "# port gets its own secure-link hostname.\n", "# Check os.environ first, then fall back to /etc/environment (Jupyter kernels\n", "# may not inherit /etc/environment depending on how the notebook server starts).\n", "_etc_env = read_etc_environment()\n", "BREV_ENV_ID = os.environ.get(\"BREV_ENV_ID\") or _etc_env.get(\"BREV_ENV_ID\", \"\")\n", "BREV_PUBLIC_HOST = \"\"\n", "BREV_PUBLIC_PORT = \"443\"\n", "BREV_BASE_URL = \"\"\n", "if BREV_ENV_ID:\n", " # Ensure BREV_ENV_ID is in os.environ so Section 7.1 picks it up.\n", " os.environ[\"BREV_ENV_ID\"] = BREV_ENV_ID\n", " proxy_port = os.environ.get(\"PROXY_PORT\", \"7777\")\n", " BREV_PUBLIC_HOST, BREV_PUBLIC_PORT, brev_link_source = require_brev_link_for_port(\n", " proxy_port,\n", " host_override=os.environ.get(\"BREV_PUBLIC_HOST\", \"\"),\n", " port_override=os.environ.get(\"BREV_PUBLIC_PORT\", \"\"),\n", " )\n", " # The resolved link is deliberately not exported. BREV_PUBLIC_HOST/PORT\n", " # are the operator's override, which Section 7.1 consults, so writing this\n", " # run's answer back would make a rerun in the same kernel reuse a stale\n", " # host after the link changed instead of resolving the new one.\n", " # BREV_LINK_DOMAIN is informational only — nothing constructs a hostname\n", " # from it any more.\n", " os.environ[\"BREV_LINK_DOMAIN\"] = BREV_PUBLIC_HOST.split(\".\", 1)[-1]\n", " BREV_BASE_URL = (f\"https://{BREV_PUBLIC_HOST}\" if BREV_PUBLIC_PORT == \"443\"\n", " else f\"https://{BREV_PUBLIC_HOST}:{BREV_PUBLIC_PORT}\")\n", " print(f\"\\n=== Brev Environment Detected ===\")\n", " print(f\" BREV_ENV_ID: {BREV_ENV_ID}\")\n", " print(f\" Secure link: {BREV_PUBLIC_HOST}:{BREV_PUBLIC_PORT} (from {brev_link_source})\")\n", " print(f\" All browser-facing URLs route through HAProxy ingress (port {proxy_port})\")\n", " print(f\" UI will be available at: {BREV_BASE_URL}\")\n", "else:\n", " BREV_ENV_ID = \"\" # ensure defined for later cells" ] }, { "cell_type": "markdown", "id": "f2e17e24", "metadata": {}, "source": [ "## 7.1 Deployment Helpers\n", "\n", "Run this cell once before deploying. It defines the deployment logic the notebook uses in Sections 8, 13 and 14 — generating the profile's `generated.env`, creating the data directories, and driving `docker compose` directly.\n", "\n", "Nothing here needs editing. The values you can change live in Section 1, plus the profile's `overrides.env` under `deploy/docker/developer-profiles/dev-profile-/` if you want to adjust something this notebook does not expose.\n", "\n", "**What gets written to `generated.env`**, on top of a copy of the profile's `overrides.env`:\n", "\n", "| Group | Variables |\n", "| --- | --- |\n", "| Paths and network | `VSS_APPS_DIR`, `VSS_DATA_DIR`, `HOST_IP`, `EXTERNAL_IP`, `VST_CONFIG_PATH` |\n", "| Credentials | `NGC_CLI_API_KEY`, `NVIDIA_API_KEY`, `OPENAI_API_KEY` |\n", "| Selection | `HARDWARE_PROFILE`, `MODE`, `INSTALL_PROPRIETARY_CODECS` |\n", "| Brev secure links | `BREV_*`, `HAPROXY_PORT`, `VSS_PUBLIC_*` |\n", "| LLM / VLM | `LLM_MODE`, `VLM_MODE`, `*_NAME`, `*_BASE_URL`, `*_DEVICE_ID`, `*_MODEL_TYPE`, `VLM_PORT` |\n", "| Hardware tuning | `RTVI_VLLM_GPU_MEMORY_UTILIZATION`, `RTVI_VLM_MAX_MODEL_LEN`, `RT_VLM_DEVICE_ID`, `RTVI_VLM_MODEL_PATH` |\n", "| Edge | `PERCEPTION_DOCKERFILE_PREFIX` |\n", "| SBSA images | `VSS_RT_CV_TAG`, `VSS_RT_EMBED_TAG`, `VSS_RT_VLM_TAG`, `VSS_VIDEO_SUMMARIZATION_TAG` |\n", "\n", "Supported hardware: `H100`, `L40S`, `RTXPRO4500BW`, `RTXPRO6000BW`, `DGX-SPARK`.\n", "\n", "`DGX-SPARK` takes `base`, `alerts` and `search`, but not `lvs`. Its single GPU is already carrying the perception pipeline under `search`, so that profile requires **both** a remote LLM and a remote VLM; the cell then routes the agent straight at the VLM endpoint (`VLM_MODEL_TYPE=nim`, `VLM_AGENT_MEDIA_MODE=remote`), drops `rtvi-vlm` from `COMPOSE_PROFILES`, and collapses every placement onto device 0.\n", "\n", "On a Brev Launchable, `search` with a local VLM needs at least 2 GPUs (RT-VLM sits next to RT-CV and RT-Embed); the cell counts GPUs and stops early, pointing at `USE_REMOTE_VLM` instead of failing during model startup.\n", "\n", "Two more things happen on `DGX-SPARK`:\n", "\n", "- **Cache cleaner.** Edge boards share one memory pool between CPU, GPU and the OS page cache, and without a periodic `drop_caches` the first inference frame OOMs. `prepare_deployment()` installs `/usr/local/bin/sys-cache-cleaner.sh` and starts it unless `pgrep` already finds it. It is deliberately not a systemd unit, so a reboot drops it and re-running the deploy cell restarts it.\n", "- **`-sbsa` managed images.** The generic manifests carry the Tegra DeepStream build, which crash-loops on GB10's SBSA driver. `compose_env()` exports `VSS_CONTAINER_TAG_SUFFIX` for Compose interpolation, and because a service that reads a tag as plain configuration never sees that suffix, `generate_env()` also pins the four suffixed keys in `generated.env` (host env wins, then an uncommented env-file value, then the profile's commented `-sbsa` line, then the derived tag). Set `VSS_USE_SBSA_IMAGES=true` to force the same on other hardware." ] }, { "cell_type": "code", "execution_count": null, "id": "86640a85", "metadata": {}, "outputs": [], "source": [ "import os, re, json, glob, shutil, subprocess, time, urllib.request\n", "\n", "DOCKER_DIR = os.path.join(REPO_DIR, \"deploy\", \"docker\")\n", "DATA_DIR = os.path.join(DOCKER_DIR, \"data-dir\")\n", "ALL_PROFILES = (\"base\", \"lvs\", \"search\", \"alerts\")\n", "EDGE_HARDWARE_PROFILES = (\"DGX-SPARK\",)\n", "# ARM64 hosts need the -sbsa managed image variants; the generic manifests carry\n", "# the Tegra DeepStream build, which crash-loops on GB10's SBSA driver.\n", "SBSA_HARDWARE_PROFILES = (\"DGX-SPARK\",)\n", "SBSA_TAG_KEYS = (\"VSS_RT_CV_TAG\", \"VSS_RT_EMBED_TAG\", \"VSS_RT_VLM_TAG\",\n", " \"VSS_VIDEO_SUMMARIZATION_TAG\")\n", "MODE_ENV_VALUES = {\"verification\": \"2d_cv\", \"real-time\": \"2d_vlm\"}\n", "\n", "\n", "def profile_path(profile, *parts):\n", " return os.path.join(DOCKER_DIR, \"developer-profiles\", f\"dev-profile-{profile}\", *parts)\n", "\n", "\n", "def _host_env(name, default=\"\"):\n", " return os.environ.get(name, default)\n", "\n", "\n", "def _sudo_prefix():\n", " return [\"sudo\"] if os.geteuid() != 0 and shutil.which(\"sudo\") else []\n", "\n", "\n", "def mask_secret(secret):\n", " secret = str(secret)\n", " if len(secret) <= 6:\n", " return \"******\"\n", " return f\"{secret[:3]}{'*' * (len(secret) - 6)}{secret[-3:]}\"\n", "\n", "\n", "# ============================================================\n", "# env-file primitives\n", "# ============================================================\n", "\n", "def _strip_quotes(value):\n", " # One leading/trailing double quote, then one single quote — the quotes do\n", " # not have to pair up.\n", " if value.startswith('\"'):\n", " value = value[1:]\n", " if value.endswith('\"'):\n", " value = value[:-1]\n", " if value.startswith(\"'\"):\n", " value = value[1:]\n", " if value.endswith(\"'\"):\n", " value = value[:-1]\n", " return value\n", "\n", "\n", "def read_env_value(path, name):\n", " if not os.path.isfile(path):\n", " return \"\"\n", " prefix = f\"{name}=\"\n", " with open(path, encoding=\"utf-8\") as f:\n", " for line in f:\n", " if line.startswith(prefix):\n", " return _strip_quotes(line.rstrip(\"\\n\").split(\"=\", 1)[1])\n", " return \"\"\n", "\n", "\n", "def env_file_has_var(path, name):\n", " if not os.path.isfile(path):\n", " return False\n", " prefix = f\"{name}=\"\n", " with open(path, encoding=\"utf-8\") as f:\n", " return any(line.startswith(prefix) for line in f)\n", "\n", "\n", "def env_value_from_files(name, *paths):\n", " \"\"\"Later files win over earlier ones.\"\"\"\n", " value = \"\"\n", " for path in paths:\n", " if env_file_has_var(path, name):\n", " value = read_env_value(path, name)\n", " return value\n", "\n", "\n", "def env_var_defined_in_files(name, *paths):\n", " return any(env_file_has_var(p, name) for p in paths)\n", "\n", "\n", "def commented_sbsa_value(name, *paths):\n", " \"\"\"Value of a commented SBSA alternate, e.g. `#VSS_RT_CV_TAG=\"develop-latest-sbsa\"`.\n", " Profiles pin the SBSA build that way when it does not follow the shared\n", " VSS_CONTAINER_TAG channel. Later files win.\"\"\"\n", " commented = re.compile(rf\"^#[ \\t]*{re.escape(name)}=\")\n", " value = \"\"\n", " for path in paths:\n", " if not os.path.isfile(path):\n", " continue\n", " with open(path, encoding=\"utf-8\") as f:\n", " for line in f:\n", " if commented.match(line) and \"sbsa\" in line:\n", " value = _strip_quotes(line.rstrip(\"\\n\").split(\"=\", 1)[1])\n", " break\n", " return value\n", "\n", "\n", "def use_sbsa_images(hardware_profile):\n", " return (hardware_profile in SBSA_HARDWARE_PROFILES\n", " or _host_env(\"VSS_USE_SBSA_IMAGES\") == \"true\")\n", "\n", "\n", "def set_env_var(path, name, value, mask=False):\n", " \"\"\"Update an active assignment, else uncomment and update a commented one,\n", " else append. Profiles ship alternatives commented out, so the middle case\n", " is how a hardware or mode selection takes effect.\"\"\"\n", " value = \"\" if value is None else str(value)\n", " with open(path, encoding=\"utf-8\") as f:\n", " lines = f.readlines()\n", "\n", " active = re.compile(rf\"^{re.escape(name)}=\")\n", " commented = re.compile(rf\"^#[ \\t]*{re.escape(name)}=\")\n", " replacement = f\"{name}={value}\\n\"\n", "\n", " if any(active.match(l) for l in lines):\n", " lines = [replacement if active.match(l) else l for l in lines]\n", " elif any(commented.match(l) for l in lines):\n", " lines = [replacement if commented.match(l) else l for l in lines]\n", " else:\n", " lines.append(replacement)\n", "\n", " with open(path, \"w\", encoding=\"utf-8\") as f:\n", " f.writelines(lines)\n", " print(f\"[INFO] Set {name}={mask_secret(value) if mask else value}\")\n", "\n", "\n", "def _rewrite_matching(path, prefix, replacement, append_if_missing=False):\n", " \"\"\"Replace every line starting with prefix. Returns True if anything changed.\"\"\"\n", " with open(path, encoding=\"utf-8\") as f:\n", " lines = f.readlines()\n", " matched = any(l.startswith(prefix) for l in lines)\n", " if matched:\n", " lines = [replacement if l.startswith(prefix) else l for l in lines]\n", " elif append_if_missing:\n", " lines.append(replacement)\n", " else:\n", " return False\n", " with open(path, \"w\", encoding=\"utf-8\") as f:\n", " f.writelines(lines)\n", " return True\n", "\n", "\n", "# ============================================================\n", "# hardware detection and tuning\n", "# ============================================================\n", "\n", "def detect_gpu_names():\n", " try:\n", " out = subprocess.run(\n", " [\"nvidia-smi\", \"--query-gpu=name\", \"--format=csv,noheader\"],\n", " capture_output=True, text=True, timeout=30,\n", " )\n", " except (OSError, subprocess.SubprocessError):\n", " return []\n", " return [l.strip() for l in out.stdout.splitlines() if l.strip()]\n", "\n", "\n", "def canonical_from_gpu_name(gpu_name):\n", " name = gpu_name.lower()\n", " if \"h100\" in name:\n", " return \"H100\"\n", " if \"l40s\" in name:\n", " return \"L40S\"\n", " if re.search(r\"rtx.*pro.*4500.*blackwell\", name):\n", " return \"RTXPRO4500BW\"\n", " if re.search(r\"rtx.*pro.*6000.*blackwell\", name):\n", " return \"RTXPRO6000BW\"\n", " if \"gb10\" in name:\n", " return \"DGX-SPARK\"\n", " return \"\"\n", "\n", "\n", "def verify_hardware_profile(hardware_profile):\n", " if _host_env(\"SKIP_HARDWARE_CHECK\").lower() == \"true\":\n", " return\n", " names = detect_gpu_names()\n", " if not names:\n", " raise RuntimeError(\n", " f\"Hardware profile '{hardware_profile}' does not match detected hardware \"\n", " \"(no NVIDIA GPU detected). Set SKIP_HARDWARE_CHECK=true to bypass.\"\n", " )\n", " # Mixed-GPU hosts must not be rejected because a different card is listed first.\n", " if not any(canonical_from_gpu_name(n) == hardware_profile for n in names):\n", " raise RuntimeError(\n", " f\"Hardware profile '{hardware_profile}' does not match any detected NVIDIA \"\n", " f\"GPU: {', '.join(names)}. Set SKIP_HARDWARE_CHECK=true to bypass.\"\n", " )\n", " print(f\"[INFO] Hardware profile {hardware_profile} matches: {', '.join(names)}\")\n", "\n", "\n", "def rtvi_vllm_gpu_memory_utilization(hardware_profile, vlm_mode, profile):\n", " # Edge boards draw CPU, GPU and page cache from one pool, so RT-VLM takes a\n", " # smaller slice than on a dGPU. alerts needs it whether or not the LLM is\n", " # co-resident, because the CV pipeline is on the same GPU either way.\n", " if hardware_profile == \"DGX-SPARK\" and (profile == \"alerts\" or vlm_mode == \"local_shared\"):\n", " return \"0.35\"\n", " if hardware_profile in (\"L40S\", \"RTXPRO4500BW\"):\n", " return \"0.8\"\n", " # vLLM claims the whole fraction whether it needs it or not and refuses to\n", " # start unless free >= fraction x total, so a co-located LLM needs the rest\n", " # of the card left free.\n", " return \"0.4\" if vlm_mode == \"local_shared\" else \"0.7\"\n", "\n", "\n", "CACHE_CLEANER_PATH = \"/usr/local/bin/sys-cache-cleaner.sh\"\n", "CACHE_CLEANER_SCRIPT = \"\"\"#!/bin/bash\n", "set -e\n", "echo 0 | tee /proc/sys/vm/nr_hugepages\n", "echo \"Starting cache cleaner\"\n", "while true; do\n", " sync && echo 3 | tee /proc/sys/vm/drop_caches > /dev/null\n", " sleep 3\n", "done\n", "\"\"\"\n", "\n", "\n", "def cache_cleaner_running():\n", " \"\"\"None when pgrep is missing, so the caller can skip the check instead of\n", " treating an unanswerable question as a failure.\"\"\"\n", " try:\n", " return subprocess.run([\"pgrep\", \"-f\", CACHE_CLEANER_PATH],\n", " capture_output=True).returncode == 0\n", " except OSError:\n", " return None\n", "\n", "\n", "def ensure_cache_cleaner(hardware_profile):\n", " \"\"\"Edge boards share one memory pool between CPU, GPU and the OS page cache.\n", " Without a periodic drop_caches the cache pins enough of it that the first\n", " inference frame OOMs, so the prerequisites list the cleaner as a platform\n", " requirement. It is deliberately not a systemd unit, so a reboot drops it and\n", " every deployment has to check again.\"\"\"\n", " if hardware_profile not in EDGE_HARDWARE_PROFILES:\n", " return\n", " if cache_cleaner_running():\n", " print(f\"[INFO] Cache cleaner already running ({CACHE_CLEANER_PATH})\")\n", " return\n", "\n", " print(f\"[INFO] {hardware_profile}: installing and starting the cache cleaner...\")\n", " sudo = _sudo_prefix()\n", " subprocess.run(sudo + [\"tee\", CACHE_CLEANER_PATH], input=CACHE_CLEANER_SCRIPT,\n", " text=True, check=True, stdout=subprocess.DEVNULL)\n", " subprocess.run(sudo + [\"chmod\", \"+x\", CACHE_CLEANER_PATH], check=True)\n", " subprocess.Popen(sudo + [CACHE_CLEANER_PATH], stdout=subprocess.DEVNULL,\n", " stderr=subprocess.DEVNULL, start_new_session=True)\n", " time.sleep(2)\n", " if cache_cleaner_running() is False:\n", " raise RuntimeError(\n", " f\"Cache cleaner did not start. Run it manually before deploying: \"\n", " f\"sudo -b {CACHE_CLEANER_PATH}\"\n", " )\n", " print(\"[INFO] Cache cleaner started (stops on reboot; re-run the deploy cell \"\n", " \"to restart it)\")\n", "\n", "\n", "def rtvi_vlm_max_model_len(hardware_profile):\n", " return \"18000\" if hardware_profile == \"RTXPRO4500BW\" else \"\"\n", "\n", "\n", "# ============================================================\n", "# LLM / VLM mode derivation\n", "# ============================================================\n", "\n", "def remote_model_name(base_url, expected_type):\n", " \"\"\"Auto-detect the served model. Only safe when the endpoint serves exactly\n", " one model: aggregate endpoints list every NIM and the first entry is not a\n", " meaningful default.\"\"\"\n", " url = f\"{base_url}/v1/models\"\n", " try:\n", " with urllib.request.urlopen(url, timeout=15) as resp:\n", " body = json.loads(resp.read().decode())\n", " except Exception as exc:\n", " raise RuntimeError(f\"Failed to retrieve model list from {url}: {exc}\") from None\n", "\n", " models = [m.get(\"id\") for m in body.get(\"data\", []) if m.get(\"id\")]\n", " if not models:\n", " raise RuntimeError(f\"No models returned from {url}\")\n", " if len(models) > 1:\n", " listing = \"\\n\".join(f\" {m}\" for m in models)\n", " raise RuntimeError(\n", " f\"{url} returns {len(models)} models — auto-select is unsafe. Point \"\n", " f\"REMOTE_{expected_type.upper()}_ENDPOINT_URL at a single-model \"\n", " f\"endpoint. Available:\\n{listing}\"\n", " )\n", " return models[0]\n", "\n", "\n", "def derive_modes(profile, hardware_profile, use_remote_llm, use_remote_vlm,\n", " llm_endpoint_url, vlm_endpoint_url):\n", " source_env = profile_path(profile, \".env\")\n", " overrides_env = profile_path(profile, \"overrides.env\")\n", "\n", " llm_base_url = llm_endpoint_url if use_remote_llm else \"\"\n", " vlm_base_url = vlm_endpoint_url if use_remote_vlm else \"\"\n", " if use_remote_llm and not llm_base_url:\n", " raise RuntimeError(\"REMOTE_LLM_ENDPOINT_URL must be set when USE_REMOTE_LLM=True\")\n", " if use_remote_vlm and not vlm_base_url:\n", " raise RuntimeError(\"REMOTE_VLM_ENDPOINT_URL must be set when USE_REMOTE_VLM=True\")\n", " llm_is_remote = bool(llm_base_url)\n", " vlm_is_remote = bool(vlm_base_url)\n", "\n", " if hardware_profile in EDGE_HARDWARE_PROFILES:\n", " # Edge platforms are single-GPU; device IDs are not configurable.\n", " llm_device_id = vlm_device_id = \"0\"\n", " else:\n", " llm_device_id = env_value_from_files(\"LLM_DEVICE_ID\", source_env, overrides_env)\n", " vlm_device_id = env_value_from_files(\"VLM_DEVICE_ID\", source_env, overrides_env)\n", "\n", " def _id_list(name):\n", " raw = env_value_from_files(name, source_env, overrides_env).replace(\" \", \"\")\n", " return f\",{raw},\"\n", "\n", " fixed_shared = _id_list(\"FIXED_SHARED_DEVICE_IDS\")\n", " reserved = _id_list(\"RESERVED_DEVICE_IDS\")\n", "\n", " def _mode(is_remote, own_id, other_id, other_is_remote):\n", " if is_remote:\n", " return \"remote\"\n", " if not own_id:\n", " return \"local\"\n", " if f\",{own_id},\" in reserved or f\",{own_id},\" in fixed_shared:\n", " return \"local_shared\"\n", " if not other_is_remote and own_id == other_id:\n", " return \"local_shared\"\n", " return \"local\"\n", "\n", " llm_mode = _mode(llm_is_remote, llm_device_id, vlm_device_id, vlm_is_remote)\n", " vlm_mode = _mode(vlm_is_remote, vlm_device_id, llm_device_id, llm_is_remote)\n", "\n", " # L40S (48 GB) has no shared-LLM profile.\n", " if hardware_profile == \"L40S\":\n", " if llm_mode == \"local_shared\":\n", " raise RuntimeError(\"On L40S, the device ID for the LLM cannot be shared with other services\")\n", " if (llm_mode != \"remote\" and vlm_mode != \"remote\"\n", " and llm_device_id and llm_device_id == vlm_device_id):\n", " raise RuntimeError(\"On L40S, the LLM and VLM cannot share the same GPU\")\n", "\n", " if hardware_profile not in EDGE_HARDWARE_PROFILES:\n", " for label, mode, dev in ((\"LLM\", llm_mode, llm_device_id), (\"VLM\", vlm_mode, vlm_device_id)):\n", " if mode != \"remote\" and dev and f\",{dev},\" in reserved:\n", " raise RuntimeError(\n", " f\"Device ID {dev} is reserved and cannot be assigned to {label} for this profile\"\n", " )\n", "\n", " return {\n", " \"llm_mode\": llm_mode, \"vlm_mode\": vlm_mode,\n", " \"llm_device_id\": llm_device_id, \"vlm_device_id\": vlm_device_id,\n", " \"llm_base_url\": llm_base_url, \"vlm_base_url\": vlm_base_url,\n", " }\n", "\n", "\n", "# ============================================================\n", "# alerts MODE derivatives\n", "# ============================================================\n", "\n", "def apply_alerts_mode(generated_env):\n", " mode = read_env_value(generated_env, \"MODE\")\n", " if mode not in (\"2d_cv\", \"2d_vlm\"):\n", " return\n", "\n", " subtitle = \"Vision (Alerts - CV)\" if mode == \"2d_cv\" else \"Vision (Alerts - VLM)\"\n", " if _rewrite_matching(generated_env, \"NEXT_PUBLIC_APP_SUBTITLE=\",\n", " f'NEXT_PUBLIC_APP_SUBTITLE=\"{subtitle}\"\\n'):\n", " print(f\"[INFO] Set NEXT_PUBLIC_APP_SUBTITLE for alerts (MODE={mode} → {subtitle})\")\n", "\n", " # Verification keeps both Manage Alerts editors; real-time drops the CV one.\n", " realtime, verification = \"true\", (\"true\" if mode == \"2d_cv\" else \"false\")\n", " for name, value in (\n", " (\"NEXT_PUBLIC_ALERTS_TAB_MANAGE_ALERTS_SUB_TAB_ENABLE_REALTIME_ALERTS\", realtime),\n", " (\"NEXT_PUBLIC_ALERTS_TAB_MANAGE_ALERTS_SUB_TAB_ENABLE_CV_ALERTS_VERIFICATION\", verification),\n", " ):\n", " _rewrite_matching(generated_env, f\"{name}=\", f\"{name}={value}\\n\", append_if_missing=True)\n", " print(f\"[INFO] Set Alerts Manage tabs for MODE={mode} \"\n", " f\"(real-time={realtime}, CV verification={verification})\")\n", "\n", " # Real-time drives alerts from RT-VLM's Kafka events, so drop the\n", " # overrides.env value and let the rtvi-vlm compose default (true) apply.\n", " if mode == \"2d_vlm\":\n", " with open(generated_env, encoding=\"utf-8\") as f:\n", " lines = f.readlines()\n", " if any(l.startswith(\"RTVI_VLM_KAFKA_ENABLED=\") for l in lines):\n", " with open(generated_env, \"w\", encoding=\"utf-8\") as f:\n", " f.writelines(\"#\" + l if l.startswith(\"RTVI_VLM_KAFKA_ENABLED=\") else l\n", " for l in lines)\n", " print(\"[INFO] Commented out RTVI_VLM_KAFKA_ENABLED for alerts (MODE=2d_vlm)\")\n", "\n", "\n", "# ============================================================\n", "# generated.env\n", "# ============================================================\n", "\n", "def generate_env(profile, hardware_profile, host_ip, external_ip, modes, mode_env, ngc_api_key):\n", " source_env = profile_path(profile, \".env\")\n", " overrides_env = profile_path(profile, \"overrides.env\")\n", " generated_env = profile_path(profile, \"generated.env\")\n", "\n", " print(f\"[INFO] Generating environment file for profile '{profile}'...\")\n", " if not os.path.isfile(source_env):\n", " raise FileNotFoundError(f\"Source .env file not found: {source_env}\")\n", " if not os.path.isfile(overrides_env):\n", " raise FileNotFoundError(f\"Overrides env file not found: {overrides_env}\")\n", "\n", " shutil.copyfile(overrides_env, generated_env)\n", " print(f\"[INFO] Copied {overrides_env} to {generated_env}\")\n", "\n", " # A missing final newline would merge the first appended variable into the\n", " # last existing one.\n", " with open(generated_env, \"rb\") as f:\n", " content = f.read()\n", " if content and not content.endswith(b\"\\n\"):\n", " with open(generated_env, \"ab\") as f:\n", " f.write(b\"\\n\")\n", "\n", " # Compose-wide defaults, for variables the profile does not define.\n", " compose_defaults = os.path.join(DOCKER_DIR, \"services\", \"vios\", \"compose-defaults.env\")\n", " if os.path.isfile(compose_defaults):\n", " with open(compose_defaults, encoding=\"utf-8\") as f:\n", " default_lines = f.readlines()\n", " for line in default_lines:\n", " entry = line.rstrip(\"\\n\")\n", " if not entry.strip() or entry.lstrip().startswith(\"#\"):\n", " continue\n", " if not env_var_defined_in_files(entry.split(\"=\", 1)[0], source_env, generated_env):\n", " with open(generated_env, \"a\", encoding=\"utf-8\") as out:\n", " out.write(entry + \"\\n\")\n", "\n", " set_env_var(generated_env, \"VSS_APPS_DIR\", DOCKER_DIR)\n", " set_env_var(generated_env, \"VSS_DATA_DIR\", DATA_DIR)\n", " set_env_var(generated_env, \"HOST_IP\", host_ip)\n", " # Forwarded so the agent/RTVI services install the patent-encumbered codecs\n", " # at startup. Export INSTALL_PROPRIETARY_CODECS=false to keep them off.\n", " set_env_var(generated_env, \"INSTALL_PROPRIETARY_CODECS\",\n", " _host_env(\"INSTALL_PROPRIETARY_CODECS\", \"true\"))\n", " set_env_var(generated_env, \"VST_CONFIG_PATH\",\n", " os.path.join(DOCKER_DIR, \"services\", \"vios\", \"configs\"))\n", " if external_ip:\n", " set_env_var(generated_env, \"EXTERNAL_IP\", external_ip, mask=True)\n", "\n", " # ===== Brev secure links =====\n", " # Resolved from the environment context file, so a Brev domain migration\n", " # needs no change here. Section 7 already exported the result; re-resolving\n", " # keeps this function usable on its own.\n", " brev_env_id = _host_env(\"BREV_ENV_ID\")\n", " if brev_env_id:\n", " proxy_port = _host_env(\"PROXY_PORT\", \"7777\")\n", " secure_link_host, public_port, _ = require_brev_link_for_port(\n", " proxy_port,\n", " host_override=_host_env(\"BREV_PUBLIC_HOST\"),\n", " port_override=_host_env(\"BREV_PUBLIC_PORT\"),\n", " )\n", " print(f\"[INFO] Brev environment detected ({brev_env_id}). \"\n", " f\"Setting HAProxy ingress to {secure_link_host}:{public_port}...\")\n", " for name, value in (\n", " (\"BREV_ENV_ID\", brev_env_id),\n", " (\"BREV_LINK_DOMAIN\", secure_link_host.split(\".\", 1)[-1]),\n", " (\"HAPROXY_PORT\", proxy_port),\n", " (\"VSS_PUBLIC_HTTP_PROTOCOL\", \"https\"),\n", " (\"VSS_PUBLIC_WS_PROTOCOL\", \"wss\"),\n", " (\"VSS_PUBLIC_HOST\", secure_link_host),\n", " (\"VSS_PUBLIC_PORT\", public_port),\n", " ):\n", " set_env_var(generated_env, name, value)\n", "\n", " set_env_var(generated_env, \"NGC_CLI_API_KEY\", ngc_api_key, mask=True)\n", " set_env_var(generated_env, \"HARDWARE_PROFILE\", hardware_profile)\n", " if mode_env:\n", " set_env_var(generated_env, \"MODE\", mode_env)\n", "\n", " if profile == \"alerts\":\n", " apply_alerts_mode(generated_env)\n", " if read_env_value(generated_env, \"MODE\") == \"2d_vlm\":\n", " set_env_var(generated_env, \"COMPOSE_PROFILES\", \"${COMPOSE_PROFILES_VLM}\")\n", " set_env_var(generated_env, \"ALERT_AGENT_ALWAYS_ON\", \"true\")\n", " else:\n", " set_env_var(generated_env, \"ALERT_AGENT_ALWAYS_ON\", \"false\")\n", "\n", " llm_mode, vlm_mode = modes[\"llm_mode\"], modes[\"vlm_mode\"]\n", " llm_base_url, vlm_base_url = modes[\"llm_base_url\"], modes[\"vlm_base_url\"]\n", "\n", " # ===== LLM =====\n", " set_env_var(generated_env, \"LLM_MODE\", llm_mode)\n", " if llm_mode == \"remote\":\n", " set_env_var(generated_env, \"LLM_NAME\", remote_model_name(llm_base_url, \"llm\"))\n", " set_env_var(generated_env, \"LLM_NAME_SLUG\", \"none\")\n", " if hardware_profile in EDGE_HARDWARE_PROFILES:\n", " set_env_var(generated_env, \"LLM_DEVICE_ID\", \"0\")\n", " set_env_var(generated_env, \"VLM_DEVICE_ID\", \"0\")\n", " # Search's profile .env pins a 2-GPU layout (RT-CV on 0, RT-Embed on 1).\n", " # These boards have one GPU, and a device_ids entry of \"1\" is a hard\n", " # startup failure, so collapse every placement onto device 0.\n", " if profile == \"search\":\n", " set_env_var(generated_env, \"RT_CV_DEVICE_ID\", \"0\")\n", " set_env_var(generated_env, \"RT_EMBED_DEVICE_ID\", \"0\")\n", " set_env_var(generated_env, \"FIXED_SHARED_DEVICE_IDS\", \"0\")\n", " else:\n", " if llm_mode != \"remote\" and modes[\"llm_device_id\"]:\n", " set_env_var(generated_env, \"LLM_DEVICE_ID\", modes[\"llm_device_id\"])\n", " if vlm_mode != \"remote\" and modes[\"vlm_device_id\"]:\n", " set_env_var(generated_env, \"VLM_DEVICE_ID\", modes[\"vlm_device_id\"])\n", " if llm_base_url:\n", " set_env_var(generated_env, \"LLM_BASE_URL\", llm_base_url)\n", " if llm_mode == \"remote\":\n", " llm_model_type = env_value_from_files(\"LLM_MODEL_TYPE\", source_env, overrides_env)\n", " if llm_model_type:\n", " set_env_var(generated_env, \"LLM_MODEL_TYPE\", llm_model_type)\n", " if _host_env(\"NVIDIA_API_KEY\"):\n", " set_env_var(generated_env, \"NVIDIA_API_KEY\", _host_env(\"NVIDIA_API_KEY\"), mask=True)\n", "\n", " # ===== VLM =====\n", " set_env_var(generated_env, \"VLM_MODE\", vlm_mode)\n", " if vlm_mode == \"remote\":\n", " set_env_var(generated_env, \"VLM_NAME\", remote_model_name(vlm_base_url, \"vlm\"))\n", " set_env_var(generated_env, \"VLM_NAME_SLUG\", \"none\")\n", " set_env_var(generated_env, \"VLM_BASE_URL\", vlm_base_url)\n", " set_env_var(generated_env, \"RTVI_VLM_ENDPOINT\", f\"{vlm_base_url}/v1\")\n", " set_env_var(generated_env, \"RTVI_VLM_MODEL_PATH\", \"none\")\n", " vlm_model_type = env_value_from_files(\"VLM_MODEL_TYPE\", source_env, overrides_env)\n", " if vlm_model_type:\n", " set_env_var(generated_env, \"VLM_MODEL_TYPE\", vlm_model_type)\n", " if _host_env(\"OPENAI_API_KEY\"):\n", " set_env_var(generated_env, \"OPENAI_API_KEY\", _host_env(\"OPENAI_API_KEY\"), mask=True)\n", "\n", " if vlm_mode == \"remote\":\n", " # rtvi-vlm proxies to the endpoint, so use the conventional NIM port and\n", " # openai-compat instead of the integrated local checkpoint.\n", " set_env_var(generated_env, \"VLM_PORT\", \"30082\")\n", " set_env_var(generated_env, \"RTVI_VLM_MODEL_TO_USE\", \"openai-compat\")\n", " if profile == \"lvs\":\n", " # LVS defers frame sampling to RT-VLM.\n", " configured = env_value_from_files(\n", " \"RTVI_VLM_DEFAULT_NUM_FRAMES_PER_SECOND_OR_FIXED_FRAMES_CHUNK\",\n", " source_env, overrides_env)\n", " frames = _host_env(\n", " \"RTVI_VLM_DEFAULT_NUM_FRAMES_PER_SECOND_OR_FIXED_FRAMES_CHUNK\", configured)\n", " set_env_var(generated_env,\n", " \"RTVI_VLM_DEFAULT_NUM_FRAMES_PER_SECOND_OR_FIXED_FRAMES_CHUNK\",\n", " frames or \"5\")\n", " else:\n", " custom_weights = _host_env(\"VLM_CUSTOM_WEIGHTS\") or env_value_from_files(\n", " \"VLM_CUSTOM_WEIGHTS\", source_env, overrides_env)\n", " if custom_weights:\n", " print(f\"[INFO] Using VLM custom weights path: {custom_weights}\")\n", " set_env_var(generated_env, \"VLM_CUSTOM_WEIGHTS\", custom_weights)\n", "\n", " # ===== Edge and hardware tuning =====\n", " if profile == \"alerts\" and hardware_profile in EDGE_HARDWARE_PROFILES:\n", " set_env_var(generated_env, \"PERCEPTION_DOCKERFILE_PREFIX\", \"EDGE-\")\n", "\n", " set_env_var(generated_env, \"VLM_NAME_SLUG\", \"none\")\n", " if vlm_mode != \"remote\":\n", " # Internal URL so sibling containers need no host-published ports.\n", " set_env_var(generated_env, \"VLM_BASE_URL\", \"http://rtvi-vlm:8000\")\n", " set_env_var(generated_env, \"RTVI_VLLM_GPU_MEMORY_UTILIZATION\",\n", " rtvi_vllm_gpu_memory_utilization(hardware_profile, vlm_mode, profile))\n", " max_model_len = rtvi_vlm_max_model_len(hardware_profile)\n", " if max_model_len:\n", " set_env_var(generated_env, \"RTVI_VLM_MAX_MODEL_LEN\", max_model_len)\n", "\n", " if vlm_mode == \"local_shared\":\n", " shared_id = env_value_from_files(\"SHARED_LLM_VLM_DEVICE_ID\", source_env, overrides_env)\n", " set_env_var(generated_env, \"RT_VLM_DEVICE_ID\", shared_id or modes[\"vlm_device_id\"])\n", " elif vlm_mode == \"remote\":\n", " set_env_var(generated_env, \"RT_VLM_DEVICE_ID\", \"0\")\n", " else:\n", " set_env_var(generated_env, \"RT_VLM_DEVICE_ID\", modes[\"vlm_device_id\"])\n", "\n", " if hardware_profile == \"RTXPRO4500BW\" and vlm_mode != \"remote\":\n", " set_env_var(generated_env, \"RTVI_VLM_MODEL_PATH\",\n", " \"ngc:nim/nvidia/cosmos3-nano-reasoner:bf16-final\")\n", " set_env_var(generated_env, \"VLM_NAME\", \"nim_nvidia_cosmos3-nano-reasoner_bf16-final\")\n", "\n", " # Search on edge hardware: the VLM is always a remote endpoint, and the agent\n", " # calls it directly instead of proxying through RT-VLM, which frees the GPU\n", " # that container would otherwise hold for media preprocessing.\n", " if profile == \"search\" and hardware_profile in EDGE_HARDWARE_PROFILES:\n", " set_env_var(generated_env, \"VLM_MODEL_TYPE\", \"nim\")\n", " set_env_var(generated_env, \"VLM_AGENT_MEDIA_MODE\", \"remote\")\n", " compose_profiles = read_env_value(generated_env, \"COMPOSE_PROFILES\")\n", " kept = [p for p in compose_profiles.split(\",\") if p != \"rtvi-vlm\"]\n", " if len(kept) != len(compose_profiles.split(\",\")):\n", " set_env_var(generated_env, \"COMPOSE_PROFILES\", \",\".join(kept))\n", " print(\"[INFO] Removed rtvi-vlm from COMPOSE_PROFILES (search on \"\n", " f\"{hardware_profile} calls the remote VLM directly)\")\n", "\n", " if profile == \"base\" and vlm_mode != \"remote\":\n", " set_env_var(generated_env, \"VLM_MODEL_TYPE\", \"rtvi\")\n", " compose_profiles = read_env_value(generated_env, \"COMPOSE_PROFILES\")\n", " if \"rtvi-vlm\" not in compose_profiles.split(\",\"):\n", " set_env_var(generated_env, \"COMPOSE_PROFILES\", f\"{compose_profiles},rtvi-vlm\")\n", "\n", " # ===== SBSA managed images =====\n", " # containers.env applies VSS_CONTAINER_TAG_SUFFIX during Compose\n", " # interpolation only, so a service that reads a tag as plain configuration\n", " # never sees it. Write the four suffixed keys here too, preferring an\n", " # explicit tag (host env, then an uncommented env-file line, then the\n", " # profile's commented SBSA pin) over the derived one.\n", " if use_sbsa_images(hardware_profile):\n", " base_tag = (_host_env(\"VSS_CONTAINER_TAG\")\n", " or env_value_from_files(\"VSS_CONTAINER_TAG\", source_env, generated_env))\n", " for key in SBSA_TAG_KEYS:\n", " value = _host_env(key) or env_value_from_files(key, source_env, generated_env)\n", " if not value and not base_tag:\n", " # No shared channel selected, so the profile's pinned SBSA build\n", " # is the only meaningful tag. A selected channel always wins.\n", " value = commented_sbsa_value(key, source_env, generated_env)\n", " set_env_var(generated_env, key, value or f\"{base_tag or 'develop-latest'}-sbsa\")\n", "\n", " print(f\"[INFO] Generated environment file: {generated_env}\")\n", " return generated_env\n", "\n", "\n", "# ============================================================\n", "# directories, kernel settings, Compose\n", "# ============================================================\n", "\n", "def create_directories(profile):\n", " print(\"[INFO] Creating data directories...\")\n", " for rel in (\"data_log/analytics_cache\", \"data_log/calibration_toolkit\",\n", " \"data_log/elastic/data\", \"data_log/elastic/logs\", \"data_log/kafka\",\n", " \"data_log/redis/data\", \"data_log/redis/log\",\n", " \"agent_eval/dataset\", \"agent_eval/results\"):\n", " os.makedirs(os.path.join(DATA_DIR, rel), exist_ok=True)\n", "\n", " if profile == \"alerts\":\n", " print(\"[INFO] Creating alerts-specific directories...\")\n", " os.makedirs(os.path.join(DATA_DIR, \"data_log\", \"vss_video_analytics_api\"), exist_ok=True)\n", " os.makedirs(os.path.join(DATA_DIR, \"videos\", \"dev-profile-alerts\"), exist_ok=True)\n", " engines = os.path.join(DOCKER_DIR, \"engines\")\n", " os.makedirs(os.path.join(engines, \"gdino\"), exist_ok=True)\n", " os.makedirs(os.path.join(engines, \"rtdetr-its\"), exist_ok=True)\n", " subprocess.run([\"chmod\", \"-R\", \"777\", engines], check=False)\n", " print(\"[INFO] Alerts model download runs in ds-start.sh phase 0 (perception).\")\n", "\n", " if profile == \"search\":\n", " print(\"[INFO] Creating search-specific directories...\")\n", " os.makedirs(os.path.join(DATA_DIR, \"data_log\", \"vss_video_analytics_api\"), exist_ok=True)\n", " print(\"[INFO] Search model download runs in ds-start.sh phase 0 (perception).\")\n", "\n", " for rel in (\"data_log\", \"agent_eval\"):\n", " print(f\"[INFO] Setting permissions on {rel} directory...\")\n", " subprocess.run([\"chmod\", \"-R\", \"777\", os.path.join(DATA_DIR, rel)], check=False)\n", "\n", "\n", "def apply_kernel_settings():\n", " \"\"\"IPv6 off plus larger TCP buffers, persisted across reboots.\"\"\"\n", " print(\"[INFO] Applying VSS Linux kernel settings...\")\n", " settings = \"\".join(f\"{line}\\n\" for line in (\n", " \"net.ipv6.conf.all.disable_ipv6 = 1\",\n", " \"net.ipv6.conf.default.disable_ipv6 = 1\",\n", " \"net.ipv6.conf.lo.disable_ipv6 = 1\",\n", " \"net.core.rmem_max = 5242880\",\n", " \"net.core.wmem_max = 5242880\",\n", " \"net.ipv4.tcp_rmem = 4096 87380 16777216\",\n", " \"net.ipv4.tcp_wmem = 4096 65536 16777216\",\n", " ))\n", " sudo = _sudo_prefix()\n", " subprocess.run(sudo + [\"mkdir\", \"-p\", \"/etc/sysctl.d\"], check=True)\n", " subprocess.run(sudo + [\"bash\", \"-c\", \"cat > /etc/sysctl.d/99-vss.conf\"],\n", " input=settings, text=True, check=True)\n", " subprocess.run(sudo + [\"sysctl\", \"--system\"], capture_output=True, check=False)\n", "\n", "\n", "def compose_base_args(profile):\n", " \"\"\"Compose invocation shared by up, stop and image resolution. Paths are\n", " relative to deploy/docker, so callers must run from DOCKER_DIR.\"\"\"\n", " # -f disables Compose's default file discovery, so name the base file even\n", " # when there is no overlay.\n", " args = [\"docker\", \"compose\", \"-f\", \"compose.yml\"]\n", " if _host_env(\"VSS_VIOS_PREBAKE_PACKAGES\", \"false\") == \"true\":\n", " args += [\"-f\", \"services/vios/streamprocessing/docker-compose.prebaked.yaml\"]\n", " args += [\n", " \"--env-file\", \"containers.env\",\n", " \"--env-file\", f\"developer-profiles/dev-profile-{profile}/.env\",\n", " \"--env-file\", f\"developer-profiles/dev-profile-{profile}/generated.env\",\n", " ]\n", " return args\n", "\n", "\n", "def compose_env(hardware_profile):\n", " \"\"\"Process environment for every Compose call.\n", "\n", " Compose gives the process environment precedence over all --env-file\n", " arguments, so the SBSA image suffix and the managed registry/tag pair have\n", " to be set here — writing them to generated.env would be silently overridden\n", " by containers.env. Per-image overrides in generated.env still apply, which\n", " is where generate_env() pins the four suffixed service tags.\n", " \"\"\"\n", " env = dict(os.environ)\n", "\n", " if use_sbsa_images(hardware_profile):\n", " env[\"VSS_CONTAINER_TAG_SUFFIX\"] = \"-sbsa\"\n", "\n", " # containers.env defines its values with shell parameter expansion, so let\n", " # bash evaluate it rather than parsing the lines.\n", " resolve = ('set -a; . \"$1\"; set +a; '\n", " 'printf \"%s\\\\n%s\\\\n\" \"${VSS_CONTAINER_REGISTRY}\" \"${VSS_CONTAINER_TAG}\"')\n", " result = subprocess.run(\n", " [\"bash\", \"-c\", resolve, \"bash\", os.path.join(DOCKER_DIR, \"containers.env\")],\n", " capture_output=True, text=True, env=env)\n", " if result.returncode != 0:\n", " raise RuntimeError(f\"Could not resolve containers.env:\\n{result.stderr}\")\n", " registry, tag = (result.stdout.splitlines() + [\"\", \"\"])[:2]\n", " env[\"VSS_CONTAINER_REGISTRY\"] = registry\n", " env[\"VSS_CONTAINER_TAG\"] = tag\n", " return env\n", "\n", "\n", "def show_resolved_images(profile, hardware_profile):\n", " env = compose_env(hardware_profile)\n", " if env.get(\"VSS_CONTAINER_TAG_SUFFIX\"):\n", " print(f\"[INFO] Managed container tag suffix: {env['VSS_CONTAINER_TAG_SUFFIX']}\")\n", " print(f\"[INFO] Managed container registry: {env['VSS_CONTAINER_REGISTRY']}\")\n", " print(f\"[INFO] Managed container tag: {env['VSS_CONTAINER_TAG']}\")\n", " print(\"[INFO] Resolved compose images:\")\n", " result = subprocess.run(compose_base_args(profile) + [\"config\", \"--images\"],\n", " cwd=DOCKER_DIR, capture_output=True, text=True, env=env)\n", " if result.returncode != 0:\n", " raise RuntimeError(f\"docker compose config failed:\\n{result.stderr}\")\n", " for image in sorted({l.strip() for l in result.stdout.splitlines() if l.strip()}):\n", " print(f\" {image}\")\n", "\n", "\n", "def docker_login_nvcr(ngc_api_key):\n", " print(\"[INFO] Logging into nvcr.io...\")\n", " result = subprocess.run(\n", " [\"docker\", \"login\", \"--username\", \"$oauthtoken\", \"--password\", ngc_api_key, \"nvcr.io\"],\n", " capture_output=True, text=True)\n", " if result.returncode != 0:\n", " raise RuntimeError(f\"docker login to nvcr.io failed:\\n{result.stderr}\")\n", "\n", "\n", "# ============================================================\n", "# teardown\n", "# ============================================================\n", "\n", "def compose_project_names():\n", " \"\"\"Every project name that might have a running deployment.\"\"\"\n", " names = []\n", " if _host_env(\"COMPOSE_PROJECT_NAME\"):\n", " names.append(_host_env(\"COMPOSE_PROJECT_NAME\"))\n", " for prof in ALL_PROFILES:\n", " generated = profile_path(prof, \"generated.env\")\n", " if os.path.isfile(generated):\n", " name = env_value_from_files(\n", " \"COMPOSE_PROJECT_NAME\",\n", " profile_path(prof, \".env\"), profile_path(prof, \"overrides.env\"), generated,\n", " ) or \"vss\"\n", " if name not in names:\n", " names.append(name)\n", " if not names:\n", " for prof in ALL_PROFILES:\n", " name = env_value_from_files(\n", " \"COMPOSE_PROJECT_NAME\",\n", " profile_path(prof, \".env\"), profile_path(prof, \"overrides.env\"))\n", " if name:\n", " names.append(name)\n", " break\n", " return names or [\"vss\"]\n", "\n", "\n", "def teardown_deployment():\n", " for name in compose_project_names():\n", " print(f\"[INFO] Bringing down docker compose project '{name}' (with volumes)...\")\n", " result = subprocess.run([\"docker\", \"compose\", \"-p\", name, \"down\", \"-v\", \"--remove-orphans\"],\n", " cwd=SCRIPT_DIR, capture_output=True, text=True)\n", " for line in (result.stdout + result.stderr).splitlines():\n", " if line.strip() and \"is not set\" not in line:\n", " print(f\" {line}\")\n", "\n", " print(\"[INFO] Cleaning up generated.env files from all profiles...\")\n", " for prof in ALL_PROFILES:\n", " generated = profile_path(prof, \"generated.env\")\n", " if os.path.isfile(generated):\n", " os.remove(generated)\n", " print(f\"[INFO] Deleted {generated}\")\n", "\n", " print(\"[INFO] Removing dangling docker volumes...\")\n", " dangling = subprocess.run([\"docker\", \"volume\", \"ls\", \"-q\", \"-f\", \"dangling=true\"],\n", " capture_output=True, text=True).stdout.split()\n", " if dangling:\n", " subprocess.run([\"docker\", \"volume\", \"rm\"] + dangling, capture_output=True, text=True)\n", " print(f\"[INFO] Removed {len(dangling)} dangling volume(s)\")\n", " else:\n", " print(\"[INFO] No dangling volumes to remove\")\n", "\n", " sudo = _sudo_prefix()\n", "\n", " # sdrc artifacts are bind-mounted and written by containers as root.\n", " sdrc_dir = os.path.join(DOCKER_DIR, \"services\", \"infra\", \"sdrc\")\n", " print(f\"[INFO] Cleaning up sdrc runtime artifacts in {sdrc_dir}...\")\n", " for artifact in (os.path.join(sdrc_dir, \"log\"), os.path.join(sdrc_dir, \".wdm-env\")):\n", " if os.path.isdir(artifact):\n", " subprocess.run(sudo + [\"rm\", \"-rf\", artifact], check=False)\n", " print(f\"[INFO] Deleted {artifact}\")\n", "\n", " # Every rendered sdrc config has a *.tmpl sibling; drop the rendered copy so\n", " # the next run regenerates it from the template.\n", " seen = set()\n", " for pattern in (os.path.join(DOCKER_DIR, \"**\", \"sdrc\", \"configs\", \"*.tmpl\"),\n", " os.path.join(DOCKER_DIR, \"**\", \"sdrc\", \"*\", \"configs\", \"*.tmpl\")):\n", " for tmpl in glob.glob(pattern, recursive=True):\n", " rendered = tmpl[: -len(\".tmpl\")]\n", " if rendered in seen or not os.path.isfile(rendered):\n", " continue\n", " seen.add(rendered)\n", " subprocess.run(sudo + [\"rm\", \"-f\", rendered], check=False)\n", " print(f\"[INFO] Deleted rendered sdrc config: {rendered}\")\n", "\n", " print(f\"[INFO] Deleting data directory: {DATA_DIR}...\")\n", " if os.path.isdir(DATA_DIR):\n", " subprocess.run(sudo + [\"rm\", \"-rf\", DATA_DIR], check=False)\n", " print(\"[INFO] Data directory deleted\")\n", " else:\n", " print(\"[INFO] Data directory does not exist, skipping\")\n", "\n", " print(\"[INFO] State down completed\")\n", "\n", "\n", "# ============================================================\n", "# orchestration\n", "# ============================================================\n", "\n", "def prepare_deployment(profile, hardware_profile, host_ip, external_ip,\n", " use_remote_llm, use_remote_vlm,\n", " llm_endpoint_url, vlm_endpoint_url, alerts_mode, ngc_api_key):\n", " \"\"\"Everything that has to happen before `docker compose up`.\"\"\"\n", " if hardware_profile in EDGE_HARDWARE_PROFILES:\n", " if profile not in (\"base\", \"alerts\", \"search\"):\n", " raise RuntimeError(\n", " f\"Hardware profile '{hardware_profile}' is only valid for profile base, \"\n", " f\"alerts or search, not '{profile}'\"\n", " )\n", " # One GPU, and search's perception pipeline already owns it: a local VLM\n", " # never fits, and two vLLM engines cannot share a single GPU at all.\n", " if profile == \"search\":\n", " if not (use_remote_llm and llm_endpoint_url):\n", " raise RuntimeError(\n", " f\"Search on {hardware_profile} cannot host a local LLM. Set \"\n", " \"USE_REMOTE_LLM=True with REMOTE_LLM_ENDPOINT_URL.\"\n", " )\n", " if not (use_remote_vlm and vlm_endpoint_url):\n", " raise RuntimeError(\n", " f\"Search on {hardware_profile} cannot host a local VLM. Set \"\n", " \"USE_REMOTE_VLM=True with REMOTE_VLM_ENDPOINT_URL.\"\n", " )\n", "\n", " verify_hardware_profile(hardware_profile)\n", " ensure_cache_cleaner(hardware_profile)\n", "\n", " mode_env = MODE_ENV_VALUES.get(alerts_mode, \"\") if profile == \"alerts\" else \"\"\n", " modes = derive_modes(profile, hardware_profile, use_remote_llm, use_remote_vlm,\n", " llm_endpoint_url, vlm_endpoint_url)\n", " print(f\"[INFO] LLM mode: {modes['llm_mode']} VLM mode: {modes['vlm_mode']}\")\n", "\n", " # Search brings up a local RT-VLM next to RT-CV and RT-Embed, so the profile\n", " # layout needs two GPUs. Without this the run fails much later, during\n", " # Compose or model startup.\n", " if profile == \"search\" and _host_env(\"BREV_ENV_ID\") and modes[\"vlm_mode\"] != \"remote\":\n", " gpu_count = len(detect_gpu_names())\n", " if 0 < gpu_count < 2:\n", " raise RuntimeError(\n", " f\"Search deploys a local RT-VLM and needs at least 2 GPUs, but this Brev \"\n", " f\"environment has {gpu_count} GPU(s). Set USE_REMOTE_VLM=True with \"\n", " \"REMOTE_VLM_ENDPOINT_URL to run search here.\"\n", " )\n", "\n", " # Deploying always starts from a clean slate.\n", " teardown_deployment()\n", "\n", " generate_env(profile, hardware_profile, host_ip, external_ip, modes, mode_env, ngc_api_key)\n", " create_directories(profile)\n", " apply_kernel_settings()\n", " show_resolved_images(profile, hardware_profile)\n", " docker_login_nvcr(ngc_api_key)\n", " return modes\n", "\n", "\n", "print(\"Deployment helpers loaded.\")\n", "print(f\" Deployment directory: {DOCKER_DIR}\")\n", "print(f\" Data directory: {DATA_DIR}\")" ] }, { "cell_type": "markdown", "id": "96774760", "metadata": {}, "source": [ "## 8. Deploy Profile\n", "\n", "This is the main deployment cell. It uses the helpers from Section 7.1 with the configuration from Section 1 and the network settings from Section 7, then calls `docker compose up` directly.\n", "\n", "This will:\n", "- **Tear down any existing deployment first**, including volumes and the data directory\n", "- Generate `generated.env` for the selected profile and hardware\n", "- Create the data directories and apply the VSS kernel settings\n", "- Download required models from NGC\n", "- Pull and build Docker images\n", "- Start all containers\n", "\n", "> **Deploying wipes existing data.** Every run starts from a clean slate, so uploaded videos, Elasticsearch indices and Kafka data from a previous deployment are removed. Use Section 13 to stop containers without losing data.\n", "\n", "**This cell takes 10-30 minutes** depending on network speed and whether images are cached.\n", "\n", "The cell shows a live progress summary. Full output is captured to `~/deploy_vss.log` — if something fails, check that file for details." ] }, { "cell_type": "code", "execution_count": null, "id": "e8b211e9", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, os, re, time, datetime, io, contextlib\n", "from IPython.display import display, clear_output, HTML\n", "\n", "LOG_FILE = os.path.expanduser(\"~/deploy_vss.log\")\n", "\n", "# Validate the hardware, tear down any previous deployment, write generated.env,\n", "# create data directories, apply kernel settings and log into nvcr.io.\n", "# This output is verbose and the live status display below clears the cell, so\n", "# capture it into the log file and surface a one-line summary.\n", "prep_buffer = io.StringIO()\n", "try:\n", " with contextlib.redirect_stdout(prep_buffer):\n", " prepare_deployment(\n", " profile=PROFILE,\n", " hardware_profile=HARDWARE_PROFILE,\n", " host_ip=HOST_IP,\n", " external_ip=EXTERNAL_IP if (EXTERNAL_IP and EXTERNAL_IP != HOST_IP) else \"\",\n", " use_remote_llm=USE_REMOTE_LLM,\n", " use_remote_vlm=USE_REMOTE_VLM,\n", " llm_endpoint_url=REMOTE_LLM_ENDPOINT_URL,\n", " vlm_endpoint_url=REMOTE_VLM_ENDPOINT_URL,\n", " alerts_mode=ALERTS_MODE,\n", " ngc_api_key=NGC_CLI_API_KEY,\n", " )\n", "finally:\n", " with open(LOG_FILE, \"w\") as log:\n", " log.write(prep_buffer.getvalue())\n", "\n", "_vars_set = sum(1 for l in prep_buffer.getvalue().splitlines() if l.startswith(\"[INFO] Set \"))\n", "print(f\"Environment prepared: {_vars_set} variables written to \"\n", " f\"developer-profiles/dev-profile-{PROFILE}/generated.env\")\n", "\n", "cmd = compose_base_args(PROFILE) + [\n", " \"up\", \"--detach\", \"--pull\", \"always\", \"--force-recreate\", \"--build\",\n", "]\n", "\n", "# compose_env() carries the SBSA image suffix and the managed registry/tag pair,\n", "# which Compose only honours from the process environment.\n", "deploy_env = {**compose_env(HARDWARE_PROFILE), \"NGC_CLI_API_KEY\": NGC_CLI_API_KEY}\n", "if USE_REMOTE_LLM:\n", " deploy_env[\"LLM_ENDPOINT_URL\"] = REMOTE_LLM_ENDPOINT_URL\n", "if USE_REMOTE_VLM:\n", " deploy_env[\"VLM_ENDPOINT_URL\"] = REMOTE_VLM_ENDPOINT_URL\n", "\n", "# Print the command\n", "display_cmd = \" \".join(cmd)\n", "print(f\"Command: {display_cmd}\")\n", "print(f\"Full log: {LOG_FILE}\\n\")\n", "\n", "# --- Phase detection and filtering ---\n", "\n", "# Lines matching these patterns are noise — suppress them\n", "SUPPRESS_PATTERNS = [\n", " re.compile(r\"^(\\s*[\\u2800-\\u28FF]|⠋|⠙|⠹|⠸|⠴|⠦|⠧|⠏)\"), # NGC spinner frames\n", " re.compile(r\"^\\s*M\\u2026|^\\s*$\"), # Truncated NGC progress fragments\n", " re.compile(r\"^\\s*#\\d+\\s+sha256:\"), # Docker buildkit layer download/extract progress\n", " re.compile(r\"^\\s*#\\d+\\s+extracting\\s\"), # Docker buildkit layer extraction\n", " re.compile(r\"^\\s*#\\d+\\s+\\.\\.\\.\"), # Docker buildkit continuation\n", " re.compile(r\"^time=.*level=warning\"), # Docker compose unset variable warnings\n", " re.compile(r\"^WARNING! Using --password\"), # Docker login warning\n", " re.compile(r\"Login Succeeded\"), # Docker login success (we print our own)\n", " re.compile(r\"^\\s*Getting files to download\"), # NGC download preamble\n", " re.compile(r\"^\\s*━\"), # NGC progress bars\n", " re.compile(r\"^\\s*[0-9a-f]{12}\\s+(Downloading|Extracting|Waiting|Verifying|Pull complete)\"), # Docker layer progress\n", "]\n", "\n", "# Lines matching these indicate phase transitions — always show.\n", "# Environment preparation runs in-process above, so these track Compose's own\n", "# lifecycle output.\n", "PHASE_PATTERNS = [\n", " (re.compile(r\"^\\s*Network\\s+\\S+\\s+Creat\"), \"Creating networks\"),\n", " (re.compile(r\"^\\s*Volume\\s+\\S+\\s+Creat\"), \"Creating volumes\"),\n", "]\n", "\n", "# Image pull tracking — service-level \"Pulling \" / \"Pulled \"\n", "PULLING_RE = re.compile(r\"^\\s*Pulling\\s+(\\S+)\")\n", "PULLED_RE = re.compile(r\"^\\s*Pulled\\s+(\\S+)\")\n", "\n", "# Image build tracking — \"#N [service-name step/total] COMMAND\" / \"#N DONE Ns\"\n", "BUILD_STEP_RE = re.compile(r\"^\\s*#\\d+\\s+\\[(\\S+)\\s+(\\d+/\\d+)\\]\")\n", "BUILD_DONE_RE = re.compile(r\"^\\s*#\\d+\\s+DONE\\s+[\\d.]+s\")\n", "IMAGE_BUILT_RE = re.compile(r\"^\\s*Image\\s+(\\S+)\\s+Built\")\n", "\n", "# Container lifecycle — track creating/starting/healthy\n", "CONTAINER_RE = re.compile(r\"^\\s*Container\\s+(\\S+)\\s+(Creating|Created|Starting|Started|Healthy|Waiting|Exited.*)\")\n", "\n", "phases_seen = [\"Prepared environment and logged into nvcr.io\"]\n", "images_pulling = set() # images we've seen \"Pulling\" for\n", "images_pulled = set() # images we've seen \"Pulled\" for\n", "builds = {} # service -> \"step/total\" for active builds\n", "builds_done = set() # services that finished building\n", "containers = {}\n", "errors = []\n", "start_time = time.time()\n", "\n", "def elapsed():\n", " s = int(time.time() - start_time)\n", " return f\"{s // 60}m {s % 60:02d}s\"\n", "\n", "def print_status():\n", " clear_output(wait=True)\n", " print(f\"Command: {display_cmd}\")\n", " print(f\"Full log: {LOG_FILE}\\n\")\n", "\n", " # Phases\n", " for p in phases_seen:\n", " print(f\" [done] {p}\")\n", " if phases_seen:\n", " print()\n", "\n", " # Image pull progress\n", " if images_pulling:\n", " total = len(images_pulling)\n", " done = len(images_pulled)\n", " if done < total:\n", " still_pulling = sorted(images_pulling - images_pulled)\n", " print(f\" Pulling images: {done}/{total} complete ({elapsed()})\")\n", " for img in still_pulling:\n", " print(f\" {img:<45s} pulling...\")\n", " print()\n", " else:\n", " print(f\" Pulling images: {total}/{total} complete\\n\")\n", "\n", " # Image build progress\n", " active_builds = {s: step for s, step in builds.items() if s not in builds_done}\n", " if builds:\n", " done_count = len(builds_done)\n", " total_count = len(builds)\n", " if active_builds:\n", " print(f\" Building images: {done_count}/{total_count} complete ({elapsed()})\")\n", " for svc, step in sorted(active_builds.items()):\n", " print(f\" {svc:<45s} [{step}]\")\n", " print()\n", " else:\n", " print(f\" Building images: {total_count}/{total_count} complete\\n\")\n", "\n", " # Container summary\n", " if containers:\n", " healthy = sum(1 for s in containers.values() if s == \"Healthy\")\n", " started = sum(1 for s in containers.values() if s in (\"Started\", \"Healthy\"))\n", " total = len(containers)\n", " print(f\" Containers: {started}/{total} started, {healthy}/{total} healthy ({elapsed()})\")\n", "\n", " # Show containers that aren't healthy yet\n", " pending = {n: s for n, s in containers.items() if s != \"Healthy\" and s not in (\"Exited\",)}\n", " if pending:\n", " # Only show non-trivial pending (skip init containers that exited)\n", " waiting = {n: s for n, s in pending.items() if \"Exited\" not in s}\n", " if waiting:\n", " print()\n", " for name, status in sorted(waiting.items()):\n", " print(f\" {name:<45s} {status}\")\n", " print()\n", "\n", " # Errors\n", " for e in errors:\n", " print(f\" ERROR: {e}\")\n", "\n", "# Run the process. compose_base_args() uses paths relative to deploy/docker,\n", "# so Compose must run from there.\n", "process = subprocess.Popen(\n", " cmd,\n", " stdout=subprocess.PIPE,\n", " stderr=subprocess.STDOUT,\n", " text=True,\n", " bufsize=1,\n", " cwd=DOCKER_DIR,\n", " env=deploy_env\n", ")\n", "\n", "last_refresh = 0\n", "with open(LOG_FILE, \"a\") as log:\n", " for line in process.stdout:\n", " log.write(line)\n", " log.flush()\n", " stripped = line.rstrip()\n", "\n", " # Capture errors\n", " if \"[ERROR]\" in stripped:\n", " errors.append(stripped)\n", " print_status()\n", " continue\n", "\n", " # Track image pulls (before suppression check)\n", " m_pulling = PULLING_RE.match(stripped)\n", " if m_pulling:\n", " images_pulling.add(m_pulling.group(1))\n", " now = time.time()\n", " if now - last_refresh > 2:\n", " last_refresh = now\n", " print_status()\n", " continue\n", "\n", " m_pulled = PULLED_RE.match(stripped)\n", " if m_pulled:\n", " images_pulled.add(m_pulled.group(1))\n", " now = time.time()\n", " if now - last_refresh > 2:\n", " last_refresh = now\n", " print_status()\n", " continue\n", "\n", " # Track image builds\n", " m_build = BUILD_STEP_RE.match(stripped)\n", " if m_build:\n", " svc, step = m_build.group(1), m_build.group(2)\n", " builds[svc] = step\n", " now = time.time()\n", " if now - last_refresh > 2:\n", " last_refresh = now\n", " print_status()\n", " continue\n", "\n", " m_built = IMAGE_BUILT_RE.match(stripped)\n", " if m_built:\n", " svc = m_built.group(1)\n", " builds_done.add(svc)\n", " now = time.time()\n", " if now - last_refresh > 2:\n", " last_refresh = now\n", " print_status()\n", " continue\n", "\n", " # Suppress noise\n", " if any(p.search(stripped) for p in SUPPRESS_PATTERNS):\n", " continue\n", "\n", " # Detect phase transitions\n", " for pattern, label in PHASE_PATTERNS:\n", " if pattern.search(stripped):\n", " if label not in phases_seen:\n", " phases_seen.append(label)\n", " print_status()\n", " break\n", "\n", " # Track container lifecycle\n", " m = CONTAINER_RE.match(stripped)\n", " if m:\n", " name, status = m.group(1), m.group(2)\n", " # Normalize \"Exited (0) ...\" to \"Exited\"\n", " if status.startswith(\"Exited\"):\n", " status = \"Exited\"\n", " containers[name] = status\n", " # Refresh display at most every 2 seconds to avoid flicker\n", " now = time.time()\n", " if now - last_refresh > 2:\n", " last_refresh = now\n", " print_status()\n", "\n", "process.wait()\n", "\n", "# Final status\n", "print_status()\n", "print(\"=\" * 50)\n", "if process.returncode == 0 and not errors:\n", " print(f\"Deployment complete in {elapsed()}.\")\n", "else:\n", " print(f\"\\nDeployment FAILED (exit code {process.returncode}).\")\n", " if errors:\n", " print(f\"\\n{len(errors)} error(s) found — see above.\")\n", " print(f\"\\nFull log: {LOG_FILE}\")\n", " print(f\" View with: cat {LOG_FILE}\")" ] }, { "cell_type": "markdown", "id": "f6efa616", "metadata": {}, "source": [ "## 9. Verify Deployment\n", "\n", "Check that all containers are running and core services are healthy. The health checks poll with retries since some services take a few minutes to fully start." ] }, { "cell_type": "code", "execution_count": null, "id": "295f8232", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, time, urllib.request, urllib.error, os\n", "\n", "# Show running containers\n", "print(\"=== Running Containers ===\")\n", "subprocess.run([\"docker\", \"ps\", \"--format\", \"table {{.Names}}\\t{{.Status}}\\t{{.Ports}}\"])\n", "print()\n", "\n", "# Determine proxy port (HAProxy, container: vss-haproxy-ingress)\n", "proxy_port = os.environ.get(\"PROXY_PORT\", \"7777\")\n", "\n", "# Health check endpoints by profile\n", "# Note: HAProxy (vss-haproxy-ingress) has no /health endpoint. The root path \"/\"\n", "# routes to the UI backend, so checking \"/\" validates that HAProxy is up and proxying.\n", "checks = [\n", " (\"HAProxy\", f\"http://localhost:{proxy_port}/\"),\n", " (\"Agent\", \"http://localhost:8000/health\"),\n", " (\"VST\", \"http://localhost:30888/vst/api/v1/sensor/list\"),\n", " (\"UI\", \"http://localhost:3000\"),\n", "]\n", "if PROFILE in (\"search\", \"alerts\", \"lvs\"):\n", " checks += [\n", " (\"Elasticsearch\", \"http://localhost:9200\"),\n", " (\"Kibana\", \"http://localhost:5601/kibana/api/status\"),\n", " ]\n", "if PROFILE == \"alerts\":\n", " checks.append((\"Video Analytics API\", \"http://localhost:8081/livez\"))\n", "\n", "# Poll with retries\n", "MAX_RETRIES = 30\n", "RETRY_INTERVAL = 10\n", "results = {}\n", "\n", "print(f\"=== Health Checks (up to {MAX_RETRIES * RETRY_INTERVAL}s) ===\")\n", "pending = list(checks)\n", "\n", "for attempt in range(1, MAX_RETRIES + 1):\n", " still_pending = []\n", " for name, url in pending:\n", " try:\n", " req = urllib.request.urlopen(url, timeout=5)\n", " results[name] = f\"OK ({req.getcode()})\"\n", " except Exception:\n", " still_pending.append((name, url))\n", " pending = still_pending\n", " if not pending:\n", " break\n", " waiting = \", \".join(n for n, _ in pending)\n", " print(f\" [{attempt}/{MAX_RETRIES}] Waiting for: {waiting}\")\n", " time.sleep(RETRY_INTERVAL)\n", "\n", "for name, url in pending:\n", " results[name] = \"FAILED\"\n", "\n", "print()\n", "all_ok = True\n", "for name, status in results.items():\n", " marker = \"OK\" if \"OK\" in status else \"FAIL\"\n", " if marker == \"FAIL\":\n", " all_ok = False\n", " print(f\" {name:.<30s} {status}\")\n", "\n", "# Check perception container status (no HTTP health endpoint — DeepStream pipeline)\n", "if PROFILE == \"search\":\n", " print()\n", " r = subprocess.run(\n", " [\"docker\", \"ps\", \"--filter\", \"name=vss-rtvi-cv\", \"--format\", \"{{.Names}}: {{.Status}}\"],\n", " capture_output=True, text=True\n", " )\n", " if r.stdout.strip():\n", " print(\" Perception containers:\")\n", " for line in r.stdout.strip().splitlines():\n", " print(f\" {line}\")\n", " else:\n", " print(\" WARNING: No vss-rtvi-cv containers found (required for search profile).\")\n", " all_ok = False\n", "\n", "print()\n", "if all_ok:\n", " print(\"All services healthy.\")\n", "else:\n", " print(\"Some services failed to start. Check container logs:\")\n", " compose_project_name = os.environ.get(\"COMPOSE_PROJECT_NAME\", \"vss\")\n", " print(f\" docker compose -p {compose_project_name} logs \")" ] }, { "cell_type": "markdown", "id": "4d21329b", "metadata": {}, "source": [ "## 10. Access the UI\n", "\n", "Once deployment is verified, open the VSS UI in your browser. Accessing the front-end will open up the chat interface where you can interact with the agent and should look like this:\n", "\n", "![VSS UI Chat Interface](images/vss_ui_home_page.png)\n", "\n", "Run the cell below to generate your VSS UI URL.\n", "\n", "**On Brev:** All browser-facing traffic routes through the HAProxy ingress on a single port (default 7777). Create **one** Brev secure link for port 7777 in the dashboard for the primary UI and service proxy. Kibana and Phoenix are accessible via proxy paths (`/kibana` and `/phoenix`) so no separate secure links are needed for those.\n", "\n", "**On other cloud providers:** Depending on your CSP's firewall and security group configuration, you may need to expose or forward ports to access the UI and other services from your browser. The following ports are used by VSS:\n", "\n", "| Port | Service | Profiles | Brev Secure Link |\n", "|------|---------|----------|------------------|\n", "| 7777 | HAProxy ingress (consolidates UI, Agent, VST) | all | Required (primary) |\n", "| 3000 | VSS UI | all | Not needed (behind proxy) |\n", "| 8000 | VSS Agent API | all | Not needed (behind proxy) |\n", "| 30888 | VST (Video Storage Toolkit) | all | Not needed (behind proxy) |\n", "| 5601 | Kibana | search, alerts, lvs | Not needed (behind proxy at `/kibana`) |\n", "| 6006 | Phoenix (LLM tracing/observability) | all | Not needed (behind proxy at `/phoenix`) |\n", "| 9200 | Elasticsearch | search, alerts, lvs | Not needed |\n", "| 8081 | Video Analytics API | alerts | Not needed |\n", "| 31000 | nvstreamer (WebRTC live view) | search, alerts, lvs | Required for live camera view |\n", "| 8554 | RTSP (if using test stream) | alerts | Not needed |\n", "\n", "**Brev summary:** For the **base** profile, create 1 secure link (port 7777). For **search**, **alerts**, and **lvs** profiles, create secure links for ports 7777 and 31000. Kibana and Phoenix are accessible via proxy paths — no separate secure links needed.\n", "\n", "If direct access is not possible, use SSH port forwarding or your CSP's port sharing/tunneling feature." ] }, { "cell_type": "code", "execution_count": null, "id": "6b6cb926", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import os\n", "\n", "if BREV_ENV_ID:\n", " proxy_port = os.environ.get(\"PROXY_PORT\", \"7777\")\n", " ui_url = BREV_BASE_URL # resolved from environment-context.json in Section 7\n", " print(f\"VSS UI (via Brev secure link): {ui_url}\")\n", " print()\n", " print(\"Setup:\")\n", " print(f\" 1. Ensure a Brev secure link exists for port {proxy_port}\")\n", " print(f\" 2. Open: {ui_url}\")\n", " print()\n", " print(\"All services (Agent API, VST, UI) are consolidated behind the proxy.\")\n", " print(\"No individual port forwarding is needed.\")\n", " if PROFILE in (\"search\", \"alerts\", \"lvs\"):\n", " print()\n", " print(f\"=== Additional Services ({PROFILE}) ===\")\n", " print(\"These services are accessible via the proxy (no separate secure links needed):\")\n", " print()\n", " kibana_url = f\"{ui_url}/kibana\"\n", " print(f\" Kibana: {kibana_url}\")\n", " if PROFILE in (\"search\", \"alerts\", \"lvs\"):\n", " # nvstreamer is the one service not behind the ingress, so it needs\n", " # its own secure link. Report the published one or say it is missing;\n", " # a guessed hostname is indistinguishable from an absent link.\n", " _ns_host, _ns_port = brev_link_for_port(\"31000\")\n", " if _ns_host:\n", " nvstreamer_url = (f\"https://{_ns_host}\" if _ns_port == \"443\"\n", " else f\"https://{_ns_host}:{_ns_port}\")\n", " print(f\" nvstreamer (port 31000): {nvstreamer_url}\")\n", " else:\n", " print(\" nvstreamer (port 31000): no secure link published\")\n", " print(\" ^ Create one in the Brev console: Secure Links -> \"\n", " \"+ HTTP port -> destination port 31000\")\n", " phoenix_url = f\"{ui_url}/phoenix\"\n", " print(f\" Phoenix: {phoenix_url} (optional, for LLM tracing)\")\n", "else:\n", " proxy_port = os.environ.get(\"PROXY_PORT\", \"7777\")\n", " proxy_url = f\"http://{EXTERNAL_IP or HOST_IP}:{proxy_port}\"\n", " ui_url = f\"http://{EXTERNAL_IP or HOST_IP}:3000\"\n", " print(f\"VSS UI (direct): {ui_url}\")\n", " print(f\"VSS UI (via proxy): {proxy_url}\")\n", " print()\n", " print(\"If the URL is not directly accessible, use one of these methods:\")\n", " print()\n", " print(\" SSH port forwarding (works everywhere):\")\n", " print(f\" ssh -L {proxy_port}:localhost:{proxy_port} @{EXTERNAL_IP or HOST_IP}\")\n", " print(f\" Then open: http://localhost:{proxy_port}\")\n", " print()\n", " print(\" VSCode Remote SSH:\")\n", " print(\" Connect to the instance via Remote-SSH, ports forward automatically.\")\n", " print()\n", " if PROFILE in (\"search\", \"alerts\", \"lvs\"):\n", " print(f\" Kibana (via proxy): {proxy_url}/kibana\")\n", " print(f\" Kibana (direct): http://{EXTERNAL_IP or HOST_IP}:5601/kibana\")\n", " if PROFILE in (\"search\", \"alerts\", \"lvs\"):\n", " print(f\" nvstreamer (live view): http://{EXTERNAL_IP or HOST_IP}:31000\")\n", " print(f\" Phoenix (via proxy): {proxy_url}/phoenix\")\n", " print(f\" Phoenix (direct): http://{EXTERNAL_IP or HOST_IP}:6006\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 11. Next Steps\n", "\n", "Once you can access the VSS frontend, continue the QuickStart example which involves uploading a video, engaging in Q&A, and generating a report: [Quickstart - Upload a Video](https://docs.nvidia.com/vss/latest/quickstart.html#step-2-upload-a-video)\n", "\n", "You can either use your own videos for these examples or download the [VSS Sample Data from NGC](https://docs.nvidia.com/vss/latest/quickstart.html#download-sample-data-from-ngc).\n", "\n", "Once you've gone through the QuickStart example, you can follow **Step 12** in this notebook to deploy different [Agent Workflows](https://docs.nvidia.com/vss/latest/agent-workflows.html).\n", "\n", "**Step 13** provides instructions on stopping the deployment." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 12. Profile-Specific Next Steps\n", "\n", "Quick-start instructions for your deployed profile." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "if PROFILE == \"base\":\n", " print(\"\"\"=== Base Profile — Quick Start ===\n", "\n", "1. Open the VSS UI (see Section 10 for the URL).\n", "\n", "2. Upload a video using the \"Upload\" button in the sidebar.\n", " Supported formats: MP4, MKV. Wait for the upload to complete.\n", "\n", "3. Once uploaded, select the video and start chatting about it.\n", " Try asking: \"What is happening in this video?\"\n", "\n", "4. The agent uses the VLM to analyze video frames and answers\n", " questions about the content.\n", "\"\"\")\n", "\n", "elif PROFILE == \"search\":\n", " print(\"\"\"=== Search Profile — Quick Start ===\n", "\n", "1. Open the VSS UI and switch to the \"Search\" tab.\n", "\n", "2. Upload a video using the upload button. The video will be\n", " split into chunks and embedded for semantic search. This\n", " takes a few minutes depending on video length.\n", "\n", "3. Once processing completes, use the search bar to find moments:\n", " - \"person walking\"\n", " - \"red car\"\n", " - \"someone carrying a box\"\n", "\n", "4. Click a search result to play the matching video clip.\n", "\n", "5. You can also chat about uploaded videos in the \"Chat\" tab.\n", "\n", "Note: The vss-rtvi-cv container must be running for the embedding\n", "pipeline. Check with: docker ps --filter name=vss-rtvi-cv\n", "\"\"\")\n", "\n", "elif PROFILE == \"alerts\":\n", " print(\"\"\"=== Alerts Profile — Quick Start ===\n", "\n", "1. Open the VSS UI. The alerts profile needs an RTSP camera stream\n", " to generate detections and alerts.\n", "\n", "2. Add a camera sensor:\n", " - Go to the \"Sensors\" or camera management section in the UI\n", " - Add your RTSP stream URL (e.g. rtsp://IP:8554/stream)\n", " - The perception pipeline will begin analyzing the stream\n", "\n", "3. View live detections:\n", " - Open the \"Alerts\" tab to see real-time alerts as they're generated\n", " - Click an alert to view the video clip with bounding boxes\n", "\n", "4. Open the \"Dashboard\" tab to see the Kibana analytics dashboard\n", " with detection statistics, timelines, and heatmaps.\n", "\n", "5. Use the \"Chat\" tab to ask questions about detected events:\n", " - \"What alerts happened in the last hour?\"\n", " - \"How many people were detected today?\"\n", "\"\"\")\n", "\n", "elif PROFILE == \"lvs\":\n", " print(\"\"\"=== LVS Profile — Quick Start ===\n", "\n", "1. Open the VSS UI and upload a video via the sidebar.\n", "\n", "2. Once uploaded, use the chat to request a report:\n", " - \"Generate a report for my_video.mp4\"\n", " - \"Summarize what happens in this video\"\n", "\n", "3. The agent analyzes the full video and generates a structured\n", " report with timeline summaries, detected events, and analytics.\n", "\n", "4. Reports are saved and accessible via the \"Reports\" section.\n", "\"\"\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 13. Stop Deployment\n", "\n", "Stop all containers **without deleting data or volumes**. Use this when you want to:\n", "- Free up GPU/memory resources temporarily\n", "- Change to a different profile (update `PROFILE` in Section 1, then re-run from Section 8)\n", "- Restart the deployment later by re-running Section 8" ] }, { "cell_type": "code", "execution_count": null, "id": "17282638", "metadata": {}, "outputs": [], "source": [ "import subprocess, os\n", "\n", "# generated.env only exists after a deploy. Without it Compose would interpolate\n", "# defaults, resolve a different project name and stop nothing.\n", "env_file = profile_path(PROFILE, \"generated.env\")\n", "if not os.path.isfile(env_file):\n", " print(f\"ERROR: Could not find generated.env at {env_file}\")\n", " print(\"Has the deployment been run at least once (Section 8)?\")\n", " raise FileNotFoundError(env_file)\n", "\n", "compose_project_name = (read_env_value(env_file, \"COMPOSE_PROJECT_NAME\")\n", " or os.environ.get(\"COMPOSE_PROJECT_NAME\", \"vss\"))\n", "print(f\"Using env file: {env_file}\")\n", "print(f\"Using compose project: {compose_project_name}\")\n", "print(\"Stopping all VSS containers (preserving data and volumes)...\\n\")\n", "\n", "# Compose must see the same three env files as the deploy in Section 8, or\n", "# interpolation falls back to defaults and resolves a different project.\n", "result = subprocess.run(\n", " compose_base_args(PROFILE) + [\"-p\", compose_project_name, \"stop\"],\n", " capture_output=True, text=True,\n", " cwd=DOCKER_DIR, env=compose_env(HARDWARE_PROFILE)\n", ")\n", "print(result.stdout)\n", "if result.stderr:\n", " # Filter out the harmless \"variable is not set\" warnings\n", " for line in result.stderr.splitlines():\n", " if \"is not set\" not in line:\n", " print(line)\n", "\n", "if result.returncode == 0:\n", " print(\"\\nAll containers stopped. Re-run Section 8 to start them again.\")\n", "else:\n", " print(f\"\\nStop exited with code {result.returncode}.\")\n", " print(f\"You can also stop manually: docker compose -p {compose_project_name} stop\")" ] }, { "cell_type": "markdown", "id": "c52b62f3", "metadata": {}, "source": [ "## 14. Teardown\n", "\n", "Stop all containers and **delete all data** (volumes, models, data directory). Run the cell below when you want to completely remove the deployment.\n", "\n", "This brings down every VSS Compose project with `-v --remove-orphans`, deletes the generated env files, removes dangling volumes, clears the sdrc runtime artifacts, and deletes the data directory. If Docker storage was moved to NVMe (Section 4), volume cleanup requires an extra step because Docker can't remove volumes whose data lives outside its data-root (the symlink trick). The cell handles this automatically." ] }, { "cell_type": "code", "execution_count": null, "id": "55cb5b1e", "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import subprocess, os, json\n", "\n", "# --- Native teardown (Section 7.1 helpers) ---\n", "print(\"Tearing down VSS deployment...\")\n", "teardown_deployment()\n", "\n", "# --- Clean up stuck volumes ---\n", "# When Docker's data-root is on NVMe but volumes are symlinked back to root,\n", "# `docker volume rm` fails with \"unable to remove a directory outside of the\n", "# local volume root\". Fall back to sudo rm for those, then restart Docker.\n", "\n", "result = subprocess.run([\"docker\", \"volume\", \"ls\", \"-q\"], capture_output=True, text=True)\n", "leftover = result.stdout.strip().splitlines()\n", "\n", "if leftover:\n", " print(f\"\\n{len(leftover)} leftover volume(s). Cleaning up...\")\n", " need_restart = False\n", " for vol in leftover:\n", " r = subprocess.run([\"docker\", \"volume\", \"rm\", \"-f\", vol],\n", " capture_output=True, text=True)\n", " if r.returncode == 0:\n", " print(f\" removed {vol}\")\n", " else:\n", " # Symlinked volume — remove directly from /var/lib/docker/volumes\n", " vol_path = f\"/var/lib/docker/volumes/{vol}\"\n", " r2 = subprocess.run([\"sudo\", \"rm\", \"-rf\", vol_path],\n", " capture_output=True, text=True)\n", " if r2.returncode == 0:\n", " print(f\" rm'd {vol}\")\n", " need_restart = True\n", " else:\n", " print(f\" FAILED {vol}: {r2.stderr.strip()}\")\n", "\n", " if need_restart:\n", " print(\"\\n Restarting Docker to clear volume metadata...\")\n", " subprocess.run([\"sudo\", \"systemctl\", \"restart\", \"docker\"],\n", " capture_output=True, check=True)\n", "\n", " # Verify\n", " result = subprocess.run([\"docker\", \"volume\", \"ls\", \"-q\"], capture_output=True, text=True)\n", " remaining = result.stdout.strip().splitlines()\n", " if remaining:\n", " print(f\"\\n {len(remaining)} volume(s) still stuck:\")\n", " for v in remaining:\n", " print(f\" {v}\")\n", " else:\n", " print(\"\\nAll volumes cleaned up.\")\n", "else:\n", " print(\"\\nAll volumes cleaned up.\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# To remove the deployment repo from disk:\n", "# import shutil\n", "# shutil.rmtree(REPO_DIR)\n", "# print(f\"Removed {REPO_DIR}\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 5 }