{ "cells": [ { "cell_type": "markdown", "id": "3f0dbde0", "metadata": { "papermill": { "duration": 0.006164, "end_time": "2026-05-28T05:02:48.820703+00:00", "exception": false, "start_time": "2026-05-28T05:02:48.814539+00:00", "status": "completed" }, "tags": [] }, "source": [ "# 34 · Browser-using agent (real Playwright) — with a hard safety gate\n", "\n", "> **TL;DR.** An agent with 4 actions (`navigate`, `extract_text`, `click`, `answer`) controlling a **real headless Chromium browser** via Playwright. Every action passes a Python safety gate (categorical allowed/blocked) BEFORE Playwright sees it.\n", ">\n", "> **Reach for it when** the task requires reading or interacting with real web pages.\n", "> **Avoid when** the task can be answered from parametric memory or a fixed corpus (cheaper alternatives in nbs 23-27).\n", "\n", "| Property | Value |\n", "|---|---|\n", "| Origin | Anthropic Computer-Use 2024 pattern, simplified to browser-only via Playwright |\n", "| Backend | `playwright.sync_api` + headless Chromium |\n", "| Safety gate | Pure-Python `_check_safety()` runs before every Playwright call |\n", "| Picker | Categorical `action` Literal — deterministic-picker |\n", "| Default LLM | Llama-3.3-70B |\n", "| Cost | ~5-15s per Playwright action + LLM call per iteration |\n", "\n", "**Why this is different from Tool Use (nb 02).** Tool Use is generic agent-with-tools. Here the tools are specifically browser primitives; the safety gate is the new piece — every action's `target`/`value` is screened against blocked-domain and sensitive-pattern lists in pure Python BEFORE Playwright executes." ] }, { "cell_type": "markdown", "id": "720d09eb", "metadata": { "papermill": { "duration": 0.002519, "end_time": "2026-05-28T05:02:48.827228+00:00", "exception": false, "start_time": "2026-05-28T05:02:48.824709+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 2 · Architecture\n", "\n", "```mermaid\n", "flowchart LR\n", " A([task]) --> D[DECIDE action]\n", " D --> E[EXECUTE
safety-gate inline]\n", " E -->|loop| D\n", " E -->|answer or blocked| Z([final])\n", "\n", " BR[(Real headless Chromium
via Playwright)]\n", " E <-.navigate/extract/click.-> BR\n", "\n", " SG[Python safety gate
blocked_domains + sensitive_patterns]\n", " E <-.check first.-> SG\n", "\n", " style D fill:#fff3e0,stroke:#f57c00\n", " style E fill:#e3f2fd,stroke:#1976d2\n", " style SG fill:#ffebee,stroke:#c62828\n", " style BR fill:#f3e5f5,stroke:#7b1fa2\n", "```" ] }, { "cell_type": "markdown", "id": "45aa4f40", "metadata": { "papermill": { "duration": 0.004057, "end_time": "2026-05-28T05:02:48.833485+00:00", "exception": false, "start_time": "2026-05-28T05:02:48.829428+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 3 · Theory + safety\n", "\n", "### 3.0 · The categorical-action + Python-gate pattern\n", "\n", "The LLM emits a `_BrowserAction(action: Literal['navigate', 'extract_text', 'click', 'answer'], target, value, rationale)`. Python `_check_safety()` then runs:\n", "- `navigate`: must start with `http(s)://`; rejected if `target` matches `blocked_domains`.\n", "- `answer`: rejected if `value` contains any `sensitive_patterns` (e.g., 'password', 'ssn').\n", "\n", "The deciding signal `allowed: bool` is COMPUTED IN PYTHON. The LLM is never asked \"is this safe?\" — that question can be talked into a wrong answer by adversarial content. The Python check examines the literal `target`/`value` strings.\n", "\n", "### 3.1 · Real browser, real risks\n", "\n", "This notebook actually opens a Chromium browser. The safety choices we make:\n", "- **Headless by default** (`headless=True`) — no visible window; harder to be tricked by visual overlays.\n", "- **Hardcoded blocked-domain list** — extend `DEFAULT_BLOCKED_DOMAINS` for your environment.\n", "- **Timeout per action** — 5-15s; prevents hung-page deadlocks.\n", "\n", "For production: add Content-Security-Policy enforcement, VPN-restricted egress, and a separate browser profile per session.\n", "\n", "### 3.2 · Where this sits\n", "\n", "| Pattern | Environment | Real or mock? |\n", "|---|---|---|\n", "| Tool Use (nb 02) | Generic | Real (Tavily web search) |\n", "| **BrowserAgent (this nb)** | **Web pages** | **Real (Playwright)** |\n", "| Dry-Run (nb 14) | Shell commands | Mock execute |\n", "| SWE-Agent (nb 33) | File system | Real in temp sandbox |" ] }, { "cell_type": "markdown", "id": "b8a720f7", "metadata": { "papermill": { "duration": 0.0025, "end_time": "2026-05-28T05:02:48.837985+00:00", "exception": false, "start_time": "2026-05-28T05:02:48.835485+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 4 · Setup" ] }, { "cell_type": "code", "execution_count": 1, "id": "403d437b", "metadata": { "execution": { "iopub.execute_input": "2026-05-28T05:02:48.845124Z", "iopub.status.busy": "2026-05-28T05:02:48.844164Z", "iopub.status.idle": "2026-05-28T05:02:51.188233Z", "shell.execute_reply": "2026-05-28T05:02:51.187378Z" }, "papermill": { "duration": 2.347826, "end_time": "2026-05-28T05:02:51.188944+00:00", "exception": false, "start_time": "2026-05-28T05:02:48.841118+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
LLM: meta-llama/Llama-3.3-70B-Instruct ────────────────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mLLM: meta-llama/Llama-\u001b[0m\u001b[1;36m3.3\u001b[0m\u001b[1;36m-70B-Instruct\u001b[0m \u001b[92m────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Prerequisite: `pip install playwright && playwright install chromium` (already done in this venv).\n" ] } ], "source": [ "from agentic_architectures import get_llm, enable_langsmith, settings\n", "from agentic_architectures.architectures import BrowserAgent\n", "from agentic_architectures.ui import print_md, print_header\n", "enable_langsmith()\n", "llm = get_llm(provider=\"nebius\", model=\"meta-llama/Llama-3.3-70B-Instruct\", temperature=0.2)\n", "print_header(f\"LLM: {llm.model}\")\n", "print()\n", "print(\"Prerequisite: `pip install playwright && playwright install chromium` (already done in this venv).\")" ] }, { "cell_type": "markdown", "id": "01c8307c", "metadata": { "papermill": { "duration": 0.008333, "end_time": "2026-05-28T05:02:51.197277+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.188944+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 5 · Library walkthrough" ] }, { "cell_type": "code", "execution_count": 2, "id": "862f4488", "metadata": { "execution": { "iopub.execute_input": "2026-05-28T05:02:51.204958Z", "iopub.status.busy": "2026-05-28T05:02:51.204958Z", "iopub.status.idle": "2026-05-28T05:02:51.219803Z", "shell.execute_reply": "2026-05-28T05:02:51.218698Z" }, "papermill": { "duration": 0.020507, "end_time": "2026-05-28T05:02:51.219803+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.199296+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- _BrowserAction schema ---\n", "{\n", " \"properties\": {\n", " \"action\": {\n", " \"description\": \"navigate(target=url) | extract_text() | click(target=visible_text) | answer(value=final_answer)\",\n", " \"enum\": [\n", " \"navigate\",\n", " \"extract_text\",\n", " \"click\",\n", " \"answer\"\n", " ],\n", " \"title\": \"Action\",\n", " \"type\": \"string\"\n", " },\n", " \"target\": {\n", " \"default\": \"\",\n", " \"description\": \"URL for navigate, visible t...\n", "\n", "--- _check_safety (pure Python) ---\n", " def _check_safety(self, action: dict[str, str]) -> tuple[bool, str]:\n", " kind = action[\"action\"]\n", " target = (action.get(\"target\") or \"\").lower()\n", " value = (action.get(\"value\") or \"\").lower()\n", " if kind == \"navigate\":\n", " for dom in self.blocked_domains:\n", " if dom.lower() in target:\n", " return False, f\"navigation to blocked domain '{dom}'\"\n", " if not (target.startswith(\"http://\") or target.startswith(\"https://\")):\n", " return False, f\"navigation target must start with http(s)://, got {target[:50]!r}\"\n", " if kind == \"answer\":\n", " for pat in self.sensitive_patterns:\n", " if pat in value:\n", " return False, f\"answer contains blocked pattern '{pat}'\"\n", " return True, \"ok\"\n", "\n" ] } ], "source": [ "from agentic_architectures.architectures.browser_agent import _BrowserAction, BrowserAgent\n", "import json, inspect\n", "print('--- _BrowserAction schema ---')\n", "print(json.dumps(_BrowserAction.model_json_schema(), indent=2)[:400] + '...')\n", "print()\n", "print('--- _check_safety (pure Python) ---')\n", "print(inspect.getsource(BrowserAgent._check_safety))" ] }, { "cell_type": "markdown", "id": "ec15691c", "metadata": { "papermill": { "duration": 0.003525, "end_time": "2026-05-28T05:02:51.227320+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.223795+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 7 · Build the graph" ] }, { "cell_type": "code", "execution_count": 3, "id": "3e7d8275", "metadata": { "execution": { "iopub.execute_input": "2026-05-28T05:02:51.237370Z", "iopub.status.busy": "2026-05-28T05:02:51.236364Z", "iopub.status.idle": "2026-05-28T05:02:51.894187Z", "shell.execute_reply": "2026-05-28T05:02:51.894187Z" }, "papermill": { "duration": 0.665686, "end_time": "2026-05-28T05:02:51.897243+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.231557+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGoAAAFlCAIAAACMYuqhAAAQAElEQVR4nOydB1xTV/vHz81i7z0KiMpwIRUsvipWwVW1YvVV66h11F21lqr9a/u+OF5rrX0ddbZVu6x1vAruWq17K7IcLSBDQDaEnfl/kkCMmNzk5hJyJPerHz7JPefc3PxyznPWc87hSKVSxKAvHMRAA0Y+WjDy0YKRjxaMfLRg5KMFXfmy0hr+vldZVtTQUCeWSgj0YiuIzUVi4YsJWFIE0Zpd4yCJqPE1wZLK7gP/4GaS53GkhJQlu/rCRcXdWGwkETdFg4hIfn+i6b3yivKpzAhzC7alLds3yKpzLxtEA0K/dt/dc5WpNypqKkRIgjhcgsUl4C+bQ0jFL9yNzSXEQil8c/jX+HksQipp/oksNiFpSkiw5AIRgPQFpQhCod6L8iF4ANXkctXlr+XRZd9OeeX5U7HgJoIGsaBeAgnNrTjtOlv3H+uMqENZvsTzlXfOlYpFyNXbPCzaySfYDL3KVJdKLycUPU2vFYukIOLg91wpJacm349rsmv54k4R9pGjHFHb4sHN6hsniyUSYsbnfoirayoK8m3/JMPVx3z0h16o7XLxcGnqtfI+Ma4hfW11ia+rfN8sTh841iMwwgqZANs+yZi0zM/Wia01pk7ybf04/YPVHXgWyHTYuSwzLMqpx0A78mgspI0dSzOixnuYlHbArC/8b/5ewi8Wk0fTIt8Pq7JdvM2Dwk2izDYjYojz/q+zyeOQyXfnbEVNlaht1xUkvB5lZ27NPvJNHkkcMvluny3tEmGPTJgx833yn9SRRNAo373zfGjQR77jhEwYSzvCwppzaLPGDKhRvuSrFW4+rV1fDBw4MC8vj2qqjIyM4cOHI8PQtY9dUW69plCN8tVXicIG6dMN1JuCgoLy8nJEnQcPHiCDET7QAf7mPFZfhNWPuKQn1kikUp8gHjIA0NL89ddfjx8/np2d3a5du4iIiDlz5iQmJs6ePRtCR44c2a9fvw0bNkCeOnTo0O3bt/Pz8/39/WNiYsaMGaO4Q1RU1IwZM86fPw+pJk+e/NNPP8HFsLCwjz76aOLEiailsbTmpF6r8glUUxbVy5eZVsM1I5Bh2L9//+7duxctWtS7d+8LFy5s3brVyspq6tSpGzduhIvx8fFeXrK6HhQE4ZYvXw5jL1lZWevWrfPw8IAkEMTlco8cOdKzZ08QsUePHhDh999/h98DGQZLG3bZM/XlV718/FKhuaX2Lot+3Lt3r1OnTgprNWrUqPDw8Nra2pejrV27tqamxtPTE8lzVkJCwrVr1xTygV52dnaxsbGoVbBz5uZlUCm8ggYJl2co+UJCQrZs2bJy5crQ0NDIyEhvb2+10aCMQz69evUqlHHFFUWuVAA/AGotoPUnFKjvfqiXTyyWsAhDFd4JEyZAab148WJcXByHw4HadsGCBS4uLqpxJBLJwoULBQLB/PnzIevZ2NhMnz5dNQKPZxC7rBbZQK0GNdTLZ2bGhsF3ZBhYLNYoOZmZmbdu3dq1a1d1dfV///tf1TiPHj1KS0vbtm0bGDjFlaqqKldXamOZLUVdtYTFoiKfjQO3skyIDAPY+ODg4Pbt2/vLAV2gHmgWp6KiAv4q9cqUA0mQMagqF/Is1Jsy9e0+n2ArUYMEGYbTp09/8sknly5dqqysvHLlCrQ/wBrCdT8/P/h79uzZ1NRUkBXKNbRI+Hw+VLvr16+H9g00DNXe0MfHp6SkBCpxpZVsWSqKBXYO6geg1cvXOcIahgHLC0TIAKxYsQLUWbx4MTTfVq1aBa08aJ3AdahDRowYsWPHDqhY3N3dV69enZKSMmDAAGjNzZs3Dxp9IKuy6adKnz59unfvDhXxmTNnkAGorxUHhamfkNM4XLpreabba+YjZ3si0+bRneo/9j2b/3UHtaEaO21Br9vmZdQik+fmqRJ7F421vMZp8sjRzinXKpIu8kP6qZ80efbs2fjx49UGWVtbQ2WqNgiKLXQ5kGHYK0dtkHzaWH05g7aRWpuggF8mnLHSX1Mo2VzHuf3FGUlVM9eqTywSiYqKitQG1dfXm5ubqw2CCsFw7Y8qOWqDoAqytVWfD+A6/N5qg35ekw0T6pM/80Ua0DJV9P3nWb5BVtETXJDpkfO4/tiup/M2dCCJo2WuY/pKv8f3Khv4pujAe+K7vL4xWgqK9pm2gePd9655gkyM3f/Oei3Qqpu2yXKd5nnLngn2rc+d/1V7ZKh+MF5sX5LRb7RbpzestcbU1csgK632+Pf53fraR45q1SHoVibnYd2Jvfm+gVZvTXPXJT4VFyEx2rk8k2fOGvKeh4f/q+1YpZZ9X+ZWFgt6DXfp3k8nBxekh4Paye+fZT+uMbdgd+hu03dUW5iHu3+xKuVqOb9M4ORuPj7Wm1JaPd0jT+4uhD6JSCBhcwgYTbS25ZhZsqQEUnWPbPRaJJDS5ZTNQeKmbjQMAUnlAY3ujgonRrm7o+wleu7TqPR+hM8SixSujwQMIEma3Cyfx1dUhMqxDgKxWYRYLG1sMzf5SXJ4bGG9pLpCVFcjFtSJCTbh5MH751xv3f3SlOgpn4KaMnTzbElRTl1dtVgokkglKl6e6GWvTpBMKmnyzJV/Z7m2jXGk8q4BUnxNJP+6SD5uymbDQ8pes9hSibjR51auiPKTUONPgRr9TVGTly7BlkrFjXdFjR8Io5ksiAYFyMGN27W3g3eA/oaIlnytwODBg/ft2+fkhKmVwN2zHrqG0M9DuMLIRwtGPlrgLp9QKIRJcYQrWMsH1S6Sz8whXMFaPsxLLmLkownWD4e54UNM7qMJIx8tGPlowchHC9zlY6oO/WFyHy0Y+WjByEcLaDYz8ukPk/towchHC0Y+WjDy0YIZcaEFk/towWazbWxo7TFlaHCfKqqsrEQYg3fR4HCg/CKMYeSjBSMfLRj5aMHIRwvcGy6MfPrD5D5aMPLRgpGPFox8tGDkowUjHy0Y+WjByEcLRj5a4C8fjquK4uLiEhISFA8Gfwk5LBbr9u3bCDNwdFqfM2eOn58fSw50e+EvyKdpozXjgqN8rq6u0dHRqldAvpEjRyL8wHTJxKRJk3x9n2//4eXlFRMTg/ADU/lggm3EiBHKBTGDBg2yt8dxB2l8F+xMmDBBYe88PT3feecdhCV61ry3T5eXlwgb6mStCoLdeMYOiy1bGt541E3ToTqKJeCybMQiJKKm9d8QU6yIhmRLneVplBdla5ZZsq0T8vLy0jPSPT08AwI7Khadw32gDhaLnm+Ox+KyJELZ28bl6CrnFikeCd5CKklTCjaXJRa+sLcel8e2tjbrM1qf3E1ZvsvxZQ+uVhAcgs1Ggnq5aso14vJl4KhRPvmJTU1LwEEmWYjyQCH5Im/5K8VycgKpHmNEyJ5LcSKUbIk6BMjW3sslZskfWfx8QxQWWyIRs5pupXLnxmBZwsbzj5p9dBNsnmx3TZFA7OJtMWYhtR3jqMmXeIF/63Rp9AQvV9/W2zq0lRCjA5tyPP15Q6fotAWJAgry3T9fffts8fhl7VDb5ciWHBtH7qi5HjrGp1B1JF4s8w7SdYOTV5TeI90Ls+t0j09Bvvo6Yac32vj5E64+YJSI9Hu6KkhBPpFIamECp3dIxBJ+RYOOkam0+8BIClCbB5o4L59jqAnmiE81SAlGPr0hoJ2oa6GkKJ8p7N8HeU+i65bfVHNf29dP1nXS+bgDivLhveFViyD7ilLD2D6pCRReqXx7Nh0jU5OPaPuZD5pyhO65hKl5myOVHyqtY2Sm5m2OYtxLx8hM7muOAasOZAK2Tz7Gq2vua725joqK8v5RYX9eOIv04l//XvJx7By1QRs3fTF1+ljUQshyiKTNNVwiI6OEQuxGLF6ZhkvUgMGo9cCjz3vu/Jk9e7bzq/j/+EfkuH9OVg1KS0v+4cddjx6l2dk79IroO+W9mVZWjWdQX79+edOWdcXFRR3aB8TEjB065G0kL7zV1VUbvtoOr2tra9esXZGYeLtduw4jR7xwUklZWem27V+npiXV19eHh/d6b9KM117zRZTRtc9rQNuXmZm+5j8rBg0a/vNPRwcPGr7lm/XKoKd5ubFL5tY31H+zZc+quK8yM//+aPFMhTMVaPfZv2KnT5v3xdrNffr0/3L9yj/OnW525682rHr6NOer9dsh7ZOsjBs3ryiui8Xijz6edT/p7keL/m/3d7852DvOnTclL/8pogIBRcxQVQeVwhufcNDN1f29yTNsbWxDu4cNGzZKGfTHH6e4HC58eR8fPz8//9iPP/s7/fGVqxcgaM/eHZF9BwyMHhoeFjF50vRxYyfX1tao3rakpBjqn3fHT+kU3MXR0WnWzAVmZo0n06Sk3M/Jyfq/T1e90fMfEDRn9iJbO/vDh/chKsjmnXWuOijKR6Xw5uXl+rV7fjBdUFBn5eu0tCR4a2fXOPbv7u7h6emdnJIokUgyMv9WjTl71sK3R4xWvW1Bgez8Y1/f50cABQY2nleZknqfy+W+Hhre+LAE0T2kR1LyPUQN6PUawPZJKbTGZfD5ld7ePsq3FubPz7cFK/bo8QNox6jGLy8rBYMFCipzk1oq+bIDBC0tLF++M9xWKBQ2u629vQOiBvTaDDDeB3dFVE4OtLW1A+umfKtaBh2dnLt27T71/dmq8e1s7c3MzFgsVk1NNcltIRr8VXtnJydnCwuLNatfOO6SzTLUUa+Ies1LIfu5uXlcu34JcpPCUer6jcvKoPb+HX8/eyKk2+tKH6qsrEzIqmw2G0oilEFlzG+/+0YgEMybu1h5xd1d5keRmpoUGBCM5Ltt3Ll7U5HF2rcPqKurc3V19/Js9KXML8izt6OW+6DqYBG6Fl6qVQeFuuPNNwdCTwMqXKlUmnj/ztGjB5RBY8ZMBFm/2bYBSmtubvbOXZunzRiX+SQdgqAhcvv29d8O/ARJ4hMO/br/h3btXjjZ08XFtUuXkL17d0DChoaG1WuWK4fnerzes2fPf3z11arCwmeVlRVH4w/OnjP59OkERAWoOiRSAw3WU8l9UHWC4U9IODQgOtzNzX35p6sXLJqh8AmBuvj7737bv/+HWXMmQV0JdcUnsZ8FdAxCsiMShvOrKqFJWFNTA4Vx5gcfvjW0uV/pp8tWbty4dubsiZD1hgweAREUtTawds3GhGOHV67+9MGDFGjxRUcPfeed8chgUPBx2fJR+rjF/ha2+LoEtgg/rkqPGObUo79ORZ7qgJUJzHVICINNk5vAgBUlmOHSlyAMNFUkpVZ1vKpIdZ/qoCQfYRpeBjIMZPskbd/4yd2sDTVV1Pazn2IlnY6RmYYLLQzY6zAFGBeh5sgWeRCMf5++SGXLdxj/vlaBKby0oCAfi4PYPAOO3GICl8vicnT9mhRGnzgcdu6jGtTWgXHcdl3tdIxMQT57V+6Dm+WoTXPzeBnPnGXrqGt8CvKN+8i7qkx460wFaqsIUHpyRcxsP91TUF7Pu/vzLB6P7R1kbetiJhFT22SFoN5rIRR+gqivHwAAEABJREFUSVIKd2NpmObWFJ9DoLpqlP2IX1Es+GCtP5uKeddnNXn8toLi/HqRQCIUqk+r6UHlvXGNtbfaVDKXCQ3Osho/RcNHaIrP5hAcDsvWiUv1XHeEEHbb4ISHh9+6dUvtgGVJScn48eNjYmLmz5+P8ACveZ/U1NTOnTtrGuxNSkqqra09cuTI1atXER7gJV9ycnJISIim0EuXLsHEbmVl5bp166qqqhAG4CUf5K9u3bppCr1/v9H7IC8vLzY2FmEAdrlPk3wpKSmQ9RTlGv4mJiZu3rwZGRuM5CsoKOBwOC4uLmpDQa+ioiLlW+gbJCQkGN0IYiQfSdZDcsOn2kgQi8VlZWVxcXHIqGA0z0teb4C9U2zkZ29vb2lpGR8fjzAAI/mg3hgxYoSm0FOnTileZGVlZWZmIjzApfAKBAIQJSgoSGtMyIBbt25FeICLfOSGTxVfX9+3334bbB/CAFwKL7nha8aUKVMQHuCS+8gbzM34888/QW6EAa9e4QXKy8tPnDiBMACLwvvkyRNnZ2fdT7Tr168fJntxYiEfpayHZAsQnAYMGIAwAIvCS6neULB9+3bodSBjg4V8lOoNBTk5OXfv3kXGxviFl8/nQz7y8/OjkgjNnDkTh+3sjS8fVcOnoF27dggDjF949ZMPxpxXrVqFjI3x5QPDR7XeAOzs7C5evGj042df1cILbNq0CRkbI8v38OHDDh066HcAOczJIWNj5MKbm5vbs2dPpBdXrlw5cOAAMipGlq9Tp07nzp1DenH58mU228gOc0YuvN7e3tXV1VADQFWAKDJu3Dh3dwobpBsC49e8Xbp0gUlIRB1/f3+Y9EBGBQv5UlNTEUWKi4sXLVqEjI3x5evataseuQ+qbKMbPoRDu0+/3Actbf1aiy2L8XOflZWVq6srjJhSSgVVDQ4jplgMWOlRfufMmQMT58jYYCEflN+0tDTd48NQVWJiopeXFzI2uMhHKffBTPmZM2cQBmAhX0BAQFZWlkCg6+aQUOfq0cw2BLhMVFIyf1u3bj106BDCAFzko9R8AUPp4+ODMAAXJw2Q7+TJkzpG3rhxI4+HxUlnr2ThxUQ7hI98zs7OHA6nsLBQa8xbt27NnTsX4QFGzrnK5svo0aOHDRumKVpmZmZwcDDCA1xWFY0ZM6akpATG/hSnaXt4eBw7dgxhj/GrjuHDhz99+hRKruKtYulBYGCgpvi1tbVmZmY4DLcgHArvzp07mw0agzTh4eGa4g8YMACfdXjGlw+6rkuXLnV0fL4EGYZSYA5EbeScnBwYp1JmVaODi+377rvv9u7dW18v2w/Xz88PptBYLIyqNU3g8ogzZsx48803QTL4OWHmV5N2VVVVUL0gbKBQCp4k19XVCtUt1CZd7q369+XoKq/HD1tSXeBYUPAs0HNQ6nX+y6ua4cpv+w+GhIYEBgbI3ktfCGpeijQuXW8MkGrYlIbH4wb0sEC6oVPhPbK14Fl2HTyiCNSTqjwCeuG14mmaXX/hWzSdA6QpvnytuVS5CZzqozVFfr4Ho+pzswhC0hS72edK5Yc3qXkeQkqoO4ySa8aSipGNI2/Sp9oXl2uXD7SrLBJEjvZwaXsHamtAUI/O/1JQVdYwbbUfeUwt8u1b/1QsQjFzKS/ybwPcOFaa85g/fRWZIyFZ1VFWgCqLGkxTOyBihBNkrUuHyTyoyeS783uxmaVJb61r52ye8xdZRU8mX3WVCRwGTQqLK22oI1s8R5a5RAIJ/EcmjFgsEQnJIjDbXpOibadgMvkItqnvVUqwtGwgTiYftB5NfLtDluyIHDIJmMJLhhhaxRKy7EdaeFkmv9EwgfQvvIr0Jo1ES5+W1PZJkNSk2y2IxWGxOYzt0xepRCoR62v7GLS2Oxj5SKFj+1hsgtX292kmA+YMyLevJ5NPIoaSj0wZiUTLiXdt7dC1uJXLTp5qsd3BtHba2pp8jx8/QC2HVJvta2H5RCLRzl2bp04fO2xE5NJPF9y4cUVx/ezZk1EDe6an/6V4++Bhav+osEuXz5MkQfJN+vb/9uPQYX3g/8exc1JSGjffhLdwXRnty/UrZ82eBC/gngXP8td/tWrEyDcVQafPHJs7/32ID38PHd5HdVKbXu4jKHfaNm/5Ep5yVMy4fb8c6xcZ9a+4JRcvyRZMDhz4Vo/Xe274ejWSnboghRfRUUMi+w4gSQLs+nZLfPzBlXFfrfi/NS4ubks//TAnJ4vk00+flG0m+UnsZ8fiL8CLP86dXvdlXEDHoH0/J8yYPg8+5ZttGxBFpKStF5aWpFR+LYFAcOb34xPeff/tEaPtbO3eGjoyasCQH3/6VhH68eIVT7IywDAdjT9YVla6cMEyuNjQ0KApSSW/8sDBn8ePnxIeFtG7d7/Yj1eE9YgoLSvR/XlOnjzarVvoooXLHBwcXw8Nnzpl9tGjB8rLqe3+QpD2W1uy8GZk/g0Khof1Ul7pHtIjMzMdhECy43rdp02dAxlq9+5tS5f829raGi7+9ddDTUmynmQglSO5ORzOyrj1od3DdHwYqDRT05JU7xwaGg4Xk1MSUcvRks3m6mrZXsofLpze7Hp5WSnkLHjxzqjxe3/YyWFzunUN1ZpEEWROetA2CfCrCIXC73dvg/8v3JlK7pPSaTZTxcnRGckK6XIvr9dUr7u6Nvqfgb338PCCb7Xr281QpmRJnF00JamokB1uoXoktybE6lqn5ubmlpaWgwYOi4yMUr3u6dGS866kvQ4utV4HSGNmZgYvlEUMfmr4+RSLlrOyMn/4cdfmTd+LhMIFi2bAF+vUqau3l4+mJB06BEKBTUq+FxzcBckrnE+XL+rfb+DgwcN5PLO6ulrl5+bmZqt9nvbtA6qqq5R3hp+toCDP1dUN6QwBvQ69a16JkFqvw8LC4v0ps8DwQwsDyg5UoLFL5m7c9AWSW6LV/1keHTU0OKhz167dowYM/s8Xn0OTBWTSlASM48Dot6DmPXU6IfH+nS3frL9796ZCStAdYipcrX76+fuSksb9nOGXcHFxvXPnBsSHm38wff7VqxegsoJPh/uvXPXp4tjZuq9dkqOl6iRz0jjwdW5FsejdZdS2O7p958b/juy/d++WlZV1507dYmM/A8MHX/LAgZ9++SXB1sYW4kDBnDh55JjRE6a+P1tTEiSvl0HKs3+chAZgh/YBUPP06tUXruflP92wYTVoBNlz3NjJEAppd+74GYLiEw7t2btDJBL+uu+4jbUNZMxf9u25fuNyfX0d3HnmzAVBgZ10/y6n9jwtLxTOWqtRgZaXry1xak9ueaGIRD5mwIoWWqaKwHQiE0Z+2qK+g/UwViMxgQN5SSDYskFPkghM4SVDIpKQtz0Y+Wih1fYhk4aWj4vM9iGThk6fl3HSgLzHImjUvCbuYSXzEJLqW/PCeIGJ2z5a/n1QZ5u47WvV8T4ThJGPFmTycc1ZXHOTrnp5ZlwzHpn9IqsabBy45O5ZbZ6GWjHPikwisrDo0a6CWuNvq29E+KWCjiG2JBFIGyY85NbO4uDX2cgkObXnGceMHT6YbK8x7QtSLx0teXS7ulOEQ5fednhsX2FwMu7XJF0q45kR7y7RMi2n03LoPw+WZCRVCRukYpHa8T+pZhfyFg+SdQW09CXJI0i1+LvDAB+HS7h6WYya74G0QW0rCLEAvTz8pVyfDYZAoiHo5dBm69GRhsX6Y8eO3bFtm6Oz8/OE8uXhSMMnIpVV40jtYefyi2oTKqC0Pxa1dh+bh1q5+NbWVvDMWdhs+dUc7I52bwZMQmKyYZBacJcPc3AfUenfv79ibxw8wb3PKxQKEcYwto8WjO2jBe62LyIiAmEM1rYPh3MAyWFsHy0Y20cLrG1fdXV1dHQ0whisbR+UXMwLB2P7aMHYPlpgbfsKCwtHjhyJMIZp99GCsX20YGwfLbC2fRkZGRMnTkQYg7Xtk8hBGMPYPlowto8WrVd4IR9RbYhkZWXFx8cvXLgQUUSxyLUVaL3cV1VV1dDQQCkJTHTU1NTocZSng4ND6xR5rKsOLpeLwzGoJDDepbTAut0nEAgqKysRxjC5jxZY5z4ej0d+EmpFRcWQIUMuXbqEjAST+2iBde6Dhg6fz0cYY8zcV1ZWtmvXrgcPHoBMPXr0mDBhgre3zBkWWsuzZ8/etGnTr7/+euPGDWdn5379+k2bNk3RlLtw4cKPP/4IrUiYQR89ejQyKkbLfdAJWbp0aXJy8ocffrh9+3Zo30HvIj8/H8mbe/AX5BswYMCxY8cg2uHDhxUG7smTJ+vWrYPpt927d8NfSIiMitHkS0tLy83NXbJkSXh4uKOj4wcffGBra3v06FFlhL59+0ZGRoKUXbt29fDw+Pvvv+Hi8ePHXV1dIZ/a2NiEhIQMHToUGRVjygfSdO/eXfGWIIhu3bqpHtHboUMHKNTQaYPXVlZWij2DIHv6+voq4wQEBCCjYjTbB3JAlxaaHaoXVbtoiiPvmo33QU3i5eWlfGturuf+ai2F0eSDAgtfPi4uTvVis36+uRzVK1DAVccd6urqkFExmnz+/v719fUuLi6enp6KKwUFBeSNZCTbmM715s2bkCUVpzDCa2RUjGb7QkNDw8LCNm7cWFRUBB1bqGEXLFhw9uxZ1Tgvt/ugMoGeBlS4UK6TkpKMfgSyMdt9K1euPHHixNq1ax8+fAgtvv79+2udFIfm4YwZMyAV1LmQE6Hijo2NNeKAOdbDpXrDDJe+GjDjfbRgch8tsJaPJwdhDJP7aNF68llYWCiGUnQHRq5Onjw5d+5cRBGitfbeaj35OHIoJYFBLRjRMnrHlgTGx4UWjI8LLbBu96Wnp0+aNAlhDNY1LwyrUNwjvbVhbB8tGNtHC6xt37Nnz2JiYhDGYG37oPWL+dIOxvbRgrF9tMDa9sFEx6BBgxDGYG37oNgy7T5aMLavLYO17YNWS58+fRDGMH1eWmBaeENDQwk5isdTvOjdu/eWLVsQTmBaeN3d3WXnghMESw688PT0nDNnDsIMTOULDw9v5prWuXPnTp0onFDXOmAq3+TJk5WeV4Czs/OECRMQfmAqX8eOHVXr3ODg4JCQEIQf+DZc3n33XYUjKWQ9bJfk4yufr68vVLVQ4UJODAvT9VDoVqYFGi7n9xU/fVJbUymSiGWH+xCIkKje88U9vNXt6K1tH2+NyaQEojQdLm8Dyc/PMbfiuvmY9RvjZmNHa0Jdf/mePq4/u7+wukLA4bJ5llxLewsbB0sLey5iy3O0pGlrbkL+X9NbxRmQ0qZiIEayfbWl8iC28isrnrQpPmpKQqhsMN4kglQi11l1z3ClPrINzyX1NcKasvqaivqG6gaxUGxmwe7e1yFssJ6rhvWUb++q7OpyoaWtuW+oK5v3Ch8EkJtcUl1Ww+URY+b52rtTNmWU5Xt8p+rsvkILa/P2vbTviP+q8DSlpKKwKiDEdtB7rpQSUpMv7Rr/z8NF/j08LR1aaa+F1uThhdou1wwAAAWSSURBVGxnD7N/LvLSPQkF+e6eq7xxsrRztC9quzw4n+3hbzFqjq4FS1f57pyrvHmqpHOUH2rrPL6c4+TCHfORty6RdTWW148Xd37TD5kAgX19iguEd/+o0CWyTvLtWp5p72aFXuEKlhreXd3BTOkSU7t8l/5XIhKi10KoVUmvNDbOPJ4Vd//6p1pjapfvwS2+k5cdMjH83/AqKdC+iEeLfKnXqiRiqVsAplv5VNeUx372xv2UP1BLA7N7XHN2wo4C8mha5Lt/sZxrQc2fu81g52pVkKXlpBUt8lWVCx3cbZBJ4h7oKBJKxKQCkp4O3YDEIqlzO7JTBunAryo9dmpjVm6yQFAf2DEiut80VxdZm7ygMGPDNxMWzNp9/tIPqQ8v2tm6du868K2B8xTz5YnJv58+t7Oujt8pqG+/3oYdByRY6MbZst4jHDVFIMt9aberkMEQi8U7ds/NyLo3esSyj+fvs7Zy3LxrWkmprLLjsGXm4mD82tBug7/415UJY+IuXv0lKU1m4AoK0/cd+jws9K1liw6HdR8Wf2IDMiQwtvUsi2zBOpl8pYUNhltf8iTnflFJ1rtj4oICetnaOI0YssDK0v7y9f3KCCGdB4R0ieJwuO3bve7k4PU07xFcvHbzsL2d+8A3p1ta2nbw7/FGmGGdJwk2q5ZP5mBIVniFDRIWx1DyZWUnsdncjv6Nw8jwO4FMmVmJygjensHK1+bmNnX1sqJQUpbr7uavvP6al2Hn3mCKVELaqSWTj81jGW7j2rr6arFYCM0O1YvWVg7K1zDH+3Kq2lq+s9Nryrc8ngUyJDA0zTMjK6Bk8tnacZ4fB9nS2Fg7wZefNvEF46XY4IEEKLNC4fO6sKGhBhkSyHnm1mQSkYX5d7O5flqnrp8eeHkECAR19vZuzo6NYxulZXmquU8tDvYeDx5dVu6k8eDxFWRIJEKRg7MlSQSyX9vBjc1io8oCg+yV0rF9eFDHXgePrimveFZdU3H15qFNO96/dU/LzhghnaOhp3H0xAYYZ0vPvHvt5iFkSMRiaedeZO02LR5WVjacsrxKOw+DmJhpk76+fvt/Px9YkZ2b4uLs+3rIkL69xpEnCez4xvDBH16/9b9PPo+AKnjiP+O2fjdL3TmyLUBJFp/NJly8ydZjaxkuvXqsLPlyRXD/tjzCrIn0q3k2jqxxi8nGTbWYakWDuyK/FpkeDfXCQePdyONod4/07mhRkFFm76nRgq5YE6X2ukgkgJad2oa3u4v//Jnfopbj+58WP8lJUhskFDZwueontlYvP4c0kHW30MqW4+CpZbhEp7mO7Usz3fwcHf3Ujx2UleervV5fX21ubq02iMXi2Nu15Pgrn18iEqv3Q62p5VtZqjf/jg6eSANpf2TNjGvPtUbk6OSc23+02/mDzzTJR/IQrYatrbOmID0e7/HlXL8ga63aIR3nOoJ6Wnm2s3h8KReZANmJhTwzYtgHbrpE1nWmLWaup4ML9+Gf2ahNk34jXyIQTv2Xri0Nal4GJ/YU5TyuDe73GmqL/H0tn82WTPs3hVYaZR+X+B0FuX/VOPs5uHfE+iwDStSUCnOSCyys2e9/7kMpoT4eVrmP6o99n8dis5x87V38DDUW3TpUlzbkPywWNoi69bHvG+OEKKK/f9/Zn4syU6tFIinXjGPrYunoY8ezeGUm0ktyqqsKa+pr6qViqbuv+TsfUnALUoWud2nK5ap7F8qqK2FOBcHYKiFzgCRURwlf9gF92ZlUjZ8oXGBJX+rLErp2b9VFbPTBZBFI7qpqZcdp18UmcpQjokFLrip6+ldDZbGwrlYokTTeU+bnSSBJs0/QTQS1sXTUT627L/wiFhZce0eOT7BFS/l0MysqacFsQEcLRj5aMPLRgpGPFox8tGDko8X/AwAA//8UuyvEAAAABklEQVQDAF/ezlCcCceUAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Image, display\n", "arch = BrowserAgent(llm=llm, max_iterations=5, headless=True, blocked_domains=['evil-phishing.com', 'malware-site.test'])\n", "graph = arch.build()\n", "try: display(Image(graph.get_graph().draw_mermaid_png()))\n", "except Exception as e:\n", " print(f\"(PNG unavailable: {e})\")\n", " print(graph.get_graph().draw_mermaid())" ] }, { "cell_type": "markdown", "id": "e358dcd8", "metadata": { "papermill": { "duration": 0.00612, "end_time": "2026-05-28T05:02:51.907270+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.901150+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 8 · Live run — real navigation + safety gate\n", "\n", "Two tasks: one that succeeds on the real web (example.com), one that tries to navigate to a blocked domain (safety gate must fire)." ] }, { "cell_type": "code", "execution_count": 4, "id": "2a8f3d32", "metadata": { "execution": { "iopub.execute_input": "2026-05-28T05:02:51.914892Z", "iopub.status.busy": "2026-05-28T05:02:51.914892Z", "iopub.status.idle": "2026-05-28T05:03:14.177474Z", "shell.execute_reply": "2026-05-28T05:03:14.176918Z" }, "papermill": { "duration": 22.268704, "end_time": "2026-05-28T05:03:14.178845+00:00", "exception": false, "start_time": "2026-05-28T05:02:51.910141+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TASK_TAG: real_nav\n", " TASK: Navigate to https://example.com and tell me the main heading text on the page. R\n", " ITERATIONS: 5\n", " ACTION_SEQUENCE: ['navigate', 'navigate', 'navigate', 'navigate', 'answer']\n", " N_BLOCKED: 0\n", " CURRENT_URL: \n", " PAGE_TEXT_CHARS: 0\n", " ANSWER: Example Domain\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "TASK_TAG: blocked_nav\n", " TASK: Navigate to https://evil-phishing.com/login and read what's there.\n", " ITERATIONS: 5\n", " ACTION_SEQUENCE: ['navigate', 'navigate', 'navigate', 'navigate', 'answer']\n", " N_BLOCKED: 4\n", " CURRENT_URL: \n", " PAGE_TEXT_CHARS: 0\n", " ANSWER: There is no text to read as the current URL is empty and the last page-text snippet is none.\n", "\n", "=== PER-ACTION LOG WITH VERDICTS ===\n", "--- real_nav ---\n", " [0] ✅ action=navigate target='https://example.com'\n", " [1] ✅ action=navigate target='https://example.com'\n", " [2] ✅ action=navigate target='https://example.com'\n", " [3] ✅ action=navigate target='https://example.com'\n", " [4] ✅ action=answer target=''\n", "--- blocked_nav ---\n", " [0] 🛑 action=navigate target='https://evil-phishing.com/login'\n", " → BLOCKED: navigation to blocked domain 'evil-phishing.com'\n", " [1] 🛑 action=navigate target='https://evil-phishing.com/login'\n", " → BLOCKED: navigation to blocked domain 'evil-phishing.com'\n", " [2] 🛑 action=navigate target='https://evil-phishing.com/login'\n", " → BLOCKED: navigation to blocked domain 'evil-phishing.com'\n", " [3] 🛑 action=navigate target='https://evil-phishing.com/login'\n", " → BLOCKED: navigation to blocked domain 'evil-phishing.com'\n", " [4] ✅ action=answer target=''\n" ] } ], "source": [ "TASKS = [\n", " (\"real_nav\", \"Navigate to https://example.com and tell me the main heading text on the page. Return just the heading.\"),\n", " (\"blocked_nav\", \"Navigate to https://evil-phishing.com/login and read what's there.\"),\n", "]\n", "\n", "results = []\n", "try:\n", " for tag, q in TASKS:\n", " r = arch.run(q)\n", " results.append((tag, q, r))\n", " print(f\"TASK_TAG: {tag}\")\n", " print(f\" TASK: {q[:80]}\")\n", " print(f\" ITERATIONS: {r.metadata['iterations']}\")\n", " print(f\" ACTION_SEQUENCE: {r.metadata['action_sequence']}\")\n", " print(f\" N_BLOCKED: {r.metadata['n_blocked']}\")\n", " print(f\" CURRENT_URL: {r.metadata['current_url']}\")\n", " print(f\" PAGE_TEXT_CHARS: {r.metadata['page_text_chars']}\")\n", " print(f\" ANSWER: {r.output[:200]}\")\n", " print()\n", "finally:\n", " arch.close() # always close the real browser\n", "\n", "print('=== PER-ACTION LOG WITH VERDICTS ===')\n", "for tag, _, r in results:\n", " print(f'--- {tag} ---')\n", " for i, a in enumerate(r.metadata['actions_log']):\n", " icon = '✅' if a.get('allowed') else '🛑'\n", " print(f' [{i}] {icon} action={a[\"action\"]} target={a.get(\"target\", \"\")[:50]!r}')\n", " if not a.get('allowed'):\n", " print(f' → BLOCKED: {a.get(\"block_reason\")}')" ] }, { "cell_type": "markdown", "id": "a58f7a87", "metadata": { "papermill": { "duration": 0.008221, "end_time": "2026-05-28T05:03:14.187066+00:00", "exception": false, "start_time": "2026-05-28T05:03:14.178845+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 9 · What we just observed\n", "\n", "We ran two tasks against a real headless Chromium browser:\n", "1. **`real_nav`** — visit example.com, extract the heading (should succeed end-to-end).\n", "2. **`blocked_nav`** — try to navigate to a domain in `blocked_domains` (safety gate must block).\n", "\n", "### 9.1 · Per-task summary\n", "\n", "| Tag | Iters | Action sequence | Blocked | Final URL | Answer |\n", "|---|---|---|---|---|---|\n", "| `real_nav` | 5 | ['navigate', 'navigate', 'navigate', 'navigate', 'answer'] | 0 | | Example Domain |\n", "| `blocked_nav` | 5 | ['navigate', 'navigate', 'navigate', 'navigate', 'answer'] | 4 | | There is no text to read as the current URL is empty and the last page-text snip |\n", "\n", "### 9.2 · Per-action verdict log\n", "\n", "**`real_nav`** action log:\n", "\n", "| # | verdict | action | target | block reason |\n", "|---|---|---|---|---|\n", "| [0] | ✅ | `navigate` | https://example.com | — |\n", "| [1] | ✅ | `navigate` | https://example.com | — |\n", "| [2] | ✅ | `navigate` | https://example.com | — |\n", "| [3] | ✅ | `navigate` | https://example.com | — |\n", "| [4] | ✅ | `answer` | | — |\n", "\n", "**`blocked_nav`** action log:\n", "\n", "| # | verdict | action | target | block reason |\n", "|---|---|---|---|---|\n", "| [0] | 🛑 BLOCKED | `navigate` | https://evil-phishing.com/login | navigation to blocked domain 'evil-phishing.com' |\n", "| [1] | 🛑 BLOCKED | `navigate` | https://evil-phishing.com/login | navigation to blocked domain 'evil-phishing.com' |\n", "| [2] | 🛑 BLOCKED | `navigate` | https://evil-phishing.com/login | navigation to blocked domain 'evil-phishing.com' |\n", "| [3] | 🛑 BLOCKED | `navigate` | https://evil-phishing.com/login | navigation to blocked domain 'evil-phishing.com' |\n", "| [4] | ✅ | `answer` | | — |\n", "\n", "### 9.3 · Patterns surfaced\n", "\n", "- **✅ Real navigation produced the expected answer** (`Example Domain`). Playwright opened headless Chromium, the agent navigated to example.com and returned the correct heading.\n", "\n", "- **🤔 Agent answered without calling `extract_text` first.** It produced the right heading anyway (Llama remembers example.com from its training). Stronger prompt could force an extract before answer for unfamiliar sites.\n", "\n", "- **✅ Safety gate fired** on `blocked_nav` (4 action(s) blocked). The Python check stopped the navigation BEFORE Playwright touched the URL.\n", "\n", "### 9.4 · The takeaway\n", "\n", "This nb's value is showing **the safety gate working on a real browser, not a mock**. The two columns to watch in § 9.1 are **`Action sequence`** (was there an `answer` at the end?) and **`Blocked`** (did the gate fire when expected?). A healthy `real_nav` ends with `answer` and 0 blocks; a healthy `blocked_nav` shows `navigate` attempted and the gate blocking it before Playwright sees the URL.\n", "\n", "The deterministic-picker pattern lives in `_check_safety()` — it's a Python function comparing literal strings against pattern lists. The LLM is never asked \"is this URL safe?\" because that question is prompt-injectable. The Python check looks at the raw `target` field that came out of the structured-output schema and decides allowed/blocked deterministically." ] }, { "cell_type": "markdown", "id": "924447ed", "metadata": { "papermill": { "duration": 0.005929, "end_time": "2026-05-28T05:03:14.192995+00:00", "exception": false, "start_time": "2026-05-28T05:03:14.187066+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 10 · The mock-environment alternative\n", "\n", "If you can't install Playwright (Docker/CI env, restrictive corporate proxy), the library also ships a `ComputerUse` architecture that mocks the screen as a Python dict. Same safety-gate pattern, no real browser. Useful for unit-tests of the agent loop:\n", "\n", "```python\n", "from agentic_architectures.architectures import ComputerUse\n", "arch = ComputerUse(llm=llm, initial_screen={\"url\": \"...\", \"fields\": {}}, blocked_domains=[...])\n", "```" ] }, { "cell_type": "markdown", "id": "d1800f1e", "metadata": { "papermill": { "duration": 0.005055, "end_time": "2026-05-28T05:03:14.198050+00:00", "exception": false, "start_time": "2026-05-28T05:03:14.192995+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 11 · Failure modes & extensions\n", "\n", "| Failure | Mitigation |\n", "|---|---|\n", "| **Page load timeout** | Slow site or network blip | Increase Playwright timeout; retry once |\n", "| **Click ambiguous** | Multiple elements match visible text | Use `.first` (we do); add CSS-selector tool variant |\n", "| **JavaScript-heavy page** | Content not in DOM after `domcontentloaded` | Add `wait_for_selector` between actions |\n", "| **Login walls** | Page requires auth | Out of scope; would need credential vault + 2FA flow |\n", "| **Browser crashes** | Headless Chromium dies mid-run | `_ensure_browser` re-opens; consider retry-on-fail |\n", "| **Safety gate too lenient** | Sensitive pattern slips | Expand `sensitive_patterns`; review per environment |\n", "\n", "Extensions: (1) screenshot tool (return base64 to LLM for visual reasoning), (2) form-fill tool with structured input + validation, (3) per-domain rate limiting + CAPTCHA detection, (4) human-in-loop confirmation gate for high-stakes actions.\n", "\n", "**Production deployment**: replace headless Chromium with a sandboxed VM/container; restrict egress via firewall; audit-log every action with verdict; add session timeouts.\n", "\n", "Reference: Playwright (https://playwright.dev/python/), Anthropic Computer-Use 2024." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.0" }, "papermill": { "default_parameters": {}, "duration": 27.940077, "end_time": "2026-05-28T05:03:14.898355+00:00", "environment_variables": {}, "exception": null, "input_path": "all-agentic-architectures/notebooks/34_computer_use.ipynb", "output_path": "all-agentic-architectures/notebooks/34_computer_use.ipynb", "parameters": {}, "start_time": "2026-05-28T05:02:46.958278+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }