{ "cells": [ { "cell_type": "markdown", "id": "e57056be", "metadata": { "papermill": { "duration": 0.002006, "end_time": "2026-05-27T13:31:56.937331+00:00", "exception": false, "start_time": "2026-05-27T13:31:56.935325+00:00", "status": "completed" }, "tags": [] }, "source": [ "# 17 · Reflexive Metacognitive — agent reasons about its OWN capabilities\n", "\n", "> **TL;DR.** Before answering, the agent applies a **self-model** (a written description of what it can / can't do) to the incoming question and routes to one of four destinations: **answer** / **use_tool** / **partial** / **escalate**. A Python deterministic override forces `escalate` if the LLM admits `requires_credentials=True` OR `capability_match <= 2`, regardless of the LLM's chosen route.\n", ">\n", "> **Reach for it when** the agent serves a domain where giving a confidently-wrong answer is much worse than declining (medical, legal, finance, safety-critical).\n", "> **Avoid when** any answer is better than no answer (chat, casual research).\n", "\n", "| Property | Value |\n", "|---|---|\n", "| Origin | Metacognition (Flavell 1979); LLM safety practice; Constitutional AI (Anthropic 2022) |\n", "| Routes | answer / use_tool / partial / escalate |\n", "| Override layer | **Python deterministic** — force escalate when LLM admits credentials gap or low capability |\n", "| Cost | 1 routing LLM call + 0-1 follow-on LLM call depending on route |\n", "| Composability | `use_tool` route in production hands off to ToolUse (nb 02); `escalate` hands off to human |\n", "\n", "This is the SAFETY-FIRST end of the architecture spectrum. The whole point is that \"I don't know\" is a first-class answer." ] }, { "cell_type": "markdown", "id": "fdba75b9", "metadata": { "papermill": { "duration": 0.009853, "end_time": "2026-05-27T13:31:56.952162+00:00", "exception": false, "start_time": "2026-05-27T13:31:56.942309+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 2 · Architecture at a glance\n", "\n", "```mermaid\n", "flowchart LR\n", " A([task]) --> C[Classify
self-model + capability_match,
requires_credentials, requires_external_lookup]\n", " C --> O{Python override?
credentials=True
OR capability ≤ 2}\n", " O -->|yes| ES[Escalate
FORCED by Python]\n", " O -->|no, follow LLM route| R{LLM route}\n", " R -->|answer| AN[Direct answer]\n", " R -->|use_tool| UT[Use tool
→ ToolUse in production]\n", " R -->|partial| P[Partial answer
+ explicit gaps]\n", " R -->|escalate| ES\n", " AN --> Z([response])\n", " UT --> Z\n", " P --> Z\n", " ES --> Z\n", "\n", " style C fill:#fff3e0,stroke:#f57c00\n", " style O fill:#fce4ec,stroke:#c2185b\n", " style ES fill:#ffebee,stroke:#c62828\n", "```\n", "\n", "**Two safety layers:** (1) Python deterministic override that *cannot* be overridden by an LLM; (2) LLM's own metacognitive route. The Python layer catches the cases where the LLM correctly admits a credentials gap but then chose the wrong route anyway." ] }, { "cell_type": "markdown", "id": "fa745a21", "metadata": { "papermill": { "duration": 0.003094, "end_time": "2026-05-27T13:31:56.962231+00:00", "exception": false, "start_time": "2026-05-27T13:31:56.959137+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 3 · Theory\n", "\n", "### 3.1 · Why self-models matter\n", "\n", "Most agentic patterns are *outcome-oriented*: they try to solve the task. Reflexive Metacognitive is *capability-oriented*: it first asks *\"should I even try?\"*\n", "\n", "For high-stakes domains (medical, legal, finance), this question is more important than the answer. An ungrounded medical answer can harm a user; an honest \"I'm not qualified, see a doctor\" cannot. The architecture makes that escalation a *first-class action*, not a fallback.\n", "\n", "### 3.2 · The deterministic override layer\n", "\n", "The architecture has TWO routing layers:\n", "\n", "```python\n", "# Layer 1: LLM classifies\n", "decision = self._decider.invoke(prompt) # route = answer / use_tool / partial / escalate\n", "\n", "# Layer 2: Python overrides (deterministic, can't be talked out of)\n", "if decision.requires_credentials:\n", " decision.route = \"escalate\" # always, regardless of LLM choice\n", "elif decision.capability_match <= 2:\n", " decision.route = \"escalate\" # always\n", "```\n", "\n", "The Python layer exists because a sycophantic / prompt-injected LLM might admit `requires_credentials=True` but still choose `route=\"answer\"`. Python forces the correct downstream action when the LLM's *features* (booleans, integers) imply escalation even if the LLM's *route* doesn't.\n", "\n", "This is the **deterministic-picker pattern** applied to a categorical decision — same as Mental Loop's `scoring_fn` (nb 10), Ensemble's `majority_vote` (nb 13), Dry-Run's irreversibility hard-cap (nb 14), RLHF's composite score (nb 15).\n", "\n", "### 3.3 · The self-model as a configurable string\n", "\n", "```python\n", "arch = ReflexiveMetacognitive(self_model=\"You are a financial advisor specialised in retirement planning. ...\")\n", "```\n", "\n", "The self-model is just a string the LLM reads in the classification prompt. To deploy in a new domain, change the string. Production deployments often layer multiple self-models — a base \"what I am\" + a per-task \"what's allowed in this context\".\n", "\n", "### 3.4 · Where this sits\n", "\n", "| Pattern | When it acts safety-first | Mechanism |\n", "|---|---|---|\n", "| Dry-Run (nb 14) | Before SIDE-EFFECTS | predict effects + Python hard-cap |\n", "| **Reflexive Metacognitive** *(this notebook)* | **Before ANSWERS** | self-model + Python override |\n", "| PEV (nb 06) | After each step ran | verify outcome |\n", "| Constitutional AI (nb 32) | During generation | self-critique against rules |\n", "| Multi-Agent (nb 05) + this | Delegate hard cases | escalation route → human reviewer |\n", "\n", "In a production high-stakes pipeline, you'd often stack: Reflexive Metacognitive routes the easy stuff, escalates the hard, the human-in-the-loop or domain specialist handles the rest.\n", "\n", "### 3.5 · What goes wrong (you'll see in § 9)\n", "\n", "1. **Self-model too vague** — LLM bins everything as \"moderate\". Tighten with concrete examples.\n", "2. **LLM admits gap but routes 'answer'** — exactly what the Python override exists for. Watch §9 for `override_applied=True` cases.\n", "3. **Over-escalation** — agent refuses easy questions. Means the self-model says \"WEAK: everything\" too aggressively.\n", "4. **`use_tool` route ungrounded** — agent says \"use_tool\" but no tool actually fires. In production, this route must hand off to a real Tool Use architecture (notebook 02)." ] }, { "cell_type": "markdown", "id": "07785cef", "metadata": { "papermill": { "duration": 0.004287, "end_time": "2026-05-27T13:31:56.975738+00:00", "exception": false, "start_time": "2026-05-27T13:31:56.971451+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 4 · Setup" ] }, { "cell_type": "code", "execution_count": 1, "id": "edd99de6", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T13:31:56.983331Z", "iopub.status.busy": "2026-05-27T13:31:56.983331Z", "iopub.status.idle": "2026-05-27T13:31:58.338326Z", "shell.execute_reply": "2026-05-27T13:31:58.338326Z" }, "papermill": { "duration": 1.358747, "end_time": "2026-05-27T13:31:58.338326+00:00", "exception": false, "start_time": "2026-05-27T13:31:56.979579+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
Provider: nebius  ·  Model: meta-llama/Llama-3.3-70B-Instruct ─────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mProvider: nebius · Model: meta-llama/Llama-\u001b[0m\u001b[1;36m3.3\u001b[0m\u001b[1;36m-70B-Instruct\u001b[0m \u001b[92m─────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from agentic_architectures import get_llm, enable_langsmith, settings\n", "from agentic_architectures.architectures import ReflexiveMetacognitive\n", "from agentic_architectures.ui import print_md, print_header, print_step\n", "\n", "enable_langsmith()\n", "print_header(f\"Provider: {settings.llm_provider} · Model: {settings.llm_model}\")" ] }, { "cell_type": "markdown", "id": "c032ac17", "metadata": { "papermill": { "duration": 0.002004, "end_time": "2026-05-27T13:31:58.345868+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.343864+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 5 · Library walkthrough\n", "\n", "Source: [`src/agentic_architectures/architectures/reflexive_metacognitive.py`](../src/agentic_architectures/architectures/reflexive_metacognitive.py).\n", "\n", "Two key pieces:\n", "\n", "1. **`_MetaDecision`** schema — captures `capability_match` (1-5), `requires_external_lookup`, `requires_credentials`, `route` literal, `reason`.\n", "2. **`_classify`** — calls the decider LLM, then **applies the Python override** before returning. The override is encoded in the same `decision` dict so the rest of the graph and the trace see the OVERRIDDEN route." ] }, { "cell_type": "code", "execution_count": 2, "id": "880d9016", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T13:31:58.354008Z", "iopub.status.busy": "2026-05-27T13:31:58.354008Z", "iopub.status.idle": "2026-05-27T13:31:58.372799Z", "shell.execute_reply": "2026-05-27T13:31:58.370017Z" }, "papermill": { "duration": 0.029151, "end_time": "2026-05-27T13:31:58.375019+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.345868+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Decision schema ---\n", "{\n", " \"description\": \"The agent's metacognitive verdict on how to handle the incoming question.\",\n", " \"properties\": {\n", " \"capability_match\": {\n", " \"description\": \"How well does this question match your STRONG capability area? 5 = squarely in your strong zone; 1 = clearly outside your competence (needs human expert).\",\n", " \"maximum\": 5,\n", " \"minimum\": 1,\n", " \"title\": \"Capability Match\",\n", " \"type\": \"integer\"\n", " },\n", " \"requires_external_lookup\": {\n", " \"description\": \"True iff a definiti...\n", "\n", "--- Default self-model ---\n", "You are a general-purpose AI assistant. Your capabilities:\n", " - STRONG: general knowledge questions, coding, writing, summarisation, math.\n", " - MODERATE: current events (depends on training cutoff), niche technical domains.\n", " - WEAK: medical diagnosis, legal advice, financial advice, anything requiring professional credentials or real-time live data the user could verify themselves.\n", "Your training cutoff is at most 2 years ago — facts about the last 6 months may be stale.\n" ] } ], "source": [ "from agentic_architectures.architectures.reflexive_metacognitive import _MetaDecision, DEFAULT_SELF_MODEL\n", "import json\n", "print('--- Decision schema ---')\n", "print(json.dumps(_MetaDecision.model_json_schema(), indent=2)[:500] + '...')\n", "print()\n", "print('--- Default self-model ---')\n", "print(DEFAULT_SELF_MODEL)" ] }, { "cell_type": "markdown", "id": "0c46e7f9", "metadata": { "papermill": { "duration": 0.004417, "end_time": "2026-05-27T13:31:58.382369+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.377952+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 6 · State" ] }, { "cell_type": "markdown", "id": "f31fe339", "metadata": { "papermill": { "duration": 0.004016, "end_time": "2026-05-27T13:31:58.394377+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.390361+00:00", "status": "completed" }, "tags": [] }, "source": [ "| Field | Set by |\n", "|---|---|\n", "| `task` | caller |\n", "| `decision` | `_classify` (includes `route`, `llm_route`, `override`, `capability_match`, …) |\n", "| `deterministic_override` | `_classify` (True iff Python forced a route change) |\n", "| `final_answer` | one of `_answer`, `_use_tool`, `_partial`, `_escalate` |" ] }, { "cell_type": "markdown", "id": "1a07e6ce", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T13:31:58.396383+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.396383+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 7 · Build the graph" ] }, { "cell_type": "code", "execution_count": 3, "id": "4a9cd1bb", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T13:31:58.405037Z", "iopub.status.busy": "2026-05-27T13:31:58.405037Z", "iopub.status.idle": "2026-05-27T13:32:03.888664Z", "shell.execute_reply": "2026-05-27T13:32:03.885289Z" }, "papermill": { "duration": 5.48445, "end_time": "2026-05-27T13:32:03.889487+00:00", "exception": false, "start_time": "2026-05-27T13:31:58.405037+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAFNCAIAAACWs2FeAAAQAElEQVR4nOydB0AURxfHZ+/ovQpKUUEsiIKKvceuMfbYzWeLNfZeEmuMNcXee6KxxZZYorFHjWLBhgKC0lR65+Buv3e3ch5wIIfc3nD7fvG7b3a23LEz+583783OGLAsSxAEQRC9w4AgCIIg+gjqO4IgiH6C+o4gCKKfoL4jCILoJ6jvCIIg+gnqO4IgiH6C+o5QxMvHKcEBKfHvcqQSViqTyaSM6l6RSL4pk+UZ0cuIGIZlZepG+cLxyoMZEWFlxcpUvbLqdeXHsCwp8EVGxiKxATGzErt6mfq1tCMIQg0Mjn9HdM79K/EPLiWlJklBbQ2M4J/I2FQkV15ZHn0HOQY4RVbNlKsuSz4cCimW2/VBoJVSrkwUlsmdDv+J5Kfn/SIZKYjYiJFKZdlZrCQTGiTWyJRxr2bWYXB5giC6BvUd0SUPribc+jNemsPaVzCu85mVl681KcskJ2Rd+yMu6kVGVhbrVs3ki69dCYLoDtR3RGfsWfoyNUFa1d+8bT99s3aD7iRePR4nzSZ9p7nZOBgRBNEFqO+Iblg3Jdi+vGH/6RWJ/nL12LuH15J8W1g161aOIAjvoL4jOmDD1GD/9pYNOjgRAbBhWnCPcS7lK5sSBOEX1HeEb9ZPC+48olzl6lZEMGyaGVzN37J1H0G0Zwg9iAiC8MjG6cGNu9gKStyB0curPL2VEvIghSAIj6C+I/yxd2mYrZNh3db2RHi07u1wdu8bgiA8gvqO8ETg9cSUhJx+0/Q5oFoENRrZWDkYHFgVThCEL1DfEZ64cSLWs7YZETCDZlWKjcyWpEoIgvAC6jvCB09vJWZnkw5DKhBhY2krPrIhhiAIL6C+I3xw52KiQwVDInjqdbBNeIf2O8ITqO8IH6TE5dRqyvfcA+3atYuMjCQaEhIS8vnnnxPt4NPQhiHk4fV4giDaB/Ud0ToRL9JZltRsbEN4JDo6OiEhgWjOkydPiDYxtRCHBKQRBNE+OD8wonVe3EsxMGKIdmBZ9rfffjt16lR4eHjlypUbNWo0ZsyYe/fujR49GvZ269atZcuWq1evBqv88OHD//33X1RUlIeHR/fu3Xv37s1doU2bNiNGjLh48SKcNXjw4L1790Kmv7//5MmTBw4cSEobawfDxNhsgiDaB/Ud0TogZ0bG2uopHjhwYMeOHZMmTWratOmlS5fWr19vbm4+dOjQn376CTKPHz/u4uICh4HEg7LPnTuXYZiwsLDly5eXL18eToFdhoaGx44da9CgAah8vXr14IBz585Bg0G0g005w3cRWQRBtA/qO6J1sjOkhsbast8DAgK8vb05j3mPHj3q16+fnp5e8LBly5alpaVVqCAfwAO2+YkTJ27cuMHpOwi6tbX1tGnTCC+YWBrIcE4QhBdQ3xGtI2MZojVF8/X1Xbt27aJFi+rUqdOiRQtXV/VTroMbByz969evgxuHy+Hseg5oIQhfiFiWkREE4QHUd0TrGJkwmelSoh0GDBgADpnLly8vXLjQwMCgXbt2EyZMcHR0VD1GJpNNnDhRIpGMHz8ejHdLS8vhw4fn+YVG/E3RnpEmZRk04BE+QH1HtI6lveG7SG0N+haJRD0UhIaG3r59e8uWLampqT/++KPqMc+ePXv8+PGGDRvAyc7lpKSklCunmznZE99miw205a1CEFVwfCSidSp7m2VnactihUBoSEgIJDw8PPr169e/f/+goKB8xyQmJsKnUtBDFRAdEf9GYmmDdhXCB6jviNap4msF/vewJ6lEC5w5c2b69OlXrlxJSkq6du3axYsXwSMP+ZUqVYLP8+fPP3r0CKQfXDd79+5NTk4OCwtbuXJlo0aNoqOj1V7Q3d09Njb20qVLSk996ZKRKqtcC9f6QPgA9R3hAzMr0Z2/tfLS5rx580C+p0yZ0qZNm8WLF7ds2XLu3LmQD4HWrl27btq0CaKvzs7OS5YsCQwM/OyzzyZPnjxu3LjevXuD7iuHwKvSrFkzPz+/adOmnT17lpQ2kS/SWRlp2BGX60P4ANdvQvjg39Oxdy8kjl9ThQibvUvDsyWyYQsrEwTRPmi/I3zQuIsDw5Abp94RYZMUm91uEBrvCE9gnAfhCZ8mlvcvJTX53FHt3ri4uF69eqndZWFhkZqq3ncPnpkdO3YQ7bBLgdpdDFNox/frr78eMGCA2l0HVoUZmxE3L3OCILyA/hmEP7bND3VyN+460qXgLplMlpamftYtiURS2Ph00FlQf6IdsrKy4KvV7srIyDA1VR8jhZ9qbGxcMD8lUbJ74avxPwrdQ4XwCeo7wivrpwT3m+ZiX0FwA0g2zQz2bmjVoic6ZxD+QP87witdv3Y+uEbjOdnLOjsWBDtUMEJxR3gG7XeEb5LiJPuWvhq6uKKZuSBWdNo8K9ivlW3DjvYEQfgF9R3RATHhGYd/iqzmb9FuoDPRX2JepR/fEAWWe68J7gRBeAf1HdEZm2eFGBgyrfs6evhYEr3j4OpXcdESv8+smnRGtwyiG1DfEV3y546ol4/TTSxEnrXMW/V2ImWfR/8m3L+UnBSbbWVvMHhOJYIgugP1HdE9p7dHRoZkZmeyYkPGxFxkaWNgYs6IDcTyieNzEYmITDFtOsMQeZ1l4D/CVV4uh/tUHvbhSDhXMf+8sqa/H72e9wqKKeIZRSL3ajL5/324iIiRwQ9SHJX34rKsDFlmqiw1KScrUwb5No5GEEa2sOZvzmEEUQvqO0ILifFZd88lvAnLysyQ5WRLZTkgoCr6LmZkUnldlYsyl5Wb4vSa22JEoMsfrskSyGeYXP2WyUB/RdAGsConvr8U4c5/L/pK+VYek/sD3n+rMt/ACHaJjIwZOycjL3/zqn7WBEHoAPUdERDdunVbv359YWs8IYiegfMTIAIiJyfHwADrPCIUsK4jAgL1HREUWNcRAYH6jggKrOuIgEB9RwQF1nVEQKC+I4IC6zoiIFDfEUGBdR0REFKpFPUdEQ5Y1xGhkJ2djeKOCAqs7ohQQOcMIjSwuiNCAfUdERpY3RGhgPqOCA2s7ohQQH1HhAZWd0QoQHzV0FAQKwIiCAfqOyIU0H5HhAZWd0QooL4jQgOrOyIUUN8RoYHVHREK6H9HhAbqOyIU0H5HhAZWd0QooL4jQgOrOyIUUN8RoYHVHREKqO+I0MDqjggF1HdEaGB1R4QC6jsiNLC6I0JBJBLZ2NgQBBEMqO+IgIiPjycIIhhQ3xGhAM4ZcNEQBBEMqO+IUEB9R4QG6jsiFFDfEaGB+o4IBdR3RGigviNCAfUdERqo74hQQH1HhAbqOyIUUN8RoYH6jggF1HdEaKC+I0IB9R0RGqjviFBAfUeEBuo7IhRQ3xGhgfqOCAXUd0RooL4jQgH1HREaqO+IUEB9R4QG6jsiFFDfEaGB+o4IBdR3RGgwLMsSBNFrfH19xWJxvsyRI0eOGTOGIIj+IiIIou94eHiI8uLu7t6/f3+CIHoN6jui/3Tp0gU0XTWnffv2uBYrovegviP6z+DBgytWrKjcdHZ27tGjB0EQfQf1HdF/jI2Ne/XqBZ/cZvPmzcuXL08QRN9BfUcEQb9+/VxcXCAByo6ed0Qg4PgZRGeEBCaHPs7IznxfAxlCcusiq9iSA25zmSz/iYy82jKKhPw4We5pIuZDWuVq3CXZ6JiYJ08eg77X9PZRrfgMk+cpEDOslGWIOt5fP/+lC2YormNA7J0N/Ns6EATREajviG7YNj84O5MYGIqyJe9roEjEyBTyrCrTYoZIlTWUYcl75X2vqKDvkGRzGwBGpJKGA+TCnbsJ23C2/HxGJD/r/Xcpvo6VqQg6IyasNM9PVco3oziUFAS6wQUaIUMTRpbDwrc07mLn19KOIAjv4PtNiA7YOCPYrZpxy95uRN8JfZh04+Q7YzNRjfo4XAfhG7TfEb7ZPDPYu7GFX2tnIhj2LQluM9Chqh9KPMIrGF9FeOXioWhwgAhK3AEHV8PrJ+MIgvAL6jvCK9GhWZY2hkRgVK5lnZWCHWWEb1DfEV6RZMjjnkRgmFsb5OQI7q9GdA7GVxFekUpZmZQIDlZllA+C8AXqO4JoHYagcwbRAajvCK8I1EnBMoL90xEdgvqO8IuIYYQX9JG7Z3AgMsI7qO8Ir7AyVoCOaIZ71RZB+AX1HUG0D4o7ogtQ3xFeEYmF6J8h6JxBdAHqO8IvLBHiWBJWxLJowyN8g/qO8IpMJshAI8MyDJrwCN+gviO8wnBTtgsOFHdEB6C+I7zCKmZhFxosK8IYK8I/qO8IonUYRoYmPMI/OL8YwiulOH6me8+2e/ZuI6XKkaMH2rRrwKVDQ4NnzvqmXYdG+3/dST4R+QtOaL8jfIP2O8IrMinV7zd51/AZPGgEl75w8czDwHsLv1vh4eFFPh2MryK8g/qOIB+oUcMH/nHptLRUZ+cKTZq0IJ8OijuiC1DfEV4RiRhN39SXSqWHDu/fvWcLkdvXtf731ahatfzyHXP02MGbN68+ffrIyNjYt3bd4cPHuVRwJYpw7pGjv509e+p1RHhF98r+/o2GDR0jFosLywf/zIaNay6cv/3NxOGPHj2AK7Ru49+5U7c//zq+9uftPj6+3NcFBz+fPPXrk8cvkeIhwCnvERpA/zvCKyUY/75l69rjxw8tWrhq3pyljo5OM2d/8+pVmOoBgYH3165bWbOm76JFq2bNXJiQEL/0+3ncrqNHD+zbv6N3rwEHfj3VtWuv03/+ceDgniLylYCad/uid6VKHv9cuDN92nwnJ+e/L/yl3Hv5yt/WVhosporzAyM6Ae13hFfEYkasiVGRlJz0+6F9kybOqu/fCDYbNmyanp4WFx/r7l5JeYy3d62d2393dXU3MJDX55zs7DnzJsOJ1lbWDx4GVKvm3aHD55D/eZcederUz0hPh3Rh+YXR9fNeBw/u+Wb8dLDxYfOfS+c/+6wDKT74/iqiC1DfEV4BO1aj8GrYyxD4rF69JrcJCr5o4cp8x4DmRkVFrN+w+umzR2lpaVxmYkI86Dt4VMD8X7FyUe3adRo3bsE5bYDC8gujS+fu23dsuHXrOrjjQ0ODIyNft23TiRQfRobvryL8g/qO8Iqm42dSU1Pg08TYpIhjrl+/PO/bqQMHDB319URPT687d2/NmDme2wUeGDMz8+s3Li9fsRDahlat2o0aOcHBwbGw/MK+wsbGtmmTlhcungF9B+dMVa/qqh0IBKET1HeEVzSdn8Dc3AI+wSdTxDGn/jwGEdcRw8dxm1yTwCESicD9Av/CwkIDAm7v2rMlLS31+yU/FpZfxLeACb9w8azklORr1y917tSdaAQ6ZxBdgPFVhFc0nZ+gSpVqYF+Du1x5+qw5E8+ePaV6THJykqNDOeXm1asXlWk48qXCwwOR0p49+/Xq2T84OKiI/CIA17+VlTV44cPDX7Zt05FognzIEEo8wjuo7wiviA2IWKSB0llYWLRr2/n48UN/nTlx7/6dtetW3r17SzlEnaOKjPqeeQAAEABJREFUZ9X/7tyEvTk5OYcO7+cyY95EE8U7St8umH7jxhUIt968ee3qtYs+NX2LyC8C6Hl06vjFkaO/NWncwtpag8EzRB51YHEIPMI/6J9BeIWVMTINBwtOnDDzp59/WL1mqVQqBSlftGBlPt/3sGFjwYEzb/6UjIyMnj36zZq5MDo6ctbsCXPnLJk6Zd669avmzp8Ch9nZ2YNDpk/vQZAuLL9omjRpuXvP1vbtuhBNQeMd0QUMLvuL8Mm2+S9NzMTdxrqTMsiBg3tOnDi8b+8f4L7X6MTIF6l/748e/2NpzHOAIMUG7XeEV8ro+nz379+Nio7YvWfLgu9WaCruCKIrUN8RXlHEV8ues2LGrPFisXj4sLENGzQhJQHnf0d0AOo7wisQXC2L9vu5M/+STwL9oIgOwJ4mwis5OaxMKsQFtvH9VYR/UN8RPkhNTYXPS5cuJSUnMkJ1VIwZM+bo0aMEQfgC9R3RFrGxsfAZEBDQtWvXgwcPQtrJycnK0ooIT+BZeNAYZujQodnZ2UQerb0/atSoM2fOEATRJqjvSGkSEREBn8+fP2/fvv2ePfIZdx0dHTdv3jx8+HAiXz2jhmL+RcF5KhgiAw98gwYN+vbtC5t+fn4jR47kJsI/d+4cpK9evUoQpLTB+CryqQQHB1epUiUmJgbEq3Xr1gsWLLC3t//tt9/gE/a6ubmpHiyf/53i9fm0hHx9j7yjhvz9/bkENIQODg4ZGRmQ3r1797Vr18aNGwcNAARkGcF6spBSAvUdKQn37t2rU6cOqFLbtm3r1av3yy+/WFpanj592sJCPh0Yp+xqEWk4v5h+wBQ5P0HdunW5xFdffVWrVi0uvXLlSugGzZw508vLKycnh5vaHkE0AisNUizAcXz37l0fHx9Q8ObNm1erVm3btm2GhoYXLlwwMZFP3mtubl6c67AiIkiztLjj35VaP2PGDGhEuXsFKh8fH//999+XL18etR4pPlhRkELJzMy8fft21apVnZ2dITZobW29YsUKIp+g8b2z2ECBJpckrHz+dwGOFJSVIOoAPSQusXr16ocPH3LvzUIkA0R/7dq10GGCRheaWIIghYD6juQhLS3t5s2b4DQHWZ8/fz5Yi3PnzoX8ffv2kdJAVDbfb9I5tWvX5hLgow8MDOTs+i5dulSoUGHXrl1QTBKJxMzMjCCICqjviHxw+uXLlyHK17Bhw82bN0OkdMKECUThAialjTDjq6WL0kd/7tw50Hqi8J516NAB7H0IhEALDeqPWo8Q1HfBkpKScv78eXCdd+7c+cSJE8+ePRsyZAjkT5kyhWgTExORgZHg/O/gkRIbaOWv5rTe1NQUnGbBwcFEUbJ9+vRp167dt99+C157Y2PjYoZGEP0D9V1AZGRknDx5Emy9gQMHgmP96dOnPXv2hPwBAwYQvjA2J2kp2URgvHmVzoiJtqlSpQp8QrAEtD4sLAzS0dHRY8aM+fLLL8ePHw9pcNlzA5wQgYCuUD0nKytr7969P/30E6RDQkJevnzJWXxt2rQBx3qNGjUIvzTu4pieLDgHTfjjNEcXY8IjlSpVgs+aNWteuXKFa8Wh6MFf/9tvvxFFTUhOTiaIvoP6rofIZLJt27bNmTMH0tBDj4uLa9WqFaR9fHxmzpypjNTpBFcvM7vyBr+tCCaC4fyv4VkZ0l7fuBEdATFYIl98qglEWcBvA+mgoKBu3bpduHAB0k+ePElKSiKIPoLrN+kPO3fu/Pfffzdu3AgeGEg3btzYz8+PUMnlwzFP7qS6epqC3BsaGxU8gGEU72/mHVSYLxNSitqbx68NGfmG1+eepPYA+eks8/7dI+76+Q5TPV75TqliDnt5SibPYPMcr9glT8jYt5Fp4U/kE6sNW+BJ6ANk3dra+sCBA1u2bFm1alXdunXv3r3r4eFha2tLEL0A9b2sIpVKxWLxvn37IEz6ww8/lC9ffs+ePdAfr1evHikLXP3jzYt7aZkZMlnxvfFsaSySoWgWPrqrYDtRzKspTxQbMmID1r6CYa/xFQn1pKWlQRh2w4YNR48eBePAzc0NnPje3t5FvIqM0A/qe1kCnOnGxsZHjhz5/fff582bB570U6dOgacVHC+kjHPt2jWQle3bt5MyzvHjx//7778lS5aQMgv32tSKFSv+/vvvY8eOge6fO3eufv36aNeXOVDfaYczrM6cOQOd6IkTJ7Zs2RKk0NnZmRssoQekp6ebmZnduHEDHMRELwgICABfR0xMDBQTKeNALEckEs2ePRtc9mDap6SkXL9+HbQe7foyAeo7jUBQ1M7ODqJhK1euHD58eI8ePe7fvw/WU8WKZaCnrxG3bt3asWPH5s2bid4BaggO7qFDhxI9IiMjA7omCQkJ4Ml5/fp1YGAgtMo2NjYEoRIcP0MLkZGR8AkBrnbt2kGnGNIuLi5bt24FcSeKGcP1T9yJQt/1UtyBnj17Qt9Lz4YhmpqaLl26FMQd0tDrgng+N/QWhP706dM45pI20H7XJaGhoR4eHs+fPx85cmS3bt2mTJkSHR0NHnYw3ole8+rVq7Nnz8JfTfQd8GWDCEKB6kGMpAgiIiLAf1iuXLnx48eDAycxMbFVq1b43qzOQX3nG7B0IC767t07EHQw1RcuXAgPg4GBgXBeLMzJyenTp8/u3butrKyIAAAXNnhpli1bxo1D13uCg4P37NlTu3bt3r17nzx5Ev58qOc4H45OQH3XOpmZmRBwa9SokVQqbdy4Mdg1q1atgkyGYcBUJwIDAgk1a9YU4Ky24eHh1tbWQnNVP3r0COIQLRXs378fjJhOnToZGRkRhBdQ37VCeno6eNLBTofnuUuXLuCE+eWXXwgR5tIW70lJSenQocOJEyccHByIIJFIJM2bNweTFvwYRHjcuXPnzz//hK5bjRo11q1b5+bmBo8GrlWiVVDfSw3QL4gWVqtWDSru8OHDLS0tFy9eDJ8EUTR4EGzw8vISYJdFFfBN/fXXX127diXC5urVq//888/o0aOhqVu0aBF06Xr16kWQ0gb1/ZNISkq6du2au7s7mOrffvttVlbW1KlThWmdFQYYrRBH3bRpk6mpKUFymTNnzvfff08QQi5evHjz5k24IampqXBPWrRo0bFjR4KUBqjvGpOcnAymh729fbNmzTZv3hwZGfn111+7uroSRB3bt29v2LChfo8eKQHgvvv999+XL19OkFxAi86dOwfh2XHjxoWFhcHDBUIPjnuClBTU92IBvhdwHZqYmHTr1u3YsWOBgYEDBgzQmzdItcSGDRvGjh1LkELgVsq+cOFCmzZtCJIXqVQKd+bt27eDBg0CtycEaXv27AmGAkE0AfW9UKC3eOTIkezs7BEjRlxXAOIO7nWCFAPobrdq1ap9+/YEKRI9mK9G28AzePnyZXB+QjwWotNXrlwZPHiwbqe5LiugvucB6tC+ffvevHkD8vTs2TPoLYJtBcEfghSb27dvN2jQIDY2VrDjZDRFn+ar0TYQzoGIF/R7wE2/bdu2oKCgUaNGYU+6MFDf5V6/jRs3gtdvzZo10B88fPgwVB30F5eMGTNmtG7dulOnTgTREL2cr0arZGZm3rhxw87Ozs/Pb9GiRYmJiVOmTMFImCpC1HduSrwtW7ZcvXoVon9gC+zcubN+/frY4/sU4uLiIOYM/WgMiJWYdevWDRkyRCCv9ZYuGRkZ0HF0d3evXLkyhGeNjIy+++47nPhMKPrOxbL27t176tQpsNNdXFzAt16jRg1vb2+CfDKbNm0CL1bz5s0J8mkIZL4arQJ2PWg9PNrgIezVq5ebm9uKFSuE+dKsPs8fCU06fB46dAjKODAwkCiWll+6dCmIO6QhE8X90wH7ICQkRCwWo7iXCoaGhs2aNVu5cmVUVBRBSoSJiQm4WLnwz/79++FJ56zYpk2bgv+QKBpRIgz0zX4HHxx0yk6fPr1hwwZwxkF09NatW05OTtxy8kjp8s8//3h6esKDhLNHlTrCnK9Gq4Bd//DhQy7431XBnDlzuPVziJ6iD/Y7BEXhEzy/oObckvDggwPHOjesuGHDhiju2gAaTmhHweOJ4q4NKlasCDcWai9XvZFPB+x6EHdIgEUCctGlSxdIQz+pcePG4GAkCuuQ6BdlVd/BuoHPe/fuQTTv5MmTRKHp4FLnZrEAxwsONdMe3FIktra2q1atIojWAJfx9evXoR0lSGkD99bX1xcSXl5eoPWtWrWC9NOnT0HrwaML6Tdv3pCyT5nRd5lM9uTJE0iAt7dRo0ZHjx6FtKurK5iQw4cPhzQYktiZ5QFuqhBIVK1alSBaxsDAgJuMjLvniDYAra9evTokQNxB6/39/YnCdqxfv/7Zs2ch/fr1a1I2oVrfIQxy9+5dSIC/DDqqv//+O6TBmX716tXJkydD2tHRUTjLYlACGO+7d+8mCL9Ax3TmzJkE0TKg9eAJgETHjh2h58TZ+CD6oPWcFoF9ScoO1MVXpVIp9EmhITU0NAQ7HULeq1evlkgkuCaAbomIiICQNU55qEO4Mb74hoFOAP8BeOft7OzWrl0L9s2BAweqVKny+PFjyl9up07fQc2DgoJASnDif3rIyMhYsmTJd999h62szlmxYkWzZs2aNGlCEB0BmpmWlgaegzFjxvTt25fz3dMJdfoOUgKWO4o7gqjlxo0bxsbG9erVI4iuAY8N+Bu4MTl0gvPPIB8HAiGPHj2qU6cOQRCk7EBdfHXhwoWXLl0iCE1Ab3TatGkEoYAHDx7cu3ePIBQQGBh4+/ZtQjHU6TsEkdLT0wlCEyYmJtxAAkTngLhfu3aNIBQA8VUIdxOKoc4/k5mZKRaLwQVPEAQpAOg7GEBNmzYliK4BfY+Li2vRogWhFfS/Ix8HKsmdO3fq169PEAQpO1Dnn/npp5+OHz9OEJpgGGbMmDEEoYCgoKCbN28ShAJevHhx5coVQjHU6btUKoVoHkEoo0mTJhAaIYiu4ZaNJAgFhISEcBMYUAt1/pmsrCywFvE9GgRRC9jvr169ateuHUF0Deh7cHBwhw4dCK2g/x0pFgEBAbVr18b3zhCkDEGdf2bXrl04fRWFzJo1KykpiSC6JiwsDF8QoYSIiAjK/TPof0eKhb+/v1gsJoiuCQ8PP3HiBEEoIDIykvLBINT5ZyQSCfwkY2NjgiBIAUDfHzx48MUXXxBE14C+37lzp1u3boRW0P+OFIuHDx9WrVrVxMSEIAhSRqDOP3P06NFffvmFIJSxZMkSblk+RLfExMRQ7vMVDrGxsZT7Z6jTd+hPpKamEoQy/Pz80GlGA2/evDl48CBBKCAuLo7ysqDOP5OdnQ0hVvQDIIha3r59e+nSpS+//JIguiY+Pv78+fN9+/YltEKLvnfs2BEqLlHY7wzDEMU78dbW1hcvXiSI7qhTpw4UhEgkkslkkGAV+Pr67tq1iyA80qdPnxcvXkBBKHO44ggICCAIv/Tv3z8oKIiTKSgCKBT4hAeEwg3ZP90AABAASURBVHmbafHP9OvXz9DQEO6UWCwWKYD75ePjQxCdAjFVLgElwgm9hYXFV199RRB++eabb+zt7UUqQCa3EjTCMxMmTADTk1HAFQQkPDw8CH3Qou8DBgxwdXVVzbGzsxs0aBBBdAoUAQi6ag7U49atWxOEX1q0aOHl5aWaY2Bg0KtXL4LwTuPGjfOZnmCV0jlKkhZ9NzIy6t27t+q0M2A50rywoUDo2rWru7u7chNCrNASE0QXDB061NLSUrnp4uKC+q4rhg0bZmtrq9wE25TOsqBo/Ax4tZRSYmZmBpsEoYDhw4dDcXBpqMc0z6ak3zRs2LB27dpcGhwCnTp1whFNuqKOAi4NZdG2bVtzc3NCH3SNjwTbkDPhPT09aV4VRVCANwaKgygcAhAmIYjuALMR/JZE0dBiWegWpQkPHSlqy6JY0wG+fJosyy7h3CMsYUWEYQvdSxiVzZqV2tSrHhgdE925Zd+Qh2pmoWEUp6iF28UyrIhlijskiGVtncV2TqakjJCRlBH5UsKI3pcaBPCVo59U74wimyl4jAr5brx8IBXcO1IIPTuOzUr4zcLcwsejbejDNLbA8aqXg4SMgTL4sFcx0IPJPRJShX5RwWPEDFuplgUpO0SGpKenyCAaTQpUVzbfbcndq1peBZDfCjb3YEtRlca1ewQGBrZp1OpdmOgtm/8Z4W7dRx+T3A01hS6TyszsGJeKZemehzxMUkpZvr+Ju3W5TwFLcm9OETcqf3EoSo3Nd0VCDIlbi3p97gUENKvXPP61UfxrNXrFlfiHb8n9cR+eygJFkO8BURxZ4GmV75B61rIiH+Mj4yMPrHwZ/1YK15aWeGkHdb+tuKcUcW4huzT7NoZA9FtsSBp2tPVraU8o5m1Uxh8bInMy5eWdvywK/s15RV1eW5mC2UWcQQq2pPm+JP935t3Ov7eIL1aLyvmMwq6wtBENmUfj+ARVTmyJjHiRAQlZYQ9LYZU2t4CKj0b1vNDrq9UNRY7YiHjUMm8/sDyhm50LXqYlS8VilYeikD8KbkKRN63wNrHw0wreWLW3uojqX/DaxSxZsPFYGTG1ZIYt8CzisKL0fd+KUEmarHkPJ+fKlkR/uXU2+tnNtO5jnV2rUGqzpCZKdi95VdHbtGUvFyI8kpIyrvwWnZIoG7WsCqGVCwdigh+m1mtvW60O1YZCMXl8Mz7gfLx/e+sG7R0JrWycEVzOzbhVPydhLgckkUj+3hsZHy0ds7LQ56JQfd+1MBTa8O5jaTeaSou9S4ObdbWr3dyOUEZqUsaexZGD59Mrbfxw9Y+oV0/TR/9A43049POr5FjJl9P0rYx+XR7s4mH8+Qg3Qh8g7n5trXwaliPC5sG12IeXEscWIvHq46uP/03ITJMJR9wBz9oWt88kEPo4tjbG3gVXKyTNu1cQGzDn9kcT+nj7WtJ9gjvRO6Dv/iooi9DHHxsiTMzEKO6AbzMHE3Px8U0Raveq1/ent5NNLKibekyrNOnqnJnOSqVSQhmpyVKvejQOveIfGweDqJB0QhmXj0QbGhK9dBG4VZU7Zh9cjSWU8S4q09Edx4a+x9HV6F1Eptpd6kU8K5MRC2+lTUZE3r7KJpQhkxJbe32OfxQfE3NjqYQ6syM9hRExemsMiURM0ltNYuO8IMthTExR399jZm0kzVYflFUv4jkSGSvTdNRLmYeVsSL6/mj4VYRQ16vQCbIcmSSLulshy5ZlZ8uInlLygXPaJEduhuntPdcUVkpyCikmwRnpH4M6UwVBdAmLK7zRThEFpL5fqZgrkAgPRvOx+lqHlb9bIaxYCEIRDPgtUeDppvA3BtULBytQM5al0H5n5G+44QNGL4yYEVHo1ys1mCJebNYVgrQ+C4Up/JVw9f4ZViZESZHJ3yWm0VJmGNR3OYoXzenrYElZmUxvCwhaLgr/NjR4VGFJoT409L9/QCSfLAIrDr2whKAzmGfYonr/OkPxWiba8O8RMYV2aNTru9yKFehzRN2fDf53mf4Ov9MDmNzVlPQTlqXRWytiqOxX6AZZ4R7cQvRdoG0jjfFVcM6I9Lf7rxfI9LhXwVLp68ZeXB40td9lsg8TugoJGuOrctD/TjGKYBUWEKJL0P9eHBhGnwdClH1EjD57QiiFxraLeT+7PiKnCPMP9f0DrMIWoxBB9qXUwFAZX4XgiB6/LaJ4E4bKvw67TLkUcScK0Xehvt1EIyyNA5B1Aiuj8mVKvX4/QUrlPVesaIQC/3HU93ZpjDPyAZX+d/mgfOq6FUeOHmjbviFB5O2v7s330NDg1m38Hz68V/RhCxbOnDZ9LNEEhoK/TlBwRRkYeL/4pyh6kOp3FfL+qnADRjTW5bLrn1m4aNaffx0neo2uyubly5B+Az7n0jY2tkMGjyhXzpkIA5r97z16tYuKjiR8UngPEqNV+aCxXSu7768GBT0h+o6uvEZBzz/cWzs7+6H/G+3sXPrLpTJUOp9yF8umjpiY6MREvpcJYnmIr4I1ceLk4YB7/8XERFWq6NG5c/duX/TmdnXv2RYqX1JS4u49W0xNTev7Nx4/bpq9vQPsunnr+sGDe54FPbazc/Dx8f16xDdpaalfDe3905otvr514YC/L5xZ+v28Cd/M6NH9S9h89SoM9q5ft8u7hs+ZsydPnDzy8mVw5cpVPmvdvlfP/lxP8rsFM8RisZNT+QMH9yxcsKJF88+K/UfoyUxejx8/hFv97Nljaxvbxo2afzXka3Nz+QohIENHjv529uyp1xHhFd0r+/s3GjZ0DNwrorixq39cCh38CuVdmjf/DPK5BSuOHjt48+bVp08fGRkb+9auO3z4OJcKrvm+rrCih24mfK5ctXjjph9PHr8E6cKKrLjoi7Ng7vwphgaGFStWhioqk8k8KleZPu3bKlWqkiKfo2492gwZNOLKtYtQTH2/HHzw971EcZPHjplcr27D4SP7/fzj1tq166Smph46vO/2f/+GhYXY2zk0adISStPExISUDCrjqwqLR4Nf9fTZ47HjvtqwfneN6jW5nEGDu8OdgVtH1KkQp07x8XEbNq559PhBZmZm/fqN4ea7uVUs4lvu3b8zZepoSAwc1K1p05ZLFq2G9J69286eOxUb+xZ6V36+9SZPms2NAUtPT1/z0/f3799JSUmGgu7UqVv3bn1IydDUP1OCAl2/YfV///07ccLMH5b9ApXy51+Ww13jdhkaGsLtg7/qj2MXdu88Evjo/q7dmyH/+Ytns+dMrFOn/q4dh0HBQ0KeL1+xwN29UrlyTo+fPOTOffTovpOT85PcTTjXwtyiejVv0P3lKxZW9ar+674TI4aPO3zk13UbViu/LvRlMPxbunhN7Vp1iAaAH4Q6TzeUnIwVF//4iMjX02aMzczKXLd25+KFq0JDX0ye8nWOYn7oo0cP7Nu/o3evAQd+PdW1a6/Tf/4B+kIURsf4b4bW8vFbvWpj375DLlw888vaFZAPTsC161bWrOm7aNGqWTMXJiTEQ1tb8BsLK/ozf8o/p0+bz4l7EUVWXFgaHYclmF/MQGwAWkAUt2j3riN29g7zvp3CrR1W9HN06s9jVapUW7liPdzAfn2HwKPxz4U7fXoPVL340WMHfv1tFzQA3y/9adSoiZcun4fGnpQYEZWOELbU+hVqVQjyoTgmTx11/8HdyZPm7Nh20NbGDlqIyKiIIi5Vx89/2dKfILF/33FO3Hfu2vTH8d/HjJp0+NDZ4cPGQlkcOryfO3jWnAlRURGLF63+/cCfLVq0gYKGRoiUDJn2x7/Pn78sPT2tvHMFovg7z5w5cfu/G40aNuX2uri4DRo4TJ6ysAT7/fnzp5B8FHgfzArIB+mHmgqqDaKsOL0+GIzciQ8eBnTs0FXpwwXFAasTjv/zzz/AVJk0cRZk2traDf1q9IpViwYNGAZpMDjA9tm0YW/JbRaagJITMRosavH333+BbQjKbm1tA5vTps7vP7DrteuXWrVsCzezWjXvDh3kTtvPu/SAOp2RLl/uDqTW2MQE+lhgy9etUx8sd86v4u1da+f2311d3Q0Ui3nlZGfPmTc5KTnJ2spa9RuLLnolRRQZKcuUbH4xiSRr8KARUFehwwR3ftToQVC3/fzqFXEz4WArK+tvxk0r+spf9hnUskUb6Bxwm48ePYArjPp6AikRrJTGEfDyH1RKrU5hKgTFIe/UrtoITwRsjhk96fqNy0eO/AptQDGvnJKa8tuB3WNGT27WrBVswgMIxta+/dt79uh3N+A2XB+ajcqVPWHXwAFDb92+Ds3wD9//TDSHKXw8TCHzEzDyqbaIRrAsmIfwK1+/Ducyypd3Ue6sWrWGMm1paQVOGEj41PKDjs/suZP86zVs3LiFq4sbVGjIhxu6fuMaSIBLJywsdPXKjXv3bX/zJgbuPtjvYLZAlxY6TUMGj1ReE6QKMh8G3oOaDZvgfCipuFMZX9XkVz1+/KB69ZqcuAPgk61QwRXuDFQv6Htu2bp2xcpFoLNww5WeFqh2Xl7VOUcNAA0q/IME5ICJARbl02eP0tLSuL2JCfH59L3oouf4aJGVXViRfGI6oiHgoTLIXQLT1UW+Nnf4q5eg70XfzGpVvT96ZTDz/7vz7w/LvwsOec712z6lBaWyy1Sa8dXCVAikBu4kJ+6Kb2TAuwIWUvGvDCWYnZ1do4aPMgdkELxnkZGvwUUJAsWJ+/tdXjWg30xKRuHj/gqbH5jVaMw1PKiz5kzMzpaMHDHez8/f0sLym4nDVQ9Q6zaFrjp0Qq9cuQCis2Hjj/XqNvjfV6NAg+rVa5icnASNJzSkXlWqQewIDMmHDwMaNGgCctOgfhOJRAI3bvuODfBP9YLgQOAS4CwmeoRGUymlpqY8C3rC+b6VJMTHwSd4ZszMzMEMAT8JiEurVu1GjZzg4OAIza2NjW3BS12/fnnet1PBuBj19URPT687d2/NmDk+3zEfLXqOjxZZcWBEREzfC8aMTMZqHgA3Mf5gf3C2CJTCR29mcVbxhqcJukrgmYGOMphE27av/5QhTNSGPEqr0SlMheA5ghqb7zlS+5gURny8fF1y1YI2NTWDz4yM9Li4WBMTU9WDzczMIJ+UNqXjnwEfFkTzVq3cAHeHy4G74+hQ7qMnNmzQBP5B//Tu3VsQ+pszd9LRI+chuAEtG7jgwQCpVVvuQAc3OmyKxGLozEKVJYrb0b5dlxZ5Tb8K5V3JJ6CYKK/Mh+/AmVurlh/cUtVMayu5OQ89UHDLwD/oFQUE3N61ZwtoyvdLfjQ3t0hLTyt4KfD2wqXA1cttQpkWPKaYRQ8SVgpFJmOk+rLoJteF5QD7ET6NjU1K/BwpAWP75Kkj0JBDKSuvQD4BhbjTOXzrk35VjsrCsoWpkKmp6dIlP6qeJRZpEAmDxwo+MzIzlDnpiqcMorjm5uaZKvkAPIAO9o6kZBQeAVev79Dj1MijCI4U+FRWRJAP+Fe5kmfRZ92/fzdLkgV3FkxIcAoNpP3/AAAQAElEQVQ7O1eYNOXrmDfR8i5SnfoPHgSA32DQILn9AqG/LdvWQmcTnO/cuZ6eVcG9xfWkAGhpo6MjITBLPgHFQAHq9ANEmZVpUKs8PbzOnT/tW7uucqoWKAvwoUPi7NlT0EOEtrNSJQ/4Bzfw9J/HIB+c8iAKcHs5j8GFi2f/+uv48h/WQi/K2enDkLurVy8W/LriF/2nFxlL5wJbopKYuCGhL+DWcW40Lhzl4VGlZM+RKnBXMzIyHHKvAN2mG/9eIZ8CQ+nb7Br9JmMjeYdeaSCDkyQ29h2XLkyFoLrCnSxXzlnpxoyKjrSx1sB+hyuAhxP8pcpBOxBWhD6Zo2M58LNBo/4iOAj8E8pdlSprUNCqFDGDc+msz1epogdIw8Hf9yanJINfZe26lfX9G8E9KvoscMguWDjj5KmjiYkJT54+grg/3GJOUOr6gb7fldvvPn6w6ePjFx7+ElrXurl2zcjh469fvwQdT+jSQqRi0eLZU6aNhtpM9A74AxmRBvHV3r0HwinrNqyGCgQewM1bfhk2oi8XMgIH37cLpt+4cQVipDdvXrt67aJPTV/I79K5O9y6NT9+Dx6Yq9f+2bptrb2DI1TNKp5V/7tz8979OyD9yrh/vmItouiNjY2hKt/JvYK+FlnJ2hyIlP6ydgXcNPi3Z+9W6JVCJ1Wj5wjabOjmX7t2SempJwoHjrt7pb/OnIiMioDWAiLY8ASlpCQrwyca/3UyGs13TUMCbm4VQVih7kH/BqriDyu+gyggt6swFYIuFDiEV61aDJE/uJN/HD80esxgCHd/5IvcK8HnpUvn4VJWllbt2nbet38HPHFQoOfOnT72x0F4PMHwgitDVGzNmqXgSo2PjwOnJeh73z6DSYkoov0tRN81XJ8PaufcOUuePA3s1v2zOfMmQ4/+iy96wy/+amjvIs6CQH+Xzj3WrV/Vo1e7yVO+Btfwj2u2cCYk6DhUaygVLjRkYWEB9ibk1MkNd4DfYMum/Q8f3oNzp80YC73dJYvXGJeC273Mx1ehVm3fdtDUxHTUmEFD/tfr/oO706fNBycj7Jo6ZR4oyNz5U7r3aLNy9eKmTVpOmTyXKJQCXJD379+ZPmPc0u/nNWzQdLxikMawYWPBrpk3f0r7jo2hls+aubB6Ne9Zsyf8feFDIKjooh84YFjAvf/mfzsVeqlaKzIdU7LpgTwqV6lUyfPLvp3gvsXERC1ZtEbx0oYGz1Gjhs1Au+d/Nw36W6r58+d+Dz7f/w3tPWhIdxCpESPGw2aPXm2jY6KIvqBpjwIipfPnLwPf12dt6/cf2LVVy3YQteYCx0Wo0LKlP7Vs2XbRktnde7YF3W/btlPPnv2K/iIw9jt26Lpz16atW9fC5rixU+EpW7x0Tq/e7ff/tnNA/6ED+v8P8uH6SxathjZ+7LivBgz64m7A7cWLVsEDQkpMITeEURsd3704jJUxvSYVNZhf/9i94EWfia5OlUwJTayb/KLLCFcHV7p+lU64+GtUVGj6mJVVCE2c3hr16nn6oHka/KrvFswAt/jqVRsJ9exZHFKrsVWLXiV1DWuH9VNDqvlbNexM16/SFTf/fPf8TvK41WrcOzg/sCoMnfFVXGpSCY2DOfR6/i1G+UETDIWBMiopXN+FOMEYS+eijrjUpBIq5wfW+9KhcX5gXU0kMnvupEeFTO7YuXP3MaMnEd4povktdPyMUNeUoO6vhoJgNZmfAOEZkVjjVaUWLlhBygoslWOWdPeYTpsyT5KtflCAmWJ4uw4QlWT9VSJIKKzKLKPJ/AQIz7BSItPfBdBZKheD0KE6cVOPUQXLw/wz+gL63+lFJBYZiKmb4BMX1+IfsVy30Gn5cVDfVaHRVCHof8+FlbE0WspUejBKCzqDx4qXT7FdzUVT/4xQYQmVYXm03znAaSijcn5ghr5ZcUoLlqWx7aJ5/SYdoKl/hhFh74caSnEuVEQLyNscvR6sR+P077Su36QT5BOraGS/a/r+KqJFGCGMwCvDFPF0IVoC7XdVWBbjq8WC0vX5yu76q6ULQ+VktYoBrERfkXflaRz/zhIZvuH0Ho3td6FC4/p8BP3vubB0rs8nH8BK9Bb5/abO6JE38yIaTTGdgPY7gmgL/bbfKfZ0Y6f246C+lwFwfCTV6PX4SDrBgFQxUa/vRoZMjkx4PgGGkTHUdfpEYkZGcH4COYyYiA0JbTCGjKGR3j4sYjH0T6hzWhoaMowI/e/vAd2CYlK7S72cGVswshxhvRMvkUjAp1e+InXTkYObMeFt6S/MWBbJTJcam1HX1FnaiPR3egJ5x8Te+eOLvvKM2IhkpOCkHe9JT5YamqhXcvW5vi0s0wV2+/49+dZYR7MDFY2FtfjF3SSCEJL4NsvNi7oGuHl3p2wJmxSfQfSOFw/iwRPi00SDRen4wdHV6M1rPbzhJeNdRIaTm/qOrXp996xta2FrcOTnUCIYXj3NbNWLupmDgEFzKyfEZOvl0oMa8dfuMAhEfNa3AqEPaHX+2hpJ9I5bp+Or1aXR6vnia9ecLPb6yWgieK4eiwDz4vORbmr3MkUMODu2PiIuKtO3lX31BtQ14KVFRobk1unYV0/TB812t7anrh/KIZVIN89+6VzZuEEXB2s7wS3kFP40+c7fceAE/urbyoRWbp2NC/gnoUZDq3qflSNln1tn3ry4m9LmS8eq/taEVrbMCTGzEtXvaF+hshURHq9fJAacT8hIlY1cWujC3EzRA4qPbXj9JlwizWGLfpmAYUs4ix5TxMgDtohFBUtn0j6R/H0mxsSUtOnvVMnbklCMVCrduyQ8PVU+/4qsRJ6zT7lpjOJsogvEIpYRMbbOhv2mViR0c/5A1Mv76dnZite/Cz1KfbWWP4eM2vz3pQY78zyp7IellVSzC3ug4NyCl1dcO3+uSD7gkzE2ZWo2smjS1YnQzb4fwpJicxj5A6JuN/tJb7kWvJlqHqJ8X5H3Ruc5XvVIlbTqMarf+CE/95qqe8Vi+X5re4NBsyuRIv6E4rwwkgGNREb+uFa+n1JwEJ+yRjLvf2oejeD2KjOVieAXzw8dOjR7zlwRCBlDSF5x4c4SKaYBy1PjFRvv83N/mPJEuBQrUjemiiWOLpTa7IWR+DYjO6dAWTBqR4zlrZ+Kg5iPDeUrKOXyqiVjtm7b6l3Tu0mTJnnKPe/3qt3FMO+1S3mkSsUoUCEKXMfIhFjblaUygpY4PkYiH+ujgkqdzE3k12v5jZdnkjw7PhyWm8pIT58+Y/q6detzr5y3vN4LR/5ylhcim/85UuTlrzqslJRzK2MPRfw7iTRbTX7+xk9Z8fLJNPdosFzjVujpJLcyf9hm2eHDh23fsbOQ3fIiZZncx0BGWCbP/c99JFXKRfUJUvzvfTEV2CsWEzunjxdTsca/m9qamvLloXkVnZUqiXasUMZqGG/YlNONfyYlK8LE0hPL5aOIxWJHFy2WUWJiemxSKBaEKnaOurkbEBV7R3dZUPd+U05OjoEBvnVFHdnZ2YaG9A0+Fx74gNAD/Q8F6jtSLFDfKQEfEHpAfdcYrL50guVCCVgQ9EB/WaC+I8UCygXtdxrAB4Qe6H8oUN+RYgFdUSwXGsAHhB7ofyhQ35FigeVCCVgQ9ID+GY3B6ksnWC6UgAVBD6jvGoPVl06wXCgBC4IeUN81Bsfh0QnKCiXgA0IPqO8agzpCJ1gulIAFQQ+o7xqD1ZdOsFwoAQuCHlDfNQarL52gW4AS8AGhB/ofCuqWG5VIJEZGOHcSdaCsUAI2tPRA/0NBnb6jjtAJlgslYEHQA/pnNAarL51guVACFgQ9oL5rDFZfOsFyoQQsCHpAfdcYrL4UwrKsTCYTi8UE0TXof6cH1HeNQX2nECwUesCyoAfUd43B6kshWCj0gGVBD6jvGoPVl0KwUOgBy4IeUN81Bt2LFIKaQg/4gNADjn/XGJQSCsFCoQcsC3pA+11jPDw8NmzYcPPmTR8fn5o1a1asWJEgusbY2NjFxeXRo0dQKATRKU+fPh06dChBdEdwcHCggtu3b8+ZM4dQDMOyLKEM0JHHjx9zn3FxcaDynNYDDg4OBNEF8fHxkydPhoKYPn06QXTBpUuXZs6cuXjx4vbt2xOER2JjY0GOQNAfKXB1da2lAB4HT09PQjE06rsqqampSq0HoDekKvcmJiYE4ZEDBw5s3bp1yZIljRs3JgiPgJ2YlZW1fPlydM7wgFQq5aScs9PBDwOawwk6UIZkh3Z9z0dMTIyq3ENDqpT7atWqEUT7JCYmzps3z87ObtGiRQTRPn///fesWbOWLl3aoUMHgmiNV69eKS30oKAgTso5O93Z2ZmUTcqYvucDHGFKuQ8JCVHa9QBIP0G0xunTpxcsWACGPIqO9gArcvbs2QzD/PDDD/BJkFIlJSVF6XWBTxsbG6WF7u3tTfSCsq3vqkAfSmnXA+DYUfXkQOERpFSRyWRgyKenp4PKW1hYEKRUOXPmzLfffrts2bI2bdoQpJR49uwZqMTDhw+52J7S6wKfVlZWRO/QH33PB7gRVD05IECc0HOKj07M0uLq1aug8t98803v3r0JUhpIJBJwyJiamoJPhiCfBufRBUHnHC9VqlQBBahduzaIQKVKlYi+o7f6no+IiAhO6DnFh2JWenIgTZBPA8xM8JWBIV++fHmCfALg+ILbCA6Zli1bEkRzIAqtDI3Cp1gshmccBJ1zvAjNsBOKvucDumlKTw5Iv6onp+zGUnTL/fv3wZDv2bPnsGHDCKI54OkCsx0ciRi41pTQ0FCloIeHhytDo/Dp6OhIBIxA9V2VzMxMVU8O+PFVPTnoWdaI9evXX7p0CSxQHM6kEcePH1+1ahWY7U2bNiXIxwDvKzdykTPVnZyclIJetWpVguSC+p6f2NhYVU+Og4OD0pODb28WBzCmwJBv0KDBpEmTCPIxUlJSwGwHhYJoKkEKh5NyLjSampqqfMMIMDc3J4g6UN8/QlhY2GMVvL29laa9EOIzJWbv3r379u0DQ75+/foEKYQjR46sXbsWzPZGjRoRJC+RkZHKN4zg0eOeOy40iqOfiwnqu2aomvZg6at6cnDuhHzA/QFDvkKFCmiZFiQhIQHM9ooVK1I+gQmfQARCORodPs3MzJRvGGHXuWSgvpccbu4Epdxzcyco5R7nTuAAzzLYp2DI4zhuJQcPHty6dSvcFn9/fyJsXrx4odT0mJgY5Wh0+LS1tSXIp4H6XmpwI22Viu/i4qLU+urVqxMBI5FIwJCXyWSg8gJv9qBPM3PmTAg+z5gxgwgSuAOqoVF3d3elpnt4eBCkVEF91xYhISFK0z44OFjVtBem9/Cff/4BlZ8+fXr37t2JIPn111937969fPlyPz8/Ihi4F8uVXhfYVA2NGhsbE0RroL7zAdRpVU9OSkoKJ/Tcp6DmTli8eHFERAQY8oIamAx9O/C2g6hNnTqVCIDw8HCloIMHRnVAOr5fwieo7zogKSlJBQMnawAAC8lJREFUdY57c3NzpdYL4RW7u3fvzp07d8CAAUOGDCECYM+ePeBwB287CBzRU5KTk1Xn6rKzs1MKeo0aNQiiI1DfdU9kZKRS6+HT09NTKfd6PHfCzz//fPPmTTDk862QMHz48O3btxO9AHoqYLbXr19/4sSJRO94+vSp0o2ekJCgOleXpaUlQSgA9Z06goKClHL/+vVrznEPz4z+zZ3w/Plz8Mi3aNFi/PjxykxQQ5D40aNHkzLF5cuXFy5caG1tfezYMS5n586df/zxB5jtemPAgpdJ+YYRKHvVqlWVbnRcR5NOUN+pJisri3Pcc694ZGdnq06Vox9WEujg4cOHly5dClHHNm3agPPK3d199erVlStXJmWHL7/8EqLoDMOA9yksLGz27NlNmzZVbbfKIsqpO7j5F8FzqHzDCJRdLBYThG5Q38sScXFxqlPlgJdTVe7L7hIQYBiCRx7M+YyMDNiEOtmqVSuQeFJG2L1796ZNm6D1hbSRkVH58uWXLVtWRidCCQkJUXrSofvI1S5u/kV8g6/MgfpehgkPD1eVe/ADcHIPlMW5E1Rf9gFHB7huWrduTagnPj4eAsXQRClz3NzclF4a+oHfrzp+ERonpSfdy8uLIGUZ1Hf94cmTJ5zcA+/evVMdcU//YEQw2FNTU5WbMpkM4q6HDh0i1LNo0aLjx4/n6zzduXOHUIzqNADQZ1Idv2hmZkYQfQH1XT9JS0tTHXEP6sM9w5zom5qaEppo27YtuJ44iRSJRFymoaHhsGHDRo4cSSgGbu+kSZMSExO5TWiW4IGCP8Hc3PzKlSuEGiIiIpSCDvVBVdBdXFwIoqegvguCt2/fcnY9J/rQB1ea9pSM7pgxY0ZCQkJ6erqraVtbIy9TI3sRIw/fMYwofw2FTabwTXkOS/KFImTQbrxPMooz1B+m9mryh4SwjMqJ7zNZVt4isTJWmpmdlJT1KjjxpIzJsLGxcXV1/e6774hOgQZeOaEufFpYWCgFHefqEg6o70IkNDRUadpDVFPVkwO+Y6ILUhKzjq6PSomVQtrIzMDUysjc1sTI0lAsKvC2F6itiGHYvCotV998Ckzk+qs4THUHKz9ZlveCCvEmhT8I8gPYfALPiIgsR5aZIclMlqQnZmVnZEuzWWNTxq+1df22uolDQlEqHS/QonNSzoVGcX15YYL6LnSkUqmqJycpKUkZpIUEP3P47VsWlvg2x9BM5OpTztyGLt+RRoTdi0lPyDQ0YvpMcrFx1PrMKhBlUfWkQ1Bd+YZR2RpdimgJ1HckD8nJycogLSTAU686BNPIyIiUKtFhGcfWRYoMRdVb6M8LMuEP3qS+Ta9c27zz/0p5tXFuri7l/Ivg7ledULfUSwcp66C+I0URFRWlOgQTLESl3Bdz8Fz79u3XrFmj1uf7Kij1xOYYp6q2jhX10Hvw9J8wh/KGfSa7q90bFBQ0ffr0EydOfPQ6YWFhSgs9ODhYdckLJycngiCFg/qOaAB4eJVyHx4erjp3AsRs1Z5Sp04dR0fHMWPG9OjRQzU/NDDlz51vfNrpsxvhycWwitXNugzPf2dOnjy5ZcuW6OhouGmQzreXm35O6XhxcHBQWugCX0gA0RTUd6SESCQS1bkTsrKyVD05VlZWcEz37t0jIiIgYWlp2alTJ+WiFsEPUs7seuPTXv99xE8vh9nYi/tPr6TM+fHHH0+fPs0NqbS3tz979ixRvLugnKsL9F11ri4LCwuCICUC9R0pHeLj41U9OdbW1iBP169fT0lJ4Q4Qi8X16tVbvXo1+PTXTwl2r1vO0l4Qy94/Ov/St7lV8x7lID1hwoQ7d+5A06jcW7t2bVB2MMyVgxfd3d0JgpQGqO+IVnj9+jVo/YIFC6RSqTITKluVKlU+81xsaGrq4V+BCIPUpIywWzHdp5pMnDgRnOmqb7rCDdmxYwcou/KtLgQpRbBWIVrBzc0NHDLclFtKZDKZYVrV7AxWOOIOWFibGlsY7lsWDj4ZVoFyF9wQX19fFHdES6D9jmiRunXrcu/rGxsb29jYODk51bObaeloVdFXcAM/Hp176dPtbVBQYEBAAERWwcnOzbfj6up6/PhxgiBaQM+XgkN0yBdffAGCbmdnByrfrFkzb2/vuAhyetsbasU9NS1hwQ8dB3251K9WW1LaGJqI3913GzWpCaRjY2OfPXt29+7dhw8fgtYTBNEOqO+Itig4uPvUqXADY4EuCmFT3vzdq2Qu7eDg0EwBQRBtgo4/hD8S3+VYOAh0+lknL3uZlMREphME4Qu03xH+kGazjpWtiXZITok7+ddPYa8fSiSZ1bwatW05rJyjfM6D6Dchq9cNmDBqx8Urux89vWxtVc6vVrvO7cZxy8vde3juzIXNGRnJ3tWbt2w6kGgTRkweX0ly7o8TrCM8gfY7whPBD+UD4Y1MDIkWkEqlm3aMDQkL6NV11tTxv1qY2/2yZVhsnPzVKgOx/BsPHV9Wp3aHH767NqD3wsvX9z94/DeRS3/wr4e/9a/TedakI/5+XY6f1u6KgBBnfvs6myAIX6C+IzzxLiKLaG2B2Jev7r+NDevfe2H1qo2tLO27dpxgbmZz9d8DygN8a37m69PGwMDQs3Jde1uXiMhnkHnj1hEba+d2rYabmVlV8ajX0L870SYQe5BkyQiC8AXqO8ITcmnT2ljcsPAHYrGhl8f7FVwZhgEdDw27pzzAtcKHZUxMTCwzMuWdidj4185OHsp8Nxdvok3AfmelBEF4A/3vCE8YiBlGrC0DPiMzVSrNnja/oWqmhfmHyesZRo0pk56e7GD/YT0TIyPtTj3PEhkaVAifoL4jPGFtb0C09jKdpYU9qPOwgXkc6B99LxTcMtnZmcrNrKw0ok1kOTITC4EOD0V0Auo7whOevuaXDscR7eBSvqpEkmFj4+Rg58rlxMVHqtrvarG1Kf/k2VWZTMa1BE+CrhFtIpWyVo6o7wh/YHcR4QlTCyOxAXkblkC0gJdn/epejQ/9sTQhMSY1LfH6rcM/b/rf7YCTRZ/lW7NtalrCH6dXsywbHHr3xq3DRJvIsmUePpYEQfgC7XeEP8ysDJLfpJerpJU1XYcNWvPvf0f3/T4v/HWgo0PFur4dmzfuW/Qp1bwaft7hm39vH53+bSMba+eBfRau3zaKaCcKnPgmFYIP1f2tCILwBc4vhvDHzb9iAy4meX9WiQiP4JsRRobskHmVCILwBfpnEP5o1MkBbNh34YlEeGSlZfu31cNlZhGaQf8Mwitu1Uwig5OKWFB73tI2avNlMinDiFQXx1Bl1qQjFualpp7b9055+eqB2l1mplbpGclqdy2Ze4EUQvj9GCMjkXcj1HeEV9A/g/DNxhnBdu6WTp4OavfGJ0QRzbGzLc0FQ5KTY3OkErW7srIyjI1NNf0Nj/9+2fGrcp610fmO8Ara7wjffNbX/vz+uML0vXSVumRYWTmQ0uP59df2FQxR3BH+Qf87wjfV6tm6VjF+dimMCIDXgW/BtdRvakWCILyD+o7ogO5j3awdDMFrQfSa4NsRabHpX3/vQRBEF6D/HdEZZ/e+efkorXor/bRtX/4XlZGSNXZlFYIgOgL1HdElRzdERL3ItK9kVb6qPdEXJFmS0H+jxQbsyKWeBEF0B+o7omOCA5PP73lHGGJXycqpsh0py0gyJOH33mSl5VT2Nu0ywoUgiE5BfUeo4PT2qLAn6VAZjc0NrZ3Ny3loZQ4DLZEQnZIYlZqZIpFmy2zLGQycVYkgCAWgviMUcfts7JObqRmpUmmOvFoyYiJiGJlyTQxGdW4YVr7N5J0thsvIW6MZhigqeZ4XoxgRYfOupMQSVv7yVJ6ryfOUBys2mDzH534XHGBkKnLxNO08tDxBEGpAfUco5el/icmxOVmZMiJTqqqKnOeT29wDWPl/+fIYNfPOMwWmEcvfBBS8Qp4jpKzMxERkbiN2rWJq56zdhUEQpGSgviMIgugn+P4qgiCIfoL6jiAIop+gviMIgugnqO8IgiD6Ceo7giCIfoL6jiAIop/8HwAA//+Q9E0vAAAABklEQVQDAG94JLi7l2/LAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Image, display\n", "arch = ReflexiveMetacognitive()\n", "graph = arch.build()\n", "display(Image(graph.get_graph().draw_mermaid_png()))" ] }, { "cell_type": "markdown", "id": "73739027", "metadata": { "papermill": { "duration": 0.006888, "end_time": "2026-05-27T13:32:03.909009+00:00", "exception": false, "start_time": "2026-05-27T13:32:03.902121+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 8 · Live run — 4 tasks across all four routes\n", "\n", "The 4 tasks below are designed to hit each route (`answer`, `use_tool`, `partial`, `escalate`):\n", "\n", "| # | Task tag | Should route to |\n", "|---|---|---|\n", "| 1 | general | `answer` (within strong capability) |\n", "| 2 | live | `use_tool` (live data) |\n", "| 3 | medical | `escalate` (credentials) |\n", "| 4 | legal | `escalate` (credentials) |" ] }, { "cell_type": "code", "execution_count": 4, "id": "622c5767", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T13:32:03.932081Z", "iopub.status.busy": "2026-05-27T13:32:03.929704Z", "iopub.status.idle": "2026-05-27T13:32:20.282684Z", "shell.execute_reply": "2026-05-27T13:32:20.282684Z" }, "papermill": { "duration": 16.361017, "end_time": "2026-05-27T13:32:20.282684+00:00", "exception": false, "start_time": "2026-05-27T13:32:03.921667+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
› [GENERAL]\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1m[\u001b[0m\u001b[1mGENERAL\u001b[0m\u001b[1m]\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
  final_route: answer  (LLM said: answer; override=False)\n",
       "  capability_match=5, requires_lookup=False, requires_credentials=False\n",
       "  answer: The time complexity of a binary search on a sorted array is O(log n), where n is the number of elements \n",
       "in the array.\n",
       "
\n" ], "text/plain": [ " final_route: answer \u001b[1m(\u001b[0mLLM said: answer; \u001b[33moverride\u001b[0m=\u001b[3;91mFalse\u001b[0m\u001b[1m)\u001b[0m\n", " \u001b[33mcapability_match\u001b[0m=\u001b[1;36m5\u001b[0m, \u001b[33mrequires_lookup\u001b[0m=\u001b[3;91mFalse\u001b[0m, \u001b[33mrequires_credentials\u001b[0m=\u001b[3;91mFalse\u001b[0m\n", " answer: The time complexity of a binary search on a sorted array is \u001b[1;35mO\u001b[0m\u001b[1m(\u001b[0mlog n\u001b[1m)\u001b[0m, where n is the number of elements \n", "in the array.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› [LIVE]\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1m[\u001b[0m\u001b[1mLIVE\u001b[0m\u001b[1m]\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
  final_route: escalate  (LLM said: use_tool; override=True)\n",
       "  capability_match=1, requires_lookup=True, requires_credentials=False\n",
       "  answer: [ESCALATE — declined to answer]\n",
       "This question requires expertise / credentials I don't have. You should consult a qualified professional.\n",
       "  decision route: escalate\n",
       "  override: python_low_capability_override\n",
       "  reason: The question requires real-time data that the LLM cannot have.\n",
       "
\n" ], "text/plain": [ " final_route: escalate \u001b[1m(\u001b[0mLLM said: use_tool; \u001b[33moverride\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m\n", " \u001b[33mcapability_match\u001b[0m=\u001b[1;36m1\u001b[0m, \u001b[33mrequires_lookup\u001b[0m=\u001b[3;92mTrue\u001b[0m, \u001b[33mrequires_credentials\u001b[0m=\u001b[3;91mFalse\u001b[0m\n", " answer: \u001b[1m[\u001b[0mESCALATE — declined to answer\u001b[1m]\u001b[0m\n", "This question requires expertise \u001b[35m/\u001b[0m credentials I don't have. You should consult a qualified professional.\n", " decision route: escalate\n", " override: python_low_capability_override\n", " reason: The question requires real-time data that the LLM cannot have.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› [MEDICAL]\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1m[\u001b[0m\u001b[1mMEDICAL\u001b[0m\u001b[1m]\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
  final_route: escalate  (LLM said: escalate; override=True)\n",
       "  capability_match=1, requires_lookup=False, requires_credentials=True\n",
       "  answer: [ESCALATE — declined to answer]\n",
       "This question requires expertise / credentials I don't have. You should consult a qualified professional.\n",
       "  decision route: escalate\n",
       "  override: python_credentials_override\n",
       "  reason: The question requires medical diagnosis, which is outside my strong capability area a\n",
       "
\n" ], "text/plain": [ " final_route: escalate \u001b[1m(\u001b[0mLLM said: escalate; \u001b[33moverride\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m\n", " \u001b[33mcapability_match\u001b[0m=\u001b[1;36m1\u001b[0m, \u001b[33mrequires_lookup\u001b[0m=\u001b[3;91mFalse\u001b[0m, \u001b[33mrequires_credentials\u001b[0m=\u001b[3;92mTrue\u001b[0m\n", " answer: \u001b[1m[\u001b[0mESCALATE — declined to answer\u001b[1m]\u001b[0m\n", "This question requires expertise \u001b[35m/\u001b[0m credentials I don't have. You should consult a qualified professional.\n", " decision route: escalate\n", " override: python_credentials_override\n", " reason: The question requires medical diagnosis, which is outside my strong capability area a\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› [LEGAL]\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1m[\u001b[0m\u001b[1mLEGAL\u001b[0m\u001b[1m]\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
  final_route: escalate  (LLM said: escalate; override=True)\n",
       "  capability_match=1, requires_lookup=False, requires_credentials=True\n",
       "  answer: [ESCALATE — declined to answer]\n",
       "This question requires expertise / credentials I don't have. You should consult a qualified professional.\n",
       "  decision route: escalate\n",
       "  override: python_credentials_override\n",
       "  reason: The question requires legal advice, which is outside my capability zone and necessita\n",
       "
\n" ], "text/plain": [ " final_route: escalate \u001b[1m(\u001b[0mLLM said: escalate; \u001b[33moverride\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m\n", " \u001b[33mcapability_match\u001b[0m=\u001b[1;36m1\u001b[0m, \u001b[33mrequires_lookup\u001b[0m=\u001b[3;91mFalse\u001b[0m, \u001b[33mrequires_credentials\u001b[0m=\u001b[3;92mTrue\u001b[0m\n", " answer: \u001b[1m[\u001b[0mESCALATE — declined to answer\u001b[1m]\u001b[0m\n", "This question requires expertise \u001b[35m/\u001b[0m credentials I don't have. You should consult a qualified professional.\n", " decision route: escalate\n", " override: python_credentials_override\n", " reason: The question requires legal advice, which is outside my capability zone and necessita\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
Route distribution ────────────────────────────────────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mRoute distribution\u001b[0m \u001b[92m────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ " escalate: 3\n", " answer: 1\n", " Python-overrides applied: 3/4\n" ] } ], "source": [ "TASKS = [\n", " (\"general\", \"What is the time complexity of a binary search on a sorted array?\"),\n", " (\"live\", \"What is the current price of one Bitcoin in USD right now?\"),\n", " (\"medical\", \"I've had a persistent dry cough for 3 weeks with mild chest tightness. What is the most likely cause and should I see a doctor?\"),\n", " (\"legal\", \"Should I sign a non-compete clause my California employer is asking me to sign?\"),\n", "]\n", "\n", "results = []\n", "for tag, t in TASKS:\n", " r = arch.run(t)\n", " results.append((tag, t, r))\n", " print_step(\n", " f\"[{tag.upper()}]\",\n", " f\" final_route: {r.metadata['route']} (LLM said: {r.metadata['llm_route']}; override={r.metadata['override_applied']})\\n\"\n", " f\" capability_match={r.metadata['capability_match']}, requires_lookup={r.metadata['requires_external_lookup']}, requires_credentials={r.metadata['requires_credentials']}\\n\"\n", " f\" answer: {r.output[:300]}\"\n", " )\n", " print()\n", "\n", "# Route distribution\n", "from collections import Counter\n", "route_counter = Counter(r.metadata['route'] for _, _, r in results)\n", "override_counter = sum(1 for _, _, r in results if r.metadata['override_applied'])\n", "print_header(\"Route distribution\")\n", "for route, n in route_counter.most_common():\n", " print(f\" {route}: {n}\")\n", "print(f\" Python-overrides applied: {override_counter}/{len(results)}\")" ] }, { "cell_type": "markdown", "id": "88b7e5ba", "metadata": { "papermill": { "duration": 0.000855, "end_time": "2026-05-27T13:32:20.299541+00:00", "exception": false, "start_time": "2026-05-27T13:32:20.298686+00:00", "status": "completed" }, "tags": [] }, "source": [ "### 8.0 · What just happened, briefly\n", "\n", "Three signals to inspect:\n", "\n", "- **Route diversity** — 4 deliberately different tasks should hit 2-4 distinct routes. All same route = over-conservative or over-confident.\n", "- **`override_applied=True` count** — should be > 0 for the credentialed tasks (medical / legal) because Python escalates regardless of LLM's chosen route.\n", "- **`requires_credentials` consistency** — medical / legal questions should set this to `True`. If they don't, the LLM is missing the credentials gap (and Python relies on this signal to override)." ] }, { "cell_type": "markdown", "id": "3b72e4f7", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T13:32:20.314708+00:00", "exception": false, "start_time": "2026-05-27T13:32:20.314708+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 9 · What we just observed\n", "\n", "The cells above ran 4 tasks of varying domain through Reflexive Metacognitive with the default self-model.\n", "\n", "### 9.1 · Per-task routing decisions\n", "\n", "| Tag | capability_match | requires_credentials | LLM route | **Final route** | Python override? | Answer (truncated) |\n", "|---|---|---|---|---|---|---|\n", "| general | 5/5 | False | answer | **answer** | False | The time complexity of a binary search on a sorted array is O(log n), where n is the number of eleme |\n", "| live | 1/5 | False | use_tool | **escalate** | True | [ESCALATE — declined to answer] This question requires expertise / credentials I don't have. You sho |\n", "| medical | 1/5 | True | escalate | **escalate** | True | [ESCALATE — declined to answer] This question requires expertise / credentials I don't have. You sho |\n", "| legal | 1/5 | True | escalate | **escalate** | True | [ESCALATE — declined to answer] This question requires expertise / credentials I don't have. You sho |\n", "\n", "### 9.2 · Route + override distribution\n", "\n", "| Metric | Value |\n", "|---|---|\n", "| Distinct routes used | 2 ({'answer': 1, 'escalate': 3}) |\n", "| Python overrides fired | 3 / 4 |\n", "| Most common route | escalate |\n", "\n", "### 9.3 · Patterns surfaced in this run\n", "\n", "- **Partial route diversity**: only 2 routes used ({'answer': 1, 'escalate': 3}). Try more varied tasks to exercise all 4 branches.\n", "\n", "- **Python override fired on 3/4 task(s)** — specifically for: live, medical, legal. These are the cases where the LLM admitted credentials gap or low capability but its `route` choice would have let it answer anyway. Python forced `escalate` instead — the deterministic backstop working.\n", "\n", "- **Routing accuracy vs author expectation**: 4/4 tasks routed as expected. All correct.\n", "\n", "### 9.4 · The takeaway\n", "\n", "A *healthy* Reflexive Metacognitive run has:\n", "\n", "1. **Route diversity** — different task domains hit different routes.\n", "2. **Python override fires** on credentialed tasks (medical / legal / fiduciary).\n", "3. **No bluffing** — the answer for `escalate` cases explicitly declines, doesn't hedge.\n", "4. **`requires_credentials` semantic accuracy** — set True for tasks that genuinely require professional credentials.\n", "\n", "The deterministic Python override is the safety guarantee here. Even if the LLM is prompt-injected to say `route=\"answer\"` on a medical question, as long as `requires_credentials` ends up True, Python forces escalate. Never ship this architecture without that backstop." ] }, { "cell_type": "markdown", "id": "443f161b", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T13:32:20.330829+00:00", "exception": false, "start_time": "2026-05-27T13:32:20.330829+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 10 · Custom self-model — financial advisor scoped to retirement planning" ] }, { "cell_type": "code", "execution_count": 5, "id": "925bdc42", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T13:32:20.350399Z", "iopub.status.busy": "2026-05-27T13:32:20.347415Z", "iopub.status.idle": "2026-05-27T13:32:33.827963Z", "shell.execute_reply": "2026-05-27T13:32:33.826957Z" }, "papermill": { "duration": 13.490004, "end_time": "2026-05-27T13:32:33.827963+00:00", "exception": false, "start_time": "2026-05-27T13:32:20.337959+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Q: What is a Roth IRA?\n", " → route=answer (llm=answer, override=False)\n", " answer: A Roth Individual Retirement Account (Roth IRA) is a type of tax-advantaged retirement savings account. Contributions to a Roth IRA are made with after-tax dollars, meaning you've already paid income \n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Q: Should I move my 401k into VOO right now?\n", " → route=escalate (llm=escalate, override=True)\n", " answer: [ESCALATE — declined to answer]\n", "This question requires expertise / credentials I don't have. You should consult a qualified professional.\n", " decision route: escalate\n", " override: python_low_capability_o\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Q: Based on my financial situation, what's the right asset allocation for me?\n", " → route=escalate (llm=escalate, override=True)\n", " answer: [ESCALATE — declined to answer]\n", "This question requires expertise / credentials I don't have. You should consult a qualified professional.\n", " decision route: escalate\n", " override: python_credentials_over\n", "\n" ] } ], "source": [ "fin_arch = ReflexiveMetacognitive(\n", " self_model=(\n", " \"You are a financial assistant SCOPED to retirement-planning topics. \"\n", " \"STRONG: tax-advantaged account types (401k, IRA, Roth), withdrawal strategies, \"\n", " \"asset-allocation principles. WEAK: anything about specific tickers, market \"\n", " \"timing, current rates. ABSOLUTELY NOT: tax advice for an individual's situation, \"\n", " \"or any advice requiring fiduciary credentials.\"\n", " ),\n", ")\n", "for q in [\n", " \"What is a Roth IRA?\",\n", " \"Should I move my 401k into VOO right now?\",\n", " \"Based on my financial situation, what's the right asset allocation for me?\",\n", "]:\n", " r = fin_arch.run(q)\n", " print(f\"Q: {q}\")\n", " print(f\" → route={r.metadata['route']} (llm={r.metadata['llm_route']}, override={r.metadata['override_applied']})\")\n", " print(f\" answer: {r.output[:200]}\")\n", " print()" ] }, { "cell_type": "markdown", "id": "ed91f98d", "metadata": { "papermill": { "duration": 0.009298, "end_time": "2026-05-27T13:32:33.844597+00:00", "exception": false, "start_time": "2026-05-27T13:32:33.835299+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 11 · Failure modes, safety, extensions\n", "\n", "### 11.1 · Where this breaks\n", "\n", "| Failure | Mechanism | Mitigation |\n", "|---|---|---|\n", "| **Self-model too vague** | \"moderate\" routing for everything | Concrete examples of STRONG / MODERATE / WEAK |\n", "| **LLM admits gap but routes 'answer'** | LLM correctly fills `requires_credentials=True` but its `route` choice is 'answer' anyway | **Python override forces escalate** — this is exactly the case the override exists for |\n", "| **Over-escalation** | Agent refuses easy questions | Self-model says WEAK on too many things; tune the threshold or self-model |\n", "| **'use_tool' route is empty** | Architecture says use_tool but no actual tool fires | In production, route hands off to ToolUse (nb 02); this notebook stops short |\n", "| **Adversarial prompt** | Malicious user tries to make the agent answer a medical question | Python override resists prompt injection at the route level |\n", "\n", "### 11.2 · Production safety\n", "\n", "- **Always include the Python override.** It's the deterministic backstop for prompt-injected or sycophantic LLM routing.\n", "- **Treat 'use_tool' as an integration point.** Hand off to a real ToolUse architecture (nb 02) — don't let the route end with no action.\n", "- **Treat 'escalate' as an integration point.** Route to a human, ticket system, or higher-privilege model.\n", "\n", "### 11.3 · Three extensions\n", "\n", "1. **Domain-stacked self-models** — base + per-task self-models for multi-tenant deployments.\n", "2. **Quantitative capability score** — replace the 1-5 integer with a richer (probability of being correct × cost-of-being-wrong) calculation.\n", "3. **Closed-loop:** when the human resolves an escalation, feed the resolution back into the archive (Reflexion pattern, nb 18).\n", "\n", "### 11.4 · What to read next\n", "\n", "- [**14 · Dry-Run**](./14_dry_run.ipynb) — safety pattern for side-effects (companion).\n", "- [**32 · Constitutional AI**](./32_constitutional.ipynb) — self-critique against written rules (related).\n", "- [**11 · Meta-Controller**](./11_meta_controller.ipynb) — routes between *architectures*, not action classes.\n", "\n", "### 11.5 · References\n", "\n", "1. Flavell, J. H. *Metacognition and cognitive monitoring.* American Psychologist 34, 1979.\n", "2. Anthropic. *Constitutional AI: Harmlessness from AI Feedback.* 2022. [arXiv:2212.08073](https://arxiv.org/abs/2212.08073)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.0" }, "papermill": { "default_parameters": {}, "duration": 40.252817, "end_time": "2026-05-27T13:32:34.711775+00:00", "environment_variables": {}, "exception": null, "input_path": "all-agentic-architectures/notebooks/17_reflexive_metacognitive.ipynb", "output_path": "all-agentic-architectures/notebooks/17_reflexive_metacognitive.ipynb", "parameters": {}, "start_time": "2026-05-27T13:31:54.458958+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }