{
"cells": [
{
"cell_type": "markdown",
"id": "7e1d8c80",
"metadata": {
"papermill": {
"duration": 0.001002,
"end_time": "2026-05-28T02:31:33.521950+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:33.520948+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"# 23 · Agentic RAG — agent decides WHEN and WHAT to retrieve\n",
"\n",
"> **TL;DR.** Plain RAG always retrieves first then answers. Agentic RAG treats retrieval as **a tool the agent can choose to call** (or not, or multiple times with different queries). Each loop iteration, the agent emits a structured `(action, query, answer)` decision; the architecture routes accordingly.\n",
">\n",
"> **Reach for it when** the task distribution mixes retrieval-requiring questions and parametric-memory-answerable questions, OR when one retrieval round isn't enough (follow-up queries needed).\n",
"> **Avoid when** every task needs exactly one retrieval — plain RAG is simpler and cheaper.\n",
"\n",
"| Property | Value |\n",
"|---|---|\n",
"| Origin | LangGraph reference pattern, formalised by Jeong et al. (Adaptive-RAG, 2024) |\n",
"| Tool | Single `retrieve(query, k=top_k)` over a vector store (FAISS default) |\n",
"| Loop body | decide → (retrieve|answer); route on `action` field |\n",
"| Picker | Categorical action — **no LLM-as-Scorer** |\n",
"| Default LLM | Llama-3.3-70B (cheap; the decision is structurally simple) |\n",
"| Cost | 1 decision call per loop iteration + 1 retrieval call when action=retrieve |\n",
"\n",
"**Why this is different from Tool Use (nb 02).** Tool Use exposes K diverse tools (search, calculator, etc.). Agentic RAG specialises to ONE tool — retrieval — but adds the structural commitment that the agent will route between \"I need more context\" and \"I'm ready to answer\" via an explicit `action` field, making the decision auditable."
]
},
{
"cell_type": "markdown",
"id": "4342c3a1",
"metadata": {
"papermill": {
"duration": 0.013166,
"end_time": "2026-05-28T02:31:33.535116+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:33.521950+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 2 · Architecture at a glance\n",
"\n",
"```mermaid\n",
"flowchart LR\n",
" A([task]) --> D[DECIDE
action: retrieve or answer?]\n",
" D -->|retrieve| R[RETRIEVE
vector search, top-k]\n",
" R --> D\n",
" D -->|answer| F[ANSWER
commit final response]\n",
" F --> Z([final answer])\n",
"\n",
" M[(VectorMemory
corpus)]\n",
" R <-.search.-> M\n",
"\n",
" style D fill:#e3f2fd,stroke:#1976d2\n",
" style R fill:#fff3e0,stroke:#f57c00\n",
" style F fill:#e8f5e9,stroke:#388e3c\n",
"```\n",
"\n",
"The router lives on DECIDE's structured output. The final iteration is forced to `action='answer'` regardless to avoid running over budget."
]
},
{
"cell_type": "markdown",
"id": "1b1811d8",
"metadata": {
"papermill": {
"duration": 0.0,
"end_time": "2026-05-28T02:31:33.541135+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:33.541135+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 3 · Theory\n",
"\n",
"### 3.0 · Why the action is a categorical (deterministic-picker style)\n",
"\n",
"If the decision were a numeric \"retrieval confidence score 0-1\", we'd be back to LLM-as-Scorer flatness — the model would emit `0.5` or `0.8` for every task. The fix: the LLM commits to a **categorical** `action: Literal['retrieve', 'answer']`, plus a `query` (if retrieve) and an `answer` (if answer). Python routes on the categorical. No numeric judgement, no flat-scoring risk.\n",
"\n",
"### 3.1 · Multiple retrieval rounds\n",
"\n",
"Unlike plain RAG which retrieves once and commits, Agentic RAG can loop:\n",
"1. Retrieve \"Stardust 9 payload\" → docs include the rocket facts.\n",
"2. Realise the answer needs context about the customer — retrieve again with \"Stardust customers\".\n",
"3. Now answer.\n",
"\n",
"This is essential for multi-hop questions where the right *second* query depends on what the *first* retrieval surfaced.\n",
"\n",
"### 3.2 · When the agent over-retrieves (a common failure)\n",
"\n",
"A naive agent will retrieve even for arithmetic (\"what is 17×6?\") because the prompt mentions a tool. We mitigate via:\n",
"- Explicit rule in the prompt: \"Call retrieve ONLY when you genuinely lack a fact.\"\n",
"- The DECIDE prompt always includes prior retrievals so the model can see \"I already have what I need.\"\n",
"\n",
"Even so, expect occasional over-retrieval in the captured run — § 9 surfaces the pattern.\n",
"\n",
"### 3.3 · Where this sits\n",
"\n",
"| Pattern | Retrieval strategy |\n",
"|---|---|\n",
"| Plain RAG | Always retrieve once, then answer |\n",
"| **Agentic RAG (this nb)** | **Agent decides each step: retrieve more, or answer now** |\n",
"| [Corrective RAG (nb 24)](./24_corrective_rag.ipynb) | Retrieve → grade docs → fall back to web if poor |\n",
"| [Self-RAG (nb 25)](./25_self_rag.ipynb) | Retrieve-on-demand with reflection tokens; deterministic-picker on tokens |\n",
"| [Adaptive RAG (nb 26)](./26_adaptive_rag.ipynb) | Router picks: no-RAG, single-step RAG, multi-step RAG |\n",
"| [GraphRAG (nb 27)](./27_graph_rag.ipynb) | Knowledge graph + community summaries (different retrieval shape entirely) |"
]
},
{
"cell_type": "markdown",
"id": "0a8d6889",
"metadata": {
"papermill": {
"duration": 0.004024,
"end_time": "2026-05-28T02:31:33.557129+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:33.553105+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 4 · Setup"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ad0af08e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:31:33.569224Z",
"iopub.status.busy": "2026-05-28T02:31:33.569224Z",
"iopub.status.idle": "2026-05-28T02:31:35.597432Z",
"shell.execute_reply": "2026-05-28T02:31:35.597432Z"
},
"papermill": {
"duration": 2.035513,
"end_time": "2026-05-28T02:31:35.598939+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:33.563426+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"LLM: meta-llama/Llama-3.3-70B-Instruct · Corpus size: 12 docs ───────────────────────────────────────────────────\n",
"\n"
],
"text/plain": [
"\u001b[1;36mLLM: meta-llama/Llama-\u001b[0m\u001b[1;36m3.3\u001b[0m\u001b[1;36m-70B-Instruct · Corpus size: \u001b[0m\u001b[1;36m12\u001b[0m\u001b[1;36m docs\u001b[0m \u001b[92m───────────────────────────────────────────────────\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from agentic_architectures import get_llm, enable_langsmith, settings\n",
"from agentic_architectures.architectures import AgenticRAG\n",
"from agentic_architectures.data import STARDUST_CORPUS\n",
"from agentic_architectures.ui import print_md, print_header, print_step\n",
"\n",
"enable_langsmith()\n",
"llm = get_llm(provider=\"nebius\", model=\"meta-llama/Llama-3.3-70B-Instruct\", temperature=0.2)\n",
"print_header(f\"LLM: {llm.model} · Corpus size: {len(STARDUST_CORPUS)} docs\")"
]
},
{
"cell_type": "markdown",
"id": "eda9f5e2",
"metadata": {
"papermill": {
"duration": 0.0,
"end_time": "2026-05-28T02:31:35.598939+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.598939+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 5 · Library walkthrough\n",
"\n",
"Source: [`src/agentic_architectures/architectures/agentic_rag.py`](../src/agentic_architectures/architectures/agentic_rag.py).\n",
"\n",
"The `_AgentDecision` schema is the load-bearing structured output:\n",
"\n",
"```python\n",
"class _AgentDecision(BaseModel):\n",
" action: Literal[\"retrieve\", \"answer\"]\n",
" query: str # if action='retrieve'\n",
" answer: str # if action='answer'\n",
" rationale: str\n",
"```\n",
"\n",
"Router checks `action` and routes to RETRIEVE or ANSWER. The last iteration is forced to ANSWER (prompt patch) to prevent runaway loops."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d14cd09e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:31:35.613132Z",
"iopub.status.busy": "2026-05-28T02:31:35.613132Z",
"iopub.status.idle": "2026-05-28T02:31:35.660573Z",
"shell.execute_reply": "2026-05-28T02:31:35.660573Z"
},
"papermill": {
"duration": 0.047441,
"end_time": "2026-05-28T02:31:35.660573+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.613132+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- _AgentDecision schema ---\n",
"{\n",
" \"description\": \"One step of the agentic-RAG loop.\",\n",
" \"properties\": {\n",
" \"action\": {\n",
" \"description\": \"Either 'retrieve' (need more context) or 'answer' (have enough to answer).\",\n",
" \"enum\": [\n",
" \"retrieve\",\n",
" \"answer\"\n",
" ],\n",
" \"title\": \"Action\",\n",
" \"type\": \"string\"\n",
" },\n",
" \"query\": {\n",
" \"default\": \"\",\n",
" \"description\": \"If action='retrieve', the search query \\u2014 focused, specific. Empty string if action='answer'.\",\n",
" \"title\": \"Query\",\n",
" \"type\"...\n"
]
}
],
"source": [
"from agentic_architectures.architectures.agentic_rag import _AgentDecision\n",
"import json\n",
"print('--- _AgentDecision schema ---')\n",
"print(json.dumps(_AgentDecision.model_json_schema(), indent=2)[:500] + '...')"
]
},
{
"cell_type": "markdown",
"id": "30c214b5",
"metadata": {
"papermill": {
"duration": 0.015753,
"end_time": "2026-05-28T02:31:35.676326+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.660573+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 6 · The corpus we'll query\n",
"\n",
"Fictional knowledge base about *Stardust Aerospace* — entirely made up so the model can't cheat from parametric memory."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5ea99b87",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:31:35.692157Z",
"iopub.status.busy": "2026-05-28T02:31:35.692157Z",
"iopub.status.idle": "2026-05-28T02:31:35.708094Z",
"shell.execute_reply": "2026-05-28T02:31:35.708094Z"
},
"papermill": {
"duration": 0.031768,
"end_time": "2026-05-28T02:31:35.708094+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.676326+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 1] Stardust Aerospace was founded in 2019 in Reno, Nevada by Dr. Amara Okonkwo and Jin-ho Park. The company emerged from a ...\n",
"[ 2] The Stardust 9 is the company's flagship two-stage orbital rocket. It uses methalox propellant (liquid methane + liquid ...\n",
"[ 3] The Stardust Lite is a smaller single-stage suborbital vehicle introduced in 2021, primarily used for atmospheric resear...\n",
"[ 4] Stardust's Phoenix-2 engine powers the first stage of the Stardust 9. It produces 215 kilonewtons of thrust at sea level...\n",
"[ 5] Dr. Amara Okonkwo, Stardust's CEO, holds a PhD in aerospace engineering from Caltech (2012) and previously worked at Blu...\n",
"[ 6] Stardust Aerospace's primary customers are NOAA, the European Space Agency, and three commercial smallsat constellation ...\n",
"[ 7] Each Stardust 9 launch follows a strict pre-flight checklist including a static fire test 72 hours before the planned la...\n",
"[ 8] Standard list price for a dedicated Stardust 9 launch to LEO is $4.2M, competitive with Rocket Lab's Electron ($7.5M) on...\n",
"[ 9] Stardust's manufacturing facility runs on 100% solar power (10 MW rooftop array installed in 2022). The company has comm...\n",
"[10] In April 2024, Stardust's seventh Stardust 9 launch failed during ascent due to a Phoenix-2 turbopump anomaly at T+93 se...\n",
"[11] Stardust's announced 2025-2027 roadmap includes a third vehicle, the Stardust Heavy, with a target LEO payload of 5,500 ...\n",
"[12] Stardust's engineering organization is structured into five pods: propulsion, structures, avionics, ground systems, and ...\n"
]
}
],
"source": [
"for i, doc in enumerate(STARDUST_CORPUS, 1):\n",
" print(f'[{i:2d}] {doc[:120]}{\"...\" if len(doc) > 120 else \"\"}')"
]
},
{
"cell_type": "markdown",
"id": "018f5ef4",
"metadata": {
"papermill": {
"duration": 0.015922,
"end_time": "2026-05-28T02:31:35.724016+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.708094+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 7 · Build the graph"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "46b5efac",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:31:35.739170Z",
"iopub.status.busy": "2026-05-28T02:31:35.739170Z",
"iopub.status.idle": "2026-05-28T02:31:42.289075Z",
"shell.execute_reply": "2026-05-28T02:31:42.289075Z"
},
"papermill": {
"duration": 6.565059,
"end_time": "2026-05-28T02:31:42.289075+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:35.724016+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOkAAAFNCAIAAABqr9/4AAAQAElEQVR4nOydB2AT5fvH38slabr3oEApLYWWXSxTmaWAgIJs2Us2MhXxDwKCgCD+UFERAWUIyBLZUzZlyKZAoVBKKaUtHXSlWXf/53IlTUsS2kibG+9HrZd73/cuee97zz3v846T0jSNMBgeIkUYDD/B2sXwFaxdDF/B2sXwFaxdDF/B2sXwFYFr99b53MS7edmZGq2K0qqKRQNpQv8P/B8RRXslNKIIQypskDKk06CSGdhyxkcr/EyUOIVUKtFpjM77sqBEhij9YQkJoimjdESTComdPenkJg1r5Fatrh3CmIEQZHz3xLa0h7dyC/J0EgkhtydlckJCEjoNVSwTwSgOEaCWohogJARNMR9pgiD0+yUygjISHxyH0tFIglDxgxXeASXqkkCklDDWruFspOzlfqJYKfgCoGatmlYp4TTMuZxcpXVbuDZq54YwxRGadg9tSIm/lQeKqVzD/u2u3q7eJOIzibEFl46kpyWqCBI1aucR0R4ruAgBaZdCv34RLyFRi87eYU2dkLA4syvjVnSmvZN0yOxqCKNHINq9fjL7zO60sCau7fp6IeGy44cnqYmqsUuCEUYY2n2Rrvtj4aNxy0RxRe/+m39009MJ39ZAoof32r18LOvfoxmjFwUh8aBDP376YPzSYKbJKGL4/etfpOouHEgXl3ABEnUYWGnlzAdI3PBbu5u/TWjcwQOJj5BwB5+q9uvmJyARw2Pt/vXjU4WDtHEHdyRKekzwz8/RXjqchcQKj7Wb9DC/16TKSMQ0bOlx7WQmEit81e6un5KdnGVOrvzueviPNH/PXaulr/zzAokSvmo35XFBvbcrtJPpwYMHXbt2RWVn69atc+bMQeWDTxU70ZpeXmr36UONVkO9FeWKKpDbt28jq7C6YGmATsSCfB0SJbwcR3btRIbCvrzuupycnJUrV545cyYjI6N27drvvvtu9+7dYc/q1ashNSIiYsqUKQMGDDh9+vShQ4euXr364sWLunXrjhw5EpIgQ1xcXL9+/ZYvX75gwQJ3d3dnZ+crV67A/n379m3cuDE0NBS9USoFywmE7l3Jq9nIEYkMXmo3I0Xl4iFD5cO8efNSUlJmzpxZvXp1eNwvWrQoKChozJgxarX68OHDe/fuhTwFBQWzZs1q0qQJZIaPR48eBUHv2rXL09NTJmO+GAh90KBBDRs2rFOnztChQ6tVq8bmLA/k9pL4W7lYu/xApdR5+ctR+QBmcvDgwc2aNYPtiRMntm/f3s2tpGOtUCi2bNlib2/PJoHd3b59+7Vr1yIjIwmCGQsJxcE2owrBTkG+SNcg8cFL7VI62k5RXj4DGEt4uGdlZTVq1Kh58+ZhYWEms+Xl5a1YseLy5cvPnz9n92RmFrWZzJUqD6RyQqsWo8vLy7YaRVEEWV7ffO7cuf3794+Ojp46dWpUVNTPP/+s1WpL5Hn27Bk4uBqNZuHChZDz/PnzJTLY2VXcfAdmQErxSRwigZd2VyYn1UoKlQ8uLi7Dhw8fNmzY9evXjx8/vmbNGmhvDRw40DjPkSNHwP0FFxbcBlTc4lY8Wi0lk4txVA5ftZudWS4eHgQNDh482K1bN/BoG+qJjY29e/fuq9lA4qxwgWPHjiHboc6nPbzEOGeWl/erszuZmaJC5YBUKl21atWMGTPA6Kanp0NgC4QLCoakgIAAcG1PnDiRkJAQEhIC2zt27AB34ty5cxcvXoRGGzgSJo9ZtWrVW7duXbp0CYJuqBwoyNdWCRVdkAHxVLt1W7irlOUy7NjR0XHp0qWpqakjRozo2LHj+vXrJ0+e3KNHD0h65513QMTTp0+HsC4kQYZff/0V4gmbNm369NNPO3fu/Pvvv4P7++oxoTgEH8aPH3///n30pnnxXEdRqEFLFyQ++Dr2fMXUuDa9vOu2qNCuNQ6y59fk5IfKUWIbwayHrz6+m7fs4iHxDqEykHgvv2YjZyRK+Orj955cbfWsOAsZIERgrivL1dUVGlsmk6D7F5wEVD7AkaH/wmSSSqUyF1aDQEdwsOmpeDfPZNMU3aa3NxIlPJ6v9sfix4QE9f80wGSqUqk0F7qCJEOIoAQODg6v9qK9KaB5B5E1k0nZ2dkQuDCZ5OPjAy1Ik0krZzwIqufcYaAPEiX8nmsJXu/7oysH1LJH4uPwhtSHMbljFovR02Xhd0y79Qe++9Y8ReJDo0b3r+aIWbiI79qt19K5ai3HtXMeIZGxZvaDtn18kbgRwtoidy7lntiWOnaJWIzQimlx/aYGeFUur5F0fEEgazrtX/MsITav0yD/6vWE7Pue3Z1x9XjGu8MqBdcXY0daCYSzlt7Ns8ySZK6e8v6fVkWCIzVBs/e3JI2KHr2gOhL1BNMihLaG6ZZlT9Kfqlw9ZA1audUTRE/puT0Zdy5lq/J1AbUcu37khzAvEeba0Tu+e5r2tADi9nb2pIML6egiZdZq1hYbNikhJZSuaA+7/rhEgihKv00XrSlNSpHu5Qhew+LPzCrTiGbWkIb8RGE1SpidiF19mpBIaIoyFCf0rWLaKDNJEjodXeILkFL4VrRaSeXm6ApydRoVJZMRlWs6dBmOVVsSYWqXJem++ubZzMxUlUpJadSUtsSoSYLSa8/wkVl/nJUmM22HAvUWDujWr4VuGNxNswvzMzkpCnJJJKRh7XKmBFOhhHFOvYaZqUC0fmV/vXT1R5DQNGU8ZpzW3xFM7IeUEo7OMp8Au+adPB3cxL1innmErN3yJiUlZfjw4fv27UMYW4Df82M9Wq3WXG8tpgLAVW89WLu2BVe99WDt2hZc9dYD2mVXEsHYBKxd68F217bgqrcerF3bgqveekC7JIn7Z20G1q71aDQa7O/aEKxd68E+g23BVW89WLu2BVe99WDt2hZc9daDtWtbcNVbD26r2RasXevBdte24Kq3Hqxd24Kr3nqwdm0Lrnrrwf6ubcHatR5sd20Lrnrrwdq1LXgen/XodDqsXRuCtWs92O7aFlz11gNtNaxdG4Kr3nqw3bUtuOqtR6EHYWwE1q71qNXq/Px8hLERWLvWAw7Dq68axlQYWLvWg7VrW7B2rQdr17Zg7VoP1q5twX0T1oO1a1uw3bUerF3bgrVrPVi7tgVr13qwdm0L1q71YO3aFqxd68HatS1Yu9aDtWtbsHatB2vXtmDtWg/Wrm3B2rUerF3bgrVrPVi7tgVr13qwdm0Lfq9lmRkyZMitW7cQ8yZh/atW9eh0uuvXryNMBYLH4pSZcePGeXl5gWpJkpTogfu/cePGCFOxYO2WmaZNm9auXdt4j7Oz8+DBgxGmYsHatYbhw4d7eHgYPgYFBbVq1QphKhasXWuoV69eeHg4uy2XywcMGIAwFQ7WrpWMHDnSx8cHNgIDA6OiohCmwhFmnOHMzqyc3AKtioJtaFTBb5SQBKVjfqlEStAUTVOwgSh9gKsoScLkZrfZUuxO+B+TX8KEFGjElGUg0J07d9LS0kJCQir7+zNJTFphZRaWYv4r/EoSElG6Yl/ScF4DUimh1Za8HAqFzMVb2rSTO8IUR2ja3bk8KSWpQCYnaVCmmtlDE/AjEQiP0v9QCQm/GOSLCJKgWcm+TCIY7SK6UGE080G/k5UgEw2DVEQhSv+wIhgV08yRJYR+B00VFnlZimYLFO6B8+oKU9mvREB8giKMvzwpQzpNyV8kVxA6Ha3TopCGLu37eyHMSwTVN3Hwt5TcHGrgzGAkuDelpiWoj/2ZHHtRUauJU5kKChjh2N3dK5PTkjR9pgcg4bJpUXxEB8+32rkgjJDaaknxytY9/ZCgqRbmfP1EBsLoEYh2464pwXP0rS5HgqZuS3dVAYUwegTi7+Znq0u02QWJvQOp02LtFiKUthrECijha1en0yEs3ZfgMZAYvoK1i+ErQtGuUZ+WkCEIhHmJcPxdQgzXlWD65DAs2GfgFRR0ZyMMi2DsLo2wQRIZgvF3CUIMT1Ps7xohHJ+BRiJ4muKJsUZgfxfDVwSkXRHYJIKQ4MaaAQFpVwS+IDNYHQfJXoLnqxVy/MSRtpERWVmZyCrmzP102vSxJpOGjeiz/LvF6I2AdWuEQOyu8cwwm9CqVaRGo0blDfYXjBCIdtnJZDYksl1HhKlYRB1nWPnLd4eP7HOwd4iM7FSlSjXjpIOH9uzesyM+Pq569Rrt2nbo2eNDQ59zdPTp7374Oi0ttUZwze7d+7zb6X2k9xlyc3OWffMzbD969HDx13MSHsc3bBgxeOBI48PGxNxYt37V3bsxrm7uzZu1HDJ4lKOjIyo1BHYajBCMv0uUdTzD37u3/71726SPZ/z00/pKlSqv3/CrIenosYNfL5lXMyR008bdI0eM375j04qflrFJINzZc6aPGD5+8aLv33mn7ZKlX0Jm48NqNJoZMyd6e/v+vnb76I8+3vLn+vT052zSk6TE6Z+OK1AVrPjht/nzvnn48P6UqaPKtpIkHs9ghGC0S5d1HNnOv7a0btW+datIF2eXTh3faxRetBje/v276tcPnzzpM3d3D9g/bMiYXbu2ZmYyE8V++31lq5btotq/2zii2aCBI/r2GZSfn2d82FOn/0lNTRk/bpqvr19gYNDHExl7zCYdPXpAJpWBagMCAiFp+rTZ9+Niz5w9gUqNfno+wrAIR7tlepyCCJKSEkFAhj01a4axGxRF3Yq53jiiuSEpPLwx7Lxx8yr8ffDwfmhoHUPSmNGT3n+vp/GR4bAKhcLPrxL70dPTy8fHl92OibkOZV1d3diPkMffvwocFmGsQjBjcYgyTYZRqVQ6nc7e3sGwR6GwZzfUajU899es/Qn+NS4CdregoADka2ensHDk7OwXxocFDPnBAN+NvQ2RuGKHzUhHGKsQztjzMtldOzs7kiRVqgLDHqUyn90Aq+ng4NAhqguEvYyL+FeqAqUkEkleXq6FI7u4uBoOxWJwKjw8verVazhs6BjjVFcXN1R68FgcIwQU3y0L0LDz9a0ErX7Uu3DP+QtnDKnBwTVzcnPCGxYaSDDDyclJ8OiHUrVq1b5565oh56+rV4CdHj9uqmGPn28lMM8PH8YFBdWAj3Fx954/Tys8bFAIhDUa1G8kkRS6ahCRqFKlLIuhiGO0XCkRiL9rhT1q2yYK2lXQnQbbm7esu337piHpoxETzp49sf/A3+Ah3Lx57cv5M6dOHwMahaRu7/W6dCn6z60brl77FyIVULB69WDjw7Zo0Voul3/z7QJQMKj2ywUzwRKzSb16DYADQsgCkhITE35Z9f3wkX0fxseh0kNRuK1mQLzx3YEDRkAP8A8rloI04VE+buzUrxbOYoMV8HHVyj/+2PQbyKugQFmndv0F878FhwGSOnbsmp3zAmK0eXl50A4b9dHEzu92Mz6sk5PTwq+Wr1r1fdf3W4P7Meqjj48eO8AmQUBjzeo/t2xZN3rswMePH0G77ZPpsyEShzBWIZD1yG6cfnFq5/Mhc4ORoFHm6rYujZ+wvAbCBNDu8wAAEABJREFUCGrOjwTPExYXwpnzQ4tgXRxm3V6s3pcIJc5AIEPjXcDox+8iDItQxpHRiMJzuUSGcPxd7AqKDcHEyAhxLOlE4ME4BgSjXVHYXf04Mvx8KURA4xnEsDAtdoyMEM5aeqKYNorbo0YIJ74rhnmI2N81Rjh2VwyXFPu7xgilbwLREnxNRYZg+iZEESPDGCMQ7UpIRMqEb3hJKSkRwc8sJQJpnAeHutI6JHgS7+WTJNZuIQLRrr0HUjiS53anIUFz+1y6u4/A391ZeoQTFO02ptrDmzlIuNb33yM5OVmaPlOrIIwe4bzHHTGvfUSrPnvo4WcXUMvZwVVCUcW72gj9f7R+JQf2RxNsaI02pBf2LBfVCTNilk1nYqtMWbpwaRqi8FCE/pi04QD6w9H63EUnKkzU/18ioWnq5cRmovBcEv03geNICEI/ENmwAT5uRrL6cWyuKk/bbbKjj48PwugRlHZZtix7kv1co9XSlLakdpn4qPFc+JdKNoaJoJqMoerFR+vFXJSZLuqmNZ5mX5iNKLZyI1u6+P1i+izIcKsw2iVkcolnJcW7I9179OixePHi+vXrI4wgtYv0a9t07959wYIF5XqZDxw4sHz58pUrV1avXh1VFOfPn2/WrFlSUlLlypWRuBHgIAC4rkqlEiRV3vZp48aNz58/37x5M6pAQLjw9/vvv9+0aRMSN0LT7qRJk0C4jo6O/v7+qDzZuXPno0eP4Ml+6dKl5ORkVLF8/fXXcjkTcEhLE3hoxQLC0a5Wqz158mSfPn1q1Cj3KeBwrm3btqlUKth+8uTJjh07UIXTq1cv+Hv16tUlS5YgUSIQ7f7www+gpNatW7/99tuo/IHn9ePHj9ltaDAcP348M9PKF1X8Rzp06BAYGHjnzh122R5RIQTtgpJcXV3LtID4fyEvL2/Pnj2s0WUB07tr1y5kI+BRU7NmTfg+s2fPRmKC39o9deoU0tuewYMHo4oCmmgGo8ui0+n27duHbAdJks7Ozi1atFi6dCkSDTyOka1atQr8znHjxqGKBRzN3NxcODU8pgsKCmQymUajkUgkEL1CtoYJYBPEzz//PGjQICcnJyRoeKndlJQUX1/f6Ojo5s2bI9sBXwDcFXC1EceIiYn57LPPwLFBgoZ/PsPatWv3798PG7YVLtJHG6RSLg4irVOnDivcc+fOVXz8rsLgk3bBrYSnMzymhw0bhjgAZ7VrIDQ0dNSoUYmJiUiI8Ea70Cw7ceIEaKXiHVxzcF+7Hh4eYIChhxxu+/v37yNhwQ/tJiQk/P3335GRkQSXFijgvnZZqlWrBoGIL7744tChQ0hAcF27oNr09HSFQrFs2TLEMfiiXZbNmzfb2zPvMoKubCQIOK3dmzdvTps2zc3NDaIKiHvwS7uIeWF3K/h75MiRhQsXIv7Dae1mZ2dv374dnneIk0DDEYK7iG989NFH0IYDDzgjIwPxGS5qNzY2tkePHrBRMYMTrIZ3dtcAVC9YhGfPnn3++ef87ZzionYhfLtt2zbEefirXZbatWu3adOGDZbzEQ5pF9pkK1euhI0pU6Zw1k8whu/aRfqhIF26dIGNGTNm5OXlIV7BIe0OGDDggw8+QPxBANo10LNnz7lz5yJewQntXr58Gf4ePHiQm/EEc0BzRzDabdKkCTsGbcOGDU+ePEF8wMbaVSqV7du355dkDQjJ7hqIioqaMGECXBfEeWxZ9RACS05OhigYRHARDxGkdv38/Hbt2pWfn3/v3j14sISFhSGuYjO7O3r0aIjO1KpVi6fCRQLVLouDg0NgYCB0YbDuHDexjXYhBDZ06FBXV1fEZ8AysTPOBYlcLgffl+1G5ibCXFukAoDw/vz58+vUqYMETUJCQmJi4jvvvIO4h23s7v37948dO4b4CbgKENJfvny54IULxMXFcXb+hW20C08iDk6VKQ2pqalghPbu3RsQEIBEAHi9LVu2RJzEZj7Dzp07oUfHzs4O8YfY2Fjo8+NvJ6rAwP5uaTl//vyKFSs2btyIxMSzZ89iYmIiIyMR97BZjOz06dMnT55EPAFs7UY9SGQkJSVt3boVcRKbaVcmk/FisBjwxx9/sEYXiY9KlSpBxyfiJDbzGaC1furUqXbt2iFuA5LVaDTg5iIMx8D+riUgglu1alXoRkFiJTMzMzo6unPnzoh72HIszqZNmy5duoS4CtjaBg0aiFm4QEZGxvr16xEnsaV2FQrF0aNHEScByULP2fvvv4/EjYeHBzeNLrKtz6BUKuPj42vXro04Rrdu3RYuXCiGbjNeg/3dYhQUFERFRW3ZsgW/iYQlPz//4MGD7NRXrmHjseeLFi26c+cO4gbJyckQDzp8+DAWrgF4Nq5atQpxEhtr18nJ6eLFi4gDQO/RqFGjzpw5w+VRfxWPo6MjN40usrnPkJeXl5uba/M5PyDZX3/9dd26dQjDH2xsd+G2NggXWkjIFuzZs2f79u1YuCbR6XQV/AK50mNLuwtiVavVaWlpUEHsAo/Dhw+fMGECqkAgeAmxjjlz5iCMEZMnTz558iR7UZiXHDPvQGZ0cuXKFcQZbDbdKjw83LCACLvh6urauHFjVIEsX74cTo2F+yoff/wx3NJJSUmGPaBjrg1ZtpnPMGbMmBKtIhcXl4qM9YJkvby8Jk6ciDCvEBQU1KJFC+M9YHe5NgjdlvOE3377bcNa0BRFgePr7OyMKgSwK02bNh04cCDCmGHQoEHGhrZq1apcW7XIlm21xYsXh4SEgGqR/pFUYXNuQbL9+vXjbFcnR/D394+MjGQn8cM1ioiICAwMRFzCxnEGkC/7+l/oN6+YPtguXbrMmjWrxAMRY5K+ffuyprdatWq9e/dGHKNUbbWH1wvy81++rhYe8obIhGEbbgHqlWLGOc2muPXpNPXwkWNgeMn8oFvR2SULGX0mICpilFgsJ63/bP6MgEql+vabbz6f+Is20wVOVHQckgit50zyp0ci9bE67ZmK1hX7qbS+dkrUALuDgXj5oXgeQoLolxcO3LdXYk7yyCbDpQUna9espcmoVFhpRQd95SzmsHhdDEgI5OAsC6xT2ivxmhjZ1m+fZDxTw7k1aurVr8F+Y7r4zzZkMKSyu4oVNK4yCL9QhkoteSimkfCyZiQSRFHFfipV/KRFp2BEThh/E/1G4fUtgVwu0VG0vSM5eHogye1XQZ7a8fzu5WydDmoM6bTsT3lZb0YaKqrnl5UpIQiKpolXLxZRJABzsmfqjclmSGCq0fhblbRmrypVfzlK5nzlVoE9UhnjCPgGKD4Y749ehyXtblmSpNFQrXr5evjJkdA581fao5icEV8GyblqgO/9m398+7Pwtt5hzSqoRWsTnsWrz+565uYn6z6mkuWcZrW7bsFjBwdppxGvl79gUOehrcsfjF0SjLjHqZ0Z965k9/0kEImD3T8lklLU75OqFvKYbqvFXlYqc7WiEi4gd0SuXnZ/LuPi6rN3L2XVb+mJRMP746pmpmqy0izlMa3dmPNZji7C9xNeJSDMJTtdgzhGarya0iJhuwqvorAnLxywJF7TcQZlruZ1TUdhYu8s0WgpxDHS0lRUaRrqAoOg87JVFtJNa1erpijOXcGKgKYonYZzKoFw2MuogojQaGi1ytKvFubSxxgxgLWL4SoEsvzmc6zdYhCIo36+CFsfBNu9ZB6s3ZJw068UX0sNIVOdoMZg7RaDYnomRakT7gGdZrTFgIFp7UokSJzLNjAd7jTnns9wO0m4+OLn8oWQIMu/2rR2IUAmWu1y0rMkaDGGLF8zQM20dkHyouyaYGqLg/csTYvR36UpgqLKHt+Fu1yUNzqGT5htq9GitLwUgdtqXAEe/qQV/i7SR9eQ+JAUDpXHcAAKnv2WroVpYUP7jrBRw7bbB5HrN6xGtoODjVS4HBISiQ2aaaFauhimFVqu/m58/IN+/buaS+3bZ1D9euEIYwSEfSgdwpTATFsNlSOx925bSO3/4VCEweinr1mO774xzwCe9Tt2bJ405aO2kRHZOcyE0oOH9oybMPTdLu/A3+07NrGTi377feXXS+alpDyDbNu2//HwYRxsnD9/plefTiNHfYiK+wwxMTc+nTHh/W5tBw3p8dPP/8vLy4Odl/49D0Vu3bpuOPWduzHMQS6cNVekTAjD283NzYWqHjt+CNT/wEHdoSoKCgrYpHlffvbl/Jnnzp16v3u7qI7N4JLduXOLTcrJzfl+xdIBA7t17tpyytTR+/bvgp3zF3w+ddoYw5GHDOsF18jwEVI/+3wSYt5Mkb7gq/+DJ2r3Hu2/WjQ7MTGBzbBj55aevTueOXsiMqrJDz9+g8qCZRtqRruFkzrLgEwm27v/rxo1ai1d8qODvcPRYwdBozVDQjdt3D1yxHjQ7oqflkG2YUPH9Os72NfX7/ixf3v3GgClYOf6javBVZg2dZbxAZ8kJU7/dFyBqmDFD7/Nn/fNw4f3p0wdpdVqG4U3dnZyPnX6H0POM2eOw57GEc3MFUGlhl08DvGfnX9t2bT5d6jVhV8tHz160omTR9atL1wCWiqVxty+ceTo/pU/bziw74yd3G7R13PYpCVL5t2OuTF58szf124PC6v7v+WLwBY0atTkzt1bOh3jtWRmZqSkJMPGkyeP2SI3b12LeKsppE6ZNvra9ctTJn++dvWf7m4e48YPSXrKzJ6Sy+X5+Xm7d2+f+dmXH3TrU+pfwKwiZVmCZrRLo7JeQoIgXFxcJ46fDr8Eamf//l3164dPnvSZu7sHqG3YkDG7dm2FX/5qKfgLsgMdh4UWW1vk6NEDMqkMJBgQEBgYGDR92uz7cbFw+5Ik2bZth1Oni14DDzqOjOwE+80VQaVG3wvAPcv7utGAr9Kn98DVqza3ad0+vGFEy3fatm3T4eKlc4ZUZX7+J9O/8K9UGa5UZLtOYCPz8/Nh//UbV1q1ioTL4ePjO+qjiT+u+N3T0zvirWZgsx/Gx0EGUGdQUEitmmGQEzFvbE1OS0t9q1HTmzevPX786POZ85s2aeHh4Tl2zGQXV7cdOzYh/SWG4v36DWkf2alKlTKtxkdYngBhJs4gJaxo2NaqWbgSHkVRt2KuN45obkgKD28MO2/cvGqyYM2QsFd3xsRcDw2t4+rqxn7086vk71+FPUKbNlHgddy7fxfpW35gA+ACmCsChgHxHRriRWUzJfBAu/Rv9Nhxg8ErAIdq67aNxoajakCgg4MDu+3kxEyDy9G7efXqNYScP69cDh6FRqMBjUIdwkOSqcabTDVCZdat0wBMMthj+HjjxhVPT6/q1YNhP5wRjBR7TNBrwwZvsfpmCa315hc9MjOeQUtbMecHng7shlqthl++Zu1P8K9xhlftbmFBU29zz83NuRt7G+q92BEy0uEv1AuY81OnjoFPcvrMcW9vn7p1G1guwneIMj4NVv36Azz6wFsACwLiW73mx/0H/jakSsw0gmZ8Ohce7v8cPwQKdnJ0+uCDvhgWeaAAAA3nSURBVIMHfQS2GUQJdqHHB32vX78MXp+dneK777+G/GBKwvV6hZqHK16i5t3c3A3bBm2UHivH4vxHFAoF3NYdorrAA8h4v3+lKqU/iIenF5gBqCnjna4ujE2F2xrcBnAGwJMGZzeqfefXFiklnB17XibAU9yzd0evnv27dilcuRG0VZqCLs4uAwcMH9B/GDSFwShs2LgGrDK4H2+91fSXX7578SIL2taNwpuAe/b06RP4COa2f7+hUBCsr729/VcL/md8NLKcg9JmxuKU3cEqQXBwTWi0grPFfoSbMjk5CbyoMhwhKOTwkX0N6jcyGIlHjx4aHKZ2bTrs3LkFAhTg0YKbVZoipYQgeK9eaDkplUovLx/2IzwGz0Wfem2pF9kvjh072PndbmB6wATAv3FxsaxjBtfxWUrysX8OBQeHsM5GrVq1oXUBPm5EBLN6J1xuOKOPj19l/0Lz9DQ5yc3VHf0H4A607CmZNcr/sbH90YgJZ8+egOcUuLngKkFQZur0MVCJkARiSk9/fubMCUMYxSS9eg2AshCdAE8fcv6y6vvhI/uyLQagTp36cCdAGCgoqAY0y0pTpDTQqMyeZQUgYZaGL8MdBU95aK0eOLg7SW8dl3zzZb26DcGjtRwxlJJSiEXM/XIGGF0IeB0+vO9+3F0oiJgl6d3APYO2Fzi7bGbYgFAGVD5YXPj4VqMmTZq0+Oab+dAOgTPu+nvbmLGDDh7cjf4LNIHosvcJ/3fjA3ftqpV/3Lhx9YOeURC3ysvLXTD/Wzu9X9us6TtQI7PnTIf72MIR4Pm1ZvWf9gr70WMHDh7aE1q4n0yfDTVoyNCmdRRYhXZtO5a+CE+hGFNatjtq9v8tVNgphg7rNXBwdxDWyJET4OMHPdsnP3tqroijo+OXc5c+f546cdIIiMhu2bp+zOjJ73UtfEEV+LVgSuu97PIE2wEfwxsWvWNh0VfLW7du/+WCmRDfBVm3b/9ujx79UHliej2ydfMfQVut1+RAJDJiL2dH70md+L8aiEvcis4+vjV16FxufavyZvPX8S6e0n7TzC5JZtbfxWBsy2v7hM2MZxDxEFYJvm85gzXzhJkBkGKVL8W9H85YIPE9Cq2cJwwhMmx+uANNczH6YXPM+Qy0ONfSw/AI7O8WQxj9aoLBmjWdxDvHnZPiZbo5xdf+IKxbS0+8c9xpLj5zODoys/yxKs5A4EcnxsZYGWegEeLeqlwVBEfXMBXf5bB+DKR4V9jgpEpE2HoGo0tZYXeReLsmRN2nyC/MaBdfPwznMa1dmVxCidLhlcoImZx7P5xEpAyJDbmCUCgszbww7Qw7OEtF+FIkICtFS0o5t0yzf4CzCPtMKB1ycLN0y5q+Tk2ivPJzyrCsgWBIvJfr6atAHMPDj5DLyOv/vEBiQpWv69Db20IG09r1D5F7+Mi3/+8xEhNx/yqVOZoeH1dC3KPZu143ozOQaNj89SP/IAdkcXKx2fe4A7t+fJr1XFu7qXtYc4G/yTblsfrq0bSMZNXoJUGIqzy9r9qz9mlgqFNER2+5PRIqt05n376UUS3Uof2HPpZzWtIusH9tSlJcvkZNlXW+1H8CvlLZY/G0tS6hRMqsV+DqIes/oyriNrej8y8cSi3I0+mYTnurroiJaiq+y1LlF8vJrI4rMSTQJrpijbObPyxhCGsRiJQQMgUZXMep3Yde6HW8RruFqJFSaXERTcJUWI39qrSZ1MI8hImAqmE1NDPfjSYIwkSpl+ejS/0NWeSkPd/MmJJxfYtfEUNtF9tTvHrZvn5j0ZcoZVyErTGCGDZ06OIli319/Irq9tXKJIyUyiax74oyHAoVHk2/od8pIQq/iWEDIXsnCKmgUlK6tUXkyF4uvsWLOYy9K0Kogq5IVn6qk5vc3sWK8Ev5xkbwuwExr0Gr1UqlXNQJ1i7mNWDtYvgK1i6Gr2DtYngJhKF0Oh3WLoZ/cNboIqxdjGWwdjF8BWsXw1c0Gg37LiYOgrWLsQS2uxi+grWL4StYuxi+gv1dDF/BdhfDV7B2MXwFaxfDV7C/i+Er2O5i+ArWLoavgHaxz4DhJdjuYvgK1i6Gr5Ak6evrizgJ1i7GEmB3U1JSECfB2sVYAhwGkC/iJFi7GEtg7WL4CtYuhq9g7WL4CtYuhq9g7WL4CnQIazQaxEmwdjGWwHYXw1ewdjF8BWsXw1ewdjF8BWsXw1ewdjF8BWsXw1ewdjF8hcvaLd17LTEiY+zYsRcuXGC3Cf3LKEEnsHH58mXEGax4WSFG+IwaNcrf31+ih9ADG35+fohLYO1iTBAeHt6wYUOKogx7wO42aNAAcQmsXYxpBgwYULlyZcNHHx+f/v37Iy6BtYsxTVhYWPPmzdnmEBjgWrVq1a1bF3EJrF2MWT788MMqVarAhpubG5hhxDGwdjFmqV69eosWLSBGFhoa2rhxY8QxcIxMCJza9jz+bp4yV6fT6a8mhajXXlYIfL25Kw9htNLpiM1EyOQSO3vSL1DRoouXqzeJrAJrl8ekJqr3/56ck6GWyaVSO9LR3d7Z3cHOzQ7CWohRMaMopNeLXqjFPuqQhEQUKrbfxAZNIAkjkcI9huLGG4COkJA0hV7Zb3w0JhvJnFidr8nPVOVlKVV5Gp1Gq3AgG7R0i+jgjsoI1i5f2bDwcVaa2tHFrkoDX7m9laaLCzy+kZaXni+3k/T+OMDFuwxOLNYu/4i/lb//t6cKR7vg5v5IKDy5kf4iNbtGfeeOQ0q7/BnWLs+IuZBzYmtqtQZ+Tt4KJDjuHE/wqmzXe1Ll0mTG2uUTMedyTuxMrRMZiITL7eMJlWs4dBv1+v5nrF3ecPtc7vGdKcIWLsu9s4ke3rJer7O+OL7LG/7Z/iy0ZQASATXfrpqSWHDxYJblbFi7/GD1rHgXb0dSLpbrFRDuf+lIuuU8WLs8IHpvhlpFBTT0QaLB2UNu5yjbvCTRQh6sXR5w81yWu78LEhk1mlROf6aykAFrl+vc+TdPo6YrhXogTpKblzl9dtNrN4+iNw6J5PbSv35MNpeOtct1rv2TaWfP0TeclTduvk4pj5XmUrF2uU5mmtrF1xGJEp8Qd62WUuaYDuPiecJch9JRPsFuqHzIzknfc2D5o8QbanVBrZBm7VsP9/GuBvuTUx4sW9H/49Fr/zm17tadk64uPg3rRXWOGk+SzMCJqzcOHzz2i1KZXTu0Zeu3y3dcr0RCXDmW+XZ3Ey4TtrucJvZSTrFBWW8UnU63cu24B4+u9Hzvs2kTNjk5eny/avjz9CeQJCUZL2Xb34vC63dcPOdM/17zTp7943oM49Qmp8Rt2v5FRHjnzybviGjY5e99y1B5IpGSqUkFppMQhsOkJqnKTboo/vG11OePPuw1L7Rmcxdnz/c6fezo4HY6eoshQ4M67RrUjZRKZcHVG3m6V36SdBd2nruww83VL6rNCAcHlxpBbzWN6I7KEwmBcl+YXrwa+wycRq3Sj+suHx4lXCdJWUhQBPuRIAjQ6MNHVw0ZqviHGbYVCmdlQQ5sPM9I9PMNMuyvWrk2KlekZoctYO1yGilZYiT3m0RZkKvTaSDCZbzTybFoDDhBmHgs5+dne3lWNXyUy+1ReUIgibn3yGPtchpXTzkqN5ydPEF5wwcUc1glkte4keAqaDRFDqhKlYfKE5qiZHamVYq1y2mC6jud2fsclQ+VK9VUq5Vubr5eHlXYPekZScZ21yTubpVu3z1NURSr8tuxZ1B5QmkpD187k0m4rcZpXDxJUEjW03KxbSHBjUNDmm/b9VVm1rPcvKyzF7Z/t3LoxSt7LJdqUKc99KXt2rcM3NC4h5fPXdiOyhOdhgptYro/HNtdruPkIs1IynHzL5fuieEDv42+tHPj1lkJiTe9vao1atCpZfO+lovUCmnatePE6Is7P/miGQQcBvSe9+Pq0W9yzrERmUl5EhL5B5l2nPDYc64TvTfj2qmssLbVkPiIi37q4Ez0/6SKyVTsM3Cd5l2ZLqWsp0okPtT5mrY9zY78xD4DD6hSw/7pgzQ3f7OTJmZ9FWlyP0XpIM5FmIkQQ8eYk+Mb621es2Fq/OPrJpMc7F3yldkmkxb83zFkhkdXU+2dJJWCzEZasM/AD1Z+9tCrmqdXoJPJ1IzMp6jseLi/ySny2dnPtTq1ySSVSmlnZ1/W7xBzNH7I/wU7eZgNb2O7yw8i+/gd2ZxsTrtvVoXW4eLihd4csacSK9dwtCBchP1dvhDSyKFKDad7pxORCEi4mkJK6e5jK1nOhrXLG94f7evuI7tzPAEJmrjzyQU5BSPnV39tTuzv8owD61IT7ipDW1VBQuTBxWSdSj1qYVBpMmPt8o/dK589vpfrE+jhE+KKhIIyR5dwOcneUTLki9JGsrF2ecnj2IK9q5MkUolnFVfvIH4rWJmpTbydoinQ1G7i2rZPGRp8WLs85vCG1PvXshEhkSlIFx9H70B3kj+TMjOS8l8k5xbkKikd7V1Z0WdKqdbPMwZrl/dcP5l99WRG7gstTTGzuwi4phJE60peVv3y0UWjgQ3LnutXLCdQsYXQ2Yym41NE0WLUrx7t1dHGxfYQRnKTkBJ7J7JamEO7Pt7IKrB2BcXjmIKMdI0yT0trdSXTCP0C5IY3pumX2YcuN5oVodG6+4wQDQulFwpcotcqrc+m1wyhz2YQIkFQzNGKrdxP6O+jojMy+ZDCjnT1kgfWdCQd0H8EaxfDV3C/GoavYO1i+ArWLoavYO1i+ArWLoavYO1i+Mr/AwAA//+9mlLdAAAABklEQVQDAI5o8j2Dwv/QAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import Image, display\n",
"arch = AgenticRAG(llm=llm, documents=STARDUST_CORPUS, max_iterations=4, top_k=3)\n",
"graph = arch.build()\n",
"try:\n",
" display(Image(graph.get_graph().draw_mermaid_png()))\n",
"except Exception as e:\n",
" print(f\"(mermaid PNG render unavailable: {e}; see § 2)\")\n",
" print(graph.get_graph().draw_mermaid())"
]
},
{
"cell_type": "markdown",
"id": "015f3f70",
"metadata": {
"papermill": {
"duration": 0.003175,
"end_time": "2026-05-28T02:31:42.304883+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:42.301708+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 8 · Live run — 4 tasks of varying retrieval need\n",
"\n",
"Four task types deliberately mixed:\n",
"1. **Single-fact retrieval needed** — payload number.\n",
"2. **Pure arithmetic** — agent should NOT retrieve.\n",
"3. **Multi-fact / multi-hop** — may need >1 retrieval.\n",
"4. **Out-of-corpus general knowledge** — agent should answer from parametric memory."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8dcf6b6b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:31:42.304883Z",
"iopub.status.busy": "2026-05-28T02:31:42.304883Z",
"iopub.status.idle": "2026-05-28T02:32:58.657945Z",
"shell.execute_reply": "2026-05-28T02:32:58.657945Z"
},
"papermill": {
"duration": 76.353062,
"end_time": "2026-05-28T02:32:58.657945+00:00",
"exception": false,
"start_time": "2026-05-28T02:31:42.304883+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"TASK_TAG: payload\n",
" TASK: What is the maximum payload to LEO of the Stardust 9 rocket?\n",
" RETRIEVAL_COUNT: 1\n",
" ITERATIONS_USED: 2\n",
" QUERIES: [\"Stardust 9 rocket maximum payload to LEO\"]\n",
" FINAL_ANSWER: 1,850 kg\n",
"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"TASK_TAG: arithmetic\n",
" TASK: What is 17 times 6? Return just the integer.\n",
" RETRIEVAL_COUNT: 0\n",
" ITERATIONS_USED: 1\n",
" QUERIES: []\n",
" FINAL_ANSWER: 102\n",
"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"TASK_TAG: multi-fact\n",
" TASK: Who is the CTO of Stardust Aerospace and what was their previous role before co-\n",
" RETRIEVAL_COUNT: 1\n",
" ITERATIONS_USED: 2\n",
" QUERIES: [\"Stardust Aerospace CTO previous role\"]\n",
" FINAL_ANSWER: Jin-ho Park\n",
"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"TASK_TAG: general\n",
" TASK: What is the capital of France? Return just the city name.\n",
" RETRIEVAL_COUNT: 2\n",
" ITERATIONS_USED: 3\n",
" QUERIES: [\"capital of France\", \"capital of France\"]\n",
" FINAL_ANSWER: Paris\n",
"\n",
"AGGREGATE: 4 total retrievals across 4 tasks\n"
]
}
],
"source": [
"TASKS = [\n",
" (\"payload\", \"What is the maximum payload to LEO of the Stardust 9 rocket?\"),\n",
" (\"arithmetic\", \"What is 17 times 6? Return just the integer.\"),\n",
" (\"multi-fact\", \"Who is the CTO of Stardust Aerospace and what was their previous role before co-founding the company?\"),\n",
" (\"general\", \"What is the capital of France? Return just the city name.\"),\n",
"]\n",
"\n",
"import json\n",
"results = []\n",
"for tag, q in TASKS:\n",
" r = arch.run(q)\n",
" queries = [ret['query'] for ret in r.metadata['retrievals']]\n",
" print(f\"TASK_TAG: {tag}\")\n",
" print(f\" TASK: {q[:80]}\")\n",
" print(f\" RETRIEVAL_COUNT: {r.metadata['retrieval_count']}\")\n",
" print(f\" ITERATIONS_USED: {r.metadata['iterations_used']}\")\n",
" print(f\" QUERIES: {json.dumps(queries)}\")\n",
" print(f\" FINAL_ANSWER: {r.output}\")\n",
" print()\n",
" results.append((tag, q, r))\n",
"\n",
"# Aggregate\n",
"total_retrievals = sum(r.metadata['retrieval_count'] for _, _, r in results)\n",
"print(f\"AGGREGATE: {total_retrievals} total retrievals across {len(TASKS)} tasks\")"
]
},
{
"cell_type": "markdown",
"id": "fc68ac15",
"metadata": {
"papermill": {
"duration": 0.0,
"end_time": "2026-05-28T02:32:58.662922+00:00",
"exception": false,
"start_time": "2026-05-28T02:32:58.662922+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 9 · What we just observed\n",
"\n",
"The cells above ran Agentic RAG on **4 task types** (single-fact retrieval needed, arithmetic, multi-hop, out-of-corpus general knowledge) and compared to a plain-RAG baseline that always retrieves once.\n",
"\n",
"### 9.1 · Per-task retrieval behaviour\n",
"\n",
"| Tag | Retrievals | Iterations | Final answer |\n",
"|---|---|---|---|\n",
"| `payload` | 1 | 2 | 1,850 kg |\n",
"| `arithmetic` | 0 | 1 | 102 |\n",
"| `multi-fact` | 1 | 2 | Jin-ho Park |\n",
"| `general` | 2 | 3 | Paris |\n",
"\n",
"- **Total retrievals across 4 tasks**: 4\n",
"- **Plain-RAG baseline retrievals** (1 per task): 4\n",
"- **Net savings vs always-retrieve**: 0 fewer retrieval calls (Agentic RAG used MORE — sign of over-retrieval pathology)\n",
"\n",
"### 9.2 · Agentic RAG vs plain RAG\n",
"\n",
"| Tag | Agentic answer | Plain-RAG answer |\n",
"|---|---|---|\n",
"| `payload` | 1,850 kg | The maximum payload to LEO (Low Earth Orbit) of the Stardust |\n",
"| `arithmetic` | 102 | 102 |\n",
"| `multi-fact` | Jin-ho Park | The CTO of Stardust Aerospace is Jin-ho Park, and according |\n",
"| `general` | Paris | Paris |\n",
"\n",
"### 9.3 · Patterns surfaced in this run\n",
"\n",
"- **✅ Correctly skipped retrieval on `arithmetic` task** — agent recognised this as parametric, didn't waste a call.\n",
"\n",
"- **⚠️ Over-retrieval on `general` task** (2 call(s)) — general knowledge shouldn't need retrieval. Likely the corpus's presence in the prompt anchored the agent toward retrieving.\n",
"\n",
"- **🤔 Agentic RAG used 0 MORE retrievals than plain RAG** — multi-hop tasks pushed the count above the always-one baseline. Net cost is higher but multi-hop answers should be more complete.\n",
"\n",
"### 9.4 · The takeaway\n",
"\n",
"Agentic RAG's value lives in two columns of § 9.1: **`Retrievals`** and **`Final answer`**. The architecture earns its keep when:\n",
"\n",
"1. **Retrieval count varies across tasks** (0 for parametric, 1 for single-fact, ≥2 for multi-hop). Flat retrieval count → degenerated to plain RAG.\n",
"2. **Zero-retrieval answers are still correct** (agent's parametric-memory judgement is accurate).\n",
"3. **Multi-hop retrievals produce complete answers** (not just the first-query's fact).\n",
"\n",
"Read § 9.3 for the specific patterns this run surfaced — over-retrieval on arithmetic, under-retrieval on multi-fact, etc. — and use them to tune the DECIDE prompt for your task distribution."
]
},
{
"cell_type": "markdown",
"id": "2275988e",
"metadata": {
"papermill": {
"duration": 0.0,
"end_time": "2026-05-28T02:32:58.662922+00:00",
"exception": false,
"start_time": "2026-05-28T02:32:58.662922+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 10 · Plain RAG baseline (always retrieve once, then answer)\n",
"\n",
"How would a non-agentic RAG handle the same tasks? It would retrieve for *every* question — including the arithmetic and general-knowledge ones, wasting calls and possibly biasing the answer toward retrieved (irrelevant) docs."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "eb1991e2",
"metadata": {
"execution": {
"iopub.execute_input": "2026-05-28T02:32:58.674047Z",
"iopub.status.busy": "2026-05-28T02:32:58.674047Z",
"iopub.status.idle": "2026-05-28T02:33:37.602877Z",
"shell.execute_reply": "2026-05-28T02:33:37.602877Z"
},
"papermill": {
"duration": 38.92883,
"end_time": "2026-05-28T02:33:37.602877+00:00",
"exception": false,
"start_time": "2026-05-28T02:32:58.674047+00:00",
"status": "completed"
},
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PLAIN_TASK_TAG: payload\n",
" PLAIN_ANSWER: The maximum payload to LEO (Low Earth Orbit) of the Stardust 9 rocket is 1,850 kg.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"PLAIN_TASK_TAG: arithmetic\n",
" PLAIN_ANSWER: 102\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"PLAIN_TASK_TAG: multi-fact\n",
" PLAIN_ANSWER: The CTO of Stardust Aerospace is Jin-ho Park, and according to the context, before co-founding the company, his previous role is not explicitly stated, but it is mentioned that Dr. Amara Okonkwo, the \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"PLAIN_TASK_TAG: general\n",
" PLAIN_ANSWER: Paris\n",
"PLAIN_RAG_RETRIEVALS: 4 (one per task, always)\n"
]
}
],
"source": [
"plain_results = []\n",
"for tag, q in TASKS:\n",
" # Manual plain-RAG: always retrieve top-3, always feed into prompt, answer.\n",
" docs = arch.memory.search(q, k=3)\n",
" context = \"\\n\\n\".join(f\"- {d.page_content[:200]}\" for d in docs)\n",
" prompt = f\"Answer the question using the context below. If the context isn't relevant, answer from your own knowledge.\\n\\n# Context\\n{context}\\n\\n# Question\\n{q}\\n\\nAnswer:\"\n",
" plain_ans = str(llm.invoke(prompt).content).strip()\n",
" print(f\"PLAIN_TASK_TAG: {tag}\")\n",
" print(f\" PLAIN_ANSWER: {plain_ans[:200]}\")\n",
" plain_results.append((tag, plain_ans))\n",
"print(f\"PLAIN_RAG_RETRIEVALS: {len(TASKS)} (one per task, always)\")"
]
},
{
"cell_type": "markdown",
"id": "e808ad1d",
"metadata": {
"papermill": {
"duration": 0.0,
"end_time": "2026-05-28T02:33:37.602877+00:00",
"exception": false,
"start_time": "2026-05-28T02:33:37.602877+00:00",
"status": "completed"
},
"tags": []
},
"source": [
"## 11 · Failure modes, safety, extensions\n",
"\n",
"### 11.1 · Where this breaks\n",
"\n",
"| Failure | Mechanism | Mitigation |\n",
"|---|---|---|\n",
"| **Over-retrieval** | Agent retrieves even for parametric / arithmetic questions | Stronger DECIDE prompt; reward (in a learned setting) for non-retrieval when answer is parametric |\n",
"| **Wrong query** | DECIDE writes a vague query → retrieve returns junk → answer wrong | Force the query to mention the task's specific entities; consider query-rewriting (nb 24 CRAG) |\n",
"| **Stops retrieving too early** | Agent commits to ANSWER before enough context | Add a `confidence` field to DECIDE; only allow ANSWER above threshold |\n",
"| **Stops too late (runaway loop)** | Agent keeps retrieving forever | Hard cap `max_iterations`; final-iteration forced-answer prompt patch (already in place) |\n",
"| **Hallucinated answer** | Even with retrieval, model uses parametric memory | Force `answer` to cite which retrieved doc supports each claim |\n",
"\n",
"### 11.2 · Production safety\n",
"\n",
"- **Track retrieval count per task.** Tasks consistently using >1 retrieval = candidate for multi-hop indexing (nb 27 GraphRAG).\n",
"- **Audit zero-retrieval answers.** When agent answers without retrieving, log it — these are the highest hallucination risk.\n",
"- **Cache retrievals across nearby queries.** Saves cost on multi-turn conversations.\n",
"\n",
"### 11.3 · Three extensions\n",
"\n",
"1. **Query rewriting** (nb 24 CRAG style). Before retrieve, have a small LLM call rewrite the query to be more retrieval-friendly.\n",
"2. **Multi-corpus router**. Add multiple `retrieve_X(query)` tools (one per corpus); DECIDE chooses the right tool.\n",
"3. **Confidence-gated answer**. Force a numeric or categorical confidence on each ANSWER; route low-confidence answers to a human review queue.\n",
"\n",
"### 11.4 · What to read next\n",
"\n",
"- [**02 · Tool Use**](./02_tool_use.ipynb) — the generic version of agent-with-tools.\n",
"- [**24 · Corrective RAG**](./24_corrective_rag.ipynb) — grade docs, fall back to web if poor.\n",
"- [**25 · Self-RAG**](./25_self_rag.ipynb) — retrieve-on-demand with reflection tokens.\n",
"- [**26 · Adaptive RAG**](./26_adaptive_rag.ipynb) — router decides between no-RAG / single-step / multi-step.\n",
"\n",
"### 11.5 · References\n",
"\n",
"1. Jeong, S. et al. *Adaptive-RAG.* NAACL 2024. [arXiv:2403.14403](https://arxiv.org/abs/2403.14403) — formalises the routing-over-retrieval pattern.\n",
"2. Lewis, P. et al. *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.* NeurIPS 2020. [arXiv:2005.11401](https://arxiv.org/abs/2005.11401) — the original RAG paper."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.0"
},
"papermill": {
"default_parameters": {},
"duration": 126.493174,
"end_time": "2026-05-28T02:33:38.341342+00:00",
"environment_variables": {},
"exception": null,
"input_path": "all-agentic-architectures/notebooks/23_agentic_rag.ipynb",
"output_path": "all-agentic-architectures/notebooks/23_agentic_rag.ipynb",
"parameters": {},
"start_time": "2026-05-28T02:31:31.848168+00:00",
"version": "2.7.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}