{ "cells": [ { "cell_type": "markdown", "id": "4480b938", "metadata": { "papermill": { "duration": 0.008594, "end_time": "2026-05-27T03:47:16.886696+00:00", "exception": false, "start_time": "2026-05-27T03:47:16.878102+00:00", "status": "completed" }, "tags": [] }, "source": [ "# 01 · Reflection — generate, critique, refine\n", "\n", "> **TL;DR.** Treat the LLM as both author and editor. The agent produces a draft, an LLM-as-Judge critiques it on a structured rubric, and the agent rewrites using the critique. Repeat until the critic is satisfied or a budget is hit.\n", ">\n", "> **Reach for it when:** quality matters more than latency — code generation, technical writing, multi-constraint reasoning, long-form answers.\n", "> **Avoid when:** the task is simple, real-time, or already verifiable by deterministic checks (exact-match QA, calculators). Reflection at least doubles latency and cost vs. a single forward pass.\n", "\n", "| Property | Value |\n", "|---|---|\n", "| Origin | Madaan et al., *Self-Refine: Iterative Refinement with Self-Feedback*, NeurIPS 2023 ([arXiv 2303.17651](https://arxiv.org/abs/2303.17651)) |\n", "| Reasoning type | Iterative, single-agent, two-role |\n", "| External tools needed? | No |\n", "| Memory across episodes? | No (see [Reflexion · notebook 18](./18_reflexion.ipynb) for the with-memory variant) |\n", "| Typical iteration count | 2–4 rounds |\n", "| Cost relative to single-pass | ≈ `2 + 2N` LLM calls (= 1 generate + 1 critique + N×(refine + critique)) for `N` refinement rounds |\n", "\n", "This notebook is the **canonical template** for the rest of the repo — every other notebook follows the same 11-section structure. If you're reading the repo cover-to-cover, start here." ] }, { "cell_type": "markdown", "id": "e6ac115d", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T03:47:16.893405+00:00", "exception": false, "start_time": "2026-05-27T03:47:16.893405+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 2 · Architecture at a glance\n", "\n", "```mermaid\n", "flowchart LR\n", " A([task]) --> G[Generate
Generator LLM]\n", " G --> C{Critique
Critic LLM-as-Judge}\n", " C -->|score < target AND
iter < max| R[Refine
Generator LLM]\n", " R --> C\n", " C -->|satisfied OR
budget exhausted| F[Finalize]\n", " F --> Z([final output])\n", "\n", " style G fill:#e3f2fd,stroke:#1976d2\n", " style R fill:#e3f2fd,stroke:#1976d2\n", " style C fill:#fff3e0,stroke:#f57c00\n", " style F fill:#e8f5e9,stroke:#388e3c\n", "```\n", "\n", "**What you're looking at.** Two LLM \"roles\" (Generator in blue, Critic in orange) share a single state object. The Generator writes the draft; the Critic scores it on a Pydantic-typed rubric and writes actionable feedback. A small router decides each round whether to keep refining or finalize.\n", "\n", "The Critic and Generator are usually the *same model* with different prompts. You'll see in § 11 why using a *different* (often stronger) model in the Critic seat is the single best upgrade." ] }, { "cell_type": "markdown", "id": "d5b4bb9c", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T03:47:16.909397+00:00", "exception": false, "start_time": "2026-05-27T03:47:16.909397+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 3 · Theory\n", "\n", "### 3.1 · Motivation: why a single forward pass isn't enough\n", "\n", "When you ask an LLM to \"write a Python function that sorts a list and handles edge cases\", the model commits to a particular *framing* in the first few generated tokens — picks a sort algorithm, decides what counts as an edge case, chooses a docstring style. From that point on, every subsequent token conditions on the choices already made. The model can't easily back up and reconsider.\n", "\n", "For *easy* tasks this is fine. For *hard* tasks — where the right answer requires weighing trade-offs, satisfying multiple constraints simultaneously, or noticing what's missing — that first commitment is often suboptimal.\n", "\n", "Reflection is the simplest possible fix: **let the model re-read its own work, criticize it, and rewrite.**\n", "\n", "### 3.2 · The core idea\n", "\n", "A Reflection agent maintains exactly two roles, alternating on shared state:\n", "\n", "1. **Generator** — writes a draft from scratch (round 0), or rewrites given a draft + critique (rounds 1+).\n", "2. **Critic** — reads the latest draft and produces a structured `(score, critique)` pair against a rubric.\n", "\n", "A router examines the score and decides whether to keep going:\n", "\n", "```\n", "if score >= target_score or iteration >= max_iterations:\n", " finalize()\n", "else:\n", " refine()\n", "```\n", "\n", "That's the whole pattern. No tools, no external memory, no other agents.\n", "\n", "### 3.3 · Why this works (theoretically)\n", "\n", "Three reasons:\n", "\n", "1. **Explicit-feedback channel.** A single forward pass has only one channel — the task prompt → the draft. Reflection introduces a *second* channel — *the draft itself, plus written critique* → the rewrite. The model can now condition on *what's wrong with the draft*, not just *what the task asks for*. That's information the model never had on the first pass.\n", "\n", "2. **Easier subproblem on each iteration.** \"Generate from scratch\" is harder than \"improve this specific draft, addressing these specific flaws.\" The refinement step is a strictly more constrained version of the original problem, so models can use their capacity for the *delta* rather than re-solving the whole task.\n", "\n", "3. **Adversarial framing.** The Critic's prompt explicitly says \"find what's wrong.\" This activates different reasoning patterns than \"write the best version\" — even with identical weights — because the *task* is different. Finding flaws in a finished artifact is a different cognitive operation than producing one.\n", "\n", "### 3.4 · A simple optimization view\n", "\n", "If you squint, Reflection is a **gradient-free local search** over the space of natural-language outputs:\n", "\n", "| Optimization concept | Reflection analogue |\n", "|---|---|\n", "| Objective function | The Critic's score |\n", "| Current point | The current draft |\n", "| Gradient | The Critic's written critique (a *direction* in language-space) |\n", "| Step | The Refiner's rewrite |\n", "| Step size | Implicit (model decides how much to change) |\n", "| Convergence test | `score >= target_score` |\n", "| Max budget | `max_iterations` |\n", "\n", "This view explains why Reflection plateaus: like any local-search method, it gets stuck in local optima. The Critic eventually runs out of substantive critiques and starts nitpicking, and the Generator rewrites in cosmetic rather than substantive ways. **2–4 iterations is the sweet spot for most tasks.**\n", "\n", "### 3.5 · Where Reflection sits in the agentic taxonomy\n", "\n", "| Pattern | Loop? | Memory across episodes? | External tools? | Models used |\n", "|---|---|---|---|---|\n", "| Single-pass | no | no | no | 1 |\n", "| Best-of-N (sample N once, pick best) | no | no | no | 1 |\n", "| Self-Consistency (nb 21) | no | no | no | 1 (N samples + vote) |\n", "| Chain-of-Verification (nb 20) | no | no | no | 1 (self-generated verification questions) |\n", "| **Reflection** *(this notebook)* | **yes** | no | no | 1 (two roles) |\n", "| Reflexion (nb 18) | yes | **yes** — verbal episode log | optional | 1+ |\n", "| RLHF self-improvement (nb 15) | yes | yes — best-output archive | no | 1+ |\n", "| Tree of Thoughts (nb 9) | tree-search | no | no | 1 |\n", "| LATS (nb 22) | tree-search + reward | no | yes | 1+ |\n", "\n", "Reflection is the *minimal* loop pattern. Almost every more advanced pattern in this repo can be understood as Reflection + something (memory → Reflexion; search → ToT; tools → ReAct + reflection).\n", "\n", "### 3.6 · Practical knobs\n", "\n", "- `max_iterations` — caps cost. 2–4 is the sweet spot.\n", "- `target_score` — sets the early-stop bar. Higher = more iterations on average.\n", "- **Critic model choice** — biggest lever. Same-model Critic has no extra setup cost (one model serves both roles) but suffers from sycophancy. A *different* model (even a smaller one) in the Critic seat usually beats more iterations.\n", "- **Critic rubric wording** — the Critic only finds what its rubric asks for. Generic \"score this on quality 1-10\" rubrics produce generic feedback. Domain-specific rubrics (\"does the code handle empty input?\" / \"is the contract correct?\") produce surgical feedback.\n" ] }, { "cell_type": "markdown", "id": "5c7a193e", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T03:47:16.925109+00:00", "exception": false, "start_time": "2026-05-27T03:47:16.925109+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 4 · Setup\n", "\n", "Three lines: configure the LLM, turn on LangSmith tracing, import the architecture. The provider comes from `.env` (`LLM_PROVIDER=nebius` by default) so this notebook runs **unchanged** against OpenAI, Anthropic, Groq, Ollama, … — see § 10." ] }, { "cell_type": "code", "execution_count": 1, "id": "2ea4e982", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:47:16.925109Z", "iopub.status.busy": "2026-05-27T03:47:16.925109Z", "iopub.status.idle": "2026-05-27T03:47:18.395785Z", "shell.execute_reply": "2026-05-27T03:47:18.395139Z" }, "papermill": { "duration": 1.470676, "end_time": "2026-05-27T03:47:18.395785+00:00", "exception": false, "start_time": "2026-05-27T03:47:16.925109+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
Provider: nebius  ·  Model: meta-llama/Llama-3.3-70B-Instruct ─────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mProvider: nebius · Model: meta-llama/Llama-\u001b[0m\u001b[1;36m3.3\u001b[0m\u001b[1;36m-70B-Instruct\u001b[0m \u001b[92m─────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
LangSmith tracing: enabled                                                                                         \n",
       "
\n" ], "text/plain": [ "LangSmith tracing: enabled \n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from agentic_architectures import get_llm, enable_langsmith, settings\n", "from agentic_architectures.architectures import Reflection\n", "from agentic_architectures.ui import print_md, print_header, print_step\n", "\n", "traced = enable_langsmith()\n", "print_header(f\"Provider: {settings.llm_provider} · Model: {settings.llm_model}\")\n", "print_md(f\"LangSmith tracing: {'enabled' if traced else 'disabled (no LANGSMITH_API_KEY)'}\")" ] }, { "cell_type": "markdown", "id": "d486660d", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T03:47:18.402928+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.402928+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 5 · Library walkthrough\n", "\n", "The full implementation is at [`src/agentic_architectures/architectures/reflection.py`](../src/agentic_architectures/architectures/reflection.py). The `Reflection` class inherits from the abstract `Architecture` base — every architecture in this repo conforms to the same contract:\n", "\n", "```python\n", "class Architecture(ABC):\n", " name: str # snake_case identifier (used as docs slug + registry key)\n", " description: str # one-line for README/docs tables\n", " reference: str # link to the originating paper or blog\n", "\n", " def build(self) -> CompiledStateGraph: ... # returns the LangGraph\n", " def run(self, task: str) -> ArchitectureResult: ...\n", " def diagram(self) -> str: ... # mermaid string of the COMPILED graph\n", " def explain(self) -> str: ... # the theory section in markdown\n", "```\n", "\n", "That contract is what makes architectures composable — a meta-controller (notebook 11) treats every architecture as a black-box callable.\n", "\n", "Below we print the method signatures of `Reflection` so you can see the shape before we dive into specifics." ] }, { "cell_type": "code", "execution_count": 2, "id": "6ce6f5f1", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:47:18.411662Z", "iopub.status.busy": "2026-05-27T03:47:18.411662Z", "iopub.status.idle": "2026-05-27T03:47:18.436148Z", "shell.execute_reply": "2026-05-27T03:47:18.436148Z" }, "papermill": { "duration": 0.024486, "end_time": "2026-05-27T03:47:18.436148+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.411662+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "def __init__(max_iterations, target_score):\n", "\n", "def _generate(state):\n", "\n", "def _critique(state):\n", "\n", "def _refine(state):\n", "\n", "def _finalize(state):\n", "\n", "def _should_continue(state):\n", "\n", "def build():\n", "\n", "def run(task):\n", "\n" ] } ], "source": [ "import inspect, ast\n", "from agentic_architectures.architectures import reflection as ref_mod\n", "\n", "src = inspect.getsource(ref_mod.Reflection)\n", "tree = ast.parse(src)\n", "\n", "for node in ast.walk(tree):\n", " if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):\n", " args = ', '.join(a.arg for a in node.args.args if a.arg != 'self')\n", " print(f\"def {node.name}({args}):\")\n", " ds = ast.get_docstring(node)\n", " if ds:\n", " first_line = ds.splitlines()[0]\n", " print(f' \"\"\"{first_line}\"\"\"')\n", " print()" ] }, { "cell_type": "markdown", "id": "21a926cb", "metadata": { "papermill": { "duration": 0.008712, "end_time": "2026-05-27T03:47:18.444860+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.436148+00:00", "status": "completed" }, "tags": [] }, "source": [ "### 5.1 · The four node functions\n", "\n", "Reading `reflection.py` top-to-bottom:\n", "\n", "- **`_generate(state)`** — round-0 only. Reads `state['task']` and produces the first draft. Sets `iteration=0` and initialises an empty `history` list.\n", "- **`_critique(state)`** — runs every round. Hands `(task, draft)` to an `LLMJudge` configured with the `_ReflectionCritique` schema; receives back `score: int 1-10` and `critique: str`. Appends a row to `history` so the trace is observable post-hoc.\n", "- **`_refine(state)`** — runs only when the router says so. Reads `(task, previous draft, critique)` and produces a *new* draft. **Notice it rewrites rather than patches** — the prompt explicitly says \"Don't just patch — rewrite as needed.\" Cosmetic fixes are easy; substantive ones require letting the model restructure freely.\n", "- **`_finalize(state)`** — copies the current draft into `final_output`. This is a trivial node, but having a dedicated terminal node makes the graph easier to extend (e.g., add post-processing).\n", "\n", "### 5.2 · The router\n", "\n", "```python\n", "def _should_continue(self, state):\n", " if state['score'] >= self.target_score: return 'finalize'\n", " if state['iteration'] >= self.max_iterations: return 'finalize'\n", " return 'refine'\n", "```\n", "\n", "Two stopping conditions: (1) the Critic is satisfied, or (2) we've run out of budget. The `add_conditional_edges` call in `build()` wires this router to two destinations.\n", "\n", "### 5.3 · Why the Critic uses `LLMJudge`\n", "\n", "`LLMJudge` (defined in [`evaluators/judge.py`](../src/agentic_architectures/evaluators/judge.py)) is one of the **most important pieces of shared infrastructure** in this repo. It replaces ~17 inline `with_structured_output(...)` blocks that existed in the original notebooks. It does three things:\n", "\n", "1. Wraps `llm.with_structured_output(schema)` so the Critic always returns a Pydantic instance — no string-parsing.\n", "2. Bakes in a uniform prompt template (`Rubric / Context / Candidate / Return your evaluation`).\n", "3. Warns clearly when the provider doesn't support reliable structured output.\n", "\n", "Re-using `LLMJudge` everywhere means every architecture's evaluator has the same UX, the same provider-compatibility checks, and the same trace shape in LangSmith." ] }, { "cell_type": "markdown", "id": "c149a9f9", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T03:47:18.449576+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.449576+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 6 · State & Critic schema\n", "\n", "LangGraph state is a `TypedDict` that gets *merged* (not replaced) between node returns. Each node returns a dict of just the keys it wants to update — LangGraph handles the merging. That's why `_critique` can return `{'score', 'critique', 'history'}` without overwriting `draft` or `task`.\n", "\n", "**Schema reference:**\n", "\n", "| Field | Type | Set by | Read by |\n", "|---|---|---|---|\n", "| `task` | `str` | caller | every node |\n", "| `draft` | `str` | `generate`, `refine` | `critique`, `finalize` |\n", "| `score` | `int 1-10` | `critique` | router |\n", "| `critique` | `str` | `critique` | `refine` |\n", "| `iteration` | `int` | `generate`, `refine` | router |\n", "| `history` | `list[dict]` | `critique` | user (post-hoc) |\n", "| `final_output` | `str` | `finalize` | user |\n", "\n", "The `history` field is the architecture's **observability surface**. Every critique round writes a row `{iteration, draft, score, critique}`. After the run we can iterate `history` to understand exactly what the Critic latched onto each round." ] }, { "cell_type": "code", "execution_count": 3, "id": "e87062b5", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:47:18.468432Z", "iopub.status.busy": "2026-05-27T03:47:18.468432Z", "iopub.status.idle": "2026-05-27T03:47:18.484961Z", "shell.execute_reply": "2026-05-27T03:47:18.484961Z" }, "papermill": { "duration": 0.024658, "end_time": "2026-05-27T03:47:18.484961+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.460303+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ReflectionState fields:\n", " task ForwardRef('str')\n", " draft ForwardRef('str')\n", " critique ForwardRef('str')\n", " score ForwardRef('int')\n", " iteration ForwardRef(\"Annotated[int, 'current refinement loop index, starts at 0']\")\n", " max_iterations ForwardRef('int')\n", " target_score ForwardRef('int')\n", " history ForwardRef('list[dict[str, Any]]')\n", " final_output ForwardRef('str')\n", "\n", "Critic structured-output schema:\n", "{\n", " \"description\": \"Critic output.\",\n", " \"properties\": {\n", " \"score\": {\n", " \"description\": \"Overall quality on a 1-10 scale.\",\n", " \"maximum\": 10,\n", " \"minimum\": 1,\n", " \"title\": \"Score\",\n", " \"type\": \"integer\"\n", " },\n", " \"critique\": {\n", " \"description\": \"Concrete, actionable critique of the draft.\",\n", " \"title\": \"Critique\",\n", " \"type\": \"string\"\n", " }\n", " },\n", " \"required\": [\n", " \"score\",\n", " \"critique\"\n", " ],\n", " \"title\": \"_ReflectionCritique\",\n", " \"type\": \"object\"\n", "}...\n" ] } ], "source": [ "from agentic_architectures.architectures.reflection import (\n", " ReflectionState,\n", " _ReflectionCritique,\n", ")\n", "\n", "print('ReflectionState fields:')\n", "for name, typ in ReflectionState.__annotations__.items():\n", " print(f' {name:20s} {typ}')\n", "\n", "print()\n", "print('Critic structured-output schema:')\n", "import json\n", "print(json.dumps(_ReflectionCritique.model_json_schema(), indent=2)[:600] + '...')" ] }, { "cell_type": "markdown", "id": "8c24dbbd", "metadata": { "papermill": { "duration": 0.010806, "end_time": "2026-05-27T03:47:18.519903+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.509097+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 7 · Build the graph\n", "\n", "`build()` returns a compiled LangGraph. We render the **actual compiled graph** as a PNG inline — this is the diagram derived from the real `StateGraph` object, so if it ever disagrees with the static diagram in § 2, the implementation has drifted from the documentation. (They should match here.)" ] }, { "cell_type": "code", "execution_count": 4, "id": "d0dda1ad", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:47:18.541084Z", "iopub.status.busy": "2026-05-27T03:47:18.539075Z", "iopub.status.idle": "2026-05-27T03:47:24.858917Z", "shell.execute_reply": "2026-05-27T03:47:24.858917Z" }, "papermill": { "duration": 6.331509, "end_time": "2026-05-27T03:47:24.862306+00:00", "exception": false, "start_time": "2026-05-27T03:47:18.530797+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAANsAAAGwCAIAAACvpYc6AAAQAElEQVR4nOydB3wUxdvHZ/fucukVSCGN0BFIwNCUv5TQRJSOgBCqFEUFRaQjTQEBQRARUJD2IhKFKE1AqYL00GsIhJBGernk7nb3fe42uRzJZZMgtzd3O1/5nLuzs3OXud89M/PM7DNyjuMQgYANckQg4ARRJAEviCIJeEEUScALokgCXhBFEvDCuhWZl85cPpmZHF+g1SBtIaMp4D1ZLEI0Lec4luJYRMGBltIl0wj+x8JFxFGI4vSvlAyxTFEK/I+iEdwChzTNsQxV9DY0h1iq6GbIoYfT5UJwO8foT2mOYovywzsi/VsbTos+gB6FkpIpKKWj3DtA2ay9p70zIhhDWaM/UpXL/bE+ITWhgGE4hR1t5yBzcJJTFKdW6dVB6fRCy5FekRx8/YxG9zfSNKgQErmiHHooWpeHV6juVE5zWpCd/kBTrCkZQvqCi6RZdKNOu5SM4hj9nTSF2KIyablOjlzxqUxBM4aiQJH2NMMgTQFboGIhXaGgPXyUAybVRAQ91qfIjZ/H5WZpXT3k9cLd2nT3QFbOP9EZt85l5edpvbztB33mjySPNSny0PaUW+eyvXztBk8JRDbHzuWPweq/9IpH+75eSMJYjSI3L3ioLuSGzwyWK5GtkpWs/XllvIu7fNCUACRVrEORu755zGjQ259IolHb+mW8Rw3FG6N8kCSxAkX+OCfO3lE+WEp9rM3zHyIZFTndBjsnFUIjvPm/pY9hHC0pOQKRs4JgSB+16gmSHlgr8uzBzKxU9aApUhyBDp0emPxIdedCHpIYWCvy/OG0Dv0l2p0CWnXxPPJzEpIY+Coyem2io5O8frgjkiovd/aQyaiDm5ORlMBXkY/v5bfoJmnPHBDWzj32ei6SEpgq8uyf6RSFXmot6qTvzp0758yZg6pO586dExISkBlo2c0T5s1vnZWQKDFV5L2LuV5+YrvCb9y4gapOYmJiRkYGMhvOHvLLx9ORZMB07U92hrZ5R09kHuLi4tauXXvhwgXwxTZt2jQyMjIsLGzMmDEXL16Eq3v37t26dau/vz+8nj59+v79+9WqVWvXrt348ePt7e0hw5QpU2Qyma+v7+bNm8eOHfv9999DYs+ePSHPsmXL0IsmsK7jnZgcJBkwVSSj5cJec0dmQK1Wg/hatGixatUqENb69esnTZq0f//+devWDR8+PCgoaO7cuZBtw4YNmzZtWrBggbu7e05OzldffQWZP/zwQ7ikUCju3LmTl5e3fPnyJk2aNGzYcOLEiXv27KlZ0yzrd+o2d7lxPhtJBhwVmXBPTdPIzgGZg4cPH6anpw8aNKhBgwZwumjRIjCNWq22VLYhQ4ZERETUqlWLP42Jifnnn394RVIU9eTJky1btvAm09zUrGOPWE6tRnZ2SArgqMjsDA2ikJkIDAz08PD4/PPPu3fv/vLLL4eGhoaHh5fNBoYQmmwY6IA55PXq6VnSiwCliiNHHuhdpMSr/WtLQpI4jmw4hl+xbRaUSiW01G3btt2+ffuoUaN69eq1b9++stmgTYd2vHfv3rt37z5//vyIESNKFYJEhNItD2aQNMBRkS4edmZd/xEcHAw9vz/++AM6gnXq1Jk9e/atW7eMM8C7R0VFvf3226BIHx/dpBF0JZHlgMqo5m2eTgx+4KjIgPpKhuU4LTIHMNCOjo6GA2h2X3vttcWLF8vl8ps3bxrn0Wg0KpWqRo0a/CkMho4fP44sREq8Blyz9i5IImDqj4TZsyunzWKWsrKy5s2bt2LFivj4eBjlbNy4EbqJ0JuESwEBAdeuXTt37lxubi7YURDu48ePMzMzIT+4h7Kzs2F8XbZAyAmvhw4dgnuRGbhzPofGfYXWiwTTv9XRRR57xSyKBPFNnz4d3D3QIvft2/fSpUvgmwwJCYFLffr0gXH0+++/f/fu3S+++AKMaL9+/aCj2bJlywkTJsBpp06dYJRdqkDwXL755ptQCHQ9kRl4eCvX2U2BJAOmK3aP7Xp663z22EUhSPJ8O/leyy5eLbpY/TNulQRTG9muXzV1IZNwT4WkzY0zORyLpCNHhHMEAY8adn9uSx4xJ7i8DAMGDEhJSSmbzjAMTdPQ/pq8C7w5MA2DzMDly5dhCG/ykvBH+uuvv+hyuopn9qf515HWejysn7NZNelu5PRabtVN/2ySk5Phm0ZVxM/PD5mNsr3MylDeR3pwTbVvU8L7S+sgKYF1lJV6zVyjVj8eOTfY5FVvb2+EGS9W7ge3JDZtK6H2mgdrv0LXSG9o56LXSfEBqF0rExxdZP/rJbk1y7h7ukbMDU56UHh811MkJQ5sfJqZoomcGYSkh3VEENgw80FAfeeuQ6sjCbD7u8ScdM3QGVJ8WBtZUZSV9TNiHV0V73xm4+FHNi94yGg4aBmQVLGmSFQ7lsSnJRc2buXebkA1ZHMc2pJy53JO9QDlgImSjpBmZdH6bp/POxaVrFGzfiGOEW97u1aTISsnLUF97LeniXH5CgXdLdIvsKF4yy7xxCojmp4/knnleGZ+jlauoOwd5E7ucidXuUzBqQtKAofK5BRrFFaUDzgKPmr4e+ESozWOPsrx8XBpOWKLFxzJ7SitWpdHtzSRKwqAKpPTjJbl72L1JfC3UDRF0xyjLXpfvnBDJFXwi3NcSeFyOxoxlCqXycnS5GVpWAY5uyvCIzwbt5XM8h5BrFKRBi78mfngVn5+tkajBlVxGnXJ3yKTIZYFrRU7E6jiQM2c7pLBs07LdKrVBdnVHVMsYwiMixiNbqGkLt5u8fJhw42GA1ofNBp0DqJkn70kkyNeo0WKpPmA00iupOQySmEvc3aT16xj36Kz5DyOwli3Is3NihUrqlWrNmTIEEQQC7JXgxBarVYuJ1UkKqS6hSCKFB9S3UIQRYoPqW4hNBqNQiGh9ds4QBQpBLGR4kOqWwiiSPEh1S0EUaT4kOoWAhRJ+pEiQxQpBLGR4kOqWwhQpExm9Ys5rAuiSCGIjRQfUt1CEEWKD6luIYiHXHyIIoUgNlJ8SHULQRQpPqS6hSCKFB9S3UKQfqT4EEUKQWyk+JDqFoIoUnxIdQtBFCk+pLqFIIoUH1LdQpC1P+JDFFkuDMOQZRbiQxRZLhzHBQTYeOArDCGKLBcwkHFxcYggLlLau6eKUBRF0/RzhDon/BeIIoWAgXbZjY4JZoUoUgiiSPEh/UghiCLFhyhSCKJI8SGKFIIoUnyIIoUgihQfokghiCLFhyhSCKJI8SGKFIIoUnyIIoUgihQfokghiCLFhyhSCKJI8SGKFAIUSVZaiAxRpBDERooPUaQQRJHiQ/b8MkGXLl1SU1MpPVwxTZs23bx5MyKYGbIazQQtWrSg9fCLdmUymaur69ChQxHB/BBFmgDE5+fnZ5wSEhLSuXNnRDA/RJEmaNCgQZs2bQynCoWiX79+iCAKRJGmGTx4cM2aNfnjwMDAHj16IIIoEEWaJjg4uG3btkg/3O7fvz8iiAXWY+37MaoH13JV+Rr+1LBpOgw5WFb/sSn9Fu78IY04FhXn1O3RzueRySlGyxmXYMhpKJCHTzckqtUFly5fgWJahLeE8U1Jfn7zePTsJzEuhuYMxVI0xekzGOc0fl9TJRTh6KKs18wxoL4DkhK4KpJBP8yN0xQyCjuZusDw7YE8dP8vER+lfy2jSN0xKsps/PUX5aFM3FJUGvfMu7AMBzVEFbckpW43UcKznxPpfzJc2fcyzmCyBD129rRazdjby0bMDUaSAUdFwrzduumx9Zt5tnjdHUmek7ufPrqRM3ZxLSQNcFTk91MfvNLdJzhUWq2VAFeO5dz4N+3dhcFIAmA3svlzS4qdnYzI0Zim7Vyg9T/xWwaSANgpMiW+wMWTzLaXxsldnnA/F0kA7BRZmM8iOfFJlYZl2YI8Saz5wM4aMQzHkuU2ZWA1HMtQSAKQ9pGAF9gpEvxzkjAFhHLA0kZSRJPSBTtFwgQGWURcFlpO0QyLJAB+itT9RxRZGlYLAz4ysrEE+oXbpNWWLvjZSGIfpQ1+IxsOEU1KGeKPJOAFUaR1QCsoGSOJxoN4yK0DmEVkpDG3iqX3h4y1JQx+q2xYGwmzMXfe1H379yBCFSHrvszF7ds3EKHq2MLIJiMj/ctFs6/fuBIYENyzZ//Hjx+dOPn3Txt3waX09LQ13y2/dj2moKCgRYs2kUNGBwQEQfqDB/dHjn57zbc/bd++8eSpo9Wr1+jQvsuYdz/gty++fv3KT5vX3bp13c3do03r/w2LHOPk5ATpUb/u2P5/GydNnDbn8ym9eg344P3Jp0+f+Ovvg1euXsrOzmrYoPHQoaObhYVDzg4Rutevls7/bu3Xv+85CscHDv4e/XvUgwf3atWq07FDl759BlVpIkD3RKU0wgZiZyN1I5sqdiOXLJ33KD7uqyVrFsxf/u+/p+Af/zArwzCTPhl7OebCpInTf9zws4e753vvD0t48hjpw1TA67LlCyIiuv154PSMaQt2/rL176OHIPFxQvzkKe8VFBasXrVx/tylsbF3J308ho+QZmdnl5+fFx29a9rUeb17DgCVL/xyZmFh4dTP5n6xcEVgYPCMmZPgNwA5D+w7Ba+fTp7Fy/HwkQOLl8ytV7fB9q3Ro0e9vytq++o1y1BVYHSziJIYa+PXalMcXRVFZmVlnjlzckD/oY0aNvbyqvbJxzOTkp7wl65evfzoUdz0afNbtXzF09Nr/LiJrm7uUVHbDfe2e61T+3adQJ2hoc39fGveuXMTEg8f3q+QK0CLoLDg4JDJn8y6e+822FGk300WVDhw4LBOEd38/QPt7e03rNvxycczwC7Cv3FjJ6pUqqvXLpf9kPv27W7atNnEj6Z6eHg2b9ZixLBxu3fvhE+OCGXATpEcQ1Vpjcv92Lvw2rhxKH/q7OzcvHlL/hjEAWoDBfCnoKew0Jdjrlw03FuvXkPDsbOzS25uDtI12TENGrzk5lb0YK6Pj6+fnz+0y4acDeq/ZDgGk7lq9Vf9BnSDZvr1N3QxMDIzSz+fxbIsdBtahJcEEmrWrAUk3rx1HRHKYPX9yJycbHh1cnI2pLi6uvEHoDCNRsN36Qy4u3sYjvnGvRRw163bN0rdlaFvi3mg7eYPkpOTPpo0unmzlrNmfNGoURNQfOeurcsWqFar4WP88OMa+GecnpUpiWcLq4rVK1KptIdXjVptSMnITOcPoBF3cHBYuOBr4/wyWiZcoKdXtSZNwkYMH2ec6OZqIpbB0WOHQG3QiYR3QaasIw807o6Ojl06v/HaaxHG6QH+QajS6LrXNFmNZgmqOrIpGjvH3Yc+H9JZuNyLF896e/vCce3a9aBjV6OGT00/fz7zk8QEdzcP4QJrh9T989De0KbNDRY0Li4Weo1lc8L42sXFlZcjcOz4kXLLrF0vJzeHH4YDYDITExPgB4MqDR/nF0kADP2RHEVX4VOB2oKCaoGzBgbRIMcVK7/09S2Ksvdy85YtW76ydOl8aF5hmZYt+gAAEABJREFUGLF7zy/jxg89cCBauMB+/d6BTh6MhWEQEx//8Pt134CfKPbBvbI5Q0LqpqU9BZ8OjMT/PfsP/BKg95mSkoR0llsJHqXz589cunwerr47asKpU0fBYQ4lw3hr3vxpH08eB7pElYejdP8kAH4jG5Ziq7h8f8rk2WDPhkb2BjcNDFYavxQKg2X+0pcLV7Rr12negmm9+nT69bcdnTq93qfPQOHSXF1cf9jws4O9w9jxQyKH9wXnEThxwHFTNmdEx65Dh4zavGU9dB9hCP/hB1M6d+q+/f82Lf/6C7j6zuCRFy+dmzX7E1WBCroB69Zuu3LlUu++ncG1lJeXC44qQ3+UYAx2U3brpj1w91a8PsK/8reA/QN75u3tw59OmzFRLpPPn7cU2RC7Vz/UFHIj5wUjW8cWZhFhBhmsI8zTgDS3bP3hwoV/33rL1oI0Q0eGlpGRjSWAYQ1VxZ/JnDmLv1o6b/2G1ampyUGBtebMWtQivDWyLVgOOjOSmEbEbzUapetKVukWN1e3BfOqNilnfeg6V8RGWgTdqIY8aSNdyFMNBLzAcaWFRJqnKkHb0bQ0rAeGT8dSpNUuC6tmWfKcjUV4jvWRBFsCx5gWLDGREgbLeW1iI8tAK2gZ6UdaBp1DEhFKwWpY8rw2gWABiCIJeIGdIpUOlNyOPEVeGjtHGS2TRIxd7L57Byd5QY4kqr5KFOaxTq6SaNCwU2RoO8+cDDUiPEtetvbVN72RBMBOkfXDHd2q2e1aFo8IxexcGlejprJ6gAxJAEzDPh3ekRp3Pd83xDGgvjNj8nkUw8bV/JluOpwrm26cQlMUW3zJdC5klMrPZRrNaHL6+43escRJVXreU/dZKEO5hip+9hajD/xMXOGSXHJK/uhObuKD/IZt3F7tUcEDazYDvoHITkSl37uarS5gNYWV6FYavkeBdRrPKMKUJI2UUeFyD+MMXPHdRad6uZV902fKfCZdd0fZPAoFrXSiG7V0bymlfcZtJDSemVi5cqWnp+fQoUMRQSyIP1IIrVYrl5MqEhVS3UIQRYoPqW4hNBoNH9ePIBpEkUIQGyk+pLqFIIoUH1LdQhBFig+pbiGIIsWHVLcQZGQjPkSRQhAbKT6kuoUgihQfUt1CMAxDFCkypLqFgH4kUaTIkOoWgrTa4kOqWwiiSPEh1S0EUaT4kOoWgihSfEh1C0E85OJDFCkEsZHiQ6pbCKJI8SHVLQRRpPiQ6haC9CPFhyhSCGIjxYdUd7nApDZN0xSJryouRJHlwrJsWFgYIogLUWS5yGSyS5cuIYK4kEiN5cLv+A6WEhFEhChSCBjWwOAGEUSEKFIIokjxIf1IIYgixYcoUgiiSPEhihSCKFJ8iCKFIIoUH6JIIYgixYcoUgiiSPEhihSCKFJ8iCKFIIoUH6JIIYgixYcoUgiiSPEhihRCoVAQRYoMUaQQxEaKD1GkEESR4kP2/DJBs2bNkH5HQ8MjDQzD1K5dOyoqChHMDFmNZoI2bdrQeqhiHB0dBw0ahAjmhyjSBJGRkV5eXsYpAQEBvXr1QgTzQxRpgtatWzdu3NhwqlQqe/fuTR6TFQeiSNMMGzbM19eXP/b39+/RowciiAJRpGlC9SD9cLtbt27Ozs6IIArW1hIx6M5VFavRlJuh7FbuemhEsxSLuCrcFdEqMjVWIZPJmgR3vXUu28RNyFR5VPEO8OU4MUzfBd54maJ2cwckeazH+8OgTQseqnK0lIzSqqv+xCqtF4JofytV5feS2dGI4Rzd5cNmBiEJYzWKXDPlfkBdl/YDaiAbRo2O/JKU9DB/3OIQJFWsQ5HfTYl9Y1Sgh48kRrt3zqnOH0kc+6VERWkFI5tdKxNcPRUSkSNQr4WDvaMsem0SkiRWoMiMVE3Nui5ISvgEOqYlFiBJYgWGR6tmnN2k5aWys0eFhQySJNagSA0H/5CU0LKI0SBpQmbGCHhBFEnACytQJE1zFE0WcUoFK1Aky1IcK61g4DAHKdnw56TVxhGYtZDs0n6iSAJeWEM/kqJIP1I6WIEidS0YRzaVkQrWoEgxV5HhAYWk+wu0AkXqRp2UxCQJDi8y1sYWvY2U1vfDsdIda1vBCgbz2cievSM2b9lQ+XSCCFiBIs1nI98eMLRpk2b8ce++nZ8kJpRNJ4iMNfQjzdZoDx40nD9ISkrMzMwom04QH2uwkVVvtLNzsr9aOr9DRHivPp0WLJyRnKxbjx0bew9Szpw52W9At9FjdCFT+Nb50uXzg955E07fGdJz5uxP0LOt9pG/Dg4Z2gtufG/C8MSkJ3Bw+MgBSN/x8+bX32hreEd4C7h06tQx/vTAwd8hP2SA111R26v66IiUZxFtcCWsVqudOu3Dp2mpy5et/WDCpympyVOnfwiJCoUCrm7eugEa5U8+nmnI3yws/MuFK+Bg29Y9C+YtMy7q0aO4hV/MjIjotmf3XyNHjP/iy1lI/wS38AcAyS5eMrde3Qbbt0aPHvU+KHL1mmWoKpBZRLwBC1mVOZsz/568efPaTxt3BQYGI13InqCdv2xNT0/jA521CG/dv987lSzq4J9/uLt7RA59VyaThb/cKj3t6bVrMRXetW/f7qZNm038aCoce3h4jhg2bsnSeUMGj4RjRKgIaxhr63qSVWjD7t+/6+joyMsRAFs1c/qCGjW8i08bVr6oe/du16/fCOTIn77UWBflQrgJZln22vWYFuFtDCnNmrWAxCtXyVbdlcJaVlpUwUbm5eUqlfblXbVTKlGlgeFOzZoBhlMH+4pjTqjVao1G88OPa+CfcXpGRjqqEsRDji06k1SV9ZGOjk4qVT6YJX7L9v+Ci4trobrQcJqvyi8vJ8MWPahlb28PFrpL5zdeey3COIOfrz+qNJSEpxFtcGTToH6jgoKC23du8qcwOpn48RhoylHV8fHxu3PnJoibP42JuWC4pFDYFRYWGmJCP3r4wHCpdu16Obk5MGDi/zV+KdTLs5qh21AZpBz32Br6kfAZ6SoE+gkPbw1N7bp135w4+fe582dWrFyUmpIcFFRL4JYAfafz6NFDN25eM05v167T06epa777GpQHbiMYIRkuNWrUBDqU4OVBetfP9h2bDJfeHTXh1Kmj+/bvASlfvXp53vxpH08eB605qgpkFhFfYJIXsVX4nOCdWbpkDcuxs+d8OuWzCfYODl9+sVLYZVPTz79b1zc3blq7fv0q43QYmI8d8+Hp08c7d20NbqARw8cZLjVs8NL4cRNB9+CGnLdg2qgR76HiQU+TJmHr1m67cuUSzANNnvIe9GsXzF+urEr/VcpYQdyfVR/fa9m5WqNX3JGlgYEOiGz2rC87tO+MzMmZfal3zme/v6w2kh7WMNYGd7HUOvrkyS+coZD0XCFkzgZrKL2VxACYv/n7yHlEMCfWoUhaWoGoJI01eMhZxDHS60dK9fFLq5hFlNxDJ7o/V6qPX1qFIimpdfPJajSs0e1LSPqRksE6Wm2ywa10IE/H4gj4FiTrXrAO74/URjYsi9iqbyJlG1jJLCJptCWDNbTaFEdJLfCPhLECRSrklFwurV6VTEYp7CTakbQGRdrJstKltZeGKpeR20nUQ24FP8TqfvaP76mQlEh9VOAb5IQkiRUosud7vgV56ivHcpA0OLE7Xctwr4+sjiSJ1exmvHZqrJe3fYvO1bwC7JCNkvRAfelIamaaeswXtZBUsZ4d3xHavig+O0PDsYjRltozUPdXPOuzpPSedaq8QTpX/iJgoUuckGdU9xHKD1AE/gJTF0t2hqdkMKCRuVe3Gzi5Cs/R2h7WpEie3HTEMEWKfBAXO3PGrO3bt+n+BqO/Q6dEruTrLoquVpxBdwi9FYMLWq+VkrtpfU79+cxZM9zc3D6dPMVQbMniYV6anNExKpIzZZzHUK4MIabMhzHKwDFMj34dN2zY0KBBAyRhrG/3EGdd8JyisCf3jsX88ecOZB6uXLlyO/aivb19nibZz88PmR3ZyZMnDx48KHFFWqXTi2XZOXPmwEG/fv2Q2Vi/fn16evqTJ0+ioqKQWHTt2hVeIyMjz549iySJVSpy/PjxQ4cORebk9OnT169f54+PHj2alpaGRGTz5s3Hjh1DksTK+pEgjvbt2yPzA6I/d+4cfwwDlnHjxo0aNQqJzo8//li/fv1XX30VSQZrspGffvqpYUxjVo4cOXL79m3DKfxo9+3bl5eXh0Rn2LBhO3fuFNlCWxbrUGRGhi5IOPQaIyIikPnZtGlTZmamccrjx49BlEh0wB+0cuVKpVJ5//79CxcuIAlgBYr85ZdfTpw4AQetWrVCogAGElpqthiwkVqtFmSKLISzs3NwcPC6deukMNzBvR8Jtur777//7LPPkCWYNGlSnz59/ve//yE8uHnzZsOGDe/cuVOvXj1ko+BrI5OTk2FsAQ2WpeSI9EH2K4yDLyYgR3gFYwntBrJRMFVkSkoKjG2bNm3q4FBxoGXzgZsieZYuXerkpFsZZJMjHhznbHJycmAo88cffyBLY9hzBDe6d+8Or3v27IFOl0XcUuYDLxup0WgGDhwIZgmccAgD8LSRBkaOHKlWq+EHXFhYiGwFvBQJ3aOFCxdatqU2BnNFIr0n39HR8datW9u2bUM2AS6K/OGHH+B18ODBtWtjFFgWf0Uivc8yNDQUet68j8zawUKRMFdmb2+P8AN6EfgrkgccVY0aNYKD3bt3I2vGwoqMjY2F1w4dOrzzTmV3hhMTq7CRBry8vOD1wYMH3333HbJaLKnI6Ojon3/+GQ5q1cJ0Eb91KZIHjCU/13r58mVkhVhSkbm5udOmTUMYA602nt4fYfgZnbt373766afI2rCAIqFZWbFCt3swjGMQ3lijjTTQv39/3m0Js1/IerCAIqdMmTJu3DhkDVi1IpG+gw6vSUlJM2fORFaCqIo8f1630QE4HfEcWZfF2hXJA76htm3bHj9+nLWGgGsiKZJhmF69etWoUQNZFVbajyxLt27dQJSgyAULFiC8EUORWVlZjx8/Xr16dWBgILIeeItC20poUfhDwN43btx4yZIlCGPM3iTFxMRAz7pLly7I2uA4rnnz5si2gJZKpVLxD4fAZA/CD7MbgLy8vL179yJrA76z1q1bW7WruTwcHBy++eabHTvM9Zz7f8TsNrJVq1bu7pbf9rVKFBYWtmvXzvAsou2B88jS+qKsmBsw6l27dj158iQiWAIxuu1r1qwB1wOyBmAQBl5lm5cjowdhiRiKrF69+unTpxH2pKen9+nTRwrBJDZv3rx27VqEJWK4f2F8B7YH4U1KSsqQIUOOHDmCJIBSqYQRN8IS0o/UkZiYOGrUKIvECCCUQiT378SJEw2BnXAjPj5+zJgxkpKj1PuRQFBQEJ7L9eLi4j788MPff/8dSYno6OhFixYhLBFpGcFHH32E4TT//fv3P/vss99++w1JDOhHUrju7CdePxKaCaymrW7fvj1nzhxspy4ki3jLCH+DeygAABAASURBVN566y18lo5Cp3b+/PmSlSO0V1qtFmGJeIoMDw83DspoQWJiYpYsWbJ161YkVcDniu3zJOItR507dy7CgIsXL3777bc//fQTkjB2dnbYrrITrx+p0Whyc3M9PDyQ5Th79uyGDRvWrVuHCLgi6g+FfxDJUsBM5qZNm4gcEelH8igUirZt24L/jz8VOWTAiRMntm/fvmbNGkTQ96THjx+PsETUWcSePXvm5ORkZmaCM8zf33/Pnj1IFI4ePQrv9fXXXyNp07dvX5jOButYUFAAB3K5HLpS4JW7dOkSwgYxRjZhYWG8J5L3ykKfGloNPkiNCBw+fPjAgQNEjki/tQDUg2GqAuQIr3Xr1kU4IUarDaNsFxcX40kC+HW2adMGmZ+DBw+CIpcuXYoICA0aNCg4ONg4BSzF66+/jnBCDEVCY92jRw/jCRsvLy8RbOTevXuPHz+O7QSuRRg6dCi4fgynAQEB0JQjnBBpZDN58uSWLVvyfVZoNZydnevUqYPMSXR0NPh6Fi5ciAhGvPnmmyEhIfwxtFodO3Z0dXVFOCHeWHvVqlX889pQEaGhocic/PrrrzCcxMQnjxvDhg1zdHREWBpIJLI/ctmyZTVr1lQqlWZ9Dnrnzp0wXTlr1ixEMEXnzp354Gnt27f39vZGmFGB9+fvnamxV3PVBSyjLb2WDO4rtaBJn/DMTuYci6gymuc4sJIm3rTc9JJiy15CAmuqaDmttJcFv+QUMbA6wpu/f3kaewXqmdFqOZN/bNm65Q+44h3kiymu+TLfTklR5Vwqr/6FSxO+SlE0LUf2DormHTxD2zujSiDk/fnr57TYq3khjd0avOzG8cOSZ2qj5Iw/0umDT9crpVT9GY6g/nSfv8zfDuk0Z5RsuFNfGlXqDv4dKVRSh6VzgP2nb57PiI3JPsKiiMH4ivLkb+n3LufWbuJa92U3WgEd7dIZKGRKpHxqOZI08Vs1XUrJJb4yKe6Z0kzUcyXL1EPLkDof3TibeWZfqoMLXe9lR1QR5drInV8n5GcxfSdZU6Se8ohaEe/oQg342B/hx6+rn2SmaPt/Ygv1LMyOxXG1Gzt1rMg0mO5Hpj1i0hILbUOOQN+JAenJ6qQ4NcKM3FSU9KhACnIEOg7wuxOTU2E204o8tTfF0cXq4yYa4+iiOLMPuz3bju9JdnSyqXoWoEYtO1pGnTuQIZzNdHWo8hi5wkai1PHI7FB+LnarXXKztAqlTdWzMDDKSU+poKUyrcgCldYawrFWARjGCo7LLYNKpWG0mD6BZQ60BaxGXcFTuVJpMnB98k5iUFSFZkEqisSVitwn0sO0Iina1mwKrrFkOAz7Euak4q/BtCI5luNsqx9J06TZtjwUVXHgAqm02vrpNtI+WpjKWDrTirQ9i0JReLaPEutH/gcbaWsx/HB1ZkmtH1mxsOTl3mZbkiTeHxzQLaJ4bkXamJEkcVtxQGcXyMiGh5ZzlG15D6ySStgFqSiS1VIY9thkcmRjXrYKeO6RjW6yx7Y6Xnj+OYwW2yGXeeAqDlhBl3OjGTte+fn5Xyya/cabr035bEJs7L0OEeFXrpg9poLN9Iyhxj6b+kHnrq23bd8Y9euOiM4t0fPCV/7Vq7po3P+xqBeIBWzk1WuXDx3a9/57H4eFhru7e0QOHV2jhg8yM7ayAyw68teBK1cvzZ2zJCSkbkZG2tAho9GLoFHDxi+qKCGee6WFmW1kHrx2ingd5AgHI4aPQ+bHZobaeXm5Pj5+r7zyGhz7+Pg2bNgYvQignBdVlCDPO6/9HPTsHRE5ZPTxk39BE7xn91+uLq4HDv4e/XvUgwf3atWq07FDl759BkG3dsMP30JzA/l79+3cIrz1uLETR707cOXX65s2bTZ33lTIAEpdtORzlSq/UaMm48Z8xFeTVqv94cc1Z/49mZKS1LhxWO+eA1q3blulj6ebhcJPlRSNynv8zySTPh57OeYCHEBrO3rU+/b2Dmu+W37k0FlI6dWnE/y2s7Iyf9q8zsHBoUV4mwnvT/byqgaXHjy4H/37rouXziUlPQkOCunevVfPt/qVKhlabb6oU6eOzZz9SamrW3761d8/8L9/C5WZ16bLubPKA1OFQvHHvt/q1Kn/1ZJvHR0cDx85sHjJ3Hp1G2zfGg11tytq++o1yyAbHM+e9SUc/BZ1aMni1cYlyOXy6zeuHDq8b+13W/bvPam0U365eA5/6ZtVS6CE3r3e3r7t93avRcyZO+XY8aptzsVoORw3cNGpsQoV/fXy70FMwcEhfx85/87gEcaXoP5//nkzTdO7fzvy08Yo6Bpt+ul7/tK3a5adO3f6ow8/W/TlNyDHld8sPvPvqfLeonHj0OXL1hr+1a5d18fb18tL97jWf/8Wnn9em5LBr7dqJgXE7+rq9sH7k/nTfft2g9mb+NFUOPbw8BwxbNySpfOGDB4JxwKFqPLzP508m4+4ENGxGxhLGAbJZLKDf/4xeNDwt97UBWDo/nrPa9diNm9ZD5WCrJwX2zuqWTNgyDsjdUfOLmAj79y5yafPmvUl9JR8ffzguFlY+IED0WfP/dO61asmC3Fzc4c8/PGe6F0JCfGrv9kIRrewsPAFfAuUief3S1HOWJvhnsMrUb9eUXAplmWvXY+BSjFcatasBSRCl1y4hIDAYF6OgLOzC7zm5GRDzarVauPSwkJfhnFiVnYV9lq0PX9WWerVa2g4dnFxhR5n0QnH/frrjsjhfaGth3+3bt/IzEivsLR79+6s/nbpZ1M+BzMJpy/kW9A9+v18NvL55rUNQbfgo2s0GuhzwD/jDBkVVYTJcO25ubpHKj/4aFSp9Iz0NDdXN0QoxmQfDQzB1OkfaTTqd0dPCAsLd3F2KVuTZcnOyZ45++Oeb/Vv364Tn/JivoVK2EizzNnY29uDqevS+Y3XnjXpfr7P8wy/VzVdJ+aTj2dAq2ScXjWfEQxt8HNGV3Vk8xzcuXvr1q3rS79a83LzIncjaKt6tRrCdy1YMN3b23f8uImGlBfyLejCuDzfSgtK9l9n3GrXrpeTm2PokYDJTExMqFHjeeIe+dcMVCqVSN8H4lPA1oLv39C+VwaOYTn8ZhGhCeM4834qGH3Dq0GCcXGx8K9WcG2BW7b/36bYB/d+WL/DOOTni/kW2IrX/pRjQ//z7/bdURNOnTq6b/8eaDVgVmDe/GkfTx4HrTmqOvA3Dx82FjrRUA6UAOO7yVPeW7GSxCmtFODuASfGzzu3QEP86FHcqtVfgdMtKTmxvPwxMRfXb1g98O1IEOWly+f5fykpyaJ9C+W12v/1h9ukSdi6tdvA9fj9um8KClQvNWq6YP5y/kf2HEAFgdHdvmPTxYtnnZycobRPPplZpRJ03RdJLkjz9vaZMX0BOCl79uoIDe6MafPT0p/Omj152Ih+c2aZ0BMMqJHOYbTcOBFcm337DHwR30LF/kjTkah+mh8Hna5+E4ORrfDrqjj4Y4fNDEI48dOCOEZL9Z+E16cyH9u+uO9f16HHaD+BPOV6yG3MV6L73bHYGUlaVvHY06Z47vWRlZntsTo4/P4k3bAGv9+JGXn+mBY0h2xr3R6eS9F0s2qSevKLYyuUpGlFslJbSUoQiYr3mJNKlBWYDMLQTOo95IhgTPkjG2RbUDjGtNB7yJGEqMTvr7x5bVurJ1a3FA27X5n+A0mpe8Q974pd+O2SfqQI6L8fSbl/KqbclRY21mrLZNiuIUcS4rmfjqVlFGtjHnIWx66I5PqRlXg6thzvz3Ot2MUZneNPYjGfrBSpRBAgWAvltNpymmJsqjmRK3D8ickVtLlX7GIFraDhi6ggj8lUR2c7CsmQDUEjmdJRgTDDycUOsVLaz4ainJztKshjMjWksVNeLnZ7tv0X8rI1dUMrtXepmNQPd1HlaZBkUBeyr7zpJZzHtCLD2rvY2cv+3pGCbIJju1LldlRYe1eEGQ1bOikdZIe32Ug9C7Pn28dePnayCkyk4Mz3ps8fOrrYvT7aF1kzezckqHK0Iz7Hd1XslvnxcqVMt47VpjpKJeRmcgc2Pvb0lvccX7GWKliLsfWL+OwMtUxOawqfiQihCzDAGp9SnNE6P1pGs0zJZf1jgFzxJX5C79lCKOhhlMwS6Z9Y44wzG0rQOUr1Qy7DOxry6HZE5+dAivMoFDSU6eIpHzIN981Zty2Kz0pTK+S0WsOUeqLZ8Hfph2a6mjGklOTR10bJ5udGGfgBnXH+kgKLnvQoqu1Sd+n3jS9dGtStbgUdZ+It+I3fuWdvgYkJGCXD11G9prLfRzVRJah4dRBi0IW/s/NzC565DT6Z0WDcoKHiU/rZaBq0YfaWpkElRccUTRe7PWEcRaFiERtKo2Q0p0+kEc3yJdA0r9ySPIZCylxycFaEd/CwGsPDoItHM/NzGY579sdvqFuDuExIknom/ZkM+lUzRe5Y/ioqLFT/c+qfDh3b6/d3NzIMxfNauh3eqWIhlyTrP0zRZyhJ5IyfFH/2FpqWuXgqmratQn+Jsr1FFYQKSU1NjYyM3L9/P8IPsi+iFNFqtXI5pl89UaQU0Wg0CgV23lkeokgpQmwkAS+IIgl4QRRJwAuiSAJekJENAS8YhiE2koARpNUm4AVRJAEvoB9JFEnACGIjCXhBFEnAC6JIAl4QRRLwgnjICXhBbCQBL4giCXiBsyJJ8EIpgnM/kihSipBWm4AXRJEEvHBwcHByckJYQhQpRfLy8lQqFcISokgpAk02NNwIS4gipQhRJAEvcFYk8f5IEaJIAl6QVpuAF0SRBLwgiiTgBUxqE0USMILYSAJeEEUS8IIokoAXRJEEvCCKJOAFUSQBL4giCXghk8mwVSTZYUlCDB48ODMzk6KogoKCvLy86tWrQ6JKpTp8+DDCBrLSQkJ06NDh6dOnycnJWVlZYCMT9bi64rWlLlGkhHj77beDgoKMU1iWbdeuHcIJokgJAebwrbfeUiqVhhR/f/++ffsinCCKlBYDBw4EFRpOX331VeNTHCCKlBYKhWLAgAG8mfT19cXNQCKiSAkCKgwICICD0NDQOnXqIMwg3h98uXMh9+bZnIyn6oJchtHtfE9xLFe0mXvxDuuGbdmRfo91unj3dkM23R7tbMku7HAkoymGZSGRlkEmSr9be4kKDHvHF5VplM4ijtYJxvDGRZdoKJEGrzslk1PVaypf7ujpV0eJnheiSByJ+uZJSnwByEZmJ7NTKuT2MoVSBtJiuCJFUqAtWicKSv8NFsuD0l8t0Z9ebJReh6hsHkOK7n9UkQz4RAoV3WUMBe9Dm9CLXEazLK1Va1XZBdpCLcsgiub86zj2HOeHqg5RJF7sXP44JaHQzkFePcjdw98ZWScp9zLTE7K1GjbkJefuI72rdC9RJC6kPFJHrU6Q21F1WwcgGbIBCjLUD68mcxw7blFI5e8iisSCmONZJ/ak+tWt4RmEaYCo5+bJzfT0x1lDZ9Ry86rU74wo0vLcj8kgzdWmAAAFzklEQVTf/1Ni487ByEbRFjK3jscPnx3s7F6xKIkiLcz5P7POHkpr1DEI2TrXDz94Z2ot9+oViJL4Iy2JWoXO7E+VghyBwMY+Wxc9qDAbUaQl+fHz+54BeC29MR8uPg6OLg4bP38onI0o0mLs25jMssivoReSDCGtfPJztddO5QjkIYq0GLFXc3zrVUMSw62Gy5n9aQIZiCItw9FfnlI0ha0PPDcvY/KsVpevvvi15f5NvApVzP0r+eVlIIq0DPdiclw8HJEkUdjLT0enlneVKNIyqPK01eu6I0ni5uuclaEp7yp5FtEC3DiTQ1GUg4sdMg9xj678+feG+Mc3nJ08GtZv26XDaHt73VTQqTO/HDr24/iR323eMS05JdbXu85rrwxq0bwHf9elK38eOPK9SpXdqMH/2r36DjIb3rXdU+9ncAyiTLkmiY20AA9v5ssU5qr5p2nx32/6QKMpnDBmw7DBixOT737343iG0T0LK5MrVKqc3XuXDug1/at5Z5o27rhz94KMzCS4lJh8b/uu2eHNuk+dGBUe9saevcuQOaFp6uo/pkfcRJEWIC9bQyvMtZjiYswBuUwxfNBi7+rBPjVC+veckZB4+9rNY/xVhtF07jA6KKAJGGlQHszYJSTegfR//o1yd/Pp3H6Uo6NrnZCXW4X3QuaEktNpTwpMXiKKtAAaNYPMNncLTXaAfyMnp6JOqqeHr5en/4OHlw0ZAmu+xB84Ouic86oCna16mh7v412yQiegZiNkTjjEFRSYDmFA+pEWAQykuUJKqApy4xNugO/GODE7p8QFSBkWiBuRn59dzSvAcGpn54DMCa1ruE1bQ6JIC6B0oKlMZCZcXLxqBYV17TjGONHJyU34LmisNZqSZrSwMA+ZF8rRxfR2ykSRFsDT2y4lvhCZBz/vuhdi9oUENzMYoaSU2OpegcJ3ebj73rh1gmVZ/q4bt08ic8IyrG8te5OXSD/SAjQMd2W0LDIP4NABYUXv/1qtLkhJffjHwdXLVg+GobTwXaEvdYJ5mt17l8FY517shX/+3YXMhlrFcByqE2p6goAo0gJ417KjaJSVmI/MALS/kydst1M4rFg7bMk3A2LjLvbvNcPfr4HwXfXrturR9YPbd09/Orv1jl/nDew7W59slvFXyr0MhZIq7ypZsWsZti2KV+VTddr4Iulx+/ijmrUdeoz2MXmV2EjL0OaNahqVGkkQBmnVXHlyRGRkYylCmjgolPTjq2n+TUyvj8zKTv1q1UCTlxyUzqrCXJOXfKqHTBizHr04Zi6MKO8SzAPJZCb0E1Cz4djhq8u76/65J25eQqojrbbFiL2i2r/5yUsRwSavwvedlZ1i8hIMWezsyhmo0nJ3txroxZGe8aS8S2pNoZ3CROwKudzO1aWcdZ8MuvZX3ITltVH5EBtpMUKaOlTztY/990lIKxOxH8D8eHo8T0yIF8uL/Qy3Tj6qE1bB47+kH2lJ3v6kpqZAk3QnA0mAuIvJ9o50t0gf4WxEkRZm7KKQ9MfZT2NzkU0Tdy6pMLdg+OyKn7ok/UgsWPPpfQ8/F98GtvkU2IPzSRTSDp9VqYeAiSJx4bsp92UKeb22eEW8/e/cOfFYpmBHza1VyfxEkRixY2l8WqLa2csxqNmLHC9binv/Jqmy8ms3ces+sgp/DlEkXiTcLTiwNakgj1HYKzx8nauHuCHrgkOJt9Jznqo0hRpnd8Ww6YFVjfNGFIkjSbGFR39NyXyq0apZfQRb3SywPlquUSbKxLSzfukjH+LUNEWxdxEfGZV6NrFUVt07sCWxTsvNSdG62Kksw7JaFq7KFJRvsGOP0b6K53qOiCgSa9QqFHMi62lCgSpfqy7kwG9uuAQyLQrnXHSuD6pL6/XDlvudUjLEMXoXC0UjhjVdVHEiXxJlpJCi259FpqCV9nKlI+0d5BDW7r8GjSGKJOAFmbMh4AVRJAEviCIJeEEUScALokgCXhBFEvDi/wEAAP//DeAunAAAAAZJREFUAwBRfSZi+Zh7+gAAAABJRU5ErkJggg==", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "---\n", "config:\n", " flowchart:\n", " curve: linear\n", "---\n", "graph TD;\n", "\t__start__([

__start__

]):::first\n", "\tgenerate(generate)\n", "\tcritique(critique)\n", "\trefine(refine)\n", "\tfinalize(finalize)\n", "\t__end__([

__end__

]):::last\n", "\t__start__ --> generate;\n", "\tcritique -.-> finalize;\n", "\tcritique -.-> refine;\n", "\tgenerate --> critique;\n", "\trefine --> critique;\n", "\tfinalize --> __end__;\n", "\tclassDef default fill:#f2f0ff,line-height:1.2\n", "\tclassDef first fill-opacity:0\n", "\tclassDef last fill:#bfb6fc\n", "\n" ] } ], "source": [ "from IPython.display import Image, display\n", "\n", "arch = Reflection(max_iterations=3, target_score=10)\n", "graph = arch.build()\n", "\n", "# Render the *compiled* graph as a PNG inline.\n", "display(Image(graph.get_graph().draw_mermaid_png()))\n", "\n", "# Also print the mermaid source — useful for copy-pasting into docs / READMEs.\n", "print(arch.diagram())" ] }, { "cell_type": "markdown", "id": "ca889d25", "metadata": { "papermill": { "duration": 0.01143, "end_time": "2026-05-27T03:47:24.883817+00:00", "exception": false, "start_time": "2026-05-27T03:47:24.872387+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 8 · Live run\n", "\n", "Concrete task: write a tiny Python function. The original Notebook 01 in this repo used a similar coding scenario; we keep the scenario stable so historical backlinks and blog references still make sense, but the implementation underneath is now the library's `Reflection` class.\n", "\n", "We set `target_score=10`. The Critic's rubric says \"reserve scores ≥ 9 for genuinely excellent work\" — so a perfect 10 forces at least one refinement round in almost every case." ] }, { "cell_type": "code", "execution_count": 5, "id": "a4fc3a87", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:47:24.901425Z", "iopub.status.busy": "2026-05-27T03:47:24.901425Z", "iopub.status.idle": "2026-05-27T03:48:33.767020Z", "shell.execute_reply": "2026-05-27T03:48:33.767020Z" }, "papermill": { "duration": 68.875157, "end_time": "2026-05-27T03:48:33.769027+00:00", "exception": false, "start_time": "2026-05-27T03:47:24.893870+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
Final output ──────────────────────────────────────────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mFinal output\u001b[0m \u001b[92m──────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                                                                                                                   \n",
       " ```python                                                                                                         \n",
       " def merge_sort(arr):                                                                                              \n",
       "     \"\"\"                                                                                                           \n",
       "     Recursively sorts a list of ints, floats, or strings using merge-sort.                                        \n",
       "                                                                                                                   \n",
       "     Example:                                                                                                      \n",
       "     >>> merge_sort([3, 6, 1, 8, 2, 4])                                                                            \n",
       "     [1, 2, 3, 4, 6, 8]                                                                                            \n",
       "     \"\"\"                                                                                                           \n",
       "     # Check if the input is a list                                                                                \n",
       "     if not isinstance(arr, list):                                                                                 \n",
       "         raise TypeError(\"Input must be a list\")                                                                   \n",
       "                                                                                                                   \n",
       "     # Check if the input list is empty or contains None                                                           \n",
       "     if not arr or None in arr:                                                                                    \n",
       "         raise ValueError(\"Input list must not be empty and must not contain None\")                                \n",
       "                                                                                                                   \n",
       "     # Check if all elements in the list are of the same type                                                      \n",
       "     element_type = type(arr[0])                                                                                   \n",
       "     if not all(isinstance(x, element_type) for x in arr):                                                         \n",
       "         raise TypeError(\"All elements in the list must be of the same type\")                                      \n",
       "                                                                                                                   \n",
       "     # Check if the elements are comparable                                                                        \n",
       "     try:                                                                                                          \n",
       "         arr[0] <= arr[0]                                                                                          \n",
       "     except TypeError:                                                                                             \n",
       "         raise TypeError(\"Elements in the list must be comparable\")                                                \n",
       "                                                                                                                   \n",
       "     # Base case: single-element list                                                                              \n",
       "     if len(arr) <= 1:                                                                                             \n",
       "         return arr[:]                                                                                             \n",
       "                                                                                                                   \n",
       "     # Split the list into two halves                                                                              \n",
       "     mid = len(arr) // 2                                                                                           \n",
       "     left = arr[:mid]                                                                                              \n",
       "     right = arr[mid:]                                                                                             \n",
       "                                                                                                                   \n",
       "     # Recursively sort the halves                                                                                 \n",
       "     left = merge_sort(left)                                                                                       \n",
       "     right = merge_sort(right)                                                                                     \n",
       "                                                                                                                   \n",
       "     # Merge the sorted halves                                                                                     \n",
       "     return merge(left, right)                                                                                     \n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       " def merge(left, right):                                                                                           \n",
       "     result = [0] * (len(left) + len(right))                                                                       \n",
       "     i = j = k = 0                                                                                                 \n",
       "                                                                                                                   \n",
       "     # Merge smaller elements first                                                                                \n",
       "     while i < len(left) and j < len(right):                                                                       \n",
       "         if left[i] <= right[j]:                                                                                   \n",
       "             result[k] = left[i]                                                                                   \n",
       "             i += 1                                                                                                \n",
       "         else:                                                                                                     \n",
       "             result[k] = right[j]                                                                                  \n",
       "             j += 1                                                                                                \n",
       "         k += 1                                                                                                    \n",
       "                                                                                                                   \n",
       "     # If there are remaining elements in the left or right lists, append them to the result                       \n",
       "     while i < len(left):                                                                                          \n",
       "         result[k] = left[i]                                                                                       \n",
       "         i += 1                                                                                                    \n",
       "         k += 1                                                                                                    \n",
       "                                                                                                                   \n",
       "     while j < len(right):                                                                                         \n",
       "         result[k] = right[j]                                                                                      \n",
       "         j += 1                                                                                                    \n",
       "         k += 1                                                                                                    \n",
       "                                                                                                                   \n",
       "     return result                                                                                                 \n",
       "                                                                                                                   \n",
       "\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "
\n" ], "text/plain": [ "\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;237;0;126;48;2;30;0;16m`\u001b[0m\u001b[38;2;237;0;126;48;2;30;0;16m`\u001b[0m\u001b[38;2;237;0;126;48;2;30;0;16m`\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mpython\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mdef\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mmerge_sort\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\"\"\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m Recursively sorts a list of ints, floats, or strings using merge-sort.\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m Example:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m >>> merge_sort([3, 6, 1, 8, 2, 4])\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m [1, 2, 3, 4, 6, 8]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m \"\"\"\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Check if the input is a list\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mif\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34mnot\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34misinstance\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlist\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mraise\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mTypeError\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mInput must be a list\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Check if the input list is empty or contains None\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mif\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34mnot\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34mor\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mNone\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34min\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mraise\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mValueError\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mInput list must not be empty and must not contain None\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Check if all elements in the list are of the same type\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34melement_type\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mtype\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mif\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34mnot\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mall\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34misinstance\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mx\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34melement_type\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mfor\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mx\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34min\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mraise\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mTypeError\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mAll elements in the list must be of the same type\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Check if the elements are comparable\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mtry\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mexcept\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mTypeError\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mraise\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mTypeError\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34mElements in the list must be comparable\u001b[0m\u001b[38;2;230;219;116;48;2;39;40;34m\"\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Base case: single-element list\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mif\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mreturn\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Split the list into two halves\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmid\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m/\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m/\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m2\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmid\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34marr\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmid\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Recursively sort the halves\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmerge_sort\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmerge_sort\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Merge the sorted halves\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mreturn\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mmerge\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mdef\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;166;226;46;48;2;39;40;34mmerge\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m,\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m*\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m0\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# Merge smaller elements first\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mwhile\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34mand\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mif\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34melse\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;149;144;119;48;2;39;40;34m# If there are remaining elements in the left or right lists, append them to the result\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mwhile\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mleft\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mi\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mwhile\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m<\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mlen\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m(\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m)\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m:\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mright\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m[\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m]\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mj\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mk\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m+\u001b[0m\u001b[38;2;255;70;137;48;2;39;40;34m=\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;174;129;255;48;2;39;40;34m1\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;102;217;239;48;2;39;40;34mreturn\u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34m \u001b[0m\u001b[38;2;248;248;242;48;2;39;40;34mresult\u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\n", "\n", "\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\u001b[48;2;39;40;34m \u001b[0m\n", "\u001b[48;2;39;40;34m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
4 iteration(s)  ·  final score 8/10  ·  budget 3 ──────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36m4\u001b[0m\u001b[1;36m \u001b[0m\u001b[1;36miteration\u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36ms\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m · final score \u001b[0m\u001b[1;36m8\u001b[0m\u001b[1;36m/\u001b[0m\u001b[1;36m10\u001b[0m\u001b[1;36m · budget \u001b[0m\u001b[1;36m3\u001b[0m \u001b[92m──────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "TASK = \"\"\"\\\n", "Write a Python function `merge_sort(arr)` that:\n", " * implements merge-sort recursively\n", " * works on lists of ints, floats, or strings (no mixed types)\n", " * handles an empty list, a single-element list, and lists with duplicates\n", " * does NOT modify the input list (returns a new sorted list)\n", " * includes a short docstring with one example\n", "Return only the Python code block, nothing else.\n", "\"\"\"\n", "\n", "result = arch.run(TASK)\n", "\n", "print_header(\"Final output\")\n", "print_md(\"```python\\n\" + result.output + \"\\n```\")\n", "print_header(\n", " f\"{result.metadata['iterations']} iteration(s) · final score \"\n", " f\"{result.metadata['final_score']}/10 · budget {result.metadata['max_iterations']}\"\n", ")" ] }, { "cell_type": "markdown", "id": "2394c088", "metadata": { "papermill": { "duration": 0.00752, "end_time": "2026-05-27T03:48:33.790619+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.783099+00:00", "status": "completed" }, "tags": [] }, "source": [ "### 8.1 · What just happened, briefly\n", "\n", "Look at the printed score above. If it's **less than your `target_score=10`** *and* the iteration count is at the budget cap (`3`), the loop ran the full budget without ever satisfying the Critic — that's the **plateau** discussed in § 3. If the iteration count is `1` and the score is already ≥ 10, the Critic was satisfied immediately — that's **sycophancy**, also discussed in § 3.\n", "\n", "Either outcome is a real teaching moment. Section 9 below will compute the exact trajectory and discuss it concretely.\n", "\n", "### 8.2 · Full refinement trace\n", "\n", "Every entry of `result.trace` is one critique round: the draft that was scored, the score the Critic gave, and the written critique. This is what makes the loop **observable** — you can post-mortem exactly what the Critic latched onto each round." ] }, { "cell_type": "code", "execution_count": 6, "id": "cbea27fa", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:48:33.814535Z", "iopub.status.busy": "2026-05-27T03:48:33.812526Z", "iopub.status.idle": "2026-05-27T03:48:33.849620Z", "shell.execute_reply": "2026-05-27T03:48:33.847608Z" }, "papermill": { "duration": 0.049233, "end_time": "2026-05-27T03:48:33.849620+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.800387+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
› Iteration 1  ·  score 9/10\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1mIteration \u001b[0m\u001b[1;36m1\u001b[0m\u001b[1m · score \u001b[0m\u001b[1;36m9\u001b[0m\u001b[1m/\u001b[0m\u001b[1;36m10\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Critique: The candidate's implementation of the merge-sort algorithm is excellent, with a clear and concise        \n",
       "docstring and proper handling of edge cases such as empty lists, single-element lists, and lists with duplicates.  \n",
       "The code is well-structured and readable, and the use of recursion is appropriate for this problem. One potential  \n",
       "improvement could be to add some error checking to ensure that the input list contains only elements of the same   \n",
       "type (e.g., all ints, all floats, or all strings).                                                                 \n",
       "
\n" ], "text/plain": [ "\u001b[1mCritique:\u001b[0m The candidate's implementation of the merge-sort algorithm is excellent, with a clear and concise \n", "docstring and proper handling of edge cases such as empty lists, single-element lists, and lists with duplicates. \n", "The code is well-structured and readable, and the use of recursion is appropriate for this problem. One potential \n", "improvement could be to add some error checking to ensure that the input list contains only elements of the same \n", "type (e.g., all ints, all floats, or all strings). \n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› Iteration 2  ·  score 9/10\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1mIteration \u001b[0m\u001b[1;36m2\u001b[0m\u001b[1m · score \u001b[0m\u001b[1;36m9\u001b[0m\u001b[1m/\u001b[0m\u001b[1;36m10\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Critique: The candidate's implementation of the merge-sort algorithm is excellent, with proper handling of edge    \n",
       "cases such as empty lists, single-element lists, and lists with duplicates. The code is well-structured, readable, \n",
       "and includes a clear docstring with an example. The use of recursion and the merge function is appropriate. One    \n",
       "potential improvement could be to consider adding a check for the input being a list, to ensure the function       \n",
       "behaves correctly even if the input is not a list. However, overall, the code is of high quality and effectively   \n",
       "implements the merge-sort algorithm.                                                                               \n",
       "
\n" ], "text/plain": [ "\u001b[1mCritique:\u001b[0m The candidate's implementation of the merge-sort algorithm is excellent, with proper handling of edge \n", "cases such as empty lists, single-element lists, and lists with duplicates. The code is well-structured, readable, \n", "and includes a clear docstring with an example. The use of recursion and the merge function is appropriate. One \n", "potential improvement could be to consider adding a check for the input being a list, to ensure the function \n", "behaves correctly even if the input is not a list. However, overall, the code is of high quality and effectively \n", "implements the merge-sort algorithm. \n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› Iteration 3  ·  score 9/10\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1mIteration \u001b[0m\u001b[1;36m3\u001b[0m\u001b[1m · score \u001b[0m\u001b[1;36m9\u001b[0m\u001b[1m/\u001b[0m\u001b[1;36m10\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Critique: The candidate's implementation of the merge-sort algorithm is excellent, with clear and concise code that\n",
       "handles various edge cases, including empty lists, single-element lists, and lists with duplicates. The inclusion  \n",
       "of a short docstring with an example is also commendable. However, to further improve the code, consider adding    \n",
       "more comprehensive error handling, such as checking for None or non-comparable elements in the list. Additionally, \n",
       "the merge function could be optimized by using a more efficient merging strategy, such as using a temporary list to\n",
       "store the merged result instead of appending to the result list. Overall, the candidate's work demonstrates a      \n",
       "strong understanding of the merge-sort algorithm and Python programming principles.                                \n",
       "
\n" ], "text/plain": [ "\u001b[1mCritique:\u001b[0m The candidate's implementation of the merge-sort algorithm is excellent, with clear and concise code that\n", "handles various edge cases, including empty lists, single-element lists, and lists with duplicates. The inclusion \n", "of a short docstring with an example is also commendable. However, to further improve the code, consider adding \n", "more comprehensive error handling, such as checking for None or non-comparable elements in the list. Additionally, \n", "the merge function could be optimized by using a more efficient merging strategy, such as using a temporary list to\n", "store the merged result instead of appending to the result list. Overall, the candidate's work demonstrates a \n", "strong understanding of the merge-sort algorithm and Python programming principles. \n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
› Iteration 4  ·  score 8/10\n",
       "
\n" ], "text/plain": [ "\u001b[1;35m›\u001b[0m \u001b[1mIteration \u001b[0m\u001b[1;36m4\u001b[0m\u001b[1m · score \u001b[0m\u001b[1;36m8\u001b[0m\u001b[1m/\u001b[0m\u001b[1;36m10\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Critique: The candidate's implementation of the merge-sort algorithm is mostly correct and well-structured.        \n",
       "However, there are a few areas that could be improved. Firstly, the input validation checks are overly restrictive \n",
       "and do not align with the problem statement, which allows for empty lists and lists with duplicates. Secondly, the \n",
       "merge function can be simplified by using a more Pythonic approach with lists. Lastly, the docstring could be more \n",
       "detailed and include information about the time and space complexity of the algorithm. Overall, the code is        \n",
       "readable and well-organized, but could benefit from some refinements to make it more robust and efficient.         \n",
       "
\n" ], "text/plain": [ "\u001b[1mCritique:\u001b[0m The candidate's implementation of the merge-sort algorithm is mostly correct and well-structured. \n", "However, there are a few areas that could be improved. Firstly, the input validation checks are overly restrictive \n", "and do not align with the problem statement, which allows for empty lists and lists with duplicates. Secondly, the \n", "merge function can be simplified by using a more Pythonic approach with lists. Lastly, the docstring could be more \n", "detailed and include information about the time and space complexity of the algorithm. Overall, the code is \n", "readable and well-organized, but could benefit from some refinements to make it more robust and efficient. \n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "for i, step in enumerate(result.trace, 1):\n", " print_step(f\"Iteration {i} · score {step['score']}/10\")\n", " print_md('**Critique:** ' + step['critique'])\n", " print()" ] }, { "cell_type": "markdown", "id": "3fec59df", "metadata": { "papermill": { "duration": 0.002013, "end_time": "2026-05-27T03:48:33.863775+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.861762+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 9 · What we just observed\n", "\n", "The cells above are live. The commentary below directly quotes the **actual** critiques the\n", "Nebius-hosted Llama-3.3-70B Critic produced on this run.\n", "\n", "### 9.1 · What the trace shows\n", "\n", "| Iteration | Score | Critic's main concern (one-liner) |\n", "|---|---|---|\n", "| 1 | 9/10 | One potential improvement could be to add some error checking to ensure that the input list contains only elements of the same ty… |\n", "| 2 | 9/10 | One potential improvement could be to consider adding a check for the input being a list, to ensure the function behaves correctl… |\n", "| 3 | 9/10 | However, to further improve the code, consider adding more comprehensive error handling, such as checking for None or non-compara… |\n", "| 4 | 8/10 | However, there are a few areas that could be improved. |\n", "\n", "This 4-round trace is a textbook demonstration of the Reflection failure modes we describe\n", "generically in § 11 — except now we have evidence.\n", "\n", "\n", "**(a) Plateau / regression.** Across 4 round(s) the score went 9 → 9 → 9 → 8/10. The trajectory **regressed** (final score lower than first). This is a textbook plateau: the Refiner keeps rewriting against fresh critiques, but the *quality dimension being scored* doesn't move. **2–4 rounds is the sweet spot** — more is wasted budget.\n", "\n", "**(b) Critique drift — the Critic's focus moved each round:**\n", "\n", "- Round 1 → *“One potential improvement could be to add some error checking to ensure that the input list contains only elements of the same ty…”*\n", "- Round 2 → *“One potential improvement could be to consider adding a check for the input being a list, to ensure the function behaves correctl…”*\n", "- Round 3 → *“However, to further improve the code, consider adding more comprehensive error handling, such as checking for None or non-compara…”*\n", "- Round 4 → *“However, there are a few areas that could be improved.”*\n", "\n", "In a stable rubric, the same flaws would be flagged repeatedly until fixed. Here the Critic finds *whatever is salient relative to the current draft* — that's drift, and it's why same-model Critics often need an external rubric to stay anchored.\n", "\n", "**(c) Score regression.** Round 4 scored 8/10, *worse* than round 1's 9/10. The Refiner over-corrected against accumulated nitpicks and produced a draft the Critic liked **less** than the original. **In production you want early-stop when consecutive scores fail to improve** — see § 11.3.\n", "\n", "### 9.2 · What this specific run teaches\n", "\n", "You can read this trace two ways:\n", "\n", "1. **The pessimistic read.** Same-model Reflection at a high `target_score` finds something to\n", " complain about every round, even when the draft is already good. The Critic doesn't have a\n", " stable definition of \"good\" — it has a *relative* one tied to whatever draft it's looking at.\n", "\n", "2. **The optimistic read.** Even with a generic rubric, the **first** refinement round was almost\n", " always substantive — the Refiner genuinely improves on round 1. The marginal value falls off\n", " fast after that.\n", "\n", "**Practical takeaway:** for production, use `target_score ≈ 8–9` (not 10), cap\n", "`max_iterations=2`, and write a **domain-specific rubric** so the Critic stays anchored. Generic\n", "loops at high target scores are how you burn budget for negative ROI — exactly as this run\n", "demonstrates." ] }, { "cell_type": "markdown", "id": "b8e82f58", "metadata": { "papermill": { "duration": 0.008195, "end_time": "2026-05-27T03:48:33.888541+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.880346+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 10 · Try other providers\n", "\n", "The whole notebook above ran on Nebius. Because every line uses `get_llm()` and never imports a provider-specific class, the same code runs unchanged on any provider with structured-output support: OpenAI, Anthropic, Groq, Together, Fireworks, Mistral, Google.\n", "\n", "To test, drop the relevant key into `.env`. The cell below is **gracefully gated** — it skips providers you don't have keys for instead of crashing." ] }, { "cell_type": "code", "execution_count": 7, "id": "6d82ef4b", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T03:48:33.912879Z", "iopub.status.busy": "2026-05-27T03:48:33.912879Z", "iopub.status.idle": "2026-05-27T03:48:33.930168Z", "shell.execute_reply": "2026-05-27T03:48:33.929002Z" }, "papermill": { "duration": 0.033374, "end_time": "2026-05-27T03:48:33.930168+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.896794+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[skip] openai: no API key in .env\n", "[skip] anthropic: no API key in .env\n", "[skip] groq: no API key in .env\n" ] } ], "source": [ "from agentic_architectures.llm.factory import provider_supports_structured_output\n", "\n", "PROVIDERS_TO_TRY = [\"openai\", \"anthropic\", \"groq\"]\n", "\n", "for p in PROVIDERS_TO_TRY:\n", " key = settings.api_key_for(p)\n", " if key is None or not key.get_secret_value():\n", " print(f\"[skip] {p}: no API key in .env\")\n", " continue\n", " if not provider_supports_structured_output(p):\n", " print(f\"[skip] {p}: provider doesn't support reliable structured output\")\n", " continue\n", "\n", " print_header(f\"Re-running Reflection on {p}\")\n", " other_llm = get_llm(provider=p)\n", " other_arch = Reflection(llm=other_llm, max_iterations=2, target_score=9)\n", " r = other_arch.run(\"Write a single-sentence definition of recursion that an absolute beginner could understand.\")\n", " print(r.output)\n", " print(f\" iterations: {r.metadata['iterations']}, final score: {r.metadata['final_score']}/10\")\n", " print()" ] }, { "cell_type": "markdown", "id": "5a5a7290", "metadata": { "papermill": { "duration": 0.008083, "end_time": "2026-05-27T03:48:33.951938+00:00", "exception": false, "start_time": "2026-05-27T03:48:33.943855+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 11 · Failure modes, safety, extensions\n", "\n", "### 11.1 · Where this breaks\n", "\n", "| Failure | Mechanism | Mitigation |\n", "|---|---|---|\n", "| **Sycophantic critic** | Critic shares Generator's framing → finds no flaws | Use a *different* model in the Critic seat (passes a separate `llm=` to `LLMJudge`) |\n", "| **Critique inflation** | \"Find what's wrong\" prompt finds something even in perfect drafts | Set `target_score` high enough that \"fine, ship it\" is a possible outcome (>= 9) |\n", "| **Loop oscillation** | Two reasonable drafts trade places forever | Cap `max_iterations` (already enforced) |\n", "| **Plateau without progress** | All remaining critique is cosmetic | Early-stop when consecutive scores differ by < 1 (extension idea) |\n", "| **Cost blowup** | Hard tasks blow through budget | Use `target_score` not `max_iterations` as the primary knob; set both |\n", "\n", "### 11.2 · Production safety\n", "\n", "- **Never let an unbounded Reflection loop go to production.** Always cap `max_iterations` and add a per-call timeout.\n", "- For *code* outputs, **run the code** as a hard check, not just an LLM critique. That's the PEV pattern in [notebook 06](./06_pev.ipynb).\n", "- For *factual* outputs, ground the Critic with retrieval. That's the bridge to [Self-RAG (notebook 25)](./25_self_rag.ipynb) and [Corrective RAG (notebook 24)](./24_corrective_rag.ipynb).\n", "- The Critic can hallucinate \"issues\" that don't exist. Treat critique scores as **noisy signals**, not ground truth.\n", "\n", "### 11.3 · Three extensions to try\n", "\n", "1. **Stronger Critic.** Construct the `LLMJudge` with a separate (larger or different-family) LLM and inject it into `Reflection`. Single biggest quality lever.\n", "2. **Domain-specific rubric.** Replace the generic Critic rubric with one tuned to your domain. The Critic only finds what the rubric describes.\n", "3. **Best-of-N + Reflection hybrid.** Sample `N` initial drafts in parallel, run Reflection on each, pick the highest-scoring final. Costs N× but often produces materially better outputs than just N rounds of Reflection on one draft.\n", "\n", "### 11.4 · What to read next\n", "\n", "- [**18 · Reflexion**](./18_reflexion.ipynb) — Reflection + memory across episodes. The pattern you actually want for any task you'll do *many times*.\n", "- [**20 · Chain-of-Verification**](./20_chain_of_verification.ipynb) — single-pass alternative when latency matters more than quality.\n", "- [**09 · Tree of Thoughts**](./09_tree_of_thoughts.ipynb) — search-based generalization of Reflection.\n", "- [**15 · RLHF / Self-Improvement**](./15_rlhf_self_improvement.ipynb) — Reflection where good outputs become training data.\n", "\n", "### 11.5 · References\n", "\n", "1. Madaan, A. et al. *Self-Refine: Iterative Refinement with Self-Feedback.* NeurIPS 2023. [arXiv:2303.17651](https://arxiv.org/abs/2303.17651)\n", "2. Shinn, N. et al. *Reflexion: Language Agents with Verbal Reinforcement Learning.* 2023. [arXiv:2303.11366](https://arxiv.org/abs/2303.11366)\n", "3. Saunders, W. et al. *Self-critiquing models for assisting human evaluators.* 2022. [arXiv:2206.05802](https://arxiv.org/abs/2206.05802)\n", "4. LangGraph state graph & conditional edges API — [official docs](https://langchain-ai.github.io/langgraph/)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.0" }, "papermill": { "default_parameters": {}, "duration": 80.805423, "end_time": "2026-05-27T03:48:35.044028+00:00", "environment_variables": {}, "exception": null, "input_path": "all-agentic-architectures/notebooks/01_reflection.ipynb", "output_path": "all-agentic-architectures/notebooks/01_reflection.ipynb", "parameters": {}, "start_time": "2026-05-27T03:47:14.238605+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }