{ "cells": [ { "cell_type": "markdown", "id": "76ad4adb", "metadata": { "papermill": { "duration": 0.010219, "end_time": "2026-05-27T16:32:11.526857+00:00", "exception": false, "start_time": "2026-05-27T16:32:11.516638+00:00", "status": "completed" }, "tags": [] }, "source": [ "# 19 · Self-Discover — agent composes its own reasoning structure\n", "\n", "> **TL;DR.** Instead of relying on a fixed prompt like \"think step by step\", the agent picks reasoning modules from a library, rephrases them in task-specific language, then composes them into an explicit step-by-step plan — *then* solves. Four stages: **SELECT → ADAPT → IMPLEMENT → SOLVE**.\n", ">\n", "> **Reach for it when** the task type benefits from explicit, transparent reasoning structure (logic puzzles, multi-step math, planning) and you want a *recipe* the agent can re-run on similar instances.\n", "> **Avoid when** the task is simple enough for plain CoT or when latency budget is tight (this is 4 LLM calls per task).\n", "\n", "| Property | Value |\n", "|---|---|\n", "| Origin | Zhou et al., *Self-Discover* (Google DeepMind, 2024). [arXiv:2402.03620](https://arxiv.org/abs/2402.03620) |\n", "| Stages | SELECT (pick modules) → ADAPT (rephrase per task) → IMPLEMENT (write plan) → SOLVE (execute plan) |\n", "| Module library | 16 atomic reasoning modules (curated from the paper's 39) |\n", "| LLM-as-Scorer? | **None** — no numeric judgement anywhere, so no flat-scoring pathology to fix |\n", "| Default LLM in this nb | **Qwen3-235B-Thinking** (reasoning model — per handoff §10) |\n", "| Cost | 4 structured-output LLM calls per task; ≈30-60s with Qwen-Thinking |\n", "\n", "**Why this is different from Tree of Thoughts (nb 09).** ToT enumerates *candidate next-thoughts at every reasoning step* and scores them with an LLM-as-Judge. Self-Discover enumerates *reasoning modules upfront*, designs a plan once, then runs the plan top-to-bottom. ToT is search-in-the-thought-space; Self-Discover is structure-design-then-execute.\n", "\n", "**Why this is different from Planning (nb 04).** Planning produces an execution plan over *tools or actions*. Self-Discover produces a *reasoning* plan — the steps are cognitive moves, not external calls. The two compose nicely: Self-Discover designs how to think, Planning designs what to do." ] }, { "cell_type": "markdown", "id": "2bc2ca44", "metadata": { "papermill": { "duration": 0.003269, "end_time": "2026-05-27T16:32:11.548812+00:00", "exception": false, "start_time": "2026-05-27T16:32:11.545543+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 2 · Architecture at a glance\n", "\n", "```mermaid\n", "flowchart LR\n", " A([task]) --> S[SELECT
pick 3-6 atomic
reasoning modules]\n", " S --> AD[ADAPT
rephrase modules in
task-specific language]\n", " AD --> IM[IMPLEMENT
compose modules into a
step-by-step reasoning plan]\n", " IM --> SO[SOLVE
execute the plan,
produce final answer]\n", " SO --> Z([final answer])\n", "\n", " L[(MODULE_LIBRARY
16 atomic modules)]\n", " L -.menu.-> S\n", "\n", " style S fill:#e3f2fd,stroke:#1976d2\n", " style AD fill:#fff3e0,stroke:#f57c00\n", " style IM fill:#fce4ec,stroke:#c2185b\n", " style SO fill:#e8f5e9,stroke:#388e3c\n", " style L fill:#f3e5f5,stroke:#7b1fa2\n", "```\n", "\n", "Each stage is a single structured-output LLM call. SELECT picks indices into the library; ADAPT rewrites each picked module; IMPLEMENT composes a plan; SOLVE executes." ] }, { "cell_type": "markdown", "id": "e2291b24", "metadata": { "papermill": { "duration": 0.008193, "end_time": "2026-05-27T16:32:11.567758+00:00", "exception": false, "start_time": "2026-05-27T16:32:11.559565+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 3 · Theory\n", "\n", "### 3.0 · Why discover the structure instead of fixing it?\n", "\n", "Chain-of-thought (\"think step by step\") is a fixed template. It works because LLMs are good at *unfolding* a known structure (linear reasoning). It fails when the task needs a *different* structure — make a table, consider counterexamples, set up an algorithm. The agent can't *invent* the structure if its only prompt is \"step by step\".\n", "\n", "Self-Discover treats reasoning structure as something the agent *designs*, then *follows*. The module library makes the design space explicit; the four-stage pipeline forces the agent to commit to a structure before producing an answer.\n", "\n", "### 3.1 · The four stages — why each one matters\n", "\n", "| Stage | Input | Output | What it buys |\n", "|---|---|---|---|\n", "| **SELECT** | task + module library | selected module ids + rationale | Forces commitment to a *type* of reasoning before any concrete answer is on the table — avoids fitting the modules to a guess. |\n", "| **ADAPT** | task + selected modules | each module rephrased in task-specific terms | Bridges from abstract module (\"make a table\") to concrete instruction (\"make a table with one row per musician, one column per constraint\"). |\n", "| **IMPLEMENT** | task + adapted modules | step-by-step plan with expected output per step | Sequencing decision is explicit — which adapted module first, which expected output type. Plan is reusable. |\n", "| **SOLVE** | task + plan | step outputs + final answer | Executes against the plan. The model can produce one structured solution rather than meandering. |\n", "\n", "### 3.2 · Where this sits\n", "\n", "| Pattern | Reasoning structure decided by... | When |\n", "|---|---|---|\n", "| Plain Chain-of-Thought | Fixed prompt (\"think step by step\") | Tasks linear reasoning suits |\n", "| [Reflection (nb 01)](./01_reflection.ipynb) | Fixed loop (gen → critique → refine) | Quality matters, no structure choice needed |\n", "| [Tree of Thoughts (nb 09)](./09_tree_of_thoughts.ipynb) | Fixed beam-search over next-thoughts | Search where the right framing is uncertain |\n", "| [Planning (nb 04)](./04_planning.ipynb) | LLM at runtime — over **tool calls** | Execution-time planning of external actions |\n", "| **Self-Discover (this nb)** | **LLM at runtime — over *reasoning moves*** | Task type benefits from custom cognitive structure |\n", "| [LATS (nb 22)](./22_lats.ipynb) | MCTS search + reward signals | Search with explicit value estimates |\n", "\n", "### 3.3 · No LLM-as-Scorer step → no flat-scoring pathology\n", "\n", "Recall from Mental Loop (nb 10 §11) and the deterministic-picker pattern: when an LLM is asked to emit a single numeric score, Llama-class models compress the band to `[4,4,4]` and the deciding signal becomes arbitrary.\n", "\n", "**Self-Discover has no numeric scoring anywhere.** SELECT picks a *list of indices* (categorical multi-pick), ADAPT/IMPLEMENT produce structured text, SOLVE produces an answer. Nothing scores anything. The architecture is immune to the flat-scoring pathology by construction.\n", "\n", "### 3.4 · Why a reasoning model (Qwen3-Thinking) is the right default\n", "\n", "Each stage produces a structured commitment — selected ids, adapted module text, plan steps, final answer. Reasoning models that use private `` tokens internally can deliberate before emitting the structured output, which tends to give cleaner selections and more on-topic plans. The architecture works with any chat model (we run a Llama comparison in § 10), but Qwen-Thinking is the natural pairing for the SELECT and IMPLEMENT stages." ] }, { "cell_type": "markdown", "id": "a32aeb58", "metadata": { "papermill": { "duration": 0.008154, "end_time": "2026-05-27T16:32:11.584211+00:00", "exception": false, "start_time": "2026-05-27T16:32:11.576057+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 4 · Setup" ] }, { "cell_type": "code", "execution_count": 1, "id": "df562821", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:32:11.601851Z", "iopub.status.busy": "2026-05-27T16:32:11.601851Z", "iopub.status.idle": "2026-05-27T16:32:14.433056Z", "shell.execute_reply": "2026-05-27T16:32:14.433056Z" }, "papermill": { "duration": 2.840341, "end_time": "2026-05-27T16:32:14.433056+00:00", "exception": false, "start_time": "2026-05-27T16:32:11.592715+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
Reasoning LLM: Qwen/Qwen3-235B-A22B-Thinking-2507-fast ────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;36mReasoning LLM: Qwen/Qwen3-235B-A22B-Thinking-\u001b[0m\u001b[1;36m2507\u001b[0m\u001b[1;36m-fast\u001b[0m \u001b[92m────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from agentic_architectures import get_llm, enable_langsmith, settings\n", "from agentic_architectures.architectures import SelfDiscover\n", "from agentic_architectures.ui import print_md, print_header, print_step\n", "\n", "enable_langsmith()\n", "\n", "# Per handoff §10, nb 19 defaults to Qwen3-Thinking (reasoning model).\n", "reasoning_llm = get_llm(\n", " provider=\"nebius\",\n", " model=\"Qwen/Qwen3-235B-A22B-Thinking-2507-fast\",\n", " temperature=0.4,\n", ")\n", "print_header(f\"Reasoning LLM: {reasoning_llm.model}\")" ] }, { "cell_type": "markdown", "id": "a5a29ae4", "metadata": { "papermill": { "duration": 0.009077, "end_time": "2026-05-27T16:32:14.450150+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.441073+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 5 · Library walkthrough\n", "\n", "Source: [`src/agentic_architectures/architectures/self_discover.py`](../src/agentic_architectures/architectures/self_discover.py).\n", "\n", "The architecture exposes one constant and four schemas:\n", "\n", "- **`MODULE_LIBRARY`** — list of 16 atomic reasoning modules (replaceable via `modules=` constructor arg).\n", "- **`_SelectedModules`** — Stage 1 output: indices into the library + rationale.\n", "- **`_AdaptedModules`** — Stage 2 output: one rephrased module per selected.\n", "- **`_ReasoningPlan`** — Stage 3 output: step list + final-answer format.\n", "- **`_Solution`** — Stage 4 output: per-step outputs + final answer.\n", "\n", "Each is bound to the LLM via `with_structured_output()` in `__init__`, so the four nodes are trivial wrappers around `invoke(prompt)`." ] }, { "cell_type": "code", "execution_count": 2, "id": "43b376c7", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:32:14.458772Z", "iopub.status.busy": "2026-05-27T16:32:14.458772Z", "iopub.status.idle": "2026-05-27T16:32:14.480769Z", "shell.execute_reply": "2026-05-27T16:32:14.480769Z" }, "papermill": { "duration": 0.026009, "end_time": "2026-05-27T16:32:14.480769+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.454760+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MODULE_LIBRARY size: 16\n", " [ 0] Critical thinking — question the question's assumptions and surface biases\n", " [ 1] Break the problem into sub-problems and solve each\n", " [ 2] Identify the type of problem (logic, math, classification, planning, ...)\n", " [ 3] Identify the goal — what specifically must the final answer contain\n", " [ 4] List the relevant facts, constraints, and unknowns\n", " [ 5] Consider analogies — what problem is this structurally similar to?\n", " [ 6] Brainstorm alternative interpretations of the question\n", " [ 7] Step-by-step reasoning from premises to conclusion\n", " [ 8] Consider counterexamples that would falsify a candidate answer\n", " [ 9] Make a table, list, or diagram to organize the information\n", " [10] Reverse-engineer from a candidate answer back to the premises\n", " [11] Evaluate trade-offs between competing options explicitly\n", " [12] Devise an algorithm or procedure that always finds the answer\n", " [13] Synthesize multiple perspectives or stakeholder views\n", " [14] Check consistency — does the candidate answer satisfy EVERY constraint?\n", " [15] Self-verify — solve a second way and compare the two answers\n", "\n", "--- _SelectedModules schema ---\n", "{\n", " \"description\": \"Stage 1 output.\",\n", " \"properties\": {\n", " \"selected_ids\": {\n", " \"description\": \"Indices (0-based) into the MODULE_LIBRARY of the atomic reasoning modules most relevant to this task type. Pick 3-6 \\u2014 enough to structure the reasoning, few enough to keep the plan focused.\",\n", " \"items\": {\n", " \"type\": \"integer\"\n", " },\n", " \"maxItems\": 8,\n", " \"minItems\": 2,\n", " \"ti...\n", "\n", "--- _ReasoningPlan schema ---\n", "{\n", " \"$defs\": {\n", " \"_PlanStep\": {\n", " \"properties\": {\n", " \"step_number\": {\n", " \"description\": \"1-indexed step number.\",\n", " \"minimum\": 1,\n", " \"title\": \"Step Number\",\n", " \"type\": \"integer\"\n", " },\n", " \"description\": {\n", " \"description\": \"What the solver does in this step. Reference which adapted module(s) this step draws on.\",\n", " \"title\": \"Descripti...\n" ] } ], "source": [ "from agentic_architectures.architectures.self_discover import (\n", " MODULE_LIBRARY, _SelectedModules, _ReasoningPlan,\n", ")\n", "import json\n", "\n", "print(f\"MODULE_LIBRARY size: {len(MODULE_LIBRARY)}\")\n", "for i, m in enumerate(MODULE_LIBRARY):\n", " print(f\" [{i:2d}] {m}\")\n", "print()\n", "print('--- _SelectedModules schema ---')\n", "print(json.dumps(_SelectedModules.model_json_schema(), indent=2)[:400] + '...')\n", "print()\n", "print('--- _ReasoningPlan schema ---')\n", "print(json.dumps(_ReasoningPlan.model_json_schema(), indent=2)[:400] + '...')" ] }, { "cell_type": "markdown", "id": "296c0a5a", "metadata": { "papermill": { "duration": 0.015901, "end_time": "2026-05-27T16:32:14.496670+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.480769+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 6 · State" ] }, { "cell_type": "markdown", "id": "279eccac", "metadata": { "papermill": { "duration": 0.007373, "end_time": "2026-05-27T16:32:14.511099+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.503726+00:00", "status": "completed" }, "tags": [] }, "source": [ "| Field | Set by | Notes |\n", "|---|---|---|\n", "| `task` | caller | The single task to solve |\n", "| `selected_ids` / `selected_modules` / `selection_rationale` | `_select` | Stage 1 commitment |\n", "| `adapted_modules` | `_adapt` | List of `{original, adapted}` pairs |\n", "| `plan_steps` / `final_answer_format` | `_implement` | The reasoning recipe |\n", "| `step_outputs` / `final_answer` | `_solve` | Stage 4 execution result |\n", "| `history` | every node (Annotated `operator.add`) | One entry per stage |" ] }, { "cell_type": "markdown", "id": "61b9dcf7", "metadata": { "papermill": { "duration": 0.005017, "end_time": "2026-05-27T16:32:14.521057+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.516040+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 7 · Build the graph" ] }, { "cell_type": "code", "execution_count": 3, "id": "8730ba8f", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:32:14.536760Z", "iopub.status.busy": "2026-05-27T16:32:14.534750Z", "iopub.status.idle": "2026-05-27T16:32:17.355880Z", "shell.execute_reply": "2026-05-27T16:32:17.355880Z" }, "papermill": { "duration": 2.834823, "end_time": "2026-05-27T16:32:17.355880+00:00", "exception": false, "start_time": "2026-05-27T16:32:14.521057+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHgAAAITCAIAAABDqk81AAAQAElEQVR4nOydB2AU1dbH78xsNrubSnpCQgq9t1BVEAIIqIB06UgRUD4BUR8CAuoTEAsqDxAREBAB4T2ainSlo3QSKSEJCZAESG+b3ezMd2Zn2WzCJmQ2d4ed5f4eL87O3Gn/uXPuuWXOVXAchwj2R4EIkkCElggitEQQoSWCCC0RRGiJsIvQl49n3bpaWJhjKNFzeh1HcYhiKJblKB5E0ZShhKVpxLJ8YlgDHiYNCQy8o8nQtMG4QcHQJQZ+AfZANOJYfqvSldIVm/xRBo5jXMnAvpwpAZwAHFb+NMjotxrTwhlhK03z12D8+3BDKRzD0K5qytPPJaqZW93mngg3FEY/+uDmtKS4Qm0BC7epVNEgCs3QbAliOf424FZBcho0gFstAWkRMt4wB0JykBKxBv4gZsXNC/B/mhea38ookUH38HwMhYwJKIY/DMdSxv35w1I0vyjswicw7i7ITT08FMc/QhMcfyn8Vr2O5XMGhdTuTJOOntHdfREm8Aj927q7SbGFCiUVEqXq0MfPx98VyZmbsbkXDubcv1MMD7tFZ++2L2CQG4PQq96/CTmu/Yt+TTp4Iefiz/+lx53M03gwo+ZGoupRLaHPHso4uTurUTuPrkMDkfPy329S0pKKp3xeB1UD24XOvl+0afGdKZ9V6/RyIfZU9uEtD974oraxmLUFG4U+9fv98wdyJi95KlQWKCkpWflu0pTPomiaRuKxZZ/7qYVnf3+6VAYUCkW3YQHfvpeIbMIWobd9ebdND2cr96pCg2jPgDDlDx/aorVoobd9laJ2o9v29EdPJQP+L6wo33Dy1wdIJKKFhvJ34Fuh6CmmcUfPi0eykUjECQ2OjtqTdvdWoqeY5/oFQCX+7/0ZovYSJ3RacnGzjvjbAWRHYLjq8rEcUbuIEDohNg+aI9q84Ick5ObNmy+99BISz9atW+fNm4fsQ9te3gV5rKhdRAgdeyJHpbHRXbeZuLg4ZBM271gVwuq4MwoUd0qEpRbRTJp9v8Td217t13l5eStXrjx27FhmZmajRo169erVr18/WLN69WrYGh0dPX369OHDhx89evT3338/f/58Tk5OkyZNxo8fD5sgQXx8/NChQ5cuXfrxxx/XqFHDw8Pj3LlzsP6XX37ZuHFjgwYNEG5clNStq0WN2ntXMb0I4bQFJT7BamQfFixYkJ6ePmvWrMjISHjrFy5cGBUVNWnSJJ1Ot2/fvj179vAXoNXOmTOnbdu2kBh+HjhwANTfsWOHr6+vi4sLrIGnMnLkyBYtWjRu3HjMmDHh4eFCSnvg6sbkZeqqnl5MDuUotcYF2QfIgKNGjWrfvj0sT506tVu3bt7e5TOLSqXavHmzWq0WNkGO3rZt24ULF2JiYoQmCNgdcj2SBBeGKtGJMKQihLbrSBvIhvCOZ2dnt2rVqkOHDg0bNrSarKCgYNmyZWfPnn3wwFRlyMrKMm+taC97AL1GLCeiPBRRGEIreHGRAdmH+fPnDxs27OTJkzNmzOjevfuKFSugEadcmrS0NDDKer3+k08+gZSnTp0ql8DVVboOhxI9y7iIUE9EjlYoqNxMPbIPnp6er7322tixYy9evHj48OHvv/8eCrQRI0ZYptm/fz+YbDC7YD1Q2bwsPdoiQ2CACEMqQmgvf5fMNBHmv+qAC7F3796+ffuCFW5h5Nq1a1evXn00GTwPQWXg4MGD6MlRXMiF1RXhGojI/PVauxfli/PSqwi0QK5ateq9996D7JyRkQE+GagMcsOmWrVqgTk+cuTIrVu36tatC8vbt28Hq3LixIkzZ85AqQj2xOoxw8LCrly58tdff4G/iHCTdb8YjHTL532qvgsDxrGKSQNCVWf2Zbp7M/6hKoQVpVLZtGlTsAxr166FIjElJWXChAngR4Mv4efnB1WPdevWgaZDhgwxGAybNm36+uuvwW7Mnj27sLBww4YNoH6zZs22bNnSu3fv0FBTgxd40+B0//TTT+3atTOvxMXeH1ILcg1teogQWlwPy6bFt0r03Kg5Eejp5j9vx9eP9uj2qoieUnGNSv2m1MzNKEFPN5eP84WwKJWR2JFKGg+Few1mwydJI9+PsJoAHIaKKmNeXl5QmlndBFZi2rRpyD7AkaFSY3VTcXFxRR4hVDLr1LHeV3dsZ0ZkE9E1ZFs6Z/8zI77PpMCweh6PboJiqqioyOpe4P8KFeVHgfXgbyD7AHYcLLvVTVCnr+i8Go2GYZhH1/++IfVWXMHEhaL7S20R+uzBzNO/ZT4lAw0s0ekMq95LfPNLW27cls7Z1jE+YfXVaz64iZ4yvp+d2HmQCE/DEtsH0Fw5lf3ntgdPT75eNj1+8NuhAba6ttUaErZ71d3bNwp7jgqMbOqBnJejO+9fPJLTfWRA/Va2d+NVd5DjxWNZJ3ZkeAYohr8bgZyO1MSi39al6YsNY+aFuaqr1WKFZ9juj4sTs+8ZPHyYZs95tuiEbUzxE+TYznvXz+VDk0PNuqp+kzBULLENRAfvbfvXd7LSSuBwrmpK4+micWdcVLTFcG8eBbSXGyzOSBlHjnPmXxSHhFHo/FqWM+0rfBUA7hb4acKyCc44gh/aho3j/EuTQtuCgjKUGIepC6P7OT4Rv57hDAaLSzKmN14DW5hbAm1yhTkGXTFSuKDgSFXfSdjq7jhH/AskXs7952x+ZqpeV8R/WmEoW5GErja9rvSMcJeIpjmz0g9loY2fYqCyq03fAFCINbCmkYYs/9EGfwum/eCxUaxxyUVJ63VCExgnnEd44uZPC4Q1QnpoVFG48p8IaLwUATVdW3bx9sXdaYdfaHtz9+7d119/fffu3UhWyO+rLKh8QrMqkhtEaIkgQkuE/K64ksYpR4bkaIkgQksEEVoiiNASQYSWCCK0RBChJYIILRGkwiIRJEdLBBFaIojQEkFstESQHC0RtoxUerLINEfLT2hioyWCCC0RRGiJIEJLhEajkfLDTVzIT2itVlvRRwWOjAzfQYXi0a+XHR8itEQQoSWCCC0R8qsZQv0bauFIbshPaGI6JIIILRFEaIkgQksEEVoiiNASQYSWCCK0RBChJYIILREyFVo2X86OGDEiNjbWPLmbEPWVZdnz588jOSCbto6pU6f6+/vTNM0wDPwFoUFlIXi0LJCN0O3atWvUqJHlGg8Pj9GjRyOZIKfWu7Fjx/r4lIY0ioqKevbZZ5FMkJPQzZs3b9WqlbCsVCpHjhyJ5IPM2qPHjBkTGMiHUIyMjIyJiUHyAb/XkXy94Pq5PJ3WeHTaNEMpbYxMYpyJlBLCmCAhqAwfFIUqDWxi/I8wk51FtBMTwjS1cXGxqWlpDRs0CAmpaVzJ34LlTZhi0BhnWC1zf/zUtpRwfGEluDBGBUr3hetx1VDRPb3c3R07MMr3H8QXFyIXV0pfzP8snUDWKAc/syst+Gf8VqPQxnl5+TA9xlg8grQPZ4wVpuM1Xahx6lhhPmDOOImtsAlOwc9ma5rEl3+ulOlYpudkvj/KeGR+k3HyWYRM57MUgGY4OKC+mPMOYIa/V91ZOS3BKfS3s+J9ghQ9x0Qg+fPzl/Ead5ehM8MRJrAJ/d378aH1VM++4jzzle1Yfoui2RGY8jWewvD0b/dYA3ImlREfwzk8O92g0+GJto9H6KR/tGov+TWbPBalKzq5G8+UDXjU0ReJmpNENrAGqliLsIBHaAOLTBPNOxfgMnEleHKQE77vjgkRujLA44Z/CAd4CkNw8ikntBx8VdYinGe1wCM0b8tkFnmzShirl3hyEDEdlcHxcWjx5CA8QjMMZa9p4Z4ofI52KBttANPhjH40n6NZR8rRzgpkZ5omNtr+QHZmHSpHg412Rsth9KMZR8rRRhvthI4070cbHMmPNtoxuzvS/fp3W79hNZIneISG9wv6lpADk5h4c+iwl9CTA5PpKHF09+7a9TgkHooW/o+BJ+Z1JCcnrV238sLFs1B5b9y42dDBo5o25SeZLSkp+X7N8lOnj927l9akSYtX+g5u397KKJnY2Es/rF919Wqsl3eNDu2fGz1qopubm7Dp5MmjX32z+P79e3Vq1+vXb3Cvnn3gRILN6RITPWXy9EEDqzpJu7FdwZFsNKOgrE3+VyHQPzRtxkSGYRYv+ubzJSsUjGL2nOlaLd/G/vU3n27bvumVfkM2/bi7c6eYeQve/ePP8rMl376TMvPdKdpi7bJv1n604LOEhBvTZ0wUhpiCynPnzRz32huLFn797LNdPl3y4YGDe8eOmTR0yKjAwKDDB/+uuso8HLZ55/HkaLaEY8WYjpSUW1lZmQP6v1qvbgP4Oe+DRRcvnQOliouLf9+3Z9irY/q8PADW9+7V98qVi+s3fAeKW+5+4MBvLgoXkNjLyxt+znx77qvDXz52/MjznbtB5u30XNfu3XrB+jbR7QsK8gsLC5ADgCdHC6Mqqp4+NLSWt3eNRZ/O3/jjGpCSpumWLaLd3d2vX/8HMnub6A7mlC2at05IiM/JLTOHamzsxQYNGgsqA0FBwSEhoZcun2dZ9mbCDdhkTjnp9beEZ2Yb5vHB1QdPjhZ7Ma6url99+d0vv+4AKwEWGWQaM2pi9+698/PzYOvUt8aVS5+VmeHl6WX+CcmuXosDg1suDRgf0NrVFdusqhy+ugEeoTnxjYm1akVMnjQNrOe5c2d+27vrk0UfhEdE+fr5w6a3Z8yuWTPMMnFAQJDlTx9fPyg5YV/LlV6e3vD84OUAc4FwwXKcQ9loxoXiSkQ8fXA5YuMugT+gUqk6duzUrt0zPXs/A3aja5cXhHhJYEmElGDK4VY1Go3l7rWj6u7b/0vzZq1MU5MhlJSUAOYIStf69RtdvlI6Z/J3q5eBLXpjygxkExgb/jH1sJRwBjGlYW5uDvgDK1YuBf8BCsYfN62FkrBJ4+Yg6JjRr0Ppd/nyBRAI/A3wLpZ+tajc7gMHDgcTsWz552ArYPdvV3392vghCYnxsKnvywP/+uvklq0bzl/4e+eubT9t/iEysjYylgoZGQ+OHTsC6at8mY7X8I9EPvkmTZrPmP7+uh++3frzRvgZ3brdF5+vjIiIgmXww2rXrrdp8zowKW5u7o0bNXv77Tnldvf08Px+9ZbNm394ffIIeDmg9Htn5lzBgXnhhZdy83LAxS4oKPD19Zs4YSq4LrC+fbtnmzZpAZ4feNxjRk9EkoNn7N0PHyVBo9KAadiGBDoIGz+Kr93co8dIcZPaWwVfzdAZO2eNPSwONYCGQk453MA4Ktuhelg45JTDDRyuCs4/dWfM0bx751A52ilzMxJstEPlaGzepoNh7AVHWMDUTMrQzlkYsohlHWsADeuUhaHxRXUk06FwodgSp8zS2MDW8G9wxoEdDtcejZxzeDTiOGzNpJiGG9DO6UdjBFNh6PDDDZ44eEyHUs1wJU44QloJPTYumAZzIRxoPKmiIicUWqdnA8LxTBeFR+geQ3y1Bc7mSMedzoRqYfNnfBAO8AitnY8gxgAAEABJREFU9lIHRyp/XBiPnIhzBzJbdfFGmMAZRuL03vvnDuUER2lC66tVKqXlJtrUkmpyTbiHTgr3qLdCIVNck7IYr5ITOszKbKYsjlvuMGUTmn+AK8qynKVHWrqJ4/JzdUn/5GXc1Q98q2ZAKLbwKJgDo/y9//7l4/nFhYaSslH4+XuzJsejSpvDw1QRsekr3wXUZ1w4lTvVY0RgSKQ7wodsAgyaSU1NnTBhwp49e5CsIJOSSQQRWiKI0BJBZlGWCJKjJYIILRFEaIkgNloiSI6WCCK0RBChJYIILRFEaIkgQksEEVoiiNASQSosEkFytESQHC0R8psXnORoiSBCSwQpDCWC5GiJ8Pb2LhdVQhbIT+icnJyCAocIkyQKMuesRBChJYIILRFEaImQn9DgRIMrjeSG/KrgJEdLBBFaIojQEkGElggitEQQoSWCCC0RRGiJIEJLBBFaImQqtGy+nO3Tp09ycjLDMCzLWn7Gfe7cOSQHZNPWMWXKFE9PT/5bbYahjcDKpk2bIpkgG6F79uwZGRlpuUalUg0dOhTJBDm13o0ePVqtLo3rEB4e3qtXLyQT5CR0165dGzZsKCyDARkwwPYJVqRHZu3R48aN8/DwQPzkCKH9+/dH8sFe7l1SbK7BYJw+i48awzs2lBBFhuL4/xkXhPlkuIdxZQQoPvLqw9/GNJRFeFB/dZN2jftcv3GjV+feSVeKHnGYOCGWNWf9Z7mkHC1cmXkNZ1C40hENcMZDMYPfvdu4MDE3wwD3Z7B0dh9GmuHERCK0mpgP7GNc+2ggGVEHt5qaVvDHDI507f9GGMIKZqFXz72p0tCdBwV5+2OLRiQxyddyju287xfiMuCNcIQPnEKvmh0fHK56fkgokj8/L72pcKFGvR+FMIGtMDyyPR2Mr3OoDAyaVjs/k72XXIQwgU3o5KsFHn5ONcu4Uo3O7H+AMIFNaH0xpZThsOVKYBQKXT62GMLY8mCJni1xrmDd+mJWp8NWgDnVy+7IEKErhI86Tzue6UBOFxMd4yQsCK/QzhbXmMM18xsPMR0VwrKgNCkM7Q+++WZ5sAlNOd0MIWCjKQcUmjfRziU1x3EYC3hsQmOckc5h4DAW8MRGV4hxwlmEC+JHSwTGPkPKfkqPHTf40WnY7Q3NUDS+fIhRaIc20a8M6H439Y6oXVgDx+IbevZU2Oi0tNTs7CwkGpxv6JMUOjHx5q7d286d/yst7W5EeFTv3v369hkobEpKSli0eN6t5MQWLaJHjRhvudfJk0cPHf790uXzubk5DRs0GTlyfMsW0bB+688bN/20buaMOV8s/QRkDQkJhR179Hjx/IW/Z7w9CRIMH9H3xd79Zr49p8oXiPMNfZIVlv8s/xwknjFjNuyanJz01deLAwOD27d7Rq/Xvzdrar26DRfMX1JUVLh23cqMDFNPh1ar/ffCOa1atv3Xewvg5x9/HJg9Z/rG9Tt8fHwZRlFQkH/w0N4fN+zUl+i3b9+06NP5DRs2gcew8N9LZ82e9uPGnSHBNat+eTRDU4wDtt5xomePnDt3YWFhQXBQCCyDHHv37jrz1wkQ+s+jh+7dS//qy9WBgUGw6f+mvjtoiGnol0qlWr1qs1qt9vLiJ1eCHL1z17bLVy507hSDjDFT+r8yVM2PG1OPGf36f/+7+eCh38eMnohsg3PIHM3ZUBpyHGhx+szxlJRbwopgY467cycFBA0KChZW+vr6BQQEmneCZ7P6+2UXLp41Z3NL+1uvnmnMGDx1sB7JyYnIVsCJdkw/WpzMLMv+6/239HrdhPFvgiH2cPeY+tY4YRMYX7W6TIwZV1eVsJCenvbW9PFgOubO/qRRo6agZvcX2pdN6Vq6rFKBMUG2wgnTe2HiiRWG129cvXo19rMly1u3aiusyc/P8/cLgAVPTy8wzZaJIRcLC0f+2K/T6cBAC8NKH/UlCgoK3NzchOVirbaGN55J8qrPExvkmJOTDX8FZZHRzYB/wnJQYDAUegkJpkn74uOvP3hwX1iGzO7h4WkevPvHnwfLHfb8hb+EheLi4uSUpMjI2shWGAVUWLAVhtiENnodIi4L/DmFQrFl64bcvFxwOb5ZtqRNdPu09FTY1LFjZ6VS+dkXH4PcIPGHH8+CPC7sFRVVF0zzrt3bodw7febEuXNnoFS8dy/NdDM0DUYfjmYwGNasXQFax3TtCevDakXA3yNH9t+Iv1blC8Rso7EJbZzXWYRFA49i9vsfx/1zuW+/ru/PmT5+3Bt9+gz8558ro8cOdHd3/+TfSw0lJS/16TzmtYEDBwwLDzeN9Y/p+sLIEePWb/gOTDM4cOCQdO/WG9znL778BBkf9uBBI2bMnNStR7vde7b/6935YWH8+LmaIaE9X3gZ3MRNm9ZW/Qrx9rBgG3v37aybNQJde419YkPCtv938/IVXxzcfwZhYsuSRI8aLkPexnNHpJm0Qhy0z9AZ20g5jHeFz+t40p2GA/oPxWg3EO91MIwjVsGdDkMJayhxPNPBcU7XZYgVjI1KzmelHbM9mnK+/OyQrXfOB7RHMw7ZZ+hssAbW4JB9hs42JIx21PHRzuZ0QMWQjCaVH0RoicAmtIuSUrg4lZVWuMBNOZ7pUKqo4iIDciKg1V/tiU0fbO5deBNN7gMdciK0BWynvjUQJrAJ/dzLgYwL9euaW8gp+PnzeN+ajLsPthgNmMNIrP84EfrrWnf3j2zsheTJ5eMPrhzNDm/s9sKIYIQP/IFRtn6RlJFWAnUqKwcu963CI58uPIxKYwXLMChlkj2McGN5QIsEplUMRRksLsgclkYIs2L6ySGGgWZoVKuhutcYEYPHqoK9Agzm5Oh0j0z/QyOKtWipEaLwlDu9ZWQey2WGRQajnYNe8MULFy357DNBQtCUfSg61ONY4Usajub4iEGlZ6RpihVqH8Ynxq/nk3FCAtoYXwiaoN09kdrdLiFd7OVHe3kpkX2MRzHLZhUm+4Uokawgs79JBBFaIojQEkGElghZToVKhJYCkqMlgggtEURoiSDTVUsEydESQYSWCCK0RBAbLREkR0sEEVoiiNASQYSWCJkWhvIbtktytEQQoSWC+NESQXK0RHh5ebm722WWJbsiP6Gzs7MLCgqQ3CBzzkoEEVoiiNASQYSWCPkJDU40uNJIbsivCk5ytEQQoSWCCC0RRGiJIEJLBBFaIojQEkGElggitETItGZIcrREUHIJCzhw4MDCwkKDwQB/i4qKXF1d9UbOnz+P5IBs2jp69+794MGDjIwMUBkZA56zLFunTh0kE2Qj9MiRI2vVqmW5hqbpPn36IJkgG6GhDBw0aJBSWfoJeERExIABA5BMkFMz6ZAhQ8yZmqKonj17ajQaJBNk1h49atQoQdyaNWv2798fyQeZCQ1FYng4Pz9Ct27dvL29kXx4jHt3YPPdxMtF+mLOUKUAYFWaDbeSMDMWR8IzVXCVzvWYK6nsnig+cDpy1VDR3b2bPetb8VEqrbAc2pp243xhZBOPeq3dacXjh7tRDy/sMclY/kXiHncoy9Ay1uGMpvoxoTopmuMqOQ7FmWIDcRUmqOxRMZQhP6/k2l85R3dkedRQRjb2qPhEFZxiy+e3crL1r86UjaP6xPnxk/gG0e7PDwqyutW6jb6TlJ+RSlQWR/t+PrGnK5wDzbrQZ37LUnswiCCG2o18wF6fOZBhdat1G63NMyiUZPZp0SgYKivNejhL60LrihHHEqFFo9chg966biSssUQQoXHCB4WswP23XhjyszUQyyEefqbBCtxl6zman62BTBNkAxXPnUlMB074eS4qaD0iQuOEYzlDBbOnWheaYSh8s60+RVBiTYfBwBE/2gYqmXaXmA6c0AzFEBstAayhQhttXX9+iihiOcRD0RVOr2U9R3OUs80wJg1cxRMv0xXuILLC0veVmPUbViMcJCTEd4mJvnz5ApIbxiq49U3YOmeHDB7ZrGlL5CwkJt4cOuwlJBJjFdz6JmyF4bBXxyAn4tr1OCSeSvxouoIdRBtps+n4346t/Qf2iI+/PuTVF7v1aDduwtC4uMsnTvz5cp/ne7347Afz3snOzhJ2ealP500/rZs3/10wFLA8a/a0vPy8R4+89/fdU94cA/vC323bN5mN2oIP//XhR7P27/+1R88OsHX6jNdzcrJ/WP9d125t+vXvtmLlUnPK2NhL7773Zp++XUaO7r98xZfm4AjCpSYnJ40dNxiuAS4VzgXr165bufjTBenpabDy520/oioDZxRvOmxtVHJxccnPz1u3/tvPPl2+e+cRvV7/yaIPftu7a/V3m3/csPPylQtbtm4QUjKMAm7jpZf6Hzrw16eLlsENf7NsSbmjHTi4F+65Xt0GmzbuGj/uDRB62fLPhU0KheJK7EX49/OW31Yu3wALb02fwLKGPbv+mPfBoq0/bzx9+jgku30nZea7U7TF2mXfrP1owWcJCTemz5goDPwVLvXrbz595+25cA2dO3X7dMmHoO/YMZOGDhkVGBh0+ODfgwYORyKgKppqsoLCsHqDeUHc0aMmhoWFq9Xqdm2fSU29M33aLLhuHx/fFs1b37x53ZyyTu16baLbw+vTqFHTvn0GHjmyv9wg819/3dGsWctpb/2rRg2fVi3bjB09aceOrVlZmcJWnU735hszvby8w8MjoyLrMAwDGmk0mpYtor29a9xMuAFpDhz4zUXhAhLXqhURERE18+25N+KvHTt+xPJS4exwDS/0eAnuOz7+GrIV8aaDrq5/FxEeJSzAbYNGILHwU63W5BeUdhXXqVPfvFwzJAxu++7d2+Y1LMtCPm0T3cG8pmXLNrDy0mXTmOiaNcPM8ZXUGo35pICbxi3faIhiYy82aNAYHoawPigoOCQk1HwEALYKCx4envA335r5qjqUqJqh0b2rltKWD6qSh+bqqjIvq9T83EkFBfkqlWkSJciwIP33a5bDP8u9zDmaLtsoSVtrowThrl6LA2tb5giZGVW5PLGAHWBFtd5RvK2RouW/wCJ3a40jzM0qG5dV8EL06P5ip04xlnuFBIeiKuPj69e0aQswKZYrvTylHrdXgdA0ojkp6oYXL541L4PphPINrMGdOynmlbVr1wNXBGyu8BMyOFj8gIBAVGVqR9Xdt/+X5s1amfN7UlJCaGgtZBcqbL6zblFYVqJPW+4/uAeOh8FgAJdjzy//7dKlh6urq2WCCePePH78yK+/7QTTDHVF8OdmzJwEJqXqpxg4cDjsC76KVqtNSbn17aqvXxs/JCExvvK94ElkZDw4duwI7IJEQIkrDCXjpRdfAScX3O3RYweG14qc+uY75RLAW79q5Y+XLp1/ZUB38NLA1Hz80RflHkbleHp4fr96i1qlfn3yiFFjBly4ePadmXPBX6x8r/btnm3apMXceTMPHvodiaHC+RitZt0fPkqChv8B08KRPYE6zoD+r44aOR45Cxs/vhneyK33WCvjHEl7NFYq7mGxbrppWFcAABAASURBVDoUCpqm5RcF6InDOxEVbLKeo0tKWAn6DHf+7yByLkifoUSAz0HGdUiE3dujCTx8o5KYsXcE2+BYxHJiOmcpiiKjHPFSQesdX40h/eCiqaRzlpgOzJDCUAoqaYkjQkuEdffaRUnTClIYioZmOJoSM1LJRck7KoggEtBY4y3mY6HI5m7aXJKjxQHdEawePfOyv9Wt1oWO7uoHncv7N4rqXHja2bM8xSeYAaxurSxex+q5N13dUL/JtRGhUvIzdb+sTvYLVfWbXGGv8WMCo/zwUUJBDkszyFBiMj38fyxGPkELLPfQmBtjZ5TGEYG9OIOVio85xIax+mlKIKykqDLDHISTWF6gMSk/GsJcMaBMoUtKT/TwUKVnMR+BH3RRdhwiX8XgyhdHwnVRdGkUD8srsTwgtNVRDGfQIb9QlyEzKuuQenyAQV2R7tyfOTqL+Ai8zFTpJYDQ5p9lwrVYCVoibC+t3xujjlDmu3v07IJilj91xUV//3XumWc7ljmj5QM3i2/lgJTwVNBjoPhveMpW8iyupMz1u9dQtO7q87gDyieSo5nbt2+/8cYbO3fuRLKCzP4mEURoiSBCSwQRWiKI0BJBhJYIMrmvRJAcLRFEaIkgQksEEVoiSGEoESRHSwQRWiKI0BJBbLREkBwtEURoiZDfF0EkR0sEEVoiSGEoESRHS4RGo3Fzc0NyQ35C5+fnC3NHygsy56xEEKElgggtEURoiSBCS4T8quAkR0sEEVoiiNASQYSWCCK0RBChJQLaSMuFPpYFJEdLBBFaIojQEkGElgjZfND56quv5uTkUBQFrf6FhYX+/nwQAVg+cOAAkgOyaevo2LFjRkZGenp6bm4u5OhUIx4eHkgmyEZoyNHh4WW+aod3MSYmBskE2Qjt5+fXq1cvy/7vkJCQAQMGIJkgp2bSwYMHh4aWBsTo1KlTcHAwkglyEtrNza1///4qFT+jSFBQ0KBBg5B8kFnDP2RqsBiw0Lp164iICCQf7OXebfkiKTejxKBDBqsBvx+GFrEMXmIKHEM9GsDdNI+l6Ur5uRk5hqaM8WrKhz+hkCk4DWcZTcV42HKJy4VtoRnOxZXyD1X2fT0M2QH8QhflG9YuSNR40kHhGoWSKSODOUDNowvmRT69Mf5Omf1MsXYsj4TKBqcxrX/4UB45KfUw3g1X/sRGKJrS5uvSkosgc0xciD+MFGahb8fn71yZ1n96LXd3JZInx3ffTbpSOGlRHYQVzDb6tzXpDdt6yFdl4JmXQ7z9lBsWJiGs4BT62oUsvZ5r84KI+ZUck7a9faGAQVjBKXTqdZ1C6QyTivjXdANbnpYkYqKox4Kz9Q7a1Eq0Mos5VhEs7y0ZED5IWOMK4Ci8MeGJ0BVAcVWIQygCnEJTeC/tiWKM3OmoOdqZJmCgTHUcbODN0Rinb33icIhzVNPBcbILdFohnNFIYwRrYehEOZpCjlwYImfK0cihC0OnoVxjYfUhhWFFOHCFhatszlXZgXledKyFIcchzkmyNIfZu5Phx0LzF7w3850pyM5UMo+bbWCugjsPXPnutGqCuVHJebSmMBc3mN07sXYtLz9v7bqVp08dy8rOrF+vUbduvV7s3U/YdPz4Hz+sX3UrOdHLy7tOnfpvTX0vMLB0BvmCgoJ+/WNGj5o4YvhrwhqDwdCnX5e+fQZNnDA1MzNj+YovrsRe1Gq1bdp0GDVifFhYOBIJ3kyD1UZTlFiP6NNPF8TFXpo2bda6NdsaNmzy5dKFsbGXYP3fZ09/MP+dHj1e3Lr513lzF6Wnpy79epHljm5ubh3aP3f06CHzGtilsLAwpmtPUHz6269fuHh2+rT316zeUsPbZ8obo+/cvY1EAe3RHE5x8B5LtM9x8dK5Tp1i2kS3DwgIhJz4n2XrfH358bhr1q7o9FzXgQOGQXZu3LjZlMkzTp06dvVanOW+nTt3u37jamraXeHnsWOHIyKiateue/nyheTkpPdnfdSubUcfH9/Jk6Z5enlv374JiYI30Djnv8MpNCvedDRt2mLrzxtXrFx64sSfer2+fr2GQUH8cLqEhBsNGjQ2JwOrAn+vXo213PeZjp1dXV2FTA11/z/+PAjZGZYvX7ng4uLSqmUbIRlUolo0bw1PFInCkXtYeMsh8sG99+78Xbu2HTr8O8jt7ub+yitDRo2cAIa1uLjY1VVlTqbRaOBvYWGB5b4qlapjh05Hjx0ePGgE5OK8vNzu3XojPkRNHjyzLjHRlom9vWsgUThyDwtfGIp82zw9PKE0Gz5s7JUrF0GyDRu/d3f36P/KUNik1ZbG8ykwSuzr41du9+ef7z5v/rsZGQ/+PHoILIxQWvr6+qnV6n9//KVlSoZmkBgculHJNItblcnPz9+3/5fevfpC3gQbAv/i46+B2VUoFGBDhFJRQFiOql233BGgPIRS8dTpY/BOjBwxXlhZu3a9oqKigICgmiGmMb53U+94e4nM0QhzVxZOG83nAjFGGgQFB27+h+9BdgaHbN++X27EX23apAVseqXfkGPHj2zf/lNuXu75C3+DrwY2t26d+uWOALa4Y8fOYHxycrKf79xNWNm6Vdu2bTt+9tlH6elpsH7Hzp8nTR65d+8uJAaj3cBZGD7JXnDIyB/OX/LNf5ZMfWsc/IyMrD3p9Wm9evaBZXDs7j+4t+XnDcuWfw4GIbp1+wnj37R6kOc7dZu9fwb4LTVqlM51t/DfS3ft3v7hx7Pi4i6DBw3uef/+Q9ETBecgx4M/pV87WzBybhSSP+vmxQ+eFhoYoUKYwN3w7yyN/9jbEjD3GTpRHwvmW8EpNNQMaSfqYUFYwSm0sZB2nh4WB+4zZCnOWXpYsIO1MHQiE21s9sdZycBaGNIUcpYMbez9dNgKC+tEneA8jtrWQdMc7QxfVphx1MKQZSkW59v2ROFLdYe10c4E5cCNSrzdIN5dBZAKi3WgHwvnN1l4hVa6Iuf4zhBgGKRSI4zg1KV5Zze9zhlKw39OZ0DrnU8gTqVxCu3l4+7hzfzyXTKSOVeO59asi60lWgDzmz5qTqS2wLBjZQKSLT8tjg+opewzMRRhxS6BUdYsuFmcz7lqaGihNlh8vE4ZhzJxrLFz0fiQzbFKaIpijVdiDIxiGvgmhDPhHgYxYWjKwJqulmYozsBvomlw3k0rIQHHh00BC0sZDPxKYYE2xlaBTTSfQEjLGT/WM/4P6tos56KkDXqDroj1CXIZMlP0+LHHYq8INNfO5Vz7K78onxVu2HQy4/tjGpJQNjIJbBLWU9aCxAgLgqYsy2ZnZfsH+BqMNX7wKc21JFjmjMd/RGiKo+EJGC+ANXVwm073cNlFSak9mLa9PQOC3ZEdkE0kRzN37tyZPHnyrl3ierWfOGT2N4kgQksEmThSIkiOlggitETI74oNBgMRWgrARhOhpYCYDokgQksEEVoiiNASQSosEkFytEQQoSWCCC0RxEZLBMnREiFToeU33oUILRHERksEsdESQYSWDldXVyQ3ZJmjoZMFyQ0y56xEEKElgggtEURoiSBCS4T8aoZEaIkgpkMiiNASAS1K0K6E5AbJ0RJBhJYIIrREEKElgggtEURoiSBCS4ScPugcOHAgy7L5+fl5eXl+fn6wXFRUdOjQISQHZJOjP/jgg8TERPOsZ6mpqYgPfu6LZIJs2jpGjRoVHBxsuQZydLt27ZBMkI3QderUee655ywNHWTn4cOHI5kgp9a7ESNG1KxZU1gGxVu0aFG/fn0kE+QkNKj8/PPPC5nay8sLdEfyQWbt0SNHjoyIiACtGzRo0Lx5cyQf7OjenTuUcetaYV5miUHHF1yPBq+1DB4jhD4xziJYuvLRlLBQUFBYVFTo6eGldHWBlWUO8jCoirBSCLZSbqVleoBRUAzNuSgpL39l7Wbu9aM9kX3AL/SR7WnxF4q0BQY+ig5N0QylcGE4jqUefXvMEWYs11DGYLIVpRRkoyk+jfD30YOUpqfKxOEzP0zL9HzUGrakxMCVwH842KjxZFrHeDd7Tuy8LY8Bp9CHt6VfPZMHt6byUgXVq6H2wBxoSwKyUnMzbuXpCnSQP54f7NegtRfCBDahV89O0OlY31qegXVkU4mohORL6bnphR6+zOjZkQgHGITOzdKt/zBZ46OKig5GzsU/fySBQJMW10HVprpCa4sMq99PrNXa39PXLsG1njgpV9Ly7hVNWVJdrasldOY93aZFyU2643m5HJa71+5n3cl/o3paV8uP/mlxcnirQOTshNT39/R3WzXrJqoGtgu9ek6C2lvl4atBTwFhTQM4jtr+VQqyFRuFPrrznraIdb7SrxLqdw5PvVWc90CLbMJGoS/+kesb5oGeMlTuyq1f3UU2YYvQJ3+9D7WS4Pp+yCG5cPnAzLnt8guyEG7qdKhZlA/dOsVIPLYIHXcyV+Utvy/9sMAo6V+/v4fEY4vQRflcQG3MTQFywc1XfT9Fh8Qjus8w9lQ2tNV4+roh+5CUfGnf4dUpt+Pc3Wo0rP9sjy7jVSr+XMdP/bz/jzWTX1uxfvOs9HsJwYF1OnV8tU2rl4S99uz95u+Lv7oqNS2bvRDgVwvZDf9Ij5upBeL3E5+jU64WQtMisg8PMlK+XTdVry9+c+Lq0cMWp6bfWLFmssEY65tRuBQV5e345bPB/d5f8uGpZk26bt3xcVZ2Gmw6cWb7iTPb+r/4zluvr/WtEbL/8PfIbqjd1dDCF38xF4lEtNC5mSW0wl7dBecu7lUwLmNeXRzoHxEUEDWo7+w7qdeu/POHsNVg0HfvMj48rCm0WUe3eBHqtHdSr8P6Yye3NmscA9JrNJ6Qx+tERSN7QjFUWqJoJ0+0ZCU6lrLbjFhgN8JCG7m5eQs/fWoE+/qEJt66YE5Qq2ZjYUGj5lvoi7R5IPeDzJTAgNJmgNCQBsiewO0X5oj+olS0jYY+CspelgOEy0+5EwfOmeXK3LwM8zL1yLm1xQUsa3B1La2gKpVYJ1B5FOjFED/lq2ih4S4K8u31hbCHh29keIsXuk60XOnmVlnru8rVjaYZvb70XS7WFSJ7wrGsxkP0Oy1aaI8aioxUe33ZEBJY9+zFX6MiWtIPrVPavQR/38q8CMjjNbyDk5Ivd37GtOafa8eRPWFZFBgmuvNI9JMJq6uBvjVkH8Bjg27cXb99qdNp792/tef3ZZ8vG5aaHl/5Xs2bdLscdxgqhLB86Oj6W7evILuhLdJBrbh+a9F9uKKFbtLRG7qW8zLt8nqC2zDzzU1KF/XSlaM//XpwQtK5Qf1mP7Zw69Z5bLvWfXf8+jkYd8jOfXpNQ8I8K3bg/s0s2qbxirY0/K+Zl4gYl6g2T1HTnZm4w4n+IcpB00TXiWxx1Oq1civKtbG1UO6wetRzjC19Hba8Bs/2Dbh0NDftRkZQXesd3lBh+/w/1ocfql3di4rzrW4K8o96c+J3CB9z/h1T0SaobTKMlXuPqNVs/MgvK9rrxskUcLq0o5goAAAB6klEQVQ8vG2Jf2Njn+HhrelxZ/Ibx0RY3Qq3kZNrvYkLSjml0nqRTdMKb68AhI/MrArbjnX6YqWLFb0UjNLTs8Lm3yv7Eoe+E+IXYkunku2ds6tmJbi6K8NbPS2W+tqftzx8mGHv2Dhfme2V6YkLo/IztYV5Regp4E7cfY7lbFYZVbMXvN/koMTTacjZeXArOyc1f9Li2qgaVHcATW6mbv1HyfWeq6lUK5EzknotI+t27pTPnugAGtOlJBZt/+aOh786vEUQci6uHb9l0LLVVxlhHOS46v2brIHyq+PlF+qN5M/tf+5lpxRovKjX5lfLYpjBOWx338bU+AsF0MoDHWuhTf1puzVb24+iHO2daxnaHB1Fow4v+rTq4oMwgX8g+r4f0xKvFOq1LM1QtIJilAzD0MLksOaTGv+aJ39F5kZmfq5ZqsxkqNaXzdOochS/j+VWYYZaYRt/NMsdHw5ER6W7UMbP6Ax6rkQHHTj87KhKDdXsGc/2vf0RVuz4acWJPfdu3yjWFRn0xZxWazCP+Bd0sHpaeBacxSbLIfqmv+ihbqWfYvDtluZjmhI8fH5mUUsVfriVZhBrgK5I5KKkFS5I46mIaKhpFYMtC5dDfnPOyhRZRtuVI0RoiSBCSwQRWiKI0BJBhJaI/wcAAP//qEo84QAAAAZJREFUAwCEeKcH8ZnMZwAAAABJRU5ErkJggg==", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Image, display\n", "arch = SelfDiscover(llm=reasoning_llm)\n", "graph = arch.build()\n", "display(Image(graph.get_graph().draw_mermaid_png()))" ] }, { "cell_type": "markdown", "id": "6259bf3f", "metadata": { "papermill": { "duration": 0.006722, "end_time": "2026-05-27T16:32:17.372171+00:00", "exception": false, "start_time": "2026-05-27T16:32:17.365449+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 8 · Live run — 5-musician height ordering puzzle\n", "\n", "A small logic-deduction task with a unique answer (`Dee, Cal, Bea, Alex, Ed`). We watch the agent pick reasoning modules, adapt them to musicians-and-heights language, write a step plan, and execute." ] }, { "cell_type": "code", "execution_count": 4, "id": "95cfe293", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:32:17.401095Z", "iopub.status.busy": "2026-05-27T16:32:17.401095Z", "iopub.status.idle": "2026-05-27T16:33:42.968632Z", "shell.execute_reply": "2026-05-27T16:33:42.966618Z" }, "papermill": { "duration": 85.583494, "end_time": "2026-05-27T16:33:42.974665+00:00", "exception": false, "start_time": "2026-05-27T16:32:17.391171+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "SELECTED_IDS: [1, 4, 7, 9, 14]\n", "SELECTED_COUNT: 5\n", " module: Break the problem into sub-problems and solve each\n", " module: List the relevant facts, constraints, and unknowns\n", " module: Step-by-step reasoning from premises to conclusion\n", " module: Make a table, list, or diagram to organize the information\n", " module: Check consistency — does the candidate answer satisfy EVERY constraint?\n", "\n", "PLAN_STEP_COUNT: 4\n", " step 1: List all height facts (Bea shorter than Cal, Dee taller than Cal, Ed shortest) and identify unknown positions using adap\n", " → expected_output: A structured list of given constraints and unknown relationships (e.g., Alex's position relative to others)\n", " step 2: Resolve each constraint sequentially starting with Ed as the shortest, then Bea < Cal, and Dee > Cal using adapted modul\n", " → expected_output: Partial orderings (Ed fixed at position 5, Cal > Bea, Dee > Cal)\n", " step 3: Integrate all deductions to determine Alex's position between Bea and Ed using adapted module 3.\n", " → expected_output: Complete relative sequence (Dee > Cal > Bea > Alex > Ed)\n", " step 4: Construct the final height ranking chart formalizing the sequence using adapted module 4.\n", " → expected_output: A comma-separated list of musicians from tallest to shortest\n", "\n", "FINAL_ANSWER_FORMAT: a single comma-separated list of names from tallest to shortest\n", "FINAL_ANSWER: Dee,Cal,Bea,Alex,Ed\n", "EXPECTED: Dee, Cal, Bea, Alex, Ed\n", "MATCH: True\n" ] } ], "source": [ "TASK = (\n", " \"Five musicians (Alex, Bea, Cal, Dee, Ed) line up by height for a photo.\\n\"\n", " \"Constraints:\\n\"\n", " \" - Bea is shorter than Cal but taller than Alex.\\n\"\n", " \" - Dee is taller than Cal.\\n\"\n", " \" - Ed is the shortest.\\n\"\n", " \"List the musicians from TALLEST to SHORTEST as a comma-separated list.\"\n", ")\n", "EXPECTED = \"Dee, Cal, Bea, Alex, Ed\"\n", "\n", "r = arch.run(TASK)\n", "\n", "# Normalise for comparison: strip spaces so 'Dee,Cal,...' equals 'Dee, Cal, ...'\n", "def _norm(s):\n", " return ''.join(s.split()).lower()\n", "match = _norm(r.output) == _norm(EXPECTED)\n", "\n", "print(f\"SELECTED_IDS: {r.metadata['selected_ids']}\")\n", "print(f\"SELECTED_COUNT: {len(r.metadata['selected_modules'])}\")\n", "for m in r.metadata['selected_modules']:\n", " print(f\" module: {m}\")\n", "print()\n", "print(f\"PLAN_STEP_COUNT: {r.metadata['plan_step_count']}\")\n", "for s in r.metadata['plan_steps']:\n", " print(f\" step {s['step_number']}: {s['description'][:120]}\")\n", " print(f\" → expected_output: {s['expected_output'][:120]}\")\n", "print()\n", "print(f\"FINAL_ANSWER_FORMAT: {r.metadata['final_answer_format']}\")\n", "print(f\"FINAL_ANSWER: {r.output}\")\n", "print(f\"EXPECTED: {EXPECTED}\")\n", "print(f\"MATCH: {match}\")" ] }, { "cell_type": "markdown", "id": "49539dab", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T16:33:42.982486+00:00", "exception": false, "start_time": "2026-05-27T16:33:42.982486+00:00", "status": "completed" }, "tags": [] }, "source": [ "### 8.0 · What just happened, briefly\n", "\n", "Three signals to read:\n", "\n", "1. **`SELECTED_COUNT` between 3 and 6.** Outside that range = SELECT mis-calibrated; either too narrow (loses useful modules) or too broad (plan becomes incoherent).\n", "2. **`PLAN_STEP_COUNT` between 3 and 7.** Plans with 1-2 steps mean IMPLEMENT collapsed the modules; plans with 8+ mean it over-decomposed.\n", "3. **`MATCH=True`** for the answer. The puzzle has a unique solution; if MATCH is False, look at the step outputs to see where the deduction broke." ] }, { "cell_type": "markdown", "id": "f8518d31", "metadata": { "papermill": { "duration": 0.0, "end_time": "2026-05-27T16:33:42.998632+00:00", "exception": false, "start_time": "2026-05-27T16:33:42.998632+00:00", "status": "completed" }, "tags": [] }, "source": [ "### 8.1 · Inspect the adapted modules + step outputs" ] }, { "cell_type": "code", "execution_count": 5, "id": "3c74f0d6", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:33:43.016243Z", "iopub.status.busy": "2026-05-27T16:33:43.016243Z", "iopub.status.idle": "2026-05-27T16:33:43.032803Z", "shell.execute_reply": "2026-05-27T16:33:43.030844Z" }, "papermill": { "duration": 0.018067, "end_time": "2026-05-27T16:33:43.032803+00:00", "exception": false, "start_time": "2026-05-27T16:33:43.014736+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=== ADAPTED MODULES ===\n", "[1] original: Break the problem into sub-problems and solve each\n", " adapted : Break the height lineup problem into specific musician constraints (Bea shorter than Cal, Dee taller than Cal, Ed shortest) and resolve each sequentially\n", "\n", "[2] original: List the relevant facts, constraints, and unknowns\n", " adapted : List the height facts (Bea shorter than Cal, Dee taller than Cal, Ed shortest), constraints, and unknown musician positions\n", "\n", "[3] original: Step-by-step reasoning from premises to conclusion\n", " adapted : Step-by-step deduction from height premises (Bea shorter than Cal, Dee taller than Cal, Ed shortest) to final tallest-to-shortest lineup\n", "\n", "[4] original: Make a table, list, or diagram to organize the information\n", " adapted : Construct a height ranking chart for Alex, Bea, Cal, Dee, Ed using the given constraints\n", "\n", "[5] original: Check consistency — does the candidate answer satisfy EVERY constraint?\n", " adapted : Verify the proposed height order satisfies all constraints: Bea shorter than Cal, Dee taller than Cal, and Ed being the shortest\n", "\n", "=== STEP-BY-STEP EXECUTION OUTPUT ===\n", "[step 1] Constraints: Bea < Cal, Bea > Alex, Dee > Cal, Ed shortest. Unknown: Alex's position relative to Cal/Dee and Ed's exact placement.\n", "[step 2] Ed fixed at position 5 (shortest). Cal > Bea > Alex established. Dee > Cal confirmed.\n", "[step 3] Alex must be between Bea and Ed (Bea > Alex > Ed), completing the sequence Dee > Cal > Bea > Alex > Ed.\n", "[step 4] Final ranking: Dee, Cal, Bea, Alex, Ed\n" ] } ], "source": [ "print('=== ADAPTED MODULES ===')\n", "for i, a in enumerate(r.metadata['adapted_modules'], 1):\n", " print(f\"[{i}] original: {a['original'][:80]}\")\n", " print(f\" adapted : {a['adapted'][:200]}\")\n", " print()\n", "\n", "print('=== STEP-BY-STEP EXECUTION OUTPUT ===')\n", "for i, out in enumerate(r.metadata['step_outputs'], 1):\n", " print(f\"[step {i}] {out}\")" ] }, { "cell_type": "markdown", "id": "85e533f5", "metadata": { "papermill": { "duration": 0.009973, "end_time": "2026-05-27T16:33:43.051884+00:00", "exception": false, "start_time": "2026-05-27T16:33:43.041911+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 9 · What we just observed\n", "\n", "The cells above ran the SELECT → ADAPT → IMPLEMENT → SOLVE pipeline on a 5-musician height-ordering puzzle, then re-ran the same puzzle with a plain (non-reasoning) LLM for contrast.\n", "\n", "### 9.1 · The recipe the reasoning model designed\n", "\n", "**Stage 1 — SELECT** picked these modules from the 16-module library:\n", "\n", "| id | module |\n", "|---|---|\n", "| `[1]` | Break the problem into sub-problems and solve each |\n", "| `[4]` | List the relevant facts, constraints, and unknowns |\n", "| `[7]` | Step-by-step reasoning from premises to conclusion |\n", "| `[9]` | Make a table, list, or diagram to organize the information |\n", "| `[14]` | Check consistency — does the candidate answer satisfy EVERY constraint? |\n", "\n", "**Stage 3 — IMPLEMENT** composed them into this plan:\n", "\n", "| # | description | expected output |\n", "|---|---|---|\n", "| 1 | List all height facts (Bea shorter than Cal, Dee taller than Cal, Ed shortest) and identify unknown positions using adap | A structured list of given constraints and unknown relationships (e.g., Alex's position relative to others) |\n", "| 2 | Resolve each constraint sequentially starting with Ed as the shortest, then Bea < Cal, and Dee > Cal using adapted modul | Partial orderings (Ed fixed at position 5, Cal > Bea, Dee > Cal) |\n", "| 3 | Integrate all deductions to determine Alex's position between Bea and Ed using adapted module 3. | Complete relative sequence (Dee > Cal > Bea > Alex > Ed) |\n", "| 4 | Construct the final height ranking chart formalizing the sequence using adapted module 4. | A comma-separated list of musicians from tallest to shortest |\n", "\n", "**Run summary:**\n", "\n", "| Field | Value |\n", "|---|---|\n", "| SELECTED modules | 5 of 16 |\n", "| Module ids | `[1, 4, 7, 9, 14]` |\n", "| PLAN steps | 4 |\n", "| Final-answer format | a single comma-separated list of names from tallest to shortest |\n", "| Final answer | `Dee,Cal,Bea,Alex,Ed` |\n", "| Expected | `Dee, Cal, Bea, Alex, Ed` |\n", "| Match | ✅ |\n", "\n", "### 9.2 · Reasoning vs non-reasoning LLM on the same task\n", "\n", "| Model | SELECT count | PLAN steps | Final answer | Match |\n", "|---|---|---|---|---|\n", "| Reasoning (Qwen3-Thinking) | 5 | 4 | `Dee,Cal,Bea,Alex,Ed` | ✅ |\n", "| Plain (Llama-3.3-70B) | 4 | 5 | `Dee,Cal,Bea,Alex,Ed` | ✅ |\n", "\n", "### 9.3 · Patterns surfaced in this run\n", "\n", "- **✅ SELECT picked 5 modules** — inside the recommended 3-6 range.\n", "\n", "- **✅ SOLVE produced the expected answer** — the discovered structure executed correctly.\n", "\n", "- **🟰 Both reasoning and plain LLM converged on the correct answer.** On this puzzle, the structure-discovery pipeline produced sensible recipes for both — the underlying model's reasoning depth wasn't the bottleneck. Bigger differentiation would show on harder logic-deduction tasks (e.g., 7-element ordering with multi-hop transitivity).\n", "\n", "- **Structure divergence**: the two models chose different module counts / plan lengths (5/4 vs 4/5). Self-Discover's recipe is LLM-specific, not task-specific — same task, different model = different reasoning shape.\n", "\n", "### 9.4 · The takeaway\n", "\n", "Self-Discover converts the implicit \"how should I think about this?\" choice into an *explicit, inspectable* artefact (the plan in § 9.1). Two consequences:\n", "\n", "1. **Plans are auditable.** Unlike chain-of-thought scratchpad, the recipe is structured Pydantic data — you can save it, diff it across model versions, or hand-edit it before SOLVE runs.\n", "2. **Plans are reusable.** Tasks of the same type share a recipe; SELECT/ADAPT/IMPLEMENT only need to run once per task family. § 11.3 extension #1 sketches a `discover() + solve()` split that makes this concrete and cuts cost 4×.\n", "\n", "The architecture's headline behaviour is that the *reasoning structure* itself becomes a first-class object the agent decides on, rather than an emergent property of a fixed CoT prompt. Whether that produces a better *answer* depends on the underlying model (§ 9.2 above) — but it always produces a more *transparent* one." ] }, { "cell_type": "markdown", "id": "6c1c3aba", "metadata": { "papermill": { "duration": 0.006686, "end_time": "2026-05-27T16:33:43.068076+00:00", "exception": false, "start_time": "2026-05-27T16:33:43.061390+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 10 · Contrast — same task, plain Llama-3.3-70B (no thinking)\n", "\n", "Self-Discover should work with any chat model — but a non-reasoning LLM may pick weaker modules or write a shallower plan. Re-run the same puzzle with Llama-3.3-70B and compare." ] }, { "cell_type": "code", "execution_count": 6, "id": "1636db63", "metadata": { "execution": { "iopub.execute_input": "2026-05-27T16:33:43.076777Z", "iopub.status.busy": "2026-05-27T16:33:43.076777Z", "iopub.status.idle": "2026-05-27T16:34:01.706995Z", "shell.execute_reply": "2026-05-27T16:34:01.703608Z" }, "papermill": { "duration": 18.638919, "end_time": "2026-05-27T16:34:01.706995+00:00", "exception": false, "start_time": "2026-05-27T16:33:43.068076+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LLAMA_SELECTED_COUNT: 4\n", "LLAMA_PLAN_STEP_COUNT: 5\n", "LLAMA_FINAL_ANSWER: Dee,Cal,Bea,Alex,Ed\n", "LLAMA_MATCH: True\n", "\n", "(Reasoning model picked 5 modules / 4 plan steps; Llama picked 4 / 5.)\n" ] } ], "source": [ "plain_llm = get_llm(provider=\"nebius\", model=\"meta-llama/Llama-3.3-70B-Instruct\", temperature=0.4)\n", "arch_llama = SelfDiscover(llm=plain_llm)\n", "r_llama = arch_llama.run(TASK)\n", "\n", "match_llama = _norm(r_llama.output) == _norm(EXPECTED)\n", "print(f\"LLAMA_SELECTED_COUNT: {len(r_llama.metadata['selected_modules'])}\")\n", "print(f\"LLAMA_PLAN_STEP_COUNT: {r_llama.metadata['plan_step_count']}\")\n", "print(f\"LLAMA_FINAL_ANSWER: {r_llama.output}\")\n", "print(f\"LLAMA_MATCH: {match_llama}\")\n", "print()\n", "print(f\"(Reasoning model picked {len(r.metadata['selected_modules'])} modules / \"\n", " f\"{r.metadata['plan_step_count']} plan steps; Llama picked \"\n", " f\"{len(r_llama.metadata['selected_modules'])} / {r_llama.metadata['plan_step_count']}.)\")" ] }, { "cell_type": "markdown", "id": "c1fc78e6", "metadata": { "papermill": { "duration": 0.00662, "end_time": "2026-05-27T16:34:01.727499+00:00", "exception": false, "start_time": "2026-05-27T16:34:01.720879+00:00", "status": "completed" }, "tags": [] }, "source": [ "## 11 · Failure modes, safety, extensions\n", "\n", "### 11.1 · Where this breaks\n", "\n", "| Failure | Mechanism | Mitigation |\n", "|---|---|---|\n", "| **Over-selection** | LLM picks 8+ modules; plan becomes a kitchen sink | `_SelectedModules.selected_ids` is `max_length=8` — and the prompt asks for 3-6 |\n", "| **Hallucinated module ids** | LLM returns an id outside `[0, len-1)` | The `_select` node clamps to the valid range silently before passing on |\n", "| **Plan-execution mismatch** | SOLVE doesn't actually follow the plan; produces an unstructured answer | The `_Solution.step_outputs` field enforces one entry per plan step, which forces the model to address each step explicitly |\n", "| **Module library too small / too generic** | No module fits the task type | Replace `MODULE_LIBRARY` via the `modules=` constructor arg with a domain-specific list |\n", "| **Non-reasoning model under-decomposes** | Llama (vs Qwen-Thinking) picks fewer/shallower modules | Use a reasoning model for the SELECT/IMPLEMENT stages even if SOLVE runs on a cheaper LLM |\n", "\n", "### 11.2 · Production safety\n", "\n", "- **Plan caching.** SELECT/ADAPT/IMPLEMENT depend only on task TYPE, not on the specific instance. Cache the plan per task-template, then only run SOLVE per instance — 4× cost reduction.\n", "- **Plan review.** For high-stakes tasks, surface the plan to a human reviewer before SOLVE runs. The plan is interpretable text by design.\n", "- **Module library audit.** Generic modules (\"step-by-step\") can hide bias; domain-specific modules sometimes leak unsafe heuristics. Treat `MODULE_LIBRARY` as a policy artefact.\n", "\n", "### 11.3 · Three extensions\n", "\n", "1. **Plan reuse across instances.** Split `.run()` into `.discover(task_type)` (returns a plan) and `.solve(instance, plan)` (just SOLVE). Many tasks of the same type → one plan, many solutions.\n", "2. **Hierarchical plans.** Each plan step can itself trigger a nested Self-Discover for that sub-problem. Useful when the task is genuinely tree-shaped.\n", "3. **Cross-task module learning.** Track which module subsets succeed on which task families; bias future SELECT calls toward proven combos. Lets the architecture \"learn\" without weight updates — and dovetails neatly with [Reflexion (nb 18)](./18_reflexion.ipynb)'s episodic memory.\n", "\n", "### 11.4 · What to read next\n", "\n", "- [**09 · Tree of Thoughts**](./09_tree_of_thoughts.ipynb) — search-in-thought-space, sibling reasoning architecture.\n", "- [**04 · Planning**](./04_planning.ipynb) — Planning over *actions*; composes cleanly with Self-Discover's reasoning-plan.\n", "- [**18 · Reflexion**](./18_reflexion.ipynb) — verbal memory of failures; useful for extension #3 above.\n", "- [**22 · LATS**](./22_lats.ipynb) — MCTS-style reasoning search with explicit reward signals.\n", "\n", "### 11.5 · References\n", "\n", "1. Zhou, P. et al. *Self-Discover: Large Language Models Self-Compose Reasoning Structures.* 2024. [arXiv:2402.03620](https://arxiv.org/abs/2402.03620)\n", "2. Wei, J. et al. *Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.* NeurIPS 2022. [arXiv:2201.11903](https://arxiv.org/abs/2201.11903) — the baseline Self-Discover improves on.\n", "3. Yao, S. et al. *Tree of Thoughts: Deliberate Problem Solving with Large Language Models.* NeurIPS 2023. [arXiv:2305.10601](https://arxiv.org/abs/2305.10601) — sibling structured-reasoning architecture." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.0" }, "papermill": { "default_parameters": {}, "duration": 113.816524, "end_time": "2026-05-27T16:34:02.845314+00:00", "environment_variables": {}, "exception": null, "input_path": "all-agentic-architectures/notebooks/19_self_discover.ipynb", "output_path": "all-agentic-architectures/notebooks/19_self_discover.ipynb", "parameters": {}, "start_time": "2026-05-27T16:32:09.028790+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }