{ "cells": [ { "cell_type": "markdown", "id": "4ec544ca", "metadata": {}, "source": [ "# Deep Researcher Agent\n", "\n", "This notebook focuses on the Deep Research Agent which has been built using [LangChain Deep Agents harness](https://docs.langchain.com/oss/python/deepagents/harness). It will demonstrate how to get started with the NVIDIA Deep Researcher Agent blueprint - deep researcher agent workflow: with **web search only**. You will:\n", "\n", "1. **Understand** the deep researcher architecture (orchestrator, planner, researcher). \n", "2. **Run** the deep research agent using the Python API\n", "3. **Run** the deep research agent via the NeMo Agent Toolkit CLI (`nat run`) for interactive research with a minimal config.\n", "4. Run **`nat eval`** on a benchmark dataset and interpret results.\n", "5. Use **profiling/tracing** (such as Phoenix/Weave) if configured." ] }, { "cell_type": "markdown", "id": "6e498866", "metadata": {}, "source": [ "# Table of Contents\n", "\n", "- [About the deep researcher](#about) \n", "- [Prerequisites](#prereqs) \n", "- [Simplest way to run the agent (Python API)](#simplest-run) \n", "- [Simplified config (web search only)](#config) \n", "- [Run with NAT CLI](#nat-run) \n", "- [Evaluate with NAT Eval](#nat-eval)\n", "- [Profiling and tracing](#profiling) \n", "- [Next steps](#next) \n" ] }, { "cell_type": "markdown", "id": "fcd30b49", "metadata": {}, "source": [ "\n", "## About the deep researcher\n", "\n", "The deep researcher is implemented in `src/deep_researcher_agent/agents/deep_researcher/agent.py`. It uses the [LangChain DeepAgents](https://docs.langchain.com/oss/python/deepagents/overview) library. The architecture consists of one main agent and two subagents. The system makes use of `TodoListMiddleware`, `FilesystemMiddleware` and so on for context isolation and management during the execution of a long-running agent.\n", "\n", "- **Orchestrator** – Coordinates the workflow and produces the final report. \n", "- **Planner subagent** – Builds a structured research plan and search queries. \n", "- **Researcher subagent** – Runs search queries and pulls in content from tools (e.g. web search).\n", "Read more about it in this [documentation](../source/architecture/agents/deep-researcher.md#workflow-phases)\n", "\n", "Phases: \n", "- (1) planning\n", "- (2) research loops\n", "- (3) final report.\n", "\n", "With **web search only**, the researcher uses a single tool (e.g. Tavily) to gather information." ] }, { "cell_type": "markdown", "id": "4629edb9", "metadata": {}, "source": [ "\n", "## Prerequisites\n", "\n", "- **Python 3.11–3.13**\n", "- **[uv](https://github.com/astral-sh/uv)** package manager\n", "- **LLM API key** – For your chosen provider (required for LLM inference):\n", " - **NVIDIA API Key** from [NVIDIA AI](https://build.nvidia.com/) (for NIM models)\n", "Optional requirements:\n", "- **Web search API key**:\n", " - **Tavily** – [tavily.com](https://tavily.com) (for general web search)\n", "- **Node.js 22+ and npm** (optional, for web UI mode)" ] }, { "cell_type": "markdown", "id": "aaf7e746", "metadata": {}, "source": [ "1. **Clone** the NVIDIA Deep Researcher Agent Blueprint repository\n", ">**Note**: Skip this step if you are not on brev and instead running from within the repository." ] }, { "cell_type": "code", "execution_count": null, "id": "228511a7", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "git clone https://github.com/NVIDIA-AI-Blueprints/deep-researcher-agent.git deep-researcher && cd deep-researcher" ] }, { "cell_type": "markdown", "id": "cf4b4610", "metadata": {}, "source": [ "2. **Install** dependencies\n", ">**Note**: Skip this step if you already have a virtual environment setup. \n", "\n", ">Just run:\n", ">```bash\n", ">source .venv/bin/activate\n", ">```" ] }, { "cell_type": "code", "execution_count": null, "id": "0d1d2c76", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Some dependencies (for example, annoy via nemoguardrails) are built from source\n", "# and need a C/C++ compiler. Fresh Brev/Ubuntu VMs do not ship one.\n", "if ! command -v c++ >/dev/null 2>&1; then\n", " if command -v apt-get >/dev/null 2>&1; then\n", " SUDO=\"\"\n", " [ \"$(id -u)\" -ne 0 ] && SUDO=\"sudo\"\n", " $SUDO apt-get update -qq && $SUDO apt-get install -y -qq build-essential\n", " else\n", " echo \"No C/C++ compiler found. Install one (for example, Xcode Command Line Tools on macOS) before running setup.\"\n", " fi\n", "fi\n", "if ! command -v c++ >/dev/null 2>&1; then\n", " echo \"C/C++ compiler is still missing; setup will fail.\" >&2\n", " exit 1\n", "fi\n", "c++ --version | head -n 1" ] }, { "cell_type": "code", "execution_count": null, "id": "380f9dc6", "metadata": {}, "outputs": [], "source": [ "# Run setup from the project root (where pyproject.toml and scripts/setup.sh live).\n", "import os\n", "from pathlib import Path\n", "\n", "cwd = Path.cwd().resolve()\n", "REPO_ROOT = None\n", "for candidate in [cwd, *cwd.parents]:\n", " if (candidate / \"pyproject.toml\").exists() and (candidate / \"scripts\" / \"setup.sh\").exists():\n", " REPO_ROOT = candidate\n", " break\n", "\n", "if REPO_ROOT is None:\n", " raise FileNotFoundError(\"Could not locate project root containing pyproject.toml and scripts/setup.sh\")\n", "\n", "os.environ[\"REPO_ROOT\"] = str(REPO_ROOT)\n", "os.chdir(REPO_ROOT)\n", "os.environ[\"UV_VENV_CLEAR\"] = \"1\"\n", "get_ipython().system(\"./scripts/setup.sh\")" ] }, { "cell_type": "code", "execution_count": null, "id": "b0fa591f", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "source .venv/bin/activate" ] }, { "cell_type": "markdown", "id": "c09ba6ee", "metadata": {}, "source": [ "4. **Obtain and set** API keys" ] }, { "cell_type": "markdown", "id": "632d4404", "metadata": {}, "source": [ "\n", "| API | Environment variable | Purpose |\n", "|-----|----------------------|--------|\n", "| **NVIDIA** | `NVIDIA_API_KEY` | Agent execution (integrate.api.nvidia.com) |\n", "| **Tavily** | `TAVILY_API_KEY` | Web search |\n", "| **OpenAI** | `OPENAI_API_KEY` | Frontier model in config (optional) |\n", "\n", "Set them before running (e.g. in your shell or below):\n", "\n", "```bash\n", "export TAVILY_API_KEY=your_key\n", "export NVIDIA_API_KEY=your_key\n", "export OPENAI_API_KEY=your_key # optional, for frontier model in config\n", "```\n", "\n", "**NVIDIA API key:** [NVIDIA Build](https://build.nvidia.com/) or [NGC](https://ngc.nvidia.com/) → Get API Key → Generate Key (`nvapi-`). \n", "**Tavily API key:** [tavily.com](https://tavily.com) → dashboard → create API key. " ] }, { "cell_type": "code", "execution_count": null, "id": "74aa298b", "metadata": {}, "outputs": [], "source": [ "import os\n", "from pathlib import Path\n", "\n", "if os.environ.get(\"REPO_ROOT\"):\n", " os.chdir(os.environ[\"REPO_ROOT\"])\n", "else:\n", " cwd = Path.cwd()\n", " if not (cwd / \"configs\").exists() and not (cwd / \"src\").exists():\n", " root = cwd.parent.parent\n", " if (root / \"configs\").is_dir() or (root / \"src\").is_dir():\n", " os.chdir(root)\n", "print(\"Working directory:\", os.getcwd())" ] }, { "cell_type": "code", "execution_count": null, "id": "c455d86a", "metadata": {}, "outputs": [], "source": [ "import getpass\n", "\n", "if \"NVIDIA_API_KEY\" not in os.environ:\n", " nvidia_api_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n", " os.environ[\"NVIDIA_API_KEY\"] = nvidia_api_key\n", "\n", "if \"TAVILY_API_KEY\" not in os.environ:\n", " tavily_api_key = getpass.getpass(\"Enter your Tavily API key: \")\n", " os.environ[\"TAVILY_API_KEY\"] = tavily_api_key" ] }, { "cell_type": "markdown", "id": "f2161eb7", "metadata": {}, "source": [ "\n", "## Simplest way to run the agent (Python API)\n", "\n", "You can run the deep researcher agent directly in Python without a config file: create an LLM, a web search tool, wire them with `LLMProvider`, and call `agent.run(state)`. The next section walks through what each part does and what happens under the hood." ] }, { "cell_type": "markdown", "id": "455735f8", "metadata": {}, "source": [ "### Walkthrough: what the code does\n", "\n", "- **LLM and provider:** You create one chat model (e.g. `ChatNVIDIA`) and a single `LLMProvider`. The provider is the central place that assigns which LLM runs which role. You call `provider.set_default(llm)` and then `provider.configure(LLMRole.ORCHESTRATOR, llm)` (and similarly for `PLANNER`, `RESEARCHER`). That way the orchestrator, planner, and researcher can all share the same model, or you can later point each role at a different model.\n", "\n", "- **Tools:** You pass a list of LangChain tools (here, only `TavilySearch`). The agent adds an internal `think` tool and passes the full list to both the orchestrator and the subagents. The orchestrator uses tools to track progress (`write_todos`), delegate work (`task`), and read/write files; the researcher uses the same list to run search (e.g. `tavily_search`) and `think`.\n", "\n", "- **Agent and state:** `DeepResearcherAgent(llm_provider=..., tools=[...])` builds the middleware and loads prompts; it does *not* build the LangGraph yet. `DeepResearchAgentState(messages=[HumanMessage(...)])` is the input state. When you call `agent.run(state)`, the agent builds the orchestrator graph (see below), invokes it, checks report completeness, and may retry with feedback up to 5 times.\n", "\n", "### What happens internally when you call `agent.run(state)`\n", "\n", "1. **Build the orchestrator:** `run()` calls `_build_orchestrator_agent(state)`. That method loads the orchestrator prompt (Jinja2) with the current tools and state (e.g. `available_documents`), defines a backend for the deep agent (in-memory state plus a `/shared/` route for plan and scratch files), and calls the [deepagents](https://docs.langchain.com/oss/python/deepagents/overview) library’s `create_deep_agent()` with the orchestrator LLM, tools, **subagents** (see snippet below), **middleware** (see snippet below), and store. So the “agent” you run is a LangGraph compiled from that deep agent spec.\n", "\n", "2. **Invoke the graph:** The code runs `await agent.ainvoke(state, config={...})`. The orchestrator node runs first; it may call `write_todos`, then `task(description=..., subagent_type=\"planner-agent\")`. That triggers the planner subagent, which has its own LLM call and tools (same tool list). The planner typically uses search tools and `write_file` to produce `/shared/plan.json`. Control returns to the orchestrator, which may then call `task(..., subagent_type=\"researcher-agent\")` multiple times for different queries. Each researcher run uses the researcher LLM and the same tools to run search and synthesize. The orchestrator reads plan and research outputs, runs more tools (`think`, `read_file`, `grep`), and eventually produces the final report in its last message.\n", "\n", "3. **Report completeness and retries:** After each `ainvoke`, the code calls `_is_report_complete(result)`. The report is considered complete only if the last message has at least 1500 characters, at least two `##` section headers, and a Sources/References section, and does not contain phrases like “please confirm” or “do you want me to”. If incomplete, the code appends a `HumanMessage` with feedback (e.g. “include a Sources section”) and calls `ainvoke` again, up to 5 times.\n", "\n", "4. **Return:** The final state (with all messages, including the final report) is returned as a `DeepResearchAgentState`; the last AI message content is the report." ] }, { "cell_type": "markdown", "id": "cc1f1d5b", "metadata": {}, "source": [ "### Subagent definitions (from `agent.py`)\n", "\n", "The orchestrator delegates to two subagents. They are built in `_get_subagents(state)` and passed to `create_deep_agent()`. Each subagent has its own **name**, **description** (used when the orchestrator chooses a subagent), **system_prompt** (Jinja2-rendered with `tools`, `available_documents`, etc.), **tools** (the same list: `think` + your search tools), **model** (from the provider for that role), and **middleware** (same stack as the orchestrator). Conceptually:\n", "\n", "```python\n", "# _get_subagents(state) returns a list of two dicts:\n", "[\n", " {\n", " \"name\": \"planner-agent\",\n", " \"description\": \"Content-driven research planning - iteratively builds evidence-grounded \"\n", " \"outlines through interleaved search and outline optimization\",\n", " \"system_prompt\": render_prompt_template(self._prompts[\"planner\"], tools=..., available_documents=...),\n", " \"tools\": self.all_tools, # think + your tools (e.g. tavily_search)\n", " \"model\": self.llm_provider.get(LLMRole.PLANNER),\n", " \"middleware\": self.middleware,\n", " },\n", " {\n", " \"name\": \"researcher-agent\",\n", " \"description\": \"Information gathering - executes search queries and synthesizes \"\n", " \"relevant content from available sources\",\n", " \"system_prompt\": render_prompt_template(self._prompts[\"researcher\"], current_datetime=..., tools=..., available_documents=...),\n", " \"tools\": self.all_tools,\n", " \"model\": self.llm_provider.get(LLMRole.RESEARCHER),\n", " \"middleware\": self.middleware,\n", " },\n", "]\n", "```\n", "\n", "The **planner** prompt (from `prompts/planner.j2`) tells the model to analyze the task, use search tools to discover what exists, and write a structured plan to `/shared/plan.json` (task analysis, TOC, constraints, queries). The **researcher** prompt (from `prompts/researcher.j2`) tells the model to run search for a given query, synthesize findings, and cite sources. The orchestrator’s prompt (from `prompts/orchestrator.j2`) describes the workflow steps (write_todos → planning → research → verify → synthesize → write report) and when to call each subagent." ] }, { "cell_type": "markdown", "id": "b4830031", "metadata": {}, "source": [ "### Middleware (from `agent.py`)\n", "\n", "All three agents (orchestrator, planner, researcher) use the same middleware list. It runs in order around model calls:\n", "\n", "```python\n", "self.middleware = [\n", " EmptyContentFixMiddleware(),\n", " ToolNameSanitizationMiddleware(valid_tool_names=[t.name for t in self.all_tools]),\n", " ModelRetryMiddleware(max_retries=10, backoff_factor=2.0, initial_delay=1.0),\n", "]\n", "```\n", "\n", "- **EmptyContentFixMiddleware:** Some LLM APIs reject messages with empty content. This middleware runs before the request is sent to the model: it finds any `ToolMessage` with empty `content` and replaces it with a placeholder string (e.g. `\"empty content received.\"`) so the API call does not fail.\n", "\n", "- **ToolNameSanitizationMiddleware:** LLMs sometimes emit malformed or hallucinated tool names (e.g. with `<|channel|>` suffixes or names like `open_file` instead of `read_file`). This middleware runs after the model response: it rewrites tool names in `AIMessage.tool_calls` to match the allowed `valid_tool_names` (and a small alias map), so the runtime dispatches the correct tool.\n", "\n", "- **ModelRetryMiddleware:** LangChain’s built-in retry for model calls: on failure it retries up to 10 times with exponential backoff (initial_delay 1s, backoff_factor 2.0), so transient API errors are retried automatically." ] }, { "cell_type": "code", "execution_count": null, "id": "5035a85c", "metadata": {}, "outputs": [], "source": [ "# Simplest run: one LLM, one web search tool, then agent.run(state)\n", "import logging\n", "import warnings\n", "\n", "# Suppress langchain_nvidia_ai_endpoints warnings about model tool support\n", "warnings.filterwarnings(\"ignore\", category=UserWarning, module=\"langchain_nvidia_ai_endpoints\")\n", "\n", "logging.basicConfig(level=logging.INFO, format=\"%(levelname)s %(name)s: %(message)s\")\n", "\n", "from langchain_core.messages import HumanMessage\n", "from langchain_core.tools import tool\n", "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", "from langchain_tavily import TavilySearch\n", "\n", "from deep_researcher_agent.agents.deep_researcher.agent import DeepResearcherAgent\n", "from deep_researcher_agent.agents.deep_researcher.models import DeepResearchAgentState\n", "from deep_researcher_agent.common import LLMProvider\n", "from deep_researcher_agent.common import LLMRole\n", "from deep_researcher_agent.common import VerboseTraceCallback\n", "\n", "# One LLM for all roles (orchestrator, planner, researcher)\n", "llm = ChatNVIDIA(\n", " model=\"nvidia/nemotron-3-ultra-550b-a55b\",\n", " base_url=\"https://integrate.api.nvidia.com/v1\",\n", " temperature=0.2,\n", " top_p=0.7,\n", " max_completion_tokens=16384,\n", " model_kwargs={\"chat_template_kwargs\": {\"enable_thinking\": False}},\n", ")\n", "provider = LLMProvider()\n", "provider.set_default(llm)\n", "provider.configure(LLMRole.ORCHESTRATOR, llm)\n", "provider.configure(LLMRole.RESEARCHER, llm)\n", "provider.configure(LLMRole.PLANNER, llm)\n", "\n", "# Web search tool (uses TAVILY_API_KEY from env)\n", "web_search_tool = TavilySearch(\n", " max_results=2,\n", " search_depth=\"advanced\",\n", " include_answer=True,\n", ")\n", "\n", "\n", "@tool(\"advanced_web_search_tool\")\n", "def tavily_search(query: str) -> dict:\n", " \"\"\"Search the web with Tavily using a plain text query.\"\"\"\n", " # Keep the tool signature strict to avoid malformed optional args.\n", " return web_search_tool.invoke({\"query\": query})\n", "\n", "\n", "agent = DeepResearcherAgent(\n", " llm_provider=provider,\n", " tools=[tavily_search],\n", " verbose=True,\n", " callbacks=[VerboseTraceCallback()],\n", ")\n", "state = DeepResearchAgentState(\n", " messages=[\n", " HumanMessage(\n", " content=\"Conduct a comprehensive technical comparison between NVIDIA GB300 and Google Ironwood AI accelerator platforms. Generate a structured report with the following sections: Compute Architecture, Memory Subsystem, Interconnect & Scalability, System & Power, Software Ecosystem and Benchmarks\"\n", " )\n", " ]\n", ")\n", "\n", "result = await agent.run(state)\n", "\n", "# Final report is in the last AI message\n", "from langchain_core.messages import AIMessage\n", "\n", "last_ai = next((m for m in reversed(result.messages) if isinstance(m, AIMessage) and m.content), None)\n", "if last_ai:\n", " print(last_ai.content[:2000] + (\"...\" if len(last_ai.content) > 2000 else \"\"))" ] }, { "cell_type": "markdown", "id": "408362e7", "metadata": {}, "source": [ "\n", "## Using NeMo Agent Toolkit to wrap the Agent\n", "\n", "\n", "**Why use the NeMo Agent toolkit (NAT) for agent development and optimization?** NAT gives you a single YAML-based configuration for LLMs, tools, and workflows so you can swap models and tools without code changes. The **`nat eval`** command runs your workflow on a dataset and scores it with built-in or custom evaluators, and the toolkit’s [evaluation docs](https://docs.nvidia.com/nemo/agent-toolkit/latest/improve-workflows/evaluate.html) describe how to add evaluators and benchmarks. For optimization, NAT’s pipeline (including **`nat optimize`**) supports tuning prompts and hyperparameters against benchmarks through integrations such as genetic-algorithm prompt search and numeric trial search. Profiling and tracing are built in: enable the profiler in your eval config to get latency reports, token metrics, and traces that you can send to observability backends (Phoenix, Weave, or others). For full details, see the [NeMo Agent toolkit documentation](https://docs.nvidia.com/nemo/agent-toolkit/latest/)." ] }, { "cell_type": "markdown", "id": "77c617c0", "metadata": {}, "source": [ "## Simplified config (web search only)\n", "\n", "Here, we will create a simplified configuration that uses one LLM and one tool that focuses only on web search. The web search tool uses **Tavily web search** for internet search capabilities. No paper search, no knowledge layer." ] }, { "cell_type": "code", "execution_count": null, "id": "12bcaa7c", "metadata": {}, "outputs": [], "source": [ "%%writefile config_simple_deep_researcher.yml\n", "\n", "general:\n", " telemetry:\n", " logging:\n", " console:\n", " _type: console\n", " level: INFO\n", "\n", "llms:\n", " nemotron_ultra_llm:\n", " _type: nim\n", " model_name: nvidia/nemotron-3-ultra-550b-a55b\n", " base_url: \"https://integrate.api.nvidia.com/v1\"\n", " api_key: ${NVIDIA_API_KEY}\n", " temperature: 0.2\n", " top_p: 0.7\n", " max_tokens: 16384\n", " num_retries: 5\n", " chat_template_kwargs:\n", " enable_thinking: false\n", "\n", " nemotron_ultra_writer_llm:\n", " _type: nim\n", " model_name: nvidia/nemotron-3-ultra-550b-a55b\n", " base_url: \"https://integrate.api.nvidia.com/v1\"\n", " api_key: ${NVIDIA_API_KEY}\n", " temperature: 0.2\n", " top_p: 0.7\n", " max_tokens: 32768\n", " num_retries: 5\n", " chat_template_kwargs:\n", " enable_thinking: false\n", "\n", "\n", "functions:\n", " advanced_web_search_tool:\n", " _type: tavily_web_search\n", " max_results: 5\n", " advanced_search: true\n", "\n", " deep_research_agent:\n", " _type: deep_research_agent\n", " orchestrator_llm: nemotron_ultra_llm\n", " source_router_llm: nemotron_ultra_llm\n", " planner_llm: nemotron_ultra_llm\n", " researcher_llm: nemotron_ultra_llm\n", " writer_llm: nemotron_ultra_writer_llm\n", " tools:\n", " - advanced_web_search_tool\n", "\n", "workflow:\n", " _type: deep_research_workflow\n" ] }, { "cell_type": "markdown", "id": "d71cb5f9", "metadata": {}, "source": [ "\n", "## Run with NAT CLI" ] }, { "cell_type": "markdown", "id": "beb8dd70", "metadata": {}, "source": [ "**Basic Execution**" ] }, { "cell_type": "code", "execution_count": null, "id": "d3435410", "metadata": {}, "outputs": [], "source": [ "!.venv/bin/nat run \\\n", " --config_file config_simple_deep_researcher.yml \\\n", " --input \"Conduct a comprehensive technical comparison between NVIDIA GB300 and Google Ironwood AI accelerator platforms. Generate a structured report with the following sections: Compute Architecture, Memory Subsystem, Interconnect & Scalability, System & Power, Software Ecosystem and Benchmarks\"" ] }, { "cell_type": "markdown", "id": "2777abb5", "metadata": {}, "source": [ "