{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Deep Researcher Agent – Customization\n",
"\n",
"This notebook walks through **customizing** the NVIDIA Deep Researcher Agent Blueprint deep researcher: adding tools (paper search), assigning different models to the orchestrator vs planner/researcher, locating and editing prompts, and adding the knowledge layer. \n",
"\n",
"You will:\n",
"\n",
"1. **Add a tool** – Install and configure the paper search tool (Google Scholar via Serper) and attach it to the deep research agent.\n",
"2. **Customize models** – Use a frontier model for orchestration and planning, and Nemotron for research.\n",
"3. **Locate and customize prompts** – Find where orchestrator, planner, and researcher prompts live and what variables they use.\n",
"4. **Add the knowledge layer** – Enable document retrieval so the agent can search over your ingested documents alongside web and paper search."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table of Contents\n",
"\n",
">[Prerequisites](#prereqs) \n",
">[1. Adding a tool: Paper search](#add-tool) \n",
">[2. Customizing models (orchestrator vs planner/researcher)](#customize-models) \n",
">[3. Prompts: Where they are and how to customize](#prompts) \n",
">[4. Adding the knowledge layer](#knowledge-layer) \n",
">[Putting it together](#put-together) \n",
">[Next steps](#next) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Prerequisites\n",
"\n",
"This notebook assumes you have completed the environment setup from the [0_Getting_Started_with_DEEP_RESEARCHER](0_Getting_Started_with_Deep_Researcher_Agent.ipynb) and run through [1_Deep_Researcher_Web_Search](1_Deep_Researcher_Web_Search.ipynb).\n",
"\n",
"**Ensure you have:**\n",
"- Cloned the repository and run `./scripts/setup.sh`\n",
"- Activated the virtual environment: `source .venv/bin/activate`\n",
"- API keys (you will be prompted in the next cells): **NVIDIA_API_KEY**, **TAVILY_API_KEY**; for paper search, **SERPER_API_KEY** from [serper.dev](https://serper.dev/)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from pathlib import Path\n",
"\n",
"if os.environ.get(\"REPO_ROOT\"):\n",
" os.chdir(os.environ[\"REPO_ROOT\"])\n",
"else:\n",
" cwd = Path.cwd()\n",
" if not (cwd / \"configs\").exists() and not (cwd / \"src\").exists():\n",
" root = cwd.parent.parent\n",
" if (root / \"configs\").is_dir() or (root / \"src\").is_dir():\n",
" os.chdir(root)\n",
"print(\"Working directory:\", os.getcwd())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"\n",
"if \"NVIDIA_API_KEY\" not in os.environ:\n",
" nvidia_api_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n",
" os.environ[\"NVIDIA_API_KEY\"] = nvidia_api_key\n",
"\n",
"if \"TAVILY_API_KEY\" not in os.environ:\n",
" tavily_api_key = getpass.getpass(\"Enter your Tavily API key: \")\n",
" os.environ[\"TAVILY_API_KEY\"] = tavily_api_key"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 1. Adding a tool: Paper search\n",
"\n",
"The deep researcher can use multiple search tools. The **paper search** tool queries academic papers through Google Scholar (Serper API). Adding it is a two-step process: set up a Serper API key, install the plugin, then add the function and tool list in your config.\n",
"\n",
"**Reference:** [sources/google_scholar_paper_search/README.md](../../sources/google_scholar_paper_search/README.md), [Tools and Sources](../source/customization/tools-and-sources.md) (enable/disable by editing `tools` lists)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.0 Set up Serper API key (required for paper search)\n",
"\n",
"The paper search tool uses the [Serper](https://serper.dev/) API to query Google Scholar. You must have a Serper API key before installing or using the tool.\n",
"\n",
"**Steps:**\n",
"1. Go to [serper.dev](https://serper.dev/) and create an account.\n",
"2. From your dashboard, generate an API key.\n",
"3. Add the key to your environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"\n",
"if \"SERPER_API_KEY\" not in os.environ:\n",
" serper_api_key = getpass.getpass(\"Enter your Serper API key (or press Enter to skip): \")\n",
" if serper_api_key:\n",
" os.environ[\"SERPER_API_KEY\"] = serper_api_key\n",
" print(\"SERPER_API_KEY set for this session.\")\n",
" else:\n",
" print(\"Skipped. Set SERPER_API_KEY in deploy/.env or run this cell again to use paper search.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.1 Install the paper search plugin\n",
"\n",
"From the repo root, install the plugin so it registers the `paper_search` function type with the NeMo Agent toolkit:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!uv pip install -e sources/google_scholar_paper_search"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!nat info components -t function 2>/dev/null | grep -E 'paper_search|paper_search_tool' || echo 'Run: nat info components -t function'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 Add to your configuration\n",
"\n",
"In your workflow YAML you need to:\n",
"\n",
"1. **Define the function** under `functions:` with `_type: paper_search` and options such as `max_results`, `serper_api_key` (or rely on `SERPER_API_KEY` in the environment).\n",
"2. **Add the tool to the deep research agent** by including its name (for example `paper_search_tool`) in `deep_research_agent.tools`.\n",
"\n",
"Example snippet:\n",
"\n",
"```yaml\n",
"functions:\n",
" paper_search_tool:\n",
" _type: paper_search\n",
" max_results: 5\n",
" serper_api_key: ${SERPER_API_KEY}\n",
"\n",
" deep_research_agent:\n",
" _type: deep_research_agent\n",
" orchestrator_llm: nemotron_ultra_llm\n",
" tools:\n",
" - paper_search_tool\n",
" - advanced_web_search_tool\n",
"```\n",
"\n",
"The agent’s prompts (orchestrator, planner, researcher) already mention `paper_search_tool` when that tool is present; they use the tool name to show the right instructions. No code changes are required beyond config."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile docs/notebooks/config_deep_researcher_with_paper_search.yml\n",
"\n",
"general:\n",
" telemetry:\n",
" logging:\n",
" console:\n",
" _type: console\n",
" level: INFO\n",
"\n",
"llms:\n",
" nemotron_ultra_llm:\n",
" _type: nim\n",
" model_name: nvidia/nemotron-3-ultra-550b-a55b\n",
" base_url: \"https://integrate.api.nvidia.com/v1\"\n",
" api_key: ${NVIDIA_API_KEY}\n",
" temperature: 0.2\n",
" top_p: 0.7\n",
" max_tokens: 16384\n",
" num_retries: 5\n",
" chat_template_kwargs:\n",
" enable_thinking: false\n",
"\n",
" nemotron_ultra_writer_llm:\n",
" _type: nim\n",
" model_name: nvidia/nemotron-3-ultra-550b-a55b\n",
" base_url: \"https://integrate.api.nvidia.com/v1\"\n",
" api_key: ${NVIDIA_API_KEY}\n",
" temperature: 0.2\n",
" top_p: 0.7\n",
" max_tokens: 32768\n",
" num_retries: 5\n",
" chat_template_kwargs:\n",
" enable_thinking: false\n",
"\n",
"\n",
"functions:\n",
" advanced_web_search_tool:\n",
" _type: tavily_web_search\n",
" max_results: 5\n",
" advanced_search: true\n",
"\n",
" paper_search_tool:\n",
" _type: paper_search\n",
" max_results: 5\n",
" serper_api_key: ${SERPER_API_KEY}\n",
"\n",
" deep_research_agent:\n",
" _type: deep_research_agent\n",
" orchestrator_llm: nemotron_ultra_llm\n",
" source_router_llm: nemotron_ultra_llm\n",
" planner_llm: nemotron_ultra_llm\n",
" researcher_llm: nemotron_ultra_llm\n",
" writer_llm: nemotron_ultra_writer_llm\n",
" tools:\n",
" - paper_search_tool\n",
" - advanced_web_search_tool\n",
"\n",
"workflow:\n",
" _type: deep_research_workflow\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.3 Run with paper search (optional)\n",
"\n",
"If you have `SERPER_API_KEY` set in `deploy/.env`, you can run the deep research workflow with paper search using the default CLI config (which already includes `paper_search_tool`). This may take several minutes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!nat run --config_file docs/notebooks/config_deep_researcher_with_paper_search.yml --input \"Recent advances in transformer architecture for LLMs\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.4 What to expect\n",
"\n",
"When you run the workflow with paper search enabled, check the logs for tool invocations. You should see **`paper_search_tool`** (or `paper_search`) listed in tool-call messages as the planner and researcher run. For example: `[Tool Start] paper_search_tool` or `→ paper_search_tool` in the trace output. The agent may call it alongside `advanced_web_search_tool` for academic-style queries. If paper search is never called, the query may not have triggered academic search in the plan; try a more explicitly scholarly prompt (for example, \"recent peer-reviewed advances in …\")."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 2. Customizing models by deep-research role\n",
"\n",
"The deep research agent has five configurable LLM roles:\n",
"\n",
"| Role | Config key | Purpose |\n",
"|------|------------|--------|\n",
"| **Orchestrator** | `orchestrator_llm` | Coordinates the workflow and supplies the default for omitted roles (required). |\n",
"| **Source router** | `source_router_llm` | Selects the source-routing strategy (optional; defaults to orchestrator). |\n",
"| **Planner** | `planner_llm` | Builds the research plan and queries (optional; defaults to orchestrator). |\n",
"| **Researcher** | `researcher_llm` | Runs search and synthesizes content (optional; defaults to orchestrator). |\n",
"| **Writer** | `writer_llm` | Produces the final report (optional; defaults to orchestrator). |\n",
"\n",
"You can assign **different models** per role. For example: use a **frontier model** for orchestration and planning, and **Nemotron** for research.\n",
"\n",
"**Reference:** [Configuration Reference](../source/customization/configuration-reference.md)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.1 Example: Frontier orchestrator and planner, Nemotron researcher\n",
"\n",
"Define two (or three) LLMs in `llms:` and point the deep research agent at them:\n",
"\n",
"```yaml\n",
"llms:\n",
" nemotron_ultra_llm:\n",
" _type: nim\n",
" model_name: nvidia/nemotron-3-ultra-550b-a55b\n",
" base_url: \"https://integrate.api.nvidia.com/v1\"\n",
" temperature: 1.0\n",
" max_tokens: 128000\n",
"\n",
"\n",
" frontier_llm:\n",
" _type: openai\n",
" model_name: \"gpt-5.2\"\n",
" api_key: ${OPENAI_API_KEY}\n",
" temperature: 1.0\n",
" max_tokens: 128000\n",
"\n",
"functions:\n",
" deep_research_agent:\n",
" _type: deep_research_agent\n",
" orchestrator_llm: frontier_llm \n",
" planner_llm: frontier_llm\n",
" researcher_llm: nemotron_ultra_llm\n",
" tools:\n",
" - advanced_web_search_tool\n",
"```\n",
"\n",
"If you omit `planner_llm` or `researcher_llm`, the agent falls back to `orchestrator_llm` for that role. The registration and wiring are in `src/deep_researcher_agent/agents/deep_researcher/register.py` (it resolves each LLM ref and configures the shared `LLMProvider`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 3. Prompts: Where they are and how to customize\n",
"\n",
"The deep researcher uses **Jinja2** templates for the orchestrator, planner, and researcher. Editing these files is the main way to change behavior (tone, structure, tool prioritization, or workflow steps).\n",
"\n",
"**Reference:** [Deep Researcher README](../../src/deep_researcher_agent/agents/deep_researcher/README.md) (Prompts section), `src/deep_researcher_agent/agents/deep_researcher/agent.py` (`_load_prompts`, `render_prompt_template`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.1 Where the prompts live\n",
"\n",
"All three prompts are under `src/deep_researcher_agent/agents/deep_researcher/prompts/`:\n",
"\n",
"| File | Role | Purpose |\n",
"|------|------|--------|\n",
"| `orchestrator.j2` | Orchestrator | System instructions: workflow steps, progress tracking, when to call planner/researcher, report format. |\n",
"| `planner.j2` | Planner subagent | Task analysis, TOC, constraints, query generation; tool prioritization (knowledge, paper, web). |\n",
"| `researcher.j2` | Researcher subagent | How to gather and synthesize information; tool use order; depth and citation requirements. |\n",
"\n",
"At runtime the agent loads them with `load_prompt(AGENT_DIR / \"prompts\", name)` (see `agent.py`). No config switch is needed to “enable” a prompt; changing the file and restarting (or re-running) is enough."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"prompts_dir = Path(\"src/deep_researcher_agent/agents/deep_researcher/prompts\")\n",
"for f in sorted(prompts_dir.glob(\"*.j2\")):\n",
" print(f.name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 Template variables\n",
"\n",
"The templates are rendered with variables so the same prompt adapts to tools and context:\n",
"\n",
"| Variable | Used in | Description |\n",
"|----------|--------|-------------|\n",
"| `tools` | All three | List of tool info (name, etc.); prompts use it to show “you have paper_search_tool, advanced_web_search_tool, …” and prioritization. |\n",
"| `current_datetime` | Orchestrator, researcher | Injected at render time for date-aware answers. |\n",
"| `clarifier_result` | Orchestrator | User clarifications from the clarifier agent, if any. |\n",
"| `available_documents` | Orchestrator, planner, researcher | User-uploaded docs (e.g. from knowledge layer) with summaries; prompts tell the agent to prioritize knowledge_search for these. |\n",
"\n",
"The logic that builds these and calls `render_prompt_template` is in `agent.py` (`_build_orchestrator_agent`, `_get_subagents`). To add a new variable, add it to the `render_prompt_template(...)` call and use it in the corresponding `.j2` file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.3 How to customize\n",
"\n",
"1. **Edit the `.j2` file** for the role you care about (orchestrator, planner, or researcher).\n",
"2. **Keep existing variables** – The templates use `{{ current_datetime }}`, `{% for tool in tools %}`, `available_documents`, etc. Removing or renaming these can break rendering.\n",
"3. **Adjust wording, steps, or priorities** – For example, in `researcher.j2` you can strengthen “cite sources inline” or add domain-specific guidance; in `orchestrator.j2` you can add or reorder workflow steps (as long as the tools and subagent names stay consistent).\n",
"4. **Restart or re-run** – The agent loads prompts at initialization; after saving changes, restart the process or re-run the notebook so the new content is used."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 4. Adding the knowledge layer\n",
"\n",
"The **knowledge layer** lets the agent search over **your ingested documents** (PDFs, docs, etc.) in addition to web and paper search. It is implemented as a NAT function type `knowledge_retrieval` with pluggable backends (LlamaIndex + ChromaDB for local dev, or Foundational RAG for a hosted server).\n",
"\n",
"**Reference:** [Knowledge Layer](../source/customization/knowledge-layer.md), `sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md` (if present), `configs/config_web_default_llamaindex.yml`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.1 Install a backend\n",
"\n",
"Choose one backend and install it. For local development without extra services, use the LlamaIndex backend:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# LlamaIndex backend (local ChromaDB; no external server)\n",
"!uv pip install -e \"sources/knowledge_layer[llamaindex]\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from deep_researcher_agent.knowledge import get_retriever\n",
"\n",
"print(\"Knowledge layer OK:\", get_retriever is not None)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4.2 Add to configuration\n",
"\n",
"In your YAML:\n",
"\n",
"1. **Define the function** under `functions:` with `_type: knowledge_retrieval`, `backend: llamaindex` (or `foundational_rag`), plus backend-specific options (`collection_name`, `top_k`, `chroma_dir` for LlamaIndex; `rag_url` / `ingest_url` for Foundational RAG).\n",
"2. **Add the tool to the deep research agent** by including the function name (for example `knowledge_search`) in `deep_research_agent.tools`.\n",
"\n",
"Example for LlamaIndex (see `configs/config_web_default_llamaindex.yml`):\n",
"\n",
"```yaml\n",
"functions:\n",
" knowledge_search:\n",
" _type: knowledge_retrieval\n",
" backend: llamaindex\n",
" collection_name: ${COLLECTION_NAME:-test_collection}\n",
" top_k: 5\n",
" chroma_dir: ${DEEP_RESEARCHER_CHROMA_DIR:-/tmp/chroma_data}\n",
"\n",
" deep_research_agent:\n",
" _type: deep_research_agent\n",
" orchestrator_llm: nemotron_ultra_llm\n",
" tools:\n",
" - advanced_web_search_tool\n",
" - knowledge_search\n",
"```\n",
"\n",
"The deep researcher prompts already include branches for `knowledge_search` (and `available_documents`); once the tool is in the list, the orchestrator and subagents will be instructed to use it for user documents and internal context."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile docs/notebooks/config_deep_researcher_with_knowledge_layer.yml\n",
"\n",
"general:\n",
" telemetry:\n",
" logging:\n",
" console:\n",
" _type: console\n",
" level: INFO\n",
"\n",
"llms:\n",
" nemotron_ultra_llm:\n",
" _type: nim\n",
" model_name: nvidia/nemotron-3-ultra-550b-a55b\n",
" base_url: \"https://integrate.api.nvidia.com/v1\"\n",
" api_key: ${NVIDIA_API_KEY}\n",
" temperature: 0.2\n",
" top_p: 0.7\n",
" max_tokens: 16384\n",
" num_retries: 5\n",
" chat_template_kwargs:\n",
" enable_thinking: false\n",
"\n",
" nemotron_ultra_writer_llm:\n",
" _type: nim\n",
" model_name: nvidia/nemotron-3-ultra-550b-a55b\n",
" base_url: \"https://integrate.api.nvidia.com/v1\"\n",
" api_key: ${NVIDIA_API_KEY}\n",
" temperature: 0.2\n",
" top_p: 0.7\n",
" max_tokens: 32768\n",
" num_retries: 5\n",
" chat_template_kwargs:\n",
" enable_thinking: false\n",
"\n",
"\n",
"functions:\n",
" advanced_web_search_tool:\n",
" _type: tavily_web_search\n",
" max_results: 5\n",
" advanced_search: true\n",
"\n",
" knowledge_search:\n",
" _type: knowledge_retrieval\n",
" backend: llamaindex\n",
" collection_name: ${COLLECTION_NAME:-test_collection}\n",
" top_k: 5\n",
" chroma_dir: ${DEEP_RESEARCHER_CHROMA_DIR:-/tmp/chroma_data}\n",
"\n",
" deep_research_agent:\n",
" _type: deep_research_agent\n",
" orchestrator_llm: nemotron_ultra_llm\n",
" source_router_llm: nemotron_ultra_llm\n",
" planner_llm: nemotron_ultra_llm\n",
" researcher_llm: nemotron_ultra_llm\n",
" writer_llm: nemotron_ultra_writer_llm\n",
" tools:\n",
" - advanced_web_search_tool\n",
" - knowledge_search\n",
"\n",
"workflow:\n",
" _type: deep_research_workflow\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Putting it together\n",
"\n",
"A single config can combine all of the above:\n",
"\n",
"- **Tools:** `paper_search_tool`, `advanced_web_search_tool`, `knowledge_search` in `deep_research_agent.tools`.\n",
"- **Models:** `orchestrator_llm: frontier_llm`, `source_router_llm: nemotron_ultra_llm`, `planner_llm: nemotron_ultra_llm`, `researcher_llm: nemotron_ultra_llm`, and `writer_llm: nemotron_ultra_writer_llm` (with those LLMs defined in `llms:`).\n",
"- **Prompts:** No config change; edit `src/deep_researcher_agent/agents/deep_researcher/prompts/*.j2` as needed.\n",
"- **Knowledge layer:** Install a backend, define `knowledge_search` in `functions`, and add it to `deep_research_agent.tools`.\n",
"\n",
"See `configs/config_web_default_llamaindex.yml` and `configs/config_web_frag.yml` for full examples that mix web search, paper search, knowledge layer, and different LLMs."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Next steps\n",
"\n",
"- **Guide to Extending** (see [Extending Documentation](../source/extending/index.md))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}