{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Document Intake Agent: LLM-Ready Markdown from Any Office Document\n\n### Overview\n\nAgents in production receive files, not clean prompts: a quarterly report as docx, a deck as pptx, a spreadsheet as xlsx, a contract as pdf. Language models read none of these natively. This tutorial builds a document intake agent with LangGraph that accepts an arbitrary incoming file, routes it through the right converter, and grounds its answers in a faithful markdown version of the document instead of a lossy text scrape.\n\nThe conversion step runs as a tool call against the [hushvert conversion API](https://hushvert.com/for-developers), a hosted service for exactly the formats that are painful to convert in pure Python (native office documents and PDFs with real layouts). The same capability is available to MCP hosts like Claude Code and Cursor through a one-block config, shown near the end.\n\n### Motivation\n\nThe naive way to feed a docx or pdf to a model is to rip the raw text out and paste it into the prompt. That silently destroys the structure business documents live on:\n\n- **Tables** collapse into an undifferentiated stream of cell values, so \"which region grew fastest\" becomes a token-counting guessing game.\n- **Reading order** breaks on multi-column layouts, interleaving sentences from different columns.\n- **Headings and lists** flatten away, so the model loses the document's own organization.\n\nConverting to markdown first preserves that structure in a form models are heavily trained on: pipe tables stay tables, headings stay headings, and the agent can quote figures instead of hallucinating around them.\n\n![What each path preserves](../images/document-intake-table-loss.svg)\n\n### Key Components\n\n1. **Format router**: inspects the incoming file and decides whether to read it directly (already text), convert it (office documents, PDF), or refuse it with a useful message. The routing table comes from the converter's live formats endpoint, not a hardcoded list.\n2. **Conversion tool**: a small client for the hushvert conversion API following its submit, upload, poll, download flow.\n3. **Analyst node**: gpt-4o-mini answering questions grounded only in the converted markdown.\n4. **LangGraph state graph** wiring the three together with one conditional edge."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Agent Architecture\n\n![Document Intake Agent](../images/document-intake-agent.svg)\n\nThe routing decision is deterministic on purpose. A ReAct-style agent could decide \"should I convert this?\" with an LLM call on every run, but format routing is a lookup, not a judgment call: putting it in a conditional edge makes the behavior testable and saves a model round trip on every document."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Setting Up the Environment\n\n### Install dependencies"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "!pip install -q langgraph langchain-openai langchain-core requests python-docx python-dotenv"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Configure API keys\n\nTwo keys are needed:\n\n- **OpenAI**: for the analyst model (gpt-4o-mini).\n- **hushvert**: create a free account and an API key at [hushvert.com/developers/keys](https://hushvert.com/developers/keys). Sign-in is a one-time email code (no password), and keys (they look like `hv_live_...`) require a confirmed email. New accounts include a free monthly allowance of server conversions, so this tutorial runs end to end without payment."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "import os\nfrom getpass import getpass\n\nfrom dotenv import load_dotenv\n\n# check for .env file at the repo root\nif os.path.exists(\"../.env\"):\n    load_dotenv(\"../.env\")\n\nif not os.environ.get(\"OPENAI_API_KEY\"):\n    os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\nif not os.environ.get(\"HUSHVERT_API_KEY\"):\n    os.environ[\"HUSHVERT_API_KEY\"] = getpass(\"Enter your hushvert API key (hv_live_...): \")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Import libraries"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "import pathlib\nimport re\nimport time\nimport zipfile\n\nimport requests\nfrom typing_extensions import TypedDict\n\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_openai import ChatOpenAI\nfrom langgraph.graph import StateGraph, START, END"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## The Conversion Layer\n\n### Discover what the API converts\n\nThe formats endpoint is public and returns every server-side pair as a slug like `docx-to-md`, together with its size cap and credit cost. We fetch it once and treat it as the routing table, so the agent's idea of \"convertible\" can never drift from what the API actually supports."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "HUSHVERT_API = \"https://hushvert.com/api\"\n\npairs = requests.get(f\"{HUSHVERT_API}/v1/formats\", timeout=30).json()[\"pairs\"]\nserver_pairs = {p[\"pair\"] for p in pairs}\n\nmd_inputs = sorted(p[\"from\"] for p in pairs if p[\"to\"] == \"md\")\nprint(f\"{len(pairs)} server pairs available\")\nprint(\"Convertible to markdown:\", \", \".join(md_inputs))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### The conversion client\n\nThe API is a three-step flow, the standard shape for services that move real file bytes:\n\n1. **Submit** the pair and byte count. The response carries a `jobId` and a presigned `uploadUrl`.\n2. **Upload** the bytes with a plain HTTP PUT. They go directly to object storage, not through the API host.\n3. **Poll** the job until it reports `done`, then download the result from the presigned `downloadUrl`.\n\nThe optional `Idempotency-Key` header makes retries safe: a resubmission with the same key returns the same job and is never charged twice. Pass a stable key if you wrap this call in retry logic.\n\n![The conversion flow](../images/document-intake-conversion-flow.svg)"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "def convert_file(path: str, to: str, idempotency_key: str | None = None) -> bytes:\n    \"\"\"Convert a local file via the hushvert API: submit, upload, poll, download.\"\"\"\n    src = pathlib.Path(path)\n    data = src.read_bytes()\n    pair = f\"{src.suffix.lstrip('.').lower()}-to-{to}\"\n\n    headers = {\"Authorization\": f\"Bearer {os.environ['HUSHVERT_API_KEY']}\"}\n    if idempotency_key:\n        headers[\"Idempotency-Key\"] = idempotency_key\n\n    # 1. Declare the pair and byte count; get a job id and a presigned upload URL.\n    submit = requests.post(\n        f\"{HUSHVERT_API}/v1/conversions\",\n        json={\"pair\": pair, \"bytes\": len(data)},\n        headers=headers,\n        timeout=30,\n    )\n    submit.raise_for_status()\n    job = submit.json()\n\n    # 2. Upload the bytes straight to object storage.\n    requests.put(job[\"uploadUrl\"], data=data, timeout=300).raise_for_status()\n\n    # 3. Poll until done, then download the result.\n    while True:\n        poll = requests.get(\n            f\"{HUSHVERT_API}/v1/conversions/{job['jobId']}\",\n            headers=headers,\n            timeout=30,\n        )\n        poll.raise_for_status()\n        state = poll.json()\n        if state[\"status\"] == \"done\":\n            result = requests.get(state[\"downloadUrl\"], timeout=300)\n            result.raise_for_status()\n            return result.content\n        if state[\"status\"] == \"failed\":\n            raise RuntimeError(f\"Conversion failed: {state.get('error')}\")\n        time.sleep(2)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Building the Agent\n\n### Define the agent state\n\nThe state carries the file path and question in, and the route, markdown, and answer out. Every node reads what it needs and returns only the keys it changes."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "class AgentState(TypedDict):\n    file_path: str\n    question: str\n    source_format: str\n    route: str\n    markdown: str\n    answer: str"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Node: inspect and route\n\nThree outcomes: the file is already text (read it), the live formats list says it converts to markdown (convert it), or neither (refuse with guidance). Nothing here calls a model."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "NATIVE_TEXT = {\"md\", \"markdown\", \"txt\"}\n\n\ndef inspect_file(state: AgentState) -> dict:\n    ext = pathlib.Path(state[\"file_path\"]).suffix.lstrip(\".\").lower()\n    if ext in NATIVE_TEXT:\n        route = \"native\"\n    elif f\"{ext}-to-md\" in server_pairs:\n        route = \"convert\"\n    else:\n        route = \"unsupported\"\n    return {\"source_format\": ext, \"route\": route}"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Node: convert to markdown\n\nThe node is a thin wrapper around the client above: convert to `md`, decode, store in state."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "def convert_document(state: AgentState) -> dict:\n    markdown = convert_file(state[\"file_path\"], to=\"md\").decode(\"utf-8\")\n    return {\"markdown\": markdown}"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Nodes: read native text and refuse unsupported formats\n\nFiles that are already markdown or plain text skip conversion entirely. Anything unroutable gets a refusal that tells the sender exactly what would have worked, built from the same live formats list."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "def read_native(state: AgentState) -> dict:\n    text = pathlib.Path(state[\"file_path\"]).read_text(encoding=\"utf-8\")\n    return {\"markdown\": text}\n\n\ndef refuse_unsupported(state: AgentState) -> dict:\n    supported = sorted({p.split(\"-to-\")[0] for p in server_pairs if p.endswith(\"-to-md\")})\n    return {\n        \"answer\": (\n            f\"'{state['source_format']}' is not a format I can read. \"\n            f\"Send markdown or plain text directly, or one of: {', '.join(supported)}.\"\n        )\n    }"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Node: analyze with the LLM\n\nThe analyst is deliberately constrained: answer from the document, quote figures exactly, admit when the document does not contain the answer. Grounding instructions like these only work when the document the model sees actually preserves the structure the question is about, which is the entire point of the conversion stage."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)\n\nanalyst_prompt = ChatPromptTemplate.from_template(\n    \"You are a careful document analyst. Answer the question using only the \"\n    \"document below. Quote figures exactly as they appear. If the document \"\n    \"does not contain the answer, say so.\\n\\n\"\n    \"Document (markdown):\\n{markdown}\\n\\n\"\n    \"Question: {question}\"\n)\n\n\ndef analyze(state: AgentState) -> dict:\n    chain = analyst_prompt | llm\n    response = chain.invoke(\n        {\"markdown\": state[\"markdown\"], \"question\": state[\"question\"]}\n    )\n    return {\"answer\": response.content}"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Assemble the graph\n\nOne conditional edge does all the routing; both reading paths converge on the analyst."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "workflow = StateGraph(AgentState)\n\nworkflow.add_node(\"inspect\", inspect_file)\nworkflow.add_node(\"convert\", convert_document)\nworkflow.add_node(\"read_native\", read_native)\nworkflow.add_node(\"refuse\", refuse_unsupported)\nworkflow.add_node(\"analyze\", analyze)\n\nworkflow.add_edge(START, \"inspect\")\nworkflow.add_conditional_edges(\n    \"inspect\",\n    lambda state: state[\"route\"],\n    {\"convert\": \"convert\", \"native\": \"read_native\", \"unsupported\": \"refuse\"},\n)\nworkflow.add_edge(\"convert\", \"analyze\")\nworkflow.add_edge(\"read_native\", \"analyze\")\nworkflow.add_edge(\"refuse\", END)\nworkflow.add_edge(\"analyze\", END)\n\napp = workflow.compile()"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Visualize the workflow"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "from IPython.display import Image, display\n\ndisplay(Image(app.get_graph().draw_mermaid_png()))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Usage Example\n\n### Create a realistic sample document\n\nTo keep the tutorial self-contained we generate the incoming file ourselves: a small quarterly report in docx with a heading, a paragraph, and a real Word table. This is exactly the kind of document that breaks naive extraction."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "from docx import Document\n\ndoc = Document()\ndoc.add_heading(\"Q3 Financial Summary\", level=1)\ndoc.add_paragraph(\n    \"Revenue grew in three of four regions. The board flagged the West region \"\n    \"for review after its second consecutive quarterly decline.\"\n)\n\ntable = doc.add_table(rows=5, cols=3)\ntable.style = \"Table Grid\"\nrows = [\n    (\"Region\", \"Revenue ($M)\", \"Change vs Q2\"),\n    (\"North\", \"128.4\", \"+6.1%\"),\n    (\"South\", \"95.2\", \"+2.8%\"),\n    (\"East\", \"143.9\", \"+11.4%\"),\n    (\"West\", \"61.7\", \"-4.3%\"),\n]\nfor r, row in enumerate(rows):\n    for c, text in enumerate(row):\n        table.rows[r].cells[c].text = text\n\ndoc.save(\"q3_report.docx\")\nprint(\"Wrote q3_report.docx\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### Run the agent\n\nOne invocation: the graph inspects the file, converts it to markdown through the API, and answers a question that can only be answered correctly by reading the table."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "result = app.invoke(\n    {\n        \"file_path\": \"q3_report.docx\",\n        \"question\": \"Which region grew fastest in Q3, and which region should worry us?\",\n    }\n)\n\nprint(result[\"answer\"])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "### What the model actually saw\n\nThe reason the answer is grounded: the table survived conversion as a GFM pipe table, so rows and columns still mean something."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "print(result[\"markdown\"])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Comparison with Naive Extraction\n\nThe intake agent's whole value is what it refuses to lose, so let's measure the loss directly. The cell below implements the \"just rip the text out\" baseline for docx: unzip the container, strip the XML tags, keep whatever falls out."
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "def naive_docx_text(path: str) -> str:\n    \"\"\"The 'just rip the text out' baseline: read the XML, strip the tags.\"\"\"\n    with zipfile.ZipFile(path) as z:\n        xml = z.read(\"word/document.xml\").decode(\"utf-8\")\n    return \" \".join(re.sub(r\"<[^>]+>\", \" \", xml).split())\n\nprint(naive_docx_text(\"q3_report.docx\")[:320])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "Every number survives, but the table does not: `North 128.4 +6.1% South 95.2 +2.8% ...` is a word soup where the association between region, revenue, and change now depends on the model counting tokens correctly. In the converted markdown the same data is a pipe table the model can read row by row.\n\nThe failure gets worse on PDFs. Glyph-level text extraction has no notion of layout, so a two-column page comes out interleaved, with sentence fragments from the left column spliced into the right. Conversion chains that reconstruct the document, rather than scrape its glyphs, preserve column reading order; that difference, not raw text recall, is usually what separates a usable ingestion pipeline from a broken one.\n\n![Two-column reading order](../images/document-intake-reading-order.svg)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Using the Same Capability from Claude Code, Cursor, and Other MCP Hosts\n\nThe graph above is the pattern for embedding document intake inside your own Python agents. If your agent is an MCP host (Claude Code, Cursor, Cline, Zed, Claude Desktop), you do not need any of this plumbing: the same conversion API ships as an MCP server, [`@hushvert/mcp`](https://www.npmjs.com/package/@hushvert/mcp), and the whole integration is one config block:\n\n```json\n{\n  \"mcpServers\": {\n    \"hushvert\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@hushvert/mcp\"],\n      \"env\": { \"HUSHVERT_API_KEY\": \"hv_live_your_key_here\" }\n    }\n  }\n}\n```\n\nThe server exposes `convert_file`, `convert_poll`, `list_formats`, and `check_usage` tools. Ask the agent to \"convert quarterly.pptx to markdown\" and it performs the same submit, upload, poll, download round trip we implemented by hand, then writes the result next to the input.\n\nFor a general introduction to the protocol itself, see the [MCP tutorial](https://github.com/NirDiamant/GenAI_Agents/blob/main/all_agents_tutorials/mcp-tutorial.ipynb) in this repo."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Additional Considerations\n\n- **Metering and quotas.** Server conversions are billed per use: a free monthly allowance, then prepaid credits. `GET /v1/usage` returns the remaining allowance and balance; have batch jobs check it up front. Each pair also carries a size cap (returned by the formats endpoint as `freeMaxBytes`), and oversized uploads are rejected with HTTP 413 rather than charged.\n- **Only pay for what needs a server.** Images, HEIC, audio, archives, and PDF page operations convert client-side in the browser with the open-source [`@hushvert/engine`](https://github.com/hushvert/engine) package (MIT, WebAssembly, the file never leaves the device). If you are building a web app rather than a Python agent, route those pairs through the engine for free and keep the API for the office-document formats a browser cannot do. The MCP server enforces this split: it refuses client-convertible pairs and points to the engine.\n- **Privacy and retention.** Uploaded inputs are deleted the moment a conversion finishes; outputs expire after about an hour. Do not treat the download URL as storage: fetch the result and persist it yourself.\n- **Failure handling.** A job ends `done` or `failed`, and `failed` carries a specific error. A scanned PDF with no text layer, for example, fails up front with a clear message and no charge, instead of returning empty markdown. Combine that with an `Idempotency-Key` and retries become boring, which is what you want.\n- **Where this pattern goes next.** Here the converted markdown feeds a single question. In a real pipeline it is what you chunk and embed for RAG, diff against last quarter's version, or archive next to the original."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Conclusion\n\nWe built an agent whose first move with any incoming file is to normalize it into LLM-ready markdown, and whose answers are grounded in structure that naive extraction throws away. Three ideas are worth keeping even if you swap every component:\n\n1. **Route deterministically.** Format routing is a lookup against a live capability list, not a judgment call for the model.\n2. **Convert, do not scrape.** Tables, headings, and reading order are where document questions are answered; a converter that reconstructs the document beats an extractor that strips it.\n3. **One capability, two surfaces.** The same conversion service serves a Python graph you own and, through MCP, agents you do not (Claude Code, Cursor, and friends), so document intake stays consistent across your stack."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## References\n\n- [hushvert conversion API](https://hushvert.com/for-developers): API documentation, key creation, and format list\n- [`@hushvert/mcp`](https://www.npmjs.com/package/@hushvert/mcp): the MCP server from the config section\n- [`@hushvert/engine`](https://github.com/hushvert/engine): the open-source browser-side engine for client-convertible formats"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}