{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ce62e879-c21f-4a27-8e08-547fb5e88e18",
   "metadata": {},
   "source": [
    "# 部署你的第一个语言模型：本地或云端\n",
    "\n",
    "> 指导文章：[06. 开始实践：部署你的第一个语言模型](https://github.com/Hoper-J/LLM-Guide-and-Demos-zh_CN/blob/master/Guide/06.%20开始实践：部署你的第一个语言模型.md)\n",
    "\n",
    "在之前的章节中，我们已经了解了 Hugging Face 中 `AutoModel` 系列的不同类。现在，我们将使用一个参数量较小的模型，为你进行演示，并向你展示如何使用 FastAPI 和 Flask 将模型部署为 API 服务。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18464b86-548c-4d85-904d-4023e1ad60dc",
   "metadata": {},
   "source": [
    "## 安装库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2acccf64-20c1-4864-bb30-daca602dc459",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: transformers in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (4.56.2)\n",
      "Requirement already satisfied: filelock in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (3.29.0)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.34.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.36.2)\n",
      "Requirement already satisfied: numpy>=1.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2.4.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (26.1)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (6.0.3)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2026.4.4)\n",
      "Requirement already satisfied: requests in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2.33.1)\n",
      "Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.22.2)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.7.0)\n",
      "Requirement already satisfied: tqdm>=4.27 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (4.67.3)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (2025.3.0)\n",
      "Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (1.4.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (4.15.0)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (3.4.7)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (3.11)\n",
      "Requirement already satisfied: urllib3<3,>=1.26 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (2.6.3)\n",
      "Requirement already satisfied: certifi>=2023.5.7 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (2026.2.25)\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: sentencepiece in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (0.2.1)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install transformers\n",
    "%pip install sentencepiece"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b458dcd-25eb-45f6-9684-aa18309be8ff",
   "metadata": {},
   "source": [
    "## 设置模型下载镜像\n",
    "\n",
    "注意，需要在导入 transformers 等模块前进行设置才能起效。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c4fc3c3e-6978-4e08-b174-7349b300df25",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b6c3128-d166-4287-b0e5-99d364c22661",
   "metadata": {},
   "source": [
    "## 选择并加载模型\n",
    "\n",
    "我们选择一个参数量较小的模型，如 `distilgpt2`，这是 GPT-2 的精简版本（或者说蒸馏），只有约 8820 万参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5d765488-51f4-4579-9ed4-44cb68b3ba27",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "# 指定模型名称\n",
    "model_name = \"distilgpt2\"\n",
    "\n",
    "# 加载 Tokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "\n",
    "# 加载预训练模型\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbf8bf8c-139c-437b-a383-6ba260a059db",
   "metadata": {},
   "source": [
    "## 移动模型到合适的设备\n",
    "\n",
    "如果你的计算机有 GPU，可将模型移动到 GPU，加快推理速度。如果你使用的是 Apple 芯片的 Mac，可以移动到 `mps` 上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e12e9eab-6af4-4952-a591-d300b726f0cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GPT2LMHeadModel(\n",
       "  (transformer): GPT2Model(\n",
       "    (wte): Embedding(50257, 768)\n",
       "    (wpe): Embedding(1024, 768)\n",
       "    (drop): Dropout(p=0.1, inplace=False)\n",
       "    (h): ModuleList(\n",
       "      (0-5): 6 x GPT2Block(\n",
       "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (attn): GPT2Attention(\n",
       "          (c_attn): Conv1D(nf=2304, nx=768)\n",
       "          (c_proj): Conv1D(nf=768, nx=768)\n",
       "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
       "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
       "        )\n",
       "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (mlp): GPT2MLP(\n",
       "          (c_fc): Conv1D(nf=3072, nx=768)\n",
       "          (c_proj): Conv1D(nf=768, nx=3072)\n",
       "          (act): NewGELUActivation()\n",
       "          (dropout): Dropout(p=0.1, inplace=False)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "  )\n",
       "  (lm_head): Linear(in_features=768, out_features=50257, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() \n",
    "                      else \"mps\" if torch.backends.mps.is_available() \n",
    "                      else \"cpu\")\n",
    "model.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7471945b-5703-4f1f-b006-3de1c82b2e1f",
   "metadata": {},
   "source": [
    "## 进行推理\n",
    "\n",
    "现在，我们可以使用模型进行文本生成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "776b0ea0-c50a-4a0c-a579-0e9c5dc6c15a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "模型生成的文本：\n",
      "Hello GPT.\n",
      "\n",
      "This article was originally published on The Conversation. Read the original article and follow us on Twitter @theconversation.\n"
     ]
    }
   ],
   "source": [
    "# 设置模型为评估模式\n",
    "model.eval()\n",
    "\n",
    "# 输入文本\n",
    "input_text = \"Hello GPT\"\n",
    "\n",
    "# 编码输入文本\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "inputs = {key: value.to(device) for key, value in inputs.items()}\n",
    "\n",
    "# 生成文本\n",
    "with torch.no_grad():\n",
    "    outputs = model.generate(\n",
    "        **inputs,\n",
    "        max_length=200,\n",
    "        num_beams=5,\n",
    "        no_repeat_ngram_size=2,\n",
    "        early_stopping=False\n",
    "    )\n",
    "\n",
    "# 解码生成的文本\n",
    "generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "print(\"模型生成的文本：\")\n",
    "print(generated_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38d85917-79de-449b-a939-35dd4533d079",
   "metadata": {},
   "source": [
    "## 部署模型为 API 服务（可选）\n",
    "\n",
    "如果你希望将模型部署为一个 API 服务，供其他应用调用，可以使用 `fastapi` 框架。\n",
    "\n",
    "### 使用 FastAPI 部署模型\n",
    "#### 安装 `fastapi` 和 `uvicorn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "63857f59-416b-4aab-8a90-ddbab8704b58",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: fastapi in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (0.136.0)\n",
      "Requirement already satisfied: uvicorn in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (0.44.0)\n",
      "Requirement already satisfied: starlette>=0.46.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from fastapi) (0.52.1)\n",
      "Requirement already satisfied: pydantic>=2.9.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from fastapi) (2.12.3)\n",
      "Requirement already satisfied: typing-extensions>=4.8.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from fastapi) (4.15.0)\n",
      "Requirement already satisfied: typing-inspection>=0.4.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from fastapi) (0.4.2)\n",
      "Requirement already satisfied: annotated-doc>=0.0.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from fastapi) (0.0.4)\n",
      "Requirement already satisfied: click>=7.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from uvicorn) (8.3.2)\n",
      "Requirement already satisfied: h11>=0.8 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from uvicorn) (0.16.0)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from pydantic>=2.9.0->fastapi) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.41.4 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from pydantic>=2.9.0->fastapi) (2.41.4)\n",
      "Requirement already satisfied: anyio<5,>=3.6.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from starlette>=0.46.0->fastapi) (4.13.0)\n",
      "Requirement already satisfied: idna>=2.8 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette>=0.46.0->fastapi) (3.11)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install fastapi uvicorn"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6b96fe34-56dc-44e8-955e-d1192e964d66",
   "metadata": {},
   "source": [
    "#### 创建 API 服务\n",
    "\n",
    "把下面这段代码保存到 `app_fastapi.py` 文件中（如果克隆了仓库，也可以直接运行后续的代码）：\n",
    "\n",
    "```python\n",
    "from fastapi import FastAPI, HTTPException\n",
    "from pydantic import BaseModel\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "import torch\n",
    "\n",
    "# 定义请求体的数据模型\n",
    "class PromptRequest(BaseModel):\n",
    "    prompt: str\n",
    "\n",
    "app = FastAPI()\n",
    "\n",
    "# 加载模型和分词器\n",
    "model_name = \"distilgpt2\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model.to(device)\n",
    "\n",
    "@app.post(\"/generate\")\n",
    "def generate_text(request: PromptRequest):\n",
    "    prompt = request.prompt\n",
    "    if not prompt:\n",
    "        raise HTTPException(status_code=400, detail=\"No prompt provided\")\n",
    "\n",
    "    inputs = tokenizer(prompt, return_tensors=\"pt\").to(device)\n",
    "    with torch.no_grad():\n",
    "        outputs = model.generate(\n",
    "            **inputs,\n",
    "            max_length=200,\n",
    "            num_beams=5,\n",
    "            no_repeat_ngram_size=2,\n",
    "            early_stopping=True\n",
    "        )\n",
    "    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "    return {\"generated_text\": generated_text}\n",
    "\n",
    "```\n",
    "\n",
    "然后在命令行执行以下命令：\n",
    "\n",
    "```bash\n",
    "uvicorn app_fastapi:app --host 0.0.0.0 --port 8000\n",
    "```\n",
    "\n",
    "你应该可以看到：\n",
    "![image-20240914112627874](../Guide/assets/image-20240914131113829.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "bfded818",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FastAPI ready on http://127.0.0.1:8000\n"
     ]
    }
   ],
   "source": [
    "import subprocess, time, requests, sys\n",
    "\n",
    "# 后台启动 FastAPI 服务\n",
    "proc_fastapi = subprocess.Popen(\n",
    "    [sys.executable, \"-m\", \"uvicorn\", \"app_fastapi:app\", \"--port\", \"8000\"],\n",
    "    stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,\n",
    ")\n",
    "\n",
    "# 等待服务就绪\n",
    "for _ in range(60):\n",
    "    try:\n",
    "        if requests.get(\"http://127.0.0.1:8000/docs\").ok:\n",
    "            print(\"FastAPI ready on http://127.0.0.1:8000\")\n",
    "            break\n",
    "    except requests.ConnectionError:\n",
    "        time.sleep(1)\n",
    "else:\n",
    "    raise RuntimeError(\"FastAPI 启动超时\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4676f1cf-3edf-481a-a57b-df47b24b6bbe",
   "metadata": {},
   "source": [
    "#### 通过 API 调用模型\n",
    "\n",
    "现在，你可以访问 [http://localhost:8000/docs](http://localhost:8000/docs) 交互使用，或者通过发送 HTTP 请求来调用模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "17726267-d159-40d3-a768-4d663b8a0d52",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'generated_text': 'Hello GPT.\\n\\nThis article was originally published on The Conversation. Read the original article.'}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.post(\n",
    "    \"http://localhost:8000/generate\",\n",
    "    json={\"prompt\": \"Hello GPT\"}\n",
    ")\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b215876-62b0-42d3-83db-ba4f063605ea",
   "metadata": {},
   "source": [
    "#### 停掉 FastAPI 服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f0150646",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FastAPI stopped\n"
     ]
    }
   ],
   "source": [
    "proc_fastapi.terminate()\n",
    "proc_fastapi.wait()\n",
    "print(\"FastAPI stopped\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0e8ab4-c604-4f10-a2f3-6554d1d6db63",
   "metadata": {},
   "source": [
    "### 使用 Flask 部署模型\n",
    "\n",
    "#### 安装 Flask\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "939cd948-806f-4185-a45d-d0b21a55d7d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: flask in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (3.1.3)\n",
      "Requirement already satisfied: blinker>=1.9.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (1.9.0)\n",
      "Requirement already satisfied: click>=8.1.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (8.3.2)\n",
      "Requirement already satisfied: itsdangerous>=2.2.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (2.2.0)\n",
      "Requirement already satisfied: jinja2>=3.1.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (3.1.6)\n",
      "Requirement already satisfied: markupsafe>=2.1.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (3.0.3)\n",
      "Requirement already satisfied: werkzeug>=3.1.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from flask) (3.1.8)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install flask"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcbfe1db-7990-4b5a-a7e9-f057054796fc",
   "metadata": {},
   "source": [
    "#### 创建 API 服务\n",
    "\n",
    "把下面这段代码保存到 `app_flask.py` 文件中（如果克隆了仓库，也可以直接运行后续的代码）：\n",
    "\n",
    "```python\n",
    "from flask import Flask, request, jsonify\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "import torch\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "# 加载模型和分词器\n",
    "model_name = \"distilgpt2\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model.to(device)\n",
    "\n",
    "@app.route('/generate', methods=['POST'])\n",
    "def generate():\n",
    "    prompt = request.json.get('prompt')\n",
    "    if not prompt:\n",
    "        return jsonify({'error': 'No prompt provided'}), 400\n",
    "\n",
    "    inputs = tokenizer(prompt, return_tensors=\"pt\").to(device)\n",
    "    with torch.no_grad():\n",
    "        outputs = model.generate(\n",
    "            **inputs,\n",
    "            max_length=200,\n",
    "            num_beams=5,\n",
    "            no_repeat_ngram_size=2,\n",
    "            early_stopping=True\n",
    "        )\n",
    "    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
    "    return jsonify({'generated_text': generated_text})\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    app.run(host='0.0.0.0', port=8000)\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "然后在命令行执行以下命令：\n",
    "\n",
    "```bash\n",
    "python app_flask.py\n",
    "```\n",
    "\n",
    "你应该可以看到：\n",
    "![image-20240914112627874](../Guide/assets/20240914143358.png)\n",
    "\n",
    "不过为了不切出 Notebook，也可以直接运行下面的代码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cc0da06d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Flask ready on http://127.0.0.1:8000\n"
     ]
    }
   ],
   "source": [
    "import subprocess, time, requests, sys\n",
    "\n",
    "# 后台启动 Flask 服务\n",
    "proc_flask = subprocess.Popen(\n",
    "    [sys.executable, \"app_flask.py\"],\n",
    "    stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,\n",
    ")\n",
    "\n",
    "for _ in range(60):\n",
    "    try:\n",
    "        r = requests.post(\"http://127.0.0.1:8000/generate\", json={\"prompt\": \"ping\"})\n",
    "        if r.ok:\n",
    "            print(\"Flask ready on http://127.0.0.1:8000\")\n",
    "            break\n",
    "    except requests.ConnectionError:\n",
    "        time.sleep(1)\n",
    "else:\n",
    "    raise RuntimeError(\"Flask 启动超时\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7672d01a-b824-4278-b986-a8d804d650fd",
   "metadata": {},
   "source": [
    "#### 通过 API 调用模型\n",
    "\n",
    "现在，你可以通过发送 HTTP 请求来调用模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "7f211f84-0284-4545-90b9-578fe8bdea1b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'generated_text': 'Hello GPT.\\n\\nThis article was originally published on The Conversation. Read the original article.'}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.post(\n",
    "    \"http://localhost:8000/generate\",\n",
    "    json={\"prompt\": \"Hello GPT\"}\n",
    ")\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1115fd45-02cc-4250-ada8-25501dbf0056",
   "metadata": {},
   "source": [
    "#### 停掉 Flask 服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "cc049748",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Flask stopped\n"
     ]
    }
   ],
   "source": [
    "proc_flask.terminate()\n",
    "proc_flask.wait()\n",
    "print(\"Flask stopped\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ai",
   "language": "python",
   "name": "ai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}