{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xvSGDbExff_I"
   },
   "source": [
    "# 作业 - Bert (Extractive Question Answering)\n",
    "\n",
    "> 本作业迁移自李宏毅老师 2023 年机器学习春的作业 7，原链接：[ML2023_HW7_Question_Answering](https://colab.research.google.com/drive/1m0fQjJfkK9vAovxPj9Nd3-hQuxezB2w1)\n",
    ">\n",
    "> 课程主页：[Machine Learning 2023 Spring](https://speech.ee.ntu.edu.tw/~hylee/ml/2023-spring.php)\n",
    ">\n",
    "> 指导文章：[22a. 微调 LLM：实现抽取式问答](../Guide/22a.%20微调%20LLM：实现抽取式问答.md) | 作业文章：[22b. 作业 - Bert 微调抽取式问答](../Guide/22b.%20作业%20-%20Bert%20微调抽取式问答.md)\n",
    "\n",
    "请根据**提示**完成相应的代码修改，从而达到学习的目的。\n",
    "\n",
    "[作业 PPT](https://github.com/Hoper-J/AI-Guide-and-Demos-zh_CN/blob/master/Demos/PDF/HW_BERT_Question_Answering.pdf) | [Kaggle 加入链接](https://www.kaggle.com/t/e001cad568dc4d77b6a5e762172f44d6)\n",
    "\n",
    "在线链接：[Kaggle](https://www.kaggle.com/code/aidemos/21-bert) | [Colab](https://colab.research.google.com/drive/1zHR2Cztmo49j3yrdT3GgkzNcAcHd7M0M?usp=sharing)\n",
    "\n",
    "\n",
    "## 作业提示\n",
    "\n",
    "> 如果理解了每个提示到底需要做什么，那么恭喜你已经完成了学习。\n",
    ">\n",
    "> 搜索代码文件中的 TODO 可以快速索引到需要修改的地方。\n",
    "\n",
    "- **Simple baseline**\n",
    "  - 直接运行当前代码文件\n",
    "- **Medium baseline**\n",
    "  - 使用学习率调度器：线性衰减，warmup 等，选择一个即可\n",
    "- **Strong baseline**\n",
    "  - 修改模型的预处理过程 (TODO: Preprocessing)\n",
    "  - 尝试替换为其他预训练模型，在 [Hugging Face](https://huggingface.co/models?language=zh&sort=trending) 中进行选择\n",
    "- **Boss baseline**\n",
    "  - 修改后处理部分 (TODO: Postprocessing)\n",
    "  - 尝试梯度累积\n",
    "\n",
    "## Kaggle 加入“比赛”\n",
    "\n",
    "访问 [Kaggle 加入链接](https://www.kaggle.com/t/e001cad568dc4d77b6a5e762172f44d6)，点击右上角的 `Late Submission`，然后点击 `I Understand and Accept` 同意：\n",
    "\n",
    "![image-20241110191905666](../Guide/assets/image-20241110191905666.png)\n",
    "\n",
    "样例代码运行后会生成文件：`result.csv`，如果想要提交查看分数，再次点击 `Late Submission`，上传后点击提交：\n",
    "\n",
    "![image-20241110192134533](../Guide/assets/image-20241110192134533.png)\n",
    "\n",
    "你可以在 `Submissions` 中看到自己的提交成绩：\n",
    "\n",
    "![image-20241110192753327](../Guide/assets/image-20241110192753327.png)\n",
    "\n",
    "> [!note]\n",
    ">\n",
    "> 在评估效果时最好仅参考 Public Score，Private Score 是测试集上的分数，因为比赛已经过了Deadline，所以显示了出来，比赛进行的过程中这个分数是隐藏的。\n",
    "\n",
    "点击 `Leaderboard` 你将能够看到曾经参与者的成绩：\n",
    "\n",
    "![image-20241110192649655](../Guide/assets/image-20241110192649655.png)\n",
    "\n",
    "\n",
    "## 任务目标：中文抽取式问答\n",
    "\n",
    "学习如何使用 transformers 微调预训练模型，以完成下游抽取式问答任务。\n",
    "\n",
    "- **任务描述**: 通过 fine-tune BERT 模型，使其能够从给定的段落中抽取出问题的具体答案。\n",
    "  - **目标**: 模型将预测答案的开始（Start）和结束（End）位置，并从文本中提取该片段作为答案。\n",
    "    \n",
    "    ![image-20240920002112364](../Guide/assets/e6a56c94334632bae17b286f42264de4.png)\n",
    "    \n",
    "  - **输入**: 段落 + 问题\n",
    "  - **输出**: 答案\n",
    " \n",
    "- **TODO**\n",
    "  - 微调预训练的中文 BERT 模型\n",
    "  - 调整超参数（如 `doc_stride`）\n",
    "  - 应用线性学习率衰减策略\n",
    "  - 尝试其他预训练模型\n",
    "  - 改进数据的预处理（preprocessing）\n",
    "  - 改进模型的后处理（postprocessing）\n",
    "\n",
    "- **训练技巧**\n",
    "  - 自动混合精度（Automatic Mixed Precision）\n",
    "  - 梯度累积（Gradient Accumulation）\n",
    "  - 集成方法（Ensemble）\n",
    "\n",
    "- **预计训练时间**（在 Tesla T4 上启用自动混合精度时的运行时间）：\n",
    "  - Simple baseline：8 分钟\n",
    "  - Medium baseline：8 分钟\n",
    "  - Strong baseline：25 分钟\n",
    "  - Boss baseline：2 小时\n",
    "\n",
    "## 性能指标（EM）\n",
    "\n",
    "- **准确率 (Exact Match)**: 该指标用于衡量模型的预测答案与真实答案**完全一致**的比例。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NYAHsHNbzdKm"
   },
   "source": [
    "## 前置准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TJ1fSAJE2oaC"
   },
   "source": [
    "### 下载数据集\n",
    "\n",
    "#### Kaggle\n",
    "\n",
    "> To use the Kaggle API, sign up for a Kaggle account at [https://www.kaggle.com](https://www.kaggle.com/). Then go to the 'Account' tab of your user profile (`https://www.kaggle.com/<username>/account`) and select 'Create API Token'. This will trigger the download of `kaggle.json`, a file containing your API credentials. Place this file in the location `~/.kaggle/kaggle.json` (on Windows in the location `C:\\Users\\<Windows-username>\\.kaggle\\kaggle.json` - you can check the exact location, sans drive, with `echo %HOMEPATH%`). You can define a shell environment variable `KAGGLE_CONFIG_DIR` to change this location to `$KAGGLE_CONFIG_DIR/kaggle.json` (on Windows it will be `%KAGGLE_CONFIG_DIR%\\kaggle.json`).\n",
    ">\n",
    "> -\\- [Official Kaggle API](https://github.com/Kaggle/kaggle-api)\n",
    "\n",
    "替换\\<username\\>为你自己的用户名，`https://www.kaggle.com/<username>/account`，然后点击 `Create New API Token`，将下载下来的文件放去应该放的位置：\n",
    "\n",
    "- Mac 和 Linux 放在 `~/.kaggle`\n",
    "- Windows 放在 `C:\\Users\\<Windows-username>\\.kaggle`\n",
    "\n",
    "```bash\n",
    "pip install kaggle\n",
    "# 你需要先在 Kaggle -> Account -> Create New API Token 中下载 kaggle.json\n",
    "# mv kaggle.json ~/.kaggle/kaggle.json\n",
    "kaggle competitions download -c ml2023spring-hw7\n",
    "unzip ml2023spring-hw7.zip   \n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YPrc4Eie9Yo5",
    "outputId": "1640c875-58d1-4288-e382-e9fa9db39b9f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "数据集已存在，跳过下载\n"
     ]
    }
   ],
   "source": [
    "# 或者通过以下命令进行下载\n",
    "import os\n",
    "\n",
    "if all(os.path.exists(f) for f in [\"hw7_train.json\", \"hw7_dev.json\", \"hw7_test.json\"]):\n",
    "    print(\"数据集已存在，跳过下载\")\n",
    "else:\n",
    "    !wget -nc https://github.com/Hoper-J/HUNG-YI_LEE_Machine-Learning_Homework/raw/refs/heads/master/HW07/ml2023spring-hw7.zip\n",
    "    !unzip -o ml2023spring-hw7.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TevOvhC03m0h"
   },
   "source": [
    "### 安装库 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tbxWFX_jpDom",
    "outputId": "138488c2-96fa-4c41-d450-9d0a1b74f36c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: transformers in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (4.56.2)\n",
      "Requirement already satisfied: filelock in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (3.29.0)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.34.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.36.2)\n",
      "Requirement already satisfied: numpy>=1.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2.4.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (26.1)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (6.0.3)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2026.4.4)\n",
      "Requirement already satisfied: requests in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (2.33.1)\n",
      "Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.22.2)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (0.7.0)\n",
      "Requirement already satisfied: tqdm>=4.27 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from transformers) (4.67.3)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (2025.3.0)\n",
      "Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (1.4.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (4.15.0)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (3.4.7)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (3.11)\n",
      "Requirement already satisfied: urllib3<3,>=1.26 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (2.6.3)\n",
      "Requirement already satisfied: certifi>=2023.5.7 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->transformers) (2026.2.25)\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Looking in indexes: http://mirrors.aliyun.com/pypi/simple\n",
      "Requirement already satisfied: accelerate in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (1.13.0)\n",
      "Requirement already satisfied: numpy>=1.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (2.4.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (26.1)\n",
      "Requirement already satisfied: psutil in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (7.2.2)\n",
      "Requirement already satisfied: pyyaml in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (6.0.3)\n",
      "Requirement already satisfied: torch>=2.0.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (2.7.1)\n",
      "Requirement already satisfied: huggingface_hub>=0.21.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (0.36.2)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from accelerate) (0.7.0)\n",
      "Requirement already satisfied: filelock in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (3.29.0)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (2025.3.0)\n",
      "Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (1.4.3)\n",
      "Requirement already satisfied: requests in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (2.33.1)\n",
      "Requirement already satisfied: tqdm>=4.42.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (4.67.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from huggingface_hub>=0.21.0->accelerate) (4.15.0)\n",
      "Requirement already satisfied: setuptools in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (82.0.1)\n",
      "Requirement already satisfied: sympy>=1.13.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (1.14.0)\n",
      "Requirement already satisfied: networkx in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (3.6.1)\n",
      "Requirement already satisfied: jinja2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (3.1.6)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.6.77 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.77)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.6.77 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.77)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.6.80 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.80)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==9.5.1.17 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (9.5.1.17)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.6.4.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.4.1)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.3.0.4 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (11.3.0.4)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.7.77 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (10.3.7.77)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.7.1.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (11.7.1.2)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.5.4.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.5.4.2)\n",
      "Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (0.6.3)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (2.26.2)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.6.77 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.77)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12==12.6.85 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (12.6.85)\n",
      "Requirement already satisfied: nvidia-cufile-cu12==1.11.1.6 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (1.11.1.6)\n",
      "Requirement already satisfied: triton==3.3.1 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from torch>=2.0.0->accelerate) (3.3.1)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from sympy>=1.13.3->torch>=2.0.0->accelerate) (1.3.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from jinja2->torch>=2.0.0->accelerate) (3.0.3)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->huggingface_hub>=0.21.0->accelerate) (3.4.7)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->huggingface_hub>=0.21.0->accelerate) (3.11)\n",
      "Requirement already satisfied: urllib3<3,>=1.26 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->huggingface_hub>=0.21.0->accelerate) (2.6.3)\n",
      "Requirement already satisfied: certifi>=2023.5.7 in /root/autodl-tmp/AI-Guide-and-Demos-zh_CN/.venv/lib/python3.12/site-packages (from requests->huggingface_hub>=0.21.0->accelerate) (2026.2.25)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install transformers\n",
    "%pip install accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XapBp31gytyD"
   },
   "source": [
    "## 开始微调\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8dKM4yCh4LI_"
   },
   "source": [
    "### 导入包\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "WOTHHtWJoahe"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np\n",
    "import random\n",
    "import torch\n",
    "from torch.utils.data import DataLoader, Dataset\n",
    "from torch.optim import AdamW\n",
    "\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "\n",
    "# 为了可重复性，固定随机数种子\n",
    "def same_seeds(seed):\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.cuda.is_available():\n",
    "        torch.cuda.manual_seed(seed)\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "    np.random.seed(seed)\n",
    "    random.seed(seed)\n",
    "    torch.backends.cudnn.benchmark = False\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "same_seeds(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2YgXHuVLp_6j"
   },
   "source": [
    "### 加载模型和分词器\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220,
     "referenced_widgets": [
      "641665cb9ec6433a81719d57eeaf8298",
      "ba9c1cd3cfe04add90aa2edf2aa87378",
      "1e20837bf23548249678da0a07745d58",
      "c1d24cb7607c44b989774113a2baf675",
      "c9144705a33848a6b142f9887da6ac49",
      "ebfa4ba9f18b4945947d326581d3b8e9",
      "28fc746a3c0548f4a4ff669b5dbb7320",
      "d26e50974c52452389c5abb001056546",
      "9eefbb580f0146658d18658f210d3a67",
      "c813a6f6ce09424582c06a3ac8a7d712",
      "ff6cd33dca3c44ba9832b2d705fd2de1",
      "8b264499afca4694849b08e2091f6826",
      "3eaf82623d964c2abb92b4509ac5ca9d",
      "aa3ff7831fbf47efa404805886ca4696",
      "6be63de77ed14eee9c37e93d3a75f743",
      "5195ed241f09440eaf84a7e0b3dc8155",
      "33fa92f225be477d8cd176a0a4e8894e",
      "b80166eeee984661a611a02d1eae4458",
      "20cd68a7e03f4353aa99fcefc1f16d36",
      "fb908d9c3e374983aa47478a95411ba7",
      "304609dbefa74735baa08ce7aadff92d",
      "250451c12cee4aacb91a32d7da4ab861",
      "2c37badbe0504073b31abd67a8c0b808",
      "bfd7d35860c14dd3b8e5eea07032311d",
      "db2c74f92f6d4a2c8bfdcb91d5bd3d64",
      "b0af063597ea4de5a8c8bf4f370f782a",
      "5b1ac1a0f04f4c29b3c1127ec3d95e81",
      "b91c8359a6444497b3665e66afbe237d",
      "1181117783fd4bca9d7f5f296507aff3",
      "941a345ba82a44d5ad597a15c0af0d5c",
      "23afbee21a644745b24b8ffefed6b286",
      "a075ee85c2d34ab3b2d011c63dc02286",
      "052cccb5d69d476f9caac95848e0353a"
     ]
    },
    "id": "xyBCYGjAp3ym",
    "outputId": "81f25c6d-cd7f-4be1-e085-f8a54bd1c27d",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of BertForQuestionAnswering were not initialized from the model checkpoint at bert-base-chinese and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "# 设置模型下载镜像\n",
    "os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'\n",
    "\n",
    "from transformers import (\n",
    "  AutoTokenizer,\n",
    "  AutoModelForQuestionAnswering,\n",
    ")\n",
    "\n",
    "model = AutoModelForQuestionAnswering.from_pretrained(\"bert-base-chinese\").to(device)\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-chinese\")\n",
    "\n",
    "# 你可以忽略警告消息（它弹出是因为新的 QA 预测头是随机初始化的）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3Td-GTmk5OW4"
   },
   "source": [
    "## 数据部分\n",
    "\n",
    "> 使用两个繁体中文阅读理解数据集：[DRCD](https://github.com/DRCKnowledgeTeam/DRCD) 和 [ODSQA](https://github.com/Chia-Hsuan-Lee/ODSQA)。\n",
    "\n",
    "- **训练集（DRCD + DRCD-backtrans）**：包含 15,329 个段落和 26,918 个问题。一个段落可能对应多个问题。\n",
    "- **开发集（DRCD + DRCD-backtrans）**：包含 1,255 个段落和 2,863 个问题。用于验证。\n",
    "- **测试集（DRCD + ODSQA）**：包含 1,606 个段落和 3,504 个问题。测试集的段落没有提供答案，需要模型进行预测。\n",
    "\n",
    "所有数据集的格式相同：\n",
    "\n",
    "- `id`：问题编号\n",
    "- `paragraph_id`：段落编号\n",
    "- `question_text`：问题文本\n",
    "- `answer_text`：答案文本\n",
    "- `answer_start`：答案在段落中的起始字符位置\n",
    "- `answer_end`：答案在段落中的结束字符位置\n",
    "\n",
    "![数据集格式](../Guide/assets/4215768313590de87aab01adcad78c90.png)\n",
    "\n",
    "### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "NvX7hlepogvu"
   },
   "outputs": [],
   "source": [
    "def read_data(file):\n",
    "    with open(file, 'r', encoding=\"utf-8\") as reader:\n",
    "        data = json.load(reader)\n",
    "    return data[\"questions\"], data[\"paragraphs\"]\n",
    "\n",
    "train_questions, train_paragraphs = read_data(\"hw7_train.json\")\n",
    "dev_questions, dev_paragraphs = read_data(\"hw7_dev.json\")\n",
    "test_questions, test_paragraphs = read_data(\"hw7_test.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fm0rpTHq0e4N"
   },
   "source": [
    "### 分词处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rTZ6B70Hoxie",
    "outputId": "db13805e-02a9-4cbe-e6d2-66c0e3860c2b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Token indices sequence length is longer than the specified maximum sequence length for this model (566 > 512). Running this sequence through the model will result in indexing errors\n"
     ]
    }
   ],
   "source": [
    "# 分别对问题和段落进行分词\n",
    "# 「add_special_tokens」设置为 False，因为在数据集的 __getitem__ 中，当合并分词后的问题和段落时会添加特殊标记\n",
    "\n",
    "train_questions_tokenized = tokenizer([train_question[\"question_text\"] for train_question in train_questions], add_special_tokens=False)\n",
    "dev_questions_tokenized = tokenizer([dev_question[\"question_text\"] for dev_question in dev_questions], add_special_tokens=False)\n",
    "test_questions_tokenized = tokenizer([test_question[\"question_text\"] for test_question in test_questions], add_special_tokens=False)\n",
    "\n",
    "train_paragraphs_tokenized = tokenizer(train_paragraphs, add_special_tokens=False)\n",
    "dev_paragraphs_tokenized = tokenizer(dev_paragraphs, add_special_tokens=False)\n",
    "test_paragraphs_tokenized = tokenizer(test_paragraphs, add_special_tokens=False)\n",
    "\n",
    "# 你可以忽略警告消息，因为在传递给模型之前，将在数据集的 __getitem__ 中进一步处理分词后的序列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ws8c8_4d5UCI"
   },
   "source": [
    "### 数据集（Medium / Strong 相关）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "Xjooag-Swnuh"
   },
   "outputs": [],
   "source": [
    "class QA_Dataset(Dataset):\n",
    "    \"\"\"\n",
    "    自定义的问答数据集类，用于处理问答任务的数据。\n",
    "\n",
    "    参数:\n",
    "        split (str): 数据集的类型，'train'、'dev' 或 'test'。\n",
    "        questions (list): 问题列表，每个元素是一个字典，包含问题的详细信息。\n",
    "        tokenized_questions (BatchEncoding): 分词后的问题，由 tokenizer 生成。\n",
    "        tokenized_paragraphs (BatchEncoding): 分词后的段落列表，由 tokenizer 生成。\n",
    "    \n",
    "    属性（即 __init_() 中的 self.xxx）：\n",
    "        max_question_len (int): 问题的最大长度（以分词后的 token 数计）。\n",
    "        max_paragraph_len (int): 段落的最大长度（以分词后的 token 数计）。\n",
    "        doc_stride (int): 段落窗口滑动步长。\n",
    "        max_seq_len (int): 输入序列的最大长度。\n",
    "    \"\"\"\n",
    "    def __init__(self, split, questions, tokenized_questions, tokenized_paragraphs):\n",
    "        self.split = split\n",
    "        self.questions = questions\n",
    "        self.tokenized_questions = tokenized_questions\n",
    "        self.tokenized_paragraphs = tokenized_paragraphs\n",
    "        self.max_question_len = 60\n",
    "        self.max_paragraph_len = 150\n",
    "\n",
    "        ##### TODO: 更改 doc_stride 的值 #####\n",
    "        self.doc_stride = 150\n",
    "\n",
    "        # 输入序列长度 = [CLS] + question + [SEP] + paragraph + [SEP]\n",
    "        self.max_seq_len = 1 + self.max_question_len + 1 + self.max_paragraph_len + 1\n",
    "\n",
    "    def __len__(self):\n",
    "        \"\"\"\n",
    "        返回数据集中样本的数量。\n",
    "\n",
    "        返回:\n",
    "            int: 数据集的长度。\n",
    "        \"\"\"\n",
    "        return len(self.questions)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        \"\"\"\n",
    "        获取数据集中指定索引的样本。\n",
    "\n",
    "        参数:\n",
    "            idx (int): 样本的索引。\n",
    "\n",
    "        返回:\n",
    "            对于训练集：返回一个输入张量和对应的答案位置\n",
    "            (input_ids, token_type_ids, attention_mask, answer_start_token, answer_end_token)\n",
    "            对于验证/测试集：返回包含多个窗口的输入张量列表\n",
    "            (input_ids_list, token_type_ids_list, attention_mask_list)\n",
    "        \"\"\"\n",
    "        question = self.questions[idx]\n",
    "        tokenized_question = self.tokenized_questions[idx]\n",
    "        tokenized_paragraph = self.tokenized_paragraphs[question[\"paragraph_id\"]]\n",
    "\n",
    "        ##### TODO: 预处理 #####\n",
    "        # 提示：思考怎么防止模型学习到不应该学习的东西\n",
    "        if self.split == \"train\":\n",
    "            # 将答案在段落文本中的起始/结束位置转换为在分词后段落中的起始/结束位置\n",
    "            answer_start_token = tokenized_paragraph.char_to_token(question[\"answer_start\"])\n",
    "            answer_end_token = tokenized_paragraph.char_to_token(question[\"answer_end\"])\n",
    "\n",
    "            # 通过切片包含答案的段落部分来获得一个窗口\n",
    "            mid = (answer_start_token + answer_end_token) // 2\n",
    "            paragraph_start = max(0, min(mid - self.max_paragraph_len // 2, len(tokenized_paragraph) - self.max_paragraph_len))\n",
    "            paragraph_end = paragraph_start + self.max_paragraph_len\n",
    "\n",
    "            # 切片问题/段落并添加特殊标记（101：CLS，102：SEP）\n",
    "            input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102]\n",
    "            input_ids_paragraph = tokenized_paragraph.ids[paragraph_start : paragraph_end] + [102]\n",
    "\n",
    "            # 将答案在分词后段落中的起始/结束位置转换为窗口中的起始/结束位置\n",
    "            answer_start_token += len(input_ids_question) - paragraph_start\n",
    "            answer_end_token += len(input_ids_question) - paragraph_start\n",
    "\n",
    "            # 填充序列并获取模型的输入\n",
    "            input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "            return torch.tensor(input_ids), torch.tensor(token_type_ids), torch.tensor(attention_mask), answer_start_token, answer_end_token\n",
    "\n",
    "        # 验证/测试\n",
    "        else:\n",
    "            input_ids_list, token_type_ids_list, attention_mask_list = [], [], []\n",
    "\n",
    "            # 段落被分割成多个窗口，每个窗口的起始位置由步长 \"doc_stride\" 分隔\n",
    "            for i in range(0, len(tokenized_paragraph), self.doc_stride):\n",
    "\n",
    "                # 切片问题/段落并添加特殊标记（101：CLS，102：SEP）\n",
    "                input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102]\n",
    "                input_ids_paragraph = tokenized_paragraph.ids[i : i + self.max_paragraph_len] + [102]\n",
    "\n",
    "                # 填充序列并获取模型的输入\n",
    "                input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "\n",
    "                input_ids_list.append(input_ids)\n",
    "                token_type_ids_list.append(token_type_ids)\n",
    "                attention_mask_list.append(attention_mask)\n",
    "\n",
    "            return torch.tensor(input_ids_list), torch.tensor(token_type_ids_list), torch.tensor(attention_mask_list)\n",
    "\n",
    "    def padding(self, input_ids_question, input_ids_paragraph):\n",
    "        \"\"\"\n",
    "        对输入的序列进行填充，生成统一长度的模型输入。\n",
    "\n",
    "        参数:\n",
    "            input_ids_question (list): 问题部分的输入 ID 列表\n",
    "            input_ids_paragraph (list): 段落部分的输入 ID 列表\n",
    "\n",
    "        返回:\n",
    "            input_ids (list): 填充后的输入 ID 列表\n",
    "            token_type_ids (list): 区分问题和段落的标记列表\n",
    "            attention_mask (list): 注意力掩码列表，指示哪些位置是有效的输入\n",
    "        \"\"\"\n",
    "        # 计算需要填充的长度\n",
    "        padding_len = self.max_seq_len - len(input_ids_question) - len(input_ids_paragraph)\n",
    "        # 填充输入序列\n",
    "        input_ids = input_ids_question + input_ids_paragraph + [0] * padding_len\n",
    "        # 构造区分问题和段落的 token_type_ids\n",
    "        token_type_ids = [0] * len(input_ids_question) + [1] * len(input_ids_paragraph) + [0] * padding_len\n",
    "        # 构造注意力掩码，有效位置为 1，填充位置为 0\n",
    "        attention_mask = [1] * (len(input_ids_question) + len(input_ids_paragraph)) + [0] * padding_len\n",
    "\n",
    "        return input_ids, token_type_ids, attention_mask\n",
    "\n",
    "train_set = QA_Dataset(\"train\", train_questions, train_questions_tokenized, train_paragraphs_tokenized)\n",
    "dev_set = QA_Dataset(\"dev\", dev_questions, dev_questions_tokenized, dev_paragraphs_tokenized)\n",
    "test_set = QA_Dataset(\"test\", test_questions, test_questions_tokenized, test_paragraphs_tokenized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 这里解释一下QA_Dataset，如果觉得太长，可以只查看重点部分。\n",
    "\n",
    "1. **初始化（`__init__`）**：\n",
    "   - `split` 决定数据集的类型（训练、验证或测试）。\n",
    "   - `questions`, `tokenized_questions`, 和 `tokenized_paragraphs` 是原问题和 tokenized 后的问题和段落。\n",
    "   - `max_question_len` 和 `max_paragraph_len` 分别设定了问题和段落的最大长度。\n",
    "   - `self.doc_stride`：段落的窗口滑动步长（决定每个窗口之间的重叠部分）。\n",
    "     - `Sample code` 中将其设置为 150，和 `max_paragraph_len` 一样，意味着窗口之间完全不重叠。\n",
    "   - `self.max_seq_len`：定义了整个输入序列的最大长度（包含问题和段落）。\n",
    "\n",
    "2. **`__getitem__`**：\n",
    "   - 针对给定的索引 `idx`，获取对应问题和段落数据，返回模型需要的输入。\n",
    "   - **训练集**：定位答案的起始和结束位置，将包含答案的段落部分截取为一个窗口（中心在答案位置附近）。然后将问题和段落合并为一个输入序列，并进行填充。\n",
    "   - **验证/测试集**：将段落分成多个窗口，每个窗口之间的步长由 `self.doc_stride` 决定，然后将每个窗口作为模型的输入。验证和测试时不需要答案位置，因此只需生成多个窗口作为输入。\n",
    "\n",
    "3. **填充（`padding`）**：\n",
    "   - 输入序列可能比最大序列长度短，填充部分用 0 表示。对于问题部分和段落部分，`token_type_ids` 被用来区分它们（0 表示问题，1 表示段落）。`attention_mask` 用于标记有效的输入部分，防止模型对填充部分进行注意力计算。\n",
    "\n",
    "#### 重点\n",
    "\n",
    "- `self.doc_stride` 通过控制窗口之间的滑动步长，确保即使答案位于窗口边缘，模型也能通过多个窗口重叠的方式找到答案。\n",
    "- **训练阶段**不需要使用 `doc_stride`，因为训练时我们已经知道答案的位置，可以直接截取包含答案的窗口。但在**验证和测试**阶段，由于模型并不知道答案的位置，`doc_stride` 保证每个窗口之间有足够的重叠，避免遗漏答案。\n",
    "- 所以这里存在一个问题，训练过程中模型可能学习到：答案就在中间这一模式。这是我们在 Strong baseline 中需要解决的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5_H1kqhR8CdM"
   },
   "source": [
    "### 评估函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "SqeA3PLPxOHu"
   },
   "outputs": [],
   "source": [
    "def evaluate(data, output):\n",
    "    ##### TODO: 后处理 #####\n",
    "    # 后处理存在一个错误和改进的空间\n",
    "    # 提示：打开你的预测文件，看看有什么问题\n",
    "\n",
    "    answer = ''\n",
    "    max_prob = float('-inf')\n",
    "    num_of_windows = data[0].shape[1]\n",
    "\n",
    "    for k in range(num_of_windows):\n",
    "        # 通过选择最可能的起始位置/结束位置来获得答案\n",
    "        start_prob, start_index = torch.max(output.start_logits[k], dim=0)\n",
    "        end_prob, end_index = torch.max(output.end_logits[k], dim=0)\n",
    "\n",
    "        # 答案的概率计算为 start_prob 和 end_prob 的和\n",
    "        prob = start_prob + end_prob\n",
    "\n",
    "        # 如果计算的概率大于之前的窗口，则替换答案\n",
    "        if prob > max_prob:\n",
    "            max_prob = prob\n",
    "            # 将标记转换为字符（例如，[1920, 7032] --> \"大 金\"）\n",
    "            answer = tokenizer.decode(data[0][0][k][start_index : end_index + 1])\n",
    "\n",
    "    # 移除答案中的空格（例如，\"大 金\" --> \"大金\"）\n",
    "    return answer.replace(' ','')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rzHQit6eMnKG"
   },
   "source": [
    "### 训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 778,
     "referenced_widgets": [
      "53c1ce79c38b45fc8d1b675515642c54",
      "9370d0df1bca4b9a9c15e1d3e10142d9",
      "611f0f0c70a5471b891bd6f6b8cb7863",
      "986b627aef004704ac0806a9aa189938",
      "f2b466617f0247638e0934c8fe95ed1d",
      "f9b28a5fbcd14568983ed36f9a1ca997",
      "04c0aa3d24944485b475601f786b56f1",
      "78818b2d8cc748efbf1b5784a9ee2a85",
      "7c61a280288d4608b6ef341ce3adb7a8",
      "8710bb7686df4fb3a73e5af839be5305",
      "c7896f62a30f4eec95e3c84aef08e1b2",
      "29a7b6bb49004c3c96a6d48cd55e1ec3",
      "06e4e53bd3e748ad90d14eeec117aac9",
      "6a47a3bc614a44e7a5b0dfef50cadaf7",
      "fba6bb8f65904ae6ae4f5bca95e942e2",
      "6d5ad2f51bb3459b85201f983a499ea2",
      "9c7a21dc260a4ad7b87f2123947058a2",
      "7b7a0c46d9794d6fa66401f03684dec0",
      "11dcde71ff984d829ec9612e7f43e3a3",
      "a987cb79f4904973958a98f492c6443a",
      "8d791392eed34a9b8e155da9a4b0a0d6",
      "bab8f16e038b440a8016ef087ee12904"
     ]
    },
    "id": "3Q-B6ka7xoCM",
    "outputId": "69c6f306-8a93-4b59-a64d-41f55889a6e6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "开始训练...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "18f5266a0308442abcdd134188ad1260",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3365 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 | Step 100 | loss = 3.789, acc = 0.071\n",
      "Epoch 1 | Step 200 | loss = 2.059, acc = 0.324\n",
      "Epoch 1 | Step 300 | loss = 1.394, acc = 0.469\n",
      "Epoch 1 | Step 400 | loss = 1.246, acc = 0.555\n",
      "Epoch 1 | Step 500 | loss = 1.188, acc = 0.551\n",
      "Epoch 1 | Step 600 | loss = 0.940, acc = 0.616\n",
      "Epoch 1 | Step 700 | loss = 0.961, acc = 0.607\n",
      "Epoch 1 | Step 800 | loss = 0.868, acc = 0.621\n",
      "Epoch 1 | Step 900 | loss = 0.909, acc = 0.647\n",
      "Epoch 1 | Step 1000 | loss = 0.829, acc = 0.639\n",
      "Epoch 1 | Step 1100 | loss = 0.815, acc = 0.646\n",
      "Epoch 1 | Step 1200 | loss = 0.739, acc = 0.680\n",
      "Epoch 1 | Step 1300 | loss = 0.722, acc = 0.664\n",
      "Epoch 1 | Step 1400 | loss = 0.759, acc = 0.670\n",
      "Epoch 1 | Step 1500 | loss = 0.754, acc = 0.667\n",
      "Epoch 1 | Step 1600 | loss = 0.816, acc = 0.664\n",
      "Epoch 1 | Step 1700 | loss = 0.788, acc = 0.694\n",
      "Epoch 1 | Step 1800 | loss = 0.761, acc = 0.691\n",
      "Epoch 1 | Step 1900 | loss = 0.654, acc = 0.729\n",
      "Epoch 1 | Step 2000 | loss = 0.658, acc = 0.709\n",
      "Epoch 1 | Step 2100 | loss = 0.705, acc = 0.694\n",
      "Epoch 1 | Step 2200 | loss = 0.698, acc = 0.701\n",
      "Epoch 1 | Step 2300 | loss = 0.752, acc = 0.681\n",
      "Epoch 1 | Step 2400 | loss = 0.643, acc = 0.719\n",
      "Epoch 1 | Step 2500 | loss = 0.698, acc = 0.710\n",
      "Epoch 1 | Step 2600 | loss = 0.581, acc = 0.736\n",
      "Epoch 1 | Step 2700 | loss = 0.655, acc = 0.730\n",
      "Epoch 1 | Step 2800 | loss = 0.617, acc = 0.745\n",
      "Epoch 1 | Step 2900 | loss = 0.636, acc = 0.731\n",
      "Epoch 1 | Step 3000 | loss = 0.622, acc = 0.716\n",
      "Epoch 1 | Step 3100 | loss = 0.619, acc = 0.745\n",
      "Epoch 1 | Step 3200 | loss = 0.606, acc = 0.732\n",
      "Epoch 1 | Step 3300 | loss = 0.633, acc = 0.722\n",
      "评估开发集...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8db0c46af9a449b5920d234d69646fa9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/2863 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation | Epoch 1 | acc = 0.591\n",
      "保存模型...\n"
     ]
    }
   ],
   "source": [
    "from accelerate import Accelerator\n",
    "\n",
    "# 超参数\n",
    "num_epoch = 1\n",
    "validation = True\n",
    "logging_step = 100\n",
    "learning_rate = 1e-5\n",
    "optimizer = AdamW(model.parameters(), lr=learning_rate)\n",
    "train_batch_size = 8\n",
    "\n",
    "#### TODO: 梯度累积（可选）####\n",
    "# 注意：train_batch_size * gradient_accumulation_steps = 有效批次大小\n",
    "# 如果 CUDA 内存不足，你可以降低 train_batch_size 并提高 gradient_accumulation_steps\n",
    "# 文档：https://huggingface.co/docs/accelerate/usage_guides/gradient_accumulation\n",
    "gradient_accumulation_steps = 16\n",
    "\n",
    "# 数据加载器\n",
    "# 注意：不要更改 dev_loader / test_loader 的批次大小！\n",
    "# 虽然批次大小=1，但它实际上是由同一对 QA 的多个窗口组成的批次\n",
    "train_loader = DataLoader(train_set, batch_size=train_batch_size, shuffle=True, pin_memory=True)\n",
    "dev_loader = DataLoader(dev_set, batch_size=1, shuffle=False, pin_memory=True)\n",
    "test_loader = DataLoader(test_set, batch_size=1, shuffle=False, pin_memory=True)\n",
    "\n",
    "# 将 \"fp16_training\" 更改为 True 以支持自动混合精度训练（fp16）\n",
    "fp16_training = True\n",
    "if fp16_training:\n",
    "    accelerator = Accelerator(mixed_precision=\"fp16\")\n",
    "else:\n",
    "    accelerator = Accelerator()\n",
    "\n",
    "# 文档：https://huggingface.co/docs/accelerate/\n",
    "model, optimizer, train_loader = accelerator.prepare(model, optimizer, train_loader)\n",
    "\n",
    "model.train()\n",
    "\n",
    "print(\"开始训练...\")\n",
    "\n",
    "for epoch in range(num_epoch):\n",
    "    step = 1\n",
    "    train_loss = train_acc = 0\n",
    "\n",
    "    for data in tqdm(train_loader):\n",
    "        # 将所有数据加载到设备\n",
    "        data = [i.to(device) for i in data]\n",
    "\n",
    "        # 模型输入：input_ids, token_type_ids, attention_mask, start_positions, end_positions（注意：只有 \"input_ids\" 是必需的）\n",
    "        # 模型输出：start_logits, end_logits, loss（提供 start_positions/end_positions 时返回）\n",
    "        output = model(input_ids=data[0], token_type_ids=data[1], attention_mask=data[2], start_positions=data[3], end_positions=data[4])\n",
    "        # 选择最可能的起始位置/结束位置\n",
    "        start_index = torch.argmax(output.start_logits, dim=1)\n",
    "        end_index = torch.argmax(output.end_logits, dim=1)\n",
    "\n",
    "        # 只有当 start_index 和 end_index 都正确时，预测才正确\n",
    "        train_acc += ((start_index == data[3]) & (end_index == data[4])).float().mean()\n",
    "\n",
    "        train_loss += output.loss\n",
    "\n",
    "        accelerator.backward(output.loss)\n",
    "\n",
    "        step += 1\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        ##### TODO: 应用线性学习率衰减 #####\n",
    "\n",
    "        # 每经过 logging_step，打印训练损失和准确率\n",
    "        if step % logging_step == 0:\n",
    "            print(f\"Epoch {epoch + 1} | Step {step} | loss = {train_loss.item() / logging_step:.3f}, acc = {train_acc / logging_step:.3f}\")\n",
    "            train_loss = train_acc = 0\n",
    "\n",
    "    if validation:\n",
    "        print(\"评估开发集...\")\n",
    "        model.eval()\n",
    "        with torch.no_grad():\n",
    "            dev_acc = 0\n",
    "            for i, data in enumerate(tqdm(dev_loader)):\n",
    "                output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "                # 只有当答案文本完全匹配时，预测才正确\n",
    "                dev_acc += evaluate(data, output) == dev_questions[i][\"answer_text\"]\n",
    "            print(f\"Validation | Epoch {epoch + 1} | acc = {dev_acc / len(dev_loader):.3f}\")\n",
    "        model.train()\n",
    "\n",
    "# 将模型及其配置文件保存到目录「saved_model」\n",
    "# 即，在目录「saved_model」下有两个文件：「pytorch_model.bin」和「config.json」\n",
    "# 可以使用「model = BertForQuestionAnswering.from_pretrained(\"saved_model\")」重新加载保存的模型\n",
    "print(\"保存模型...\")\n",
    "model_save_dir = \"saved_model\"\n",
    "model.save_pretrained(model_save_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kMmdLOKBMsdE"
   },
   "source": [
    "### 测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 84,
     "referenced_widgets": [
      "9c84cbc758db46e78d7fa361a385ccce",
      "642db3bee97a4bea8a4c1079e2b7526c",
      "ed9f5d147e7844a187d8610130f1ecd9",
      "42dd646b18b24deda274c857d337c973",
      "9e2635394d5b4965b41a05a9bc37a4ec",
      "3366101d46a847dfa5700ffb652dcc86",
      "a1602e58c3a7413490ce0a93f35a7ca9",
      "d26f522a28d24494914df1dd8432af49",
      "c24b174369fb48cdbc3d4afaa4cceccb",
      "a0f74d882c63403c8116c19745e011ba",
      "55a7b680dae94cba8758288dacff2988"
     ]
    },
    "id": "U5scNKC9xz0C",
    "outputId": "841c70f2-2f3c-4c0f-866e-143f127eb6b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "评估测试集...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d487f396364648b193043e0e8b7f5558",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3524 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "完成！结果保存在 result.csv\n"
     ]
    }
   ],
   "source": [
    "print(\"评估测试集...\")\n",
    "\n",
    "result = []\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    for data in tqdm(test_loader):\n",
    "        output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "        result.append(evaluate(data, output))\n",
    "\n",
    "result_file = \"result.csv\"\n",
    "with open(result_file, 'w') as f:\n",
    "    f.write(\"ID,Answer\\n\")\n",
    "    for i, test_question in enumerate(test_questions):\n",
    "        # 将答案中的逗号替换为空字符串（因为 csv 以逗号分隔）\n",
    "        # Kaggle 中的答案也以同样的方式处理\n",
    "        f.write(f\"{test_question['id']},{result[i].replace(',','')}\\n\")\n",
    "\n",
    "print(f\"完成！结果保存在 {result_file}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 拓展\n",
    "\n",
    "### Question 1\n",
    "\n",
    ">微调（Finetune）和提示（Prompt）之间存在一些差异。上下文学习（In-context Learning）可以让预训练模型在不进行梯度下降的情况下，通过少量示例给出许多下游任务的正确预测。请回答以下问题：\n",
    ">\n",
    ">a. **Encoder-only 模型（如 BERT 系列）如何在抽取式问答任务中确定答案？**\n",
    ">\n",
    ">b. **Decoder-only 模型（如 GPT 系列）如何在抽取式问答任务中确定答案？**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0_JVNKOCywbR"
   },
   "source": [
    "### Question 2 (In-context learning)\n",
    "\n",
    "> 尝试不同的 Prompt 并观察 fine-tuning 和 in-context learning 的区别。\n",
    ">\n",
    "> 代码所下载的模型是 [facebook/xglm-1.7B](https://huggingface.co/facebook/xglm-1.7B)，实际上你也可以直接去 GPT 或者其他 AI 平台提问，这里的目的是去调整自己的 prompt，从而使模型不经过微调也能获取到正确答案。\n",
    ">\n",
    "> 对于“老手”来说，完全可以跳过这个问题。\n",
    "\n",
    "#### Prompt 对比（错误对比）\n",
    "\n",
    "1. **Prompt 示例 1**: \"根据文章找出问题的答案：{问题}\"。\n",
    "2. **Prompt 示例 2**: \"请阅读文章并回答以下问题：{问题}\"。\n",
    "3. **Prompt 示例 3**: \"请根据文章信息回答下列问题：{问题}\"。\n",
    "\n",
    "#### Prompt 对比（正确对比）\n",
    "\n",
    "> ![image-20240920173255150](../Guide/assets/9805a280ab816c41ec21aee5a223bf9d.png)\n",
    "\n",
    "- 中英对比\n",
    "- 不同 prompt 对比\n",
    "\n",
    "**样例 prompt**:\n",
    "```\n",
    "請從最後一篇的文章中找出最後一個問題的答案：\n",
    "文章：<文章1 內容>\n",
    "問題：<問題1 敘述>\n",
    "答案：<答案1>\n",
    "...\n",
    "文章：<文章n 內容>\n",
    "問題：<問題n 敘述>\n",
    "答案：\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "sGUxltxqzKpm"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import random\n",
    "import numpy as np\n",
    "\n",
    "# 为避免 CUDA_OUT_OF_MEMORY\n",
    "torch.set_default_device('cuda')\n",
    "\n",
    "# 为了可重复性，固定随机种子\n",
    "def same_seeds(seed):\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.cuda.is_available():\n",
    "        torch.cuda.manual_seed(seed)\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "    np.random.seed(seed)\n",
    "    random.seed(seed)\n",
    "    torch.backends.cudnn.benchmark = False\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "same_seeds(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 241,
     "referenced_widgets": [
      "fd1e236692ec43a18a1750a29515050d",
      "7b3c48ae548848c880439841c0b9cb6b",
      "1023013dc30a4731b61383bd59f2dc91",
      "ee0cb4c1f31346b5b180a7b826688eb4",
      "0f7a38ddfe7b42eaa5c8ffdb3c150641",
      "eb405f797a1441d280029e6e203befce",
      "7a7081fd9d5942af9148e29054b18f51",
      "eb4bdd1ff3cf4b0db273c53e32cad5de",
      "7c852afee0de49fe8dac74f3af82a725",
      "24b96f20dd5f4a6d8cf9788e97a74f73",
      "a5bf4132f3dd4c7a807f27c8106fdd59",
      "0549ffe5a6a444dd9fbf8c4513ebe7f7",
      "388d3d2566ff4a1cb68048ac78dd374f",
      "6235eaf236d643c6a163dddb86d3664d",
      "faf78b2739f841d8a213e4311926b598",
      "91f03b6ed1c94cb68970f45ffa64695e",
      "9e312a0ec6a2478fb0f55691caf6d9ae",
      "39d35e6e28264b5ca157745d0c1e1491",
      "c497cc2cfabb451aad71f5d7b19873e4",
      "7f7f2c3afc2647c086877a2f4eff6c12",
      "0386ad4eea2b4387b48cd58e95bcd4ba",
      "028ec097e62b4ee8a463708f4c3b8f24",
      "f607835a7acf4037821885704bde736d",
      "ddd59aac5a5d48ceace9ede35bf8f11b",
      "20e14c8de0f84979ac3f0d896263f49d",
      "4d7c3ea23e9b42a2ba5b0b11ee190c69",
      "5d9e4fd05f964fd7a11d66b964ea0873",
      "7fb466a306e6443287cb2fa2bea90f06",
      "1b2b45ecc27b4f28bc0194709fed0638",
      "6a2f5722362c4ec08ae8ed7d681dc7ba",
      "f51192579d014a52b132f99abf8901b2",
      "6912ca44b1864902a3e32d4a19a4972d",
      "acbfe8b88e6945ab9a5959fbb5f294de",
      "fda75dc7c2d24f7b827b2c78a6988586",
      "2d1a48d240b949ad9b465985e90557c5",
      "5a0f5ba8ad024f5b915e34e48f269437",
      "bc4b72ff74d84c9e83b328e8823044b1",
      "901f4ebc70f144c188d03a4978920441",
      "98ce66c91c5c45968a3844fb0de50624",
      "a24d04e7064048c2819ef88a50ce9093",
      "154be9baa03943d08c975dbc85c4edf8",
      "2fe4fa69d9544fd393a4d72c02245245",
      "c4da9fed45254ebdaa49312f16d9b159",
      "53a2b78a732c4522805de5c9aed92c41",
      "0440f9e26a0b420b86175bc62e212d01",
      "c9bf7b5808f74db9811f5b64e878dcd5",
      "66a35d11743141b1b20904b13a4b86ea",
      "c0145fad2cb843e9b75ac47d307cc346",
      "d61e0c79febd45c8b00e3566296d7e8e",
      "2a59b752f9414e69b8c1d24bd24159f2",
      "447bb12155cf4dc193ffea3f64c6a8f8",
      "16b30759d76a4c8f979212d1ae30c297",
      "1639410507da4b54996064d9fcc8621c",
      "e515df89088042dbbe23594e5976e10d",
      "0dccf470d87f4585af65826582399466",
      "7dee4e5320d349159422709a69af61f4",
      "86dbdec7087f4312b945c3a3940d8965",
      "cdbf6c3b5b3948229946fd51c227b51e",
      "d132ff798f284c9cbe2a73d35d776592",
      "fead01a1a86c40bcb1ba9133ef351d7c",
      "f19d9d34c978402691a89114849d2bb9",
      "7a729fb76a374da99334e7d4b4de102e",
      "cccbdce89ff545f6a66304352b320ad7",
      "cbd776799c18488eb74a8f17cc27c6fd",
      "b1cd7728deff4d50a18cb22805351330",
      "550e7aaac7fa4c3db37ceca6a5595ddb",
      "745f78a378b5412eae007f029e8c321a",
      "2c449a0e664c41a79b3a1de62e94b5b7",
      "a9a16bf8795c402a90cf4504b1cd30a1",
      "715ad9f6bdaa46298317e049404f9755",
      "667f153201d347a0a01e39b438fa736b",
      "d652cdca44e04070a420c5d1a24b4fd1",
      "b5c71e0a94014278a22ba45a6f58b77d",
      "7f928c5e620a49c1ae137868e34818a9",
      "d403f126d48746948c0f627dfdce5d65",
      "d41e81f720f44b52b9a914e4c619bfee",
      "bee7c0220d4e476dbc035fbc5dc44fa3"
     ]
    },
    "id": "wIS23s79zItf",
    "outputId": "c686be61-221a-4f55-ca79-67715343f1db"
   },
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "# 你可以尝试不同大小的模型\n",
    "# 当使用 Colab 或 Kaggle 时，超过 20 亿（2B）参数的模型可能会耗尽内存\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"facebook/xglm-1.7B\")\n",
    "model = AutoModelForCausalLM.from_pretrained(\"facebook/xglm-1.7B\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "2d97YtavzFHA"
   },
   "outputs": [],
   "source": [
    "# 清理模型输出。如果你尝试不同的提示，可能需要自行修正此函数\n",
    "def clean_text(text):\n",
    "    # 注意：当你使用单语模型时，冒号可能会变成全角\n",
    "    text = text.split(\"答案:\")[-1]\n",
    "    text = text.split(\" \")[0]\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jdf3gdP_yykH",
    "outputId": "b9a784dd-e5d7-4212-c089-e0180157bd11"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：廣州是京廣鐵路，廣深鐵路，廣茂鐵路和廣梅鐵路的終點站。2009年底，武廣客運專線投入運營，多機組列車長980公里，最高時速350公里。2011年1月7日，廣珠城際鐵路投入運營，平均時速200公里。廣州鐵路，長途巴士和渡輪直達香港。廣九快速列車從廣州火車東站出發，直達香港紅磡火車站。總長約182公里，行程大約需要兩個小時。每年都有繁忙的教練從香港的不同乘客點接載乘客。在市中心的珠江北岸有一條渡輪線路，河流居民可以直接過河而無需乘坐公共汽車或步行穿過大橋。每天都有往返南沙碼頭和蓮花山碼頭的高速雙體船。渡輪也開往香港中國客運碼頭和港澳客運碼頭。\n",
      "問題：廣珠城際鐵路平均每小時可以走多遠？\n",
      "答案：200公里\n",
      "文章：自古以來，廣州一直是華南地區的著名商人，擁有2000多年的開放貿易歷史。20世紀70年代末中國大陸改革開放後，廣州經濟發展迅速。2010年，全市地區生產總值10604.48億元，同比增長13％。它成為僅次於上海和北京的第三個進入“萬億元俱樂部”國內生產總值的城市。這也是第一個超過一萬億的經濟總量。首都。根據國務院2005年發布的報告，廣州成為中國第一個進入“發達”狀態的城市。2012年9月，廣州南沙新區獲批，成為第六個國家級開放開發新區。2015年，廣州GDP達到1810.41億元，人均GDP達到138,377.05元。國內生產總值是中國的第三位，人均國內生產總值與西班牙相當。購買力平價水平與發達國家相當。\n",
      "問題：進入國內生產總值「萬億元俱樂部」的城市第三個為？\n",
      "答案：廣州\n",
      "文章：2010年引入的廣州快速交通運輸系統是世界第二大快速運輸系統。每日載客量可達100萬人次。每小時的客流量峰值高達26,900名乘客，僅次於波哥大的快速交通系統。每10秒有一輛公共汽車，每輛公共汽車在一個方向上行駛350小時。該平台包括橋樑，是世界上最長的國家公共汽車快速運輸系統平台，長度為260米。目前，廣州市的出租車和公交車主要以液化石油氣為燃料，部分公交車採用油電，氣電混合技術。2012年底，一輛LNG燃料公共汽車開始啟動。2014年6月，引入了LNG插電式混合動力公交車取代LPG公交車。2007年1月16日，廣州市政府完全禁止在城市地區駕駛摩托車。違反禁令的機動車將被沒收。廣州市交通局聲稱，禁令的實施導致交通擁堵和車禍大大減少。廣州白雲國際機場位於白雲區與花都區交界處。它於2004年8月5日正式投入運營。它是中國第二繁忙的機場。機場取代了原先位於市中心的舊機場，無法滿足日益增長的航空需求。目前，機場有三個簡易機場，是中國第三個擁有三條跑道的民航機場。比2023年香港國際機場第三條跑道的預計完工時間提前了8年。\n",
      "問題：從哪一天開始在廣州市內騎摩托車會被沒收？\n",
      "答案：\n",
      "正確答案: 2007年1月16日\n",
      "模型輸出: 2007年1月16日\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：自古以來，廣州一直是華南地區的著名商人，擁有2000多年的開放貿易歷史。20世紀70年代末中國大陸改革開放後，廣州經濟發展迅速。2010年，全市地區生產總值10604.48億元，同比增長13％。它成為僅次於上海和北京的第三個進入“萬億元俱樂部”國內生產總值的城市。這也是第一個超過一萬億的經濟總量。首都。根據國務院2005年發布的報告，廣州成為中國第一個進入“發達”狀態的城市。2012年9月，廣州南沙新區獲批，成為第六個國家級開放開發新區。2015年，廣州GDP達到1810.41億元，人均GDP達到138,377.05元。國內生產總值是中國的第三位，人均國內生產總值與西班牙相當。購買力平價水平與發達國家相當。\n",
      "問題：進入國內生產總值「萬億元俱樂部」的城市第三個為？\n",
      "答案：廣州\n",
      "文章：多年來，廣州當局積極開展各種政府項目。自1990年以來，廣州參與了“創建國家健康城市”，並於2008年成功獲得該獎項。1998年，“創建全國文明城市”開始，2011年取得了成功。在“創造健康”期間，廣州市的衛生狀況得到了明顯改善，如清理城市衛生角，城中村，污水處理和食品安全管理。一些官員說，“創造健康”只是一個起點，沒有盡頭。 “創造”是“人心的項目”。關於在這些項目，形式主義和騷亂上花費的巨額公共資金，還有媒體問題和批評。多年來在這兩個項目上花費的公共資金數量已達數十億，這使得計算難度很大。在廣州亞運會之際，廣州市政府在2010年之前開展了大量的城市建設項目，包括：改造多條道路和人行道，重新鋪設地磚和花卉基地;和許多新老天橋的灰白色油漆;大多數街道拱廊和歷史建築的改造;正面建築的屋頂均勻配備紅色塑料屋頂，外牆經過翻新，配備空調壓縮機蓋和照明。然而，有些人質疑，在亞運會期間，許多精心準備的道路被強行翻新，當街邊商場和歷史建築被翻新時，原有的歷史和藝術感被摧毀。許多小偷參與了街道上數千棟建築的統一改造。通過腳手架，數百戶家庭被盜。項目完成後不久，揭露了涉及使用劣質材料的事件，這也遭到許多公民的批評。作為中國的中心城市之一，廣州的醫療保險和便利服務覆蓋了整個城市。大多數市政公園和所有公共廁所也是免費的。\n",
      "問題：廣州市於什麼時候獲得從1990年開始參與的「創衛」？\n",
      "答案：2008年\n",
      "文章：廣州是京廣鐵路，廣深鐵路，廣茂鐵路和廣梅鐵路的終點站。2009年底，武廣客運專線投入運營，多機組列車長980公里，最高時速350公里。2011年1月7日，廣珠城際鐵路投入運營，平均時速200公里。廣州鐵路，長途巴士和渡輪直達香港。廣九快速列車從廣州火車東站出發，直達香港紅磡火車站。總長約182公里，行程大約需要兩個小時。每年都有繁忙的教練從香港的不同乘客點接載乘客。在市中心的珠江北岸有一條渡輪線路，河流居民可以直接過河而無需乘坐公共汽車或步行穿過大橋。每天都有往返南沙碼頭和蓮花山碼頭的高速雙體船。渡輪也開往香港中國客運碼頭和港澳客運碼頭。\n",
      "問題：廣珠城際鐵路平均每小時可以走多遠？\n",
      "答案：\n",
      "正確答案: 200公里\n",
      "模型輸出: 200公里\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：公元前214年，秦始皇出兵徵兵嶺南，南海縣任仁軍在梵山和廬山建造了番禺城。這是廣州最初建成時的名稱，已在中華民國使用。公元226年和246年，吳國將膠州分為兩部分：“廣州”和“廣州”。合浦以南仍被稱為“城洲”，合浦以北被稱為國務院芷芝部的“廣州”。 “廣州”包括南海，蒼梧，榆林和合浦四個縣。這是兩國分裂的開始，也是“廣州”這個名稱的開頭，但廣州當時只提到了一個管轄區，相當於現在的廣東省。自唐代以來，廣州地區已成立為光復的起源。它於1921年正式成立，番禺市首次被評為廣州。廣州又被稱為“五羊城”，“陽城”和“遂城”，簡稱“隋”。這些別名來自五羊的傳說：有五個神仙穿著五色衣服，騎著五色羊。拿著“一根六根”的穀物來到廣州市，在把耳朵送到城市後，他們飛走了。五隻羊變成了石頭。因此，廣州也被稱為“五陽城”和“遂城”。因為它位於亞熱帶地區，氣候溫暖​​，適合種植花卉。此外，廣州人喜愛種花，買花，送花。花卉市場每年繁榮，廣州享有華城的美譽。從唐代開始，廣州就是廣州的首府。清朝以後，它是廣東省的省會。因此，省會成為廣州廣東人的總稱。在古代，廣州也有南武市，楚庭，番禺和南海的稱號。\n",
      "問題：番禺城這一個名子一直用到哪一年才改名為廣州市？\n",
      "答案：1921\n",
      "文章：自古以來，廣州一直是華南地區的著名商人，擁有2000多年的開放貿易歷史。20世紀70年代末中國大陸改革開放後，廣州經濟發展迅速。2010年，全市地區生產總值10604.48億元，同比增長13％。它成為僅次於上海和北京的第三個進入“萬億元俱樂部”國內生產總值的城市。這也是第一個超過一萬億的經濟總量。首都。根據國務院2005年發布的報告，廣州成為中國第一個進入“發達”狀態的城市。2012年9月，廣州南沙新區獲批，成為第六個國家級開放開發新區。2015年，廣州GDP達到1810.41億元，人均GDP達到138,377.05元。國內生產總值是中國的第三位，人均國內生產總值與西班牙相當。購買力平價水平與發達國家相當。\n",
      "問題：進入國內生產總值「萬億元俱樂部」的城市第三個為？\n",
      "答案：\n",
      "正確答案: 廣州\n",
      "模型輸出: 廣州\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：作為現代革命的發祥地之一，廣州是中華民國以來中國社會運動的中心之一。每次全國社會運動都有廣州人民的回應和參與。一個相對大規模的社會運動集中在廣州，這是1925年至1926年在廣州和香港舉行的首次省和港口罷工。1989年，廣州市民發起了支持天安門民主運動的運動。數百萬人聚集在海珠廣場，聚集在廣州解放紀念碑周圍。在早期，廣州媒體支持“愛國運動”的名稱。流血事件發生後，民眾發生騷亂，省市政府機構遭到襲擊，所有廣州媒體都被立即禁止，這也乾擾並封鎖了香港電視台。省政府派出軍隊和警察鎮壓遊行群眾，大量參與者潛入香港，台灣和海外。所有企事業單位也派人檢查各部門員工是否組織或參加了集會。在1999年的全國反美活動中，數十萬人和學生在該市舉行示威活動，抗議北約轟炸中國駐南斯拉夫大使館。與此同時，美國駐廣州領事館也受到一些激進示威者的破壞。廣州媒體在此過程中也紛紛效仿，但沒有提到美國駐廣州領事館受損的情況。2005年，主要道路上發生了數十萬起反日遊行。然而，在廣州的封鎖中，大學和中學也禁止學生遊行，否則他們就開除了學業。\n",
      "問題：省港大罷工是除了廣州以外還有哪個地區參與？\n",
      "答案：香港\n",
      "文章：自古以來，廣州基本上就是嶺南的政治中心。秦朝末年是南嶽的首都，漢朝征服了南嶽州，後來成為南海縣。漢末遷至龍灣和古代大壩。在三國時期，吳國布將縣長遷回番禺，後來成立了膠州府。廣州和廣州轉移後，它是廣州的州長。晉南，南北朝使用南海縣，番禺由縣統治。文帝皇帝廢棄了南海縣，成立了廣州總務辦公室，後來改為杜都政府。唐代分為十個民族，其中嶺南大智學院位於廣州。862年，嶺南分為東二西路，廣州分為嶺南東路。五代時期，廣州是南漢的首都。宋初，嶺南路重建，廣州是行政中心。明代廣州是廣州阜城，隸屬番禺縣和南海縣。清軍佔領中原後，南明邵武帝朱熹在廣州建成，很快南明就死了。清代廣州成為廣東省，廣州阜城，番禺和南海縣。廣東省和廣西省，廣東省屯門省長，廣東省屯門局長，廣東省檢察院都在廣州市南海縣境內。1858年，英法聯軍攻占廣州，廣東省長投降，外國人委員會成立，廣東省政府成為政府。\n",
      "問題：南明的首都在哪？\n",
      "答案：廣州\n",
      "文章：廣州的雨水豐富，土地肥沃。市區曾經擁有非常大的農業用地。兩千年前有水稻種植記錄。宋代的廣州是中國最大的大米市場之一，蔬菜，水果，甘蔗和鮮花也很有名。由於長期以來一直是國家港口，廣州引進了各種優質作物品種。二十世紀上半葉，由於外國農產品的入侵和戰爭的影響，廣州的農業增長放緩。在20世紀50年代中期，農業生產急劇增加，但是後來的“大躍進”和隨後的文化大革命嚴重打擊了農業生產。改革開放以來，隨著廣州作為一個大城市的發展，服務城市的郊區農業模式逐步形成。政府的政策是為城市服務，豐富農民和人民，並鼓勵出口收入。具體措施是降低糧食生產比例，增加水果，蔬菜等農副產品比重，努力發展林業，畜牧業和漁業。廣州的農業產值大幅增加，但隨著第二，三產業的快速增長，農業的經濟份額逐漸下降。1978年，廣州農業總產值佔地區生產總值的11.67％，1990年下降到8.05％。 2015年，廣州農業總產值為413億元，佔地區GDP的2.28％。城市發展和工業化進程也造成了農村勞動力的大量流失和農地的迅速衰退。清光緒五年間，廣州登記的天山山池總面積為10662.3萬畝。1990年，耕地面積為247萬畝，2006年減少到158.3萬畝。目前，廣州農產品種植地主要分佈在白雲，花都，番禺，南沙，從化和增城。1990年，農業人口為252.87萬，2008年為792,200。廣州最著名的農業特產包括梧桐五秀和各種熱帶水果，如荔枝，香蕉，木瓜和菠蘿。\n",
      "問題：1978年的廣州市農業生產總值到了哪一年占地區生總值減少了約9.39%？\n",
      "答案：\n",
      "正確答案: 2015\n",
      "模型輸出: 廣州農業總產值佔地區生總值的11.67%,1990年下降\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：廣州是京廣鐵路，廣深鐵路，廣茂鐵路和廣梅鐵路的終點站。2009年底，武廣客運專線投入運營，多機組列車長980公里，最高時速350公里。2011年1月7日，廣珠城際鐵路投入運營，平均時速200公里。廣州鐵路，長途巴士和渡輪直達香港。廣九快速列車從廣州火車東站出發，直達香港紅磡火車站。總長約182公里，行程大約需要兩個小時。每年都有繁忙的教練從香港的不同乘客點接載乘客。在市中心的珠江北岸有一條渡輪線路，河流居民可以直接過河而無需乘坐公共汽車或步行穿過大橋。每天都有往返南沙碼頭和蓮花山碼頭的高速雙體船。渡輪也開往香港中國客運碼頭和港澳客運碼頭。\n",
      "問題：廣珠城際鐵路平均每小時可以走多遠？\n",
      "答案：200公里\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：中華人民共和國成立後的工業國有化。在20世紀50年代和60年代，該行業復甦，但文化大革命再次嚴重影響了廣州的工業生產，工業發展放緩。然而，在政府的支持下，仍然建立了重工業體系。1975年以後，政局改善，政府大力支持以日用品為主的輕工業，廣州工業進入快速增長期。20世紀80年代末，廣州主要發展第三產業，工業產值比重下降。從20世紀90年代到2000年，廣州市政府試圖改變行業構成，增加對重工業的支持。汽車，石化，電子信息產品的製造已成為廣州的三大支柱產業。廣州的工業總產值在全省排名第三。其中，汽車工業成功吸引了日本三大汽車製造企業投資建廠，使廣州成為近幾年來中國重要的汽車生產基地之一。江南最大的廣州晨天汽車零部件市場位於郊區，廣州也是國內。生產日本汽車最多的城市。\n",
      "問題：中華人民共和國成立後由於什麼事件廣州的工業發展又減慢？\n",
      "答案：\n",
      "正確答案: 文化大革命\n",
      "模型輸出: 文化大革命\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：作為中國最大的城市之一，廣州有大量在廣州工作的外國人。然而，“本地人”與“外國人”之間的關係難以協調，也會引發一系列社會問題。由於廣州的能力和就業能力有限，廣州無法容納來自全國各地的農民工。其安全環境已經過嚴格測試。經常發生“兩個盜賊和一個小偷”事件，特別是在城市村莊。根據廣州社會科學院的一項調查，80％的囚犯被逮捕為“局外人”。除了廣東話的光復文化外，外國人和外國方言的排斥增加了廣州市民的仇外心理。近年來，由於政府的大力禁令，廣州的公安管理取得了顯著成效。2014年，廣東省政府明確要求控制廣州和深圳特大城市的人口規模。2008年，農民工佔廣州人口的40％，其中大多數是女性農民工，其中許多人未婚。截至2013年底，廣州註冊人口為686.7萬。根據一定的錯失率，廣州的實際流動人口約為837萬。這些數據已超過常住人口。最新統計數據顯示，廣州常住人口為832萬。其中，白雲區，番禺區和天河區流動人口超過百萬。番禺區有111.7萬戶登記移民，流動戶籍區涉及30個省，自治區，直轄市。在性別比方面，男性略多，佔總數的53.5％;女性佔總數的46.5％。在番禺流動人口中，兒童和老年人的比例極小，15歲以下為0.49％，60歲以上為1.68％，大專以上為10％。\n",
      "問題：有百分之多少的犯人被抓的是外地人？\n",
      "答案：80\n",
      "文章：2012年，全市常住人口1283萬人，戶籍人口8230萬，城鎮常住人口778.17萬人，外來人口占36％。廣州是全國人口密度高的城市之一。大多數人口集中在舊城區。四個中心區的平均居民人口密度達到每平方公里18,113人。人口密度最高的越秀區每平方公里達到34,009人，是人口密度最低的從化市的100多倍。自2001年以來，廣州的年出生率一直低於每千人9.6人，自然增長率低於每千人3.4人。公民的預期壽命高於全國平均水平，男性的預期壽命為75.25歲，女性的預期壽命為80.94歲。與此同時，人口老齡化問題也日益突出。截至2008年底，60歲以上老年人口達到1103,500人，佔全市總人口的13.4％。其中，老城區老年人口比例最高，超過18％，預計2030年老年人口將超過200萬。2010年第六次全國人口普查顯示，廣州的男性多於女性。截至2007年底，男女比例為109.46，即每100名女性109.46名男性。男性多於女性573,694。其中，0~14歲年齡組性別失衡嚴重，達到115.62，表明廣州性別比差距將在未來擴大。廣州人口以漢族為主，佔常住人口的98.3％，少數民族人口為219,900人。它屬於55個少數民族。其中，壯族，土家族，苗族，瑤族，回族，滿族，彝族，蒙古族，布依族，朝鮮族等10個民族。廣州經濟的快速發展和高就業機會吸引了大批農民工。 2008年，農民工佔該市人口的40％。如何妥善管理城市管理是市政府的一項艱鉅任務。\n",
      "問題：哪邊的人口密度為廣州市的一百倍分之一以下？\n",
      "答案：從化市\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：\n",
      "正確答案: 廣州\n",
      "模型輸出: 廣州\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：多年來，廣州當局積極開展各種政府項目。自1990年以來，廣州參與了“創建國家健康城市”，並於2008年成功獲得該獎項。1998年，“創建全國文明城市”開始，2011年取得了成功。在“創造健康”期間，廣州市的衛生狀況得到了明顯改善，如清理城市衛生角，城中村，污水處理和食品安全管理。一些官員說，“創造健康”只是一個起點，沒有盡頭。 “創造”是“人心的項目”。關於在這些項目，形式主義和騷亂上花費的巨額公共資金，還有媒體問題和批評。多年來在這兩個項目上花費的公共資金數量已達數十億，這使得計算難度很大。在廣州亞運會之際，廣州市政府在2010年之前開展了大量的城市建設項目，包括：改造多條道路和人行道，重新鋪設地磚和花卉基地;和許多新老天橋的灰白色油漆;大多數街道拱廊和歷史建築的改造;正面建築的屋頂均勻配備紅色塑料屋頂，外牆經過翻新，配備空調壓縮機蓋和照明。然而，有些人質疑，在亞運會期間，許多精心準備的道路被強行翻新，當街邊商場和歷史建築被翻新時，原有的歷史和藝術感被摧毀。許多小偷參與了街道上數千棟建築的統一改造。通過腳手架，數百戶家庭被盜。項目完成後不久，揭露了涉及使用劣質材料的事件，這也遭到許多公民的批評。作為中國的中心城市之一，廣州的醫療保險和便利服務覆蓋了整個城市。大多數市政公園和所有公共廁所也是免費的。\n",
      "問題：廣州市於什麼時候獲得從1990年開始參與的「創衛」？\n",
      "答案：2008年\n",
      "文章：廣州是中國最重要的開放城市之一。作為對外貿易的窗口，有許多來自北美，日本，韓國，歐洲，中東，非洲等國的外國人，尤其是非洲黑人。廣州甚至被稱為“第三世界”。首都主要位於廣州市環市東路秀山大廈，淘金路，花園酒店，建馬路和劍馬路;天河北路，體育東路，天河路，龍口西路，臨河中路等公司辦公室;番禺區的一些物業，如克利福德新村和麗江花園，也有更多的外國人。據廣州市公安局出入境管理支隊2014年10月統計，在廣州登記的外國人有11.8萬人，其中臨時居民71,000人，居住半年以上的47,000名永久居民。其中大多數是來自非洲的4,000多名註冊外國人;來自阿拉伯國家和地區的約1萬人雖然沒有大規模的定居點，但經常出現在小北 - 登封地區的商業區進行商業活動。在荔景街也有更多的韓國人。2013年，有超過15萬名乘客進出廣州港，其中54萬人來自非洲國家。他們中的大多數人來往於同一個人，主要是因為簽證允許短期逗留或商業需求而不是移民。近年來，廣州黑人一直捲入廣州的一些公安問題，引起廣州市民的一些不滿。針對“三非”問題，公安機關設立了四個涉外綜合執法隊伍，翻譯志願服務隊伍，調查“三個非外國”外國專業隊和四個外國隊 - 相關管理基層服務隊伍;自年初以來，廣州在200多名外國人社區建立了75個“外國人管理服務工作站”，並大力打擊“三非”外國人。它已經取得了成果。\n",
      "問題：被稱為「第三世界首都」的城市哪種外籍人士最多？\n",
      "答案：非洲黑人\n",
      "文章：自古以來，廣州基本上就是嶺南的政治中心。秦朝末年是南嶽的首都，漢朝征服了南嶽州，後來成為南海縣。漢末遷至龍灣和古代大壩。在三國時期，吳國布將縣長遷回番禺，後來成立了膠州府。廣州和廣州轉移後，它是廣州的州長。晉南，南北朝使用南海縣，番禺由縣統治。文帝皇帝廢棄了南海縣，成立了廣州總務辦公室，後來改為杜都政府。唐代分為十個民族，其中嶺南大智學院位於廣州。862年，嶺南分為東二西路，廣州分為嶺南東路。五代時期，廣州是南漢的首都。宋初，嶺南路重建，廣州是行政中心。明代廣州是廣州阜城，隸屬番禺縣和南海縣。清軍佔領中原後，南明邵武帝朱熹在廣州建成，很快南明就死了。清代廣州成為廣東省，廣州阜城，番禺和南海縣。廣東省和廣西省，廣東省屯門省長，廣東省屯門局長，廣東省檢察院都在廣州市南海縣境內。1858年，英法聯軍攻占廣州，廣東省長投降，外國人委員會成立，廣東省政府成為政府。\n",
      "問題：南明的首都在哪？\n",
      "答案：\n",
      "正確答案: 廣州\n",
      "模型輸出: 廣州\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：作為現代革命的發祥地之一，廣州是中華民國以來中國社會運動的中心之一。每次全國社會運動都有廣州人民的回應和參與。一個相對大規模的社會運動集中在廣州，這是1925年至1926年在廣州和香港舉行的首次省和港口罷工。1989年，廣州市民發起了支持天安門民主運動的運動。數百萬人聚集在海珠廣場，聚集在廣州解放紀念碑周圍。在早期，廣州媒體支持“愛國運動”的名稱。流血事件發生後，民眾發生騷亂，省市政府機構遭到襲擊，所有廣州媒體都被立即禁止，這也乾擾並封鎖了香港電視台。省政府派出軍隊和警察鎮壓遊行群眾，大量參與者潛入香港，台灣和海外。所有企事業單位也派人檢查各部門員工是否組織或參加了集會。在1999年的全國反美活動中，數十萬人和學生在該市舉行示威活動，抗議北約轟炸中國駐南斯拉夫大使館。與此同時，美國駐廣州領事館也受到一些激進示威者的破壞。廣州媒體在此過程中也紛紛效仿，但沒有提到美國駐廣州領事館受損的情況。2005年，主要道路上發生了數十萬起反日遊行。然而，在廣州的封鎖中，大學和中學也禁止學生遊行，否則他們就開除了學業。\n",
      "問題：省港大罷工是除了廣州以外還有哪個地區參與？\n",
      "答案：香港\n",
      "文章：廣州是京廣鐵路，廣深鐵路，廣茂鐵路和廣梅鐵路的終點站。2009年底，武廣客運專線投入運營，多機組列車長980公里，最高時速350公里。2011年1月7日，廣珠城際鐵路投入運營，平均時速200公里。廣州鐵路，長途巴士和渡輪直達香港。廣九快速列車從廣州火車東站出發，直達香港紅磡火車站。總長約182公里，行程大約需要兩個小時。每年都有繁忙的教練從香港的不同乘客點接載乘客。在市中心的珠江北岸有一條渡輪線路，河流居民可以直接過河而無需乘坐公共汽車或步行穿過大橋。每天都有往返南沙碼頭和蓮花山碼頭的高速雙體船。渡輪也開往香港中國客運碼頭和港澳客運碼頭。\n",
      "問題：廣珠城際鐵路平均每小時可以走多遠？\n",
      "答案：200公里\n",
      "文章：清朝末期，廣州爆發了幾次武裝起義，所有起義都以失敗告終。1911年10月10日武昌起義後，廣東省獨立。 11月10日，軍政府成立，胡漢民當選為州長。12月初，廣東省臨時省委成立。據宣布，21歲以上的廣東國民有選舉權和被選舉權。議會由120名成員組成，其中包括20名聯盟成員，21名軍團協會代表，12名海外華人代表以及師生代表。 9人，1名“自治團體”代表，57名來自各地區的代表。其中，女議員必須佔據​​10人，中國女性是第一個參與政治的人。1913年4月27日，廣東省議會成立，羅曉峰被任命為議長。1918年10月，廣州市政府辦公室成立。1921年2月，廣州正式成立，廣州市政府成立，孫克被任命為市長。1917年至1922年間，孫中山和西南各省議會代表兩次在廣州建立了法國政府。1925年，孫中山去世。同年7月，廣州國民政府成立。 7月4日，廣州市政府成立，實行市委製度。吳朝書被任命為市委主席。1929年，廣州實施了市長制。1949年10月14日，中國人民解放軍佔領了廣州。 28日，廣州軍事管理委員會成立。葉劍英是董事長。目前，廣州實行了中華人民共和國政治體制下的人民代表大會制度。市政府在廣州市委員會的領導下運作，政府駐留在越秀區。廣州省是廣東省省會，廣東省人民政府駐地。\n",
      "問題：廣東省因為哪一場戰爭而獨立？\n",
      "答案：\n",
      "正確答案: 武昌起義\n",
      "模型輸出: 1911年10月10日武昌起義\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：2010年引入的廣州快速交通運輸系統是世界第二大快速運輸系統。每日載客量可達100萬人次。每小時的客流量峰值高達26,900名乘客，僅次於波哥大的快速交通系統。每10秒有一輛公共汽車，每輛公共汽車在一個方向上行駛350小時。該平台包括橋樑，是世界上最長的國家公共汽車快速運輸系統平台，長度為260米。目前，廣州市的出租車和公交車主要以液化石油氣為燃料，部分公交車採用油電，氣電混合技術。2012年底，一輛LNG燃料公共汽車開始啟動。2014年6月，引入了LNG插電式混合動力公交車取代LPG公交車。2007年1月16日，廣州市政府完全禁止在城市地區駕駛摩托車。違反禁令的機動車將被沒收。廣州市交通局聲稱，禁令的實施導致交通擁堵和車禍大大減少。廣州白雲國際機場位於白雲區與花都區交界處。它於2004年8月5日正式投入運營。它是中國第二繁忙的機場。機場取代了原先位於市中心的舊機場，無法滿足日益增長的航空需求。目前，機場有三個簡易機場，是中國第三個擁有三條跑道的民航機場。比2023年香港國際機場第三條跑道的預計完工時間提前了8年。\n",
      "問題：從哪一天開始在廣州市內騎摩托車會被沒收？\n",
      "答案：2007年1月16日\n",
      "文章：多年來，廣州當局積極開展各種政府項目。自1990年以來，廣州參與了“創建國家健康城市”，並於2008年成功獲得該獎項。1998年，“創建全國文明城市”開始，2011年取得了成功。在“創造健康”期間，廣州市的衛生狀況得到了明顯改善，如清理城市衛生角，城中村，污水處理和食品安全管理。一些官員說，“創造健康”只是一個起點，沒有盡頭。 “創造”是“人心的項目”。關於在這些項目，形式主義和騷亂上花費的巨額公共資金，還有媒體問題和批評。多年來在這兩個項目上花費的公共資金數量已達數十億，這使得計算難度很大。在廣州亞運會之際，廣州市政府在2010年之前開展了大量的城市建設項目，包括：改造多條道路和人行道，重新鋪設地磚和花卉基地;和許多新老天橋的灰白色油漆;大多數街道拱廊和歷史建築的改造;正面建築的屋頂均勻配備紅色塑料屋頂，外牆經過翻新，配備空調壓縮機蓋和照明。然而，有些人質疑，在亞運會期間，許多精心準備的道路被強行翻新，當街邊商場和歷史建築被翻新時，原有的歷史和藝術感被摧毀。許多小偷參與了街道上數千棟建築的統一改造。通過腳手架，數百戶家庭被盜。項目完成後不久，揭露了涉及使用劣質材料的事件，這也遭到許多公民的批評。作為中國的中心城市之一，廣州的醫療保險和便利服務覆蓋了整個城市。大多數市政公園和所有公共廁所也是免費的。\n",
      "問題：廣州市於什麼時候獲得從1990年開始參與的「創衛」？\n",
      "答案：2008年\n",
      "文章：作為現代革命的發祥地之一，廣州是中華民國以來中國社會運動的中心之一。每次全國社會運動都有廣州人民的回應和參與。一個相對大規模的社會運動集中在廣州，這是1925年至1926年在廣州和香港舉行的首次省和港口罷工。1989年，廣州市民發起了支持天安門民主運動的運動。數百萬人聚集在海珠廣場，聚集在廣州解放紀念碑周圍。在早期，廣州媒體支持“愛國運動”的名稱。流血事件發生後，民眾發生騷亂，省市政府機構遭到襲擊，所有廣州媒體都被立即禁止，這也乾擾並封鎖了香港電視台。省政府派出軍隊和警察鎮壓遊行群眾，大量參與者潛入香港，台灣和海外。所有企事業單位也派人檢查各部門員工是否組織或參加了集會。在1999年的全國反美活動中，數十萬人和學生在該市舉行示威活動，抗議北約轟炸中國駐南斯拉夫大使館。與此同時，美國駐廣州領事館也受到一些激進示威者的破壞。廣州媒體在此過程中也紛紛效仿，但沒有提到美國駐廣州領事館受損的情況。2005年，主要道路上發生了數十萬起反日遊行。然而，在廣州的封鎖中，大學和中學也禁止學生遊行，否則他們就開除了學業。\n",
      "問題：省港大罷工是除了廣州以外還有哪個地區參與？\n",
      "答案：\n",
      "正確答案: 香港\n",
      "模型輸出: 香港\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：廣州是中國最重要的開放城市之一。作為對外貿易的窗口，有許多來自北美，日本，韓國，歐洲，中東，非洲等國的外國人，尤其是非洲黑人。廣州甚至被稱為“第三世界”。首都主要位於廣州市環市東路秀山大廈，淘金路，花園酒店，建馬路和劍馬路;天河北路，體育東路，天河路，龍口西路，臨河中路等公司辦公室;番禺區的一些物業，如克利福德新村和麗江花園，也有更多的外國人。據廣州市公安局出入境管理支隊2014年10月統計，在廣州登記的外國人有11.8萬人，其中臨時居民71,000人，居住半年以上的47,000名永久居民。其中大多數是來自非洲的4,000多名註冊外國人;來自阿拉伯國家和地區的約1萬人雖然沒有大規模的定居點，但經常出現在小北 - 登封地區的商業區進行商業活動。在荔景街也有更多的韓國人。2013年，有超過15萬名乘客進出廣州港，其中54萬人來自非洲國家。他們中的大多數人來往於同一個人，主要是因為簽證允許短期逗留或商業需求而不是移民。近年來，廣州黑人一直捲入廣州的一些公安問題，引起廣州市民的一些不滿。針對“三非”問題，公安機關設立了四個涉外綜合執法隊伍，翻譯志願服務隊伍，調查“三個非外國”外國專業隊和四個外國隊 - 相關管理基層服務隊伍;自年初以來，廣州在200多名外國人社區建立了75個“外國人管理服務工作站”，並大力打擊“三非”外國人。它已經取得了成果。\n",
      "問題：被稱為「第三世界首都」的城市哪種外籍人士最多？\n",
      "答案：非洲黑人\n",
      "文章：2010年7月，1000多名市民聚集在地鐵江南西站出口處舉行2010年廣州支持粵語行動請願活動。在收到有關該活動被取消的消息後，他們仍然按計劃時間到達為光復文化發行自己的活動。語音。在內地媒體封鎖消息傳出後，廣州還封鎖了與香港電視有關的新聞片段。8月1日，廣州中央公園的粵語聚會受到當局的強烈阻撓。人們被迫轉移路線。當他們到達北京路南段時，他們被大批警察包圍。參與者被帶到公共汽車並在東山體育場被帶走。它在被質疑和被定罪後被釋放。大陸請願制度允許人們向上級政府部門發出呼籲。廣東省政府經常邀請來自省內各地的請願人來到東風路正門，得到省政府的幫助。在集會時，當局將調動大量警察部隊積極處理他們。在對峙期間，交通通常會很慢，但當地官方媒體從未報導相關報導。2014年初，更重要的請願者是該省的家禽商，由於市場被迫關閉，在禽流感期間遭受了損失，以及隋毛反對茂名石化項目的年輕人。\n",
      "問題：2010年廣州撐粵語行動請願活動是在當年的什麼時候舉行的？\n",
      "答案：7月\n",
      "文章：公元前214年，秦始皇出兵徵兵嶺南，南海縣任仁軍在梵山和廬山建造了番禺城。這是廣州最初建成時的名稱，已在中華民國使用。公元226年和246年，吳國將膠州分為兩部分：“廣州”和“廣州”。合浦以南仍被稱為“城洲”，合浦以北被稱為國務院芷芝部的“廣州”。 “廣州”包括南海，蒼梧，榆林和合浦四個縣。這是兩國分裂的開始，也是“廣州”這個名稱的開頭，但廣州當時只提到了一個管轄區，相當於現在的廣東省。自唐代以來，廣州地區已成立為光復的起源。它於1921年正式成立，番禺市首次被評為廣州。廣州又被稱為“五羊城”，“陽城”和“遂城”，簡稱“隋”。這些別名來自五羊的傳說：有五個神仙穿著五色衣服，騎著五色羊。拿著“一根六根”的穀物來到廣州市，在把耳朵送到城市後，他們飛走了。五隻羊變成了石頭。因此，廣州也被稱為“五陽城”和“遂城”。因為它位於亞熱帶地區，氣候溫暖​​，適合種植花卉。此外，廣州人喜愛種花，買花，送花。花卉市場每年繁榮，廣州享有華城的美譽。從唐代開始，廣州就是廣州的首府。清朝以後，它是廣東省的省會。因此，省會成為廣州廣東人的總稱。在古代，廣州也有南武市，楚庭，番禺和南海的稱號。\n",
      "問題：番禺城這一個名子一直用到哪一年才改名為廣州市？\n",
      "答案：\n",
      "正確答案: 1921\n",
      "模型輸出: 公元前221年\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：2012年，全市常住人口1283萬人，戶籍人口8230萬，城鎮常住人口778.17萬人，外來人口占36％。廣州是全國人口密度高的城市之一。大多數人口集中在舊城區。四個中心區的平均居民人口密度達到每平方公里18,113人。人口密度最高的越秀區每平方公里達到34,009人，是人口密度最低的從化市的100多倍。自2001年以來，廣州的年出生率一直低於每千人9.6人，自然增長率低於每千人3.4人。公民的預期壽命高於全國平均水平，男性的預期壽命為75.25歲，女性的預期壽命為80.94歲。與此同時，人口老齡化問題也日益突出。截至2008年底，60歲以上老年人口達到1103,500人，佔全市總人口的13.4％。其中，老城區老年人口比例最高，超過18％，預計2030年老年人口將超過200萬。2010年第六次全國人口普查顯示，廣州的男性多於女性。截至2007年底，男女比例為109.46，即每100名女性109.46名男性。男性多於女性573,694。其中，0~14歲年齡組性別失衡嚴重，達到115.62，表明廣州性別比差距將在未來擴大。廣州人口以漢族為主，佔常住人口的98.3％，少數民族人口為219,900人。它屬於55個少數民族。其中，壯族，土家族，苗族，瑤族，回族，滿族，彝族，蒙古族，布依族，朝鮮族等10個民族。廣州經濟的快速發展和高就業機會吸引了大批農民工。 2008年，農民工佔該市人口的40％。如何妥善管理城市管理是市政府的一項艱鉅任務。\n",
      "問題：哪邊的人口密度為廣州市的一百倍分之一以下？\n",
      "答案：從化市\n",
      "文章：作為中國最大的城市之一，廣州有大量在廣州工作的外國人。然而，“本地人”與“外國人”之間的關係難以協調，也會引發一系列社會問題。由於廣州的能力和就業能力有限，廣州無法容納來自全國各地的農民工。其安全環境已經過嚴格測試。經常發生“兩個盜賊和一個小偷”事件，特別是在城市村莊。根據廣州社會科學院的一項調查，80％的囚犯被逮捕為“局外人”。除了廣東話的光復文化外，外國人和外國方言的排斥增加了廣州市民的仇外心理。近年來，由於政府的大力禁令，廣州的公安管理取得了顯著成效。2014年，廣東省政府明確要求控制廣州和深圳特大城市的人口規模。2008年，農民工佔廣州人口的40％，其中大多數是女性農民工，其中許多人未婚。截至2013年底，廣州註冊人口為686.7萬。根據一定的錯失率，廣州的實際流動人口約為837萬。這些數據已超過常住人口。最新統計數據顯示，廣州常住人口為832萬。其中，白雲區，番禺區和天河區流動人口超過百萬。番禺區有111.7萬戶登記移民，流動戶籍區涉及30個省，自治區，直轄市。在性別比方面，男性略多，佔總數的53.5％;女性佔總數的46.5％。在番禺流動人口中，兒童和老年人的比例極小，15歲以下為0.49％，60歲以上為1.68％，大專以上為10％。\n",
      "問題：有百分之多少的犯人被抓的是外地人？\n",
      "答案：80\n",
      "文章：2010年7月，1000多名市民聚集在地鐵江南西站出口處舉行2010年廣州支持粵語行動請願活動。在收到有關該活動被取消的消息後，他們仍然按計劃時間到達為光復文化發行自己的活動。語音。在內地媒體封鎖消息傳出後，廣州還封鎖了與香港電視有關的新聞片段。8月1日，廣州中央公園的粵語聚會受到當局的強烈阻撓。人們被迫轉移路線。當他們到達北京路南段時，他們被大批警察包圍。參與者被帶到公共汽車並在東山體育場被帶走。它在被質疑和被定罪後被釋放。大陸請願制度允許人們向上級政府部門發出呼籲。廣東省政府經常邀請來自省內各地的請願人來到東風路正門，得到省政府的幫助。在集會時，當局將調動大量警察部隊積極處理他們。在對峙期間，交通通常會很慢，但當地官方媒體從未報導相關報導。2014年初，更重要的請願者是該省的家禽商，由於市場被迫關閉，在禽流感期間遭受了損失，以及隋毛反對茂名石化項目的年輕人。\n",
      "問題：2010年廣州撐粵語行動請願活動是在當年的什麼時候舉行的？\n",
      "答案：\n",
      "正確答案: 7月\n",
      "模型輸出: 2010年7月\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：清朝末期，廣州爆發了幾次武裝起義，所有起義都以失敗告終。1911年10月10日武昌起義後，廣東省獨立。 11月10日，軍政府成立，胡漢民當選為州長。12月初，廣東省臨時省委成立。據宣布，21歲以上的廣東國民有選舉權和被選舉權。議會由120名成員組成，其中包括20名聯盟成員，21名軍團協會代表，12名海外華人代表以及師生代表。 9人，1名“自治團體”代表，57名來自各地區的代表。其中，女議員必須佔據​​10人，中國女性是第一個參與政治的人。1913年4月27日，廣東省議會成立，羅曉峰被任命為議長。1918年10月，廣州市政府辦公室成立。1921年2月，廣州正式成立，廣州市政府成立，孫克被任命為市長。1917年至1922年間，孫中山和西南各省議會代表兩次在廣州建立了法國政府。1925年，孫中山去世。同年7月，廣州國民政府成立。 7月4日，廣州市政府成立，實行市委製度。吳朝書被任命為市委主席。1929年，廣州實施了市長制。1949年10月14日，中國人民解放軍佔領了廣州。 28日，廣州軍事管理委員會成立。葉劍英是董事長。目前，廣州實行了中華人民共和國政治體制下的人民代表大會制度。市政府在廣州市委員會的領導下運作，政府駐留在越秀區。廣州省是廣東省省會，廣東省人民政府駐地。\n",
      "問題：廣東省因為哪一場戰爭而獨立？\n",
      "答案：武昌起義\n",
      "文章：多年來，廣州當局積極開展各種政府項目。自1990年以來，廣州參與了“創建國家健康城市”，並於2008年成功獲得該獎項。1998年，“創建全國文明城市”開始，2011年取得了成功。在“創造健康”期間，廣州市的衛生狀況得到了明顯改善，如清理城市衛生角，城中村，污水處理和食品安全管理。一些官員說，“創造健康”只是一個起點，沒有盡頭。 “創造”是“人心的項目”。關於在這些項目，形式主義和騷亂上花費的巨額公共資金，還有媒體問題和批評。多年來在這兩個項目上花費的公共資金數量已達數十億，這使得計算難度很大。在廣州亞運會之際，廣州市政府在2010年之前開展了大量的城市建設項目，包括：改造多條道路和人行道，重新鋪設地磚和花卉基地;和許多新老天橋的灰白色油漆;大多數街道拱廊和歷史建築的改造;正面建築的屋頂均勻配備紅色塑料屋頂，外牆經過翻新，配備空調壓縮機蓋和照明。然而，有些人質疑，在亞運會期間，許多精心準備的道路被強行翻新，當街邊商場和歷史建築被翻新時，原有的歷史和藝術感被摧毀。許多小偷參與了街道上數千棟建築的統一改造。通過腳手架，數百戶家庭被盜。項目完成後不久，揭露了涉及使用劣質材料的事件，這也遭到許多公民的批評。作為中國的中心城市之一，廣州的醫療保險和便利服務覆蓋了整個城市。大多數市政公園和所有公共廁所也是免費的。\n",
      "問題：廣州市於什麼時候獲得從1990年開始參與的「創衛」？\n",
      "答案：\n",
      "正確答案: 2008年\n",
      "模型輸出: 武昌起義\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：清朝末期，廣州爆發了幾次武裝起義，所有起義都以失敗告終。1911年10月10日武昌起義後，廣東省獨立。 11月10日，軍政府成立，胡漢民當選為州長。12月初，廣東省臨時省委成立。據宣布，21歲以上的廣東國民有選舉權和被選舉權。議會由120名成員組成，其中包括20名聯盟成員，21名軍團協會代表，12名海外華人代表以及師生代表。 9人，1名“自治團體”代表，57名來自各地區的代表。其中，女議員必須佔據​​10人，中國女性是第一個參與政治的人。1913年4月27日，廣東省議會成立，羅曉峰被任命為議長。1918年10月，廣州市政府辦公室成立。1921年2月，廣州正式成立，廣州市政府成立，孫克被任命為市長。1917年至1922年間，孫中山和西南各省議會代表兩次在廣州建立了法國政府。1925年，孫中山去世。同年7月，廣州國民政府成立。 7月4日，廣州市政府成立，實行市委製度。吳朝書被任命為市委主席。1929年，廣州實施了市長制。1949年10月14日，中國人民解放軍佔領了廣州。 28日，廣州軍事管理委員會成立。葉劍英是董事長。目前，廣州實行了中華人民共和國政治體制下的人民代表大會制度。市政府在廣州市委員會的領導下運作，政府駐留在越秀區。廣州省是廣東省省會，廣東省人民政府駐地。\n",
      "問題：廣東省因為哪一場戰爭而獨立？\n",
      "答案：武昌起義\n",
      "文章：2010年引入的廣州快速交通運輸系統是世界第二大快速運輸系統。每日載客量可達100萬人次。每小時的客流量峰值高達26,900名乘客，僅次於波哥大的快速交通系統。每10秒有一輛公共汽車，每輛公共汽車在一個方向上行駛350小時。該平台包括橋樑，是世界上最長的國家公共汽車快速運輸系統平台，長度為260米。目前，廣州市的出租車和公交車主要以液化石油氣為燃料，部分公交車採用油電，氣電混合技術。2012年底，一輛LNG燃料公共汽車開始啟動。2014年6月，引入了LNG插電式混合動力公交車取代LPG公交車。2007年1月16日，廣州市政府完全禁止在城市地區駕駛摩托車。違反禁令的機動車將被沒收。廣州市交通局聲稱，禁令的實施導致交通擁堵和車禍大大減少。廣州白雲國際機場位於白雲區與花都區交界處。它於2004年8月5日正式投入運營。它是中國第二繁忙的機場。機場取代了原先位於市中心的舊機場，無法滿足日益增長的航空需求。目前，機場有三個簡易機場，是中國第三個擁有三條跑道的民航機場。比2023年香港國際機場第三條跑道的預計完工時間提前了8年。\n",
      "問題：從哪一天開始在廣州市內騎摩托車會被沒收？\n",
      "答案：2007年1月16日\n",
      "文章：2012年，全市常住人口1283萬人，戶籍人口8230萬，城鎮常住人口778.17萬人，外來人口占36％。廣州是全國人口密度高的城市之一。大多數人口集中在舊城區。四個中心區的平均居民人口密度達到每平方公里18,113人。人口密度最高的越秀區每平方公里達到34,009人，是人口密度最低的從化市的100多倍。自2001年以來，廣州的年出生率一直低於每千人9.6人，自然增長率低於每千人3.4人。公民的預期壽命高於全國平均水平，男性的預期壽命為75.25歲，女性的預期壽命為80.94歲。與此同時，人口老齡化問題也日益突出。截至2008年底，60歲以上老年人口達到1103,500人，佔全市總人口的13.4％。其中，老城區老年人口比例最高，超過18％，預計2030年老年人口將超過200萬。2010年第六次全國人口普查顯示，廣州的男性多於女性。截至2007年底，男女比例為109.46，即每100名女性109.46名男性。男性多於女性573,694。其中，0~14歲年齡組性別失衡嚴重，達到115.62，表明廣州性別比差距將在未來擴大。廣州人口以漢族為主，佔常住人口的98.3％，少數民族人口為219,900人。它屬於55個少數民族。其中，壯族，土家族，苗族，瑤族，回族，滿族，彝族，蒙古族，布依族，朝鮮族等10個民族。廣州經濟的快速發展和高就業機會吸引了大批農民工。 2008年，農民工佔該市人口的40％。如何妥善管理城市管理是市政府的一項艱鉅任務。\n",
      "問題：哪邊的人口密度為廣州市的一百倍分之一以下？\n",
      "答案：\n",
      "正確答案: 從化市\n",
      "模型輸出: 越秀區\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：2010年7月，1000多名市民聚集在地鐵江南西站出口處舉行2010年廣州支持粵語行動請願活動。在收到有關該活動被取消的消息後，他們仍然按計劃時間到達為光復文化發行自己的活動。語音。在內地媒體封鎖消息傳出後，廣州還封鎖了與香港電視有關的新聞片段。8月1日，廣州中央公園的粵語聚會受到當局的強烈阻撓。人們被迫轉移路線。當他們到達北京路南段時，他們被大批警察包圍。參與者被帶到公共汽車並在東山體育場被帶走。它在被質疑和被定罪後被釋放。大陸請願制度允許人們向上級政府部門發出呼籲。廣東省政府經常邀請來自省內各地的請願人來到東風路正門，得到省政府的幫助。在集會時，當局將調動大量警察部隊積極處理他們。在對峙期間，交通通常會很慢，但當地官方媒體從未報導相關報導。2014年初，更重要的請願者是該省的家禽商，由於市場被迫關閉，在禽流感期間遭受了損失，以及隋毛反對茂名石化項目的年輕人。\n",
      "問題：2010年廣州撐粵語行動請願活動是在當年的什麼時候舉行的？\n",
      "答案：7月\n",
      "文章：作為中國最大的城市之一，廣州有大量在廣州工作的外國人。然而，“本地人”與“外國人”之間的關係難以協調，也會引發一系列社會問題。由於廣州的能力和就業能力有限，廣州無法容納來自全國各地的農民工。其安全環境已經過嚴格測試。經常發生“兩個盜賊和一個小偷”事件，特別是在城市村莊。根據廣州社會科學院的一項調查，80％的囚犯被逮捕為“局外人”。除了廣東話的光復文化外，外國人和外國方言的排斥增加了廣州市民的仇外心理。近年來，由於政府的大力禁令，廣州的公安管理取得了顯著成效。2014年，廣東省政府明確要求控制廣州和深圳特大城市的人口規模。2008年，農民工佔廣州人口的40％，其中大多數是女性農民工，其中許多人未婚。截至2013年底，廣州註冊人口為686.7萬。根據一定的錯失率，廣州的實際流動人口約為837萬。這些數據已超過常住人口。最新統計數據顯示，廣州常住人口為832萬。其中，白雲區，番禺區和天河區流動人口超過百萬。番禺區有111.7萬戶登記移民，流動戶籍區涉及30個省，自治區，直轄市。在性別比方面，男性略多，佔總數的53.5％;女性佔總數的46.5％。在番禺流動人口中，兒童和老年人的比例極小，15歲以下為0.49％，60歲以上為1.68％，大專以上為10％。\n",
      "問題：有百分之多少的犯人被抓的是外地人？\n",
      "答案：\n",
      "正確答案: 80\n",
      "模型輸出: 80%\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：中華人民共和國成立後的工業國有化。在20世紀50年代和60年代，該行業復甦，但文化大革命再次嚴重影響了廣州的工業生產，工業發展放緩。然而，在政府的支持下，仍然建立了重工業體系。1975年以後，政局改善，政府大力支持以日用品為主的輕工業，廣州工業進入快速增長期。20世紀80年代末，廣州主要發展第三產業，工業產值比重下降。從20世紀90年代到2000年，廣州市政府試圖改變行業構成，增加對重工業的支持。汽車，石化，電子信息產品的製造已成為廣州的三大支柱產業。廣州的工業總產值在全省排名第三。其中，汽車工業成功吸引了日本三大汽車製造企業投資建廠，使廣州成為近幾年來中國重要的汽車生產基地之一。江南最大的廣州晨天汽車零部件市場位於郊區，廣州也是國內。生產日本汽車最多的城市。\n",
      "問題：中華人民共和國成立後由於什麼事件廣州的工業發展又減慢？\n",
      "答案：文化大革命\n",
      "文章：廣州是中國最重要的開放城市之一。作為對外貿易的窗口，有許多來自北美，日本，韓國，歐洲，中東，非洲等國的外國人，尤其是非洲黑人。廣州甚至被稱為“第三世界”。首都主要位於廣州市環市東路秀山大廈，淘金路，花園酒店，建馬路和劍馬路;天河北路，體育東路，天河路，龍口西路，臨河中路等公司辦公室;番禺區的一些物業，如克利福德新村和麗江花園，也有更多的外國人。據廣州市公安局出入境管理支隊2014年10月統計，在廣州登記的外國人有11.8萬人，其中臨時居民71,000人，居住半年以上的47,000名永久居民。其中大多數是來自非洲的4,000多名註冊外國人;來自阿拉伯國家和地區的約1萬人雖然沒有大規模的定居點，但經常出現在小北 - 登封地區的商業區進行商業活動。在荔景街也有更多的韓國人。2013年，有超過15萬名乘客進出廣州港，其中54萬人來自非洲國家。他們中的大多數人來往於同一個人，主要是因為簽證允許短期逗留或商業需求而不是移民。近年來，廣州黑人一直捲入廣州的一些公安問題，引起廣州市民的一些不滿。針對“三非”問題，公安機關設立了四個涉外綜合執法隊伍，翻譯志願服務隊伍，調查“三個非外國”外國專業隊和四個外國隊 - 相關管理基層服務隊伍;自年初以來，廣州在200多名外國人社區建立了75個“外國人管理服務工作站”，並大力打擊“三非”外國人。它已經取得了成果。\n",
      "問題：被稱為「第三世界首都」的城市哪種外籍人士最多？\n",
      "答案：\n",
      "正確答案: 非洲黑人\n",
      "模型輸出: 非洲黑人\n",
      "\n",
      "請從最後一篇的文章中找出最後一個問題的答案\n",
      "文章：多年來，廣州當局積極開展各種政府項目。自1990年以來，廣州參與了“創建國家健康城市”，並於2008年成功獲得該獎項。1998年，“創建全國文明城市”開始，2011年取得了成功。在“創造健康”期間，廣州市的衛生狀況得到了明顯改善，如清理城市衛生角，城中村，污水處理和食品安全管理。一些官員說，“創造健康”只是一個起點，沒有盡頭。 “創造”是“人心的項目”。關於在這些項目，形式主義和騷亂上花費的巨額公共資金，還有媒體問題和批評。多年來在這兩個項目上花費的公共資金數量已達數十億，這使得計算難度很大。在廣州亞運會之際，廣州市政府在2010年之前開展了大量的城市建設項目，包括：改造多條道路和人行道，重新鋪設地磚和花卉基地;和許多新老天橋的灰白色油漆;大多數街道拱廊和歷史建築的改造;正面建築的屋頂均勻配備紅色塑料屋頂，外牆經過翻新，配備空調壓縮機蓋和照明。然而，有些人質疑，在亞運會期間，許多精心準備的道路被強行翻新，當街邊商場和歷史建築被翻新時，原有的歷史和藝術感被摧毀。許多小偷參與了街道上數千棟建築的統一改造。通過腳手架，數百戶家庭被盜。項目完成後不久，揭露了涉及使用劣質材料的事件，這也遭到許多公民的批評。作為中國的中心城市之一，廣州的醫療保險和便利服務覆蓋了整個城市。大多數市政公園和所有公共廁所也是免費的。\n",
      "問題：廣州市於什麼時候獲得從1990年開始參與的「創衛」？\n",
      "答案：2008年\n",
      "文章：六朝時期，廣州的對外貿易相當繁榮，外國海商“長期停留在廣州尋求利潤”。在隋唐時期，廣州的對外貿易發展到了頂峰。作為唐代唯一的城市，外國人口占全市人口的30％以上，成為中國對外貿易的核心。在元朝短暫的沉寂之後，廣州和明清在廣州重新出現。在清代的一站式貿易政策下，它成為當時中國唯一的外國港口，成為具有壟斷地位的國家商業中心。上海開放後，隨著國家的逐步開放，廣州逐漸失去了對外貿易中心的地位，但它仍然是全國最重要的商業城市。改革開放後，廣州百貨業蓬勃發展。 20世紀80年代，友誼商店專門為香港，澳門和外國客人銷售電器和其他進口商品。西湖路照明夜市是平民的熱門目的地。南樓位於江西路沿線，已成為華南地區最大的綜合性百貨商場。它還在大陸創建了第一家24小時便利店，華夏百貨商店成立了人民南商圈。延伸，後者於1995年建立了一條步行街。新大新公司和廣州百貨在北京路的業務也蓬勃發展，這條路也成了一條步行街。超市Wankelong於1996年在廣州開設了第一家分店，然後廣州人熟悉的超市和便利店品牌，如香港百佳超市進入廣州。由於交通壓力和天河新區的發展，20世紀90年代末人民南方商業圈開始衰落，成為電子服裝批發的集散地，城市商業區遷至東部新區。21世紀，以天河城，天環廣場，鄭家廣場，太古彙和萬靈彙為代表的天河路商業區已成為廣州的中心商業區。自1957年以來，中國出口商品交易會每年春秋兩季舉辦。目前，每個展覽都吸引了來自世界各地的20多萬商家。這是中國規模最大，時間最長，最成功的國際展覽會。隨著琶洲展館三期工程的建設，展覽能力躍居世界前列。\n",
      "問題：唐朝的對外貿易核心在哪？\n",
      "答案：廣州\n",
      "文章：長期以來，廣州大部分地方的語言主要是廣東話。因此，廣州還形成了粵語文化，包括粵劇，粵劇，南音和鹹水歌。隨著中國大陸的改革開放，來自城外和廣東省以外的大量人口迅速湧入。在這些移民中，由於不同的省/總統，有許多不同的語言，外國人和來自不同地區的人經常用普通話交流;因此，在許多大型公共場所，普通話是主要語言。在某些情況下，將增加粵語或英語廣播，種族群體仍然使用自己的方言進行相互交流。一些外國人會用當地文化交流，用粵語與當地人交流。如今，隨著普通話的推廣，外國和本地年輕人受長期使用普通話作為教學媒介和普通話的共同環境的影響。目前，廣州的語言交換主要講普通話，但當地人仍然使用廣東話。與交流。雖然普通話自21世紀初以來逐漸佔據主導地位，但它受到當地文化的影響。與此同時，由於香港和澳門的距離較近，以及廣州市民近年來開展了一系列推廣普通話的評論，它已開始保護廣東話，所以廣東話仍有一定的地位。當地。 。\n",
      "問題：在廣州許多比較大的公眾場合以甚麼為主要交流語言？\n",
      "答案：\n",
      "正確答案: 普通話\n",
      "模型輸出: 廣州的廣東話和普通話的交流\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "import json\n",
    "\n",
    "with open(\"hw7_in-context-learning-examples.json\", \"r\") as f:\n",
    "    test = json.load(f)\n",
    "\n",
    "# K-shot 学习\n",
    "# 给模型 K 个示例，使其获得更好的准确率\n",
    "# 注意：(1) 当 K >= 4 时，可能会发生 CUDA_OUT_OF_MEMORY。\n",
    "#       (2) XGLM 的最大输入长度为 2048\n",
    "K = 2\n",
    "\n",
    "question_ids = [qa[\"id\"] for qa in test[\"questions\"]]\n",
    "\n",
    "with open(\"in-context-learning-result.txt\", \"w\") as f:\n",
    "    print(\"ID,Ground-Truth,Prediction\", file = f)\n",
    "    with torch.no_grad():\n",
    "        for idx, qa in enumerate(test[\"questions\"]):\n",
    "            # 你可以尝试不同的提示\n",
    "            prompt = \"請從最後一篇的文章中找出最後一個問題的答案\\n\"\n",
    "            exist_question_indexs = [question_ids.index(qa[\"id\"])]\n",
    "\n",
    "            # K-shot 学习：给模型 K 个带有答案的示例\n",
    "            for i in range(K):\n",
    "                question_index = question_ids.index(qa[\"id\"])\n",
    "                while(question_index in exist_question_indexs):\n",
    "                    question_index = random.randint(0, len(question_ids) - 1)\n",
    "                exist_question_indexs.append(question_index)\n",
    "                paragraph_id = test[\"questions\"][question_index][\"paragraph_id\"]\n",
    "                prompt += f'文章：{test[\"paragraphs\"][paragraph_id]}\\n'\n",
    "                prompt += f'問題：{test[\"questions\"][question_index][\"question_text\"]}\\n'\n",
    "                prompt += f'答案：{test[\"questions\"][question_index][\"answer_text\"]}\\n'\n",
    "\n",
    "            # 最后一个没有答案的问题\n",
    "            paragraph_id = qa[\"paragraph_id\"]\n",
    "            prompt += f'文章：{test[\"paragraphs\"][paragraph_id]}\\n'\n",
    "            prompt += f'問題：{qa[\"question_text\"]}\\n'\n",
    "            prompt += f'答案：'\n",
    "\n",
    "            inputs = tokenizer(prompt, add_special_tokens=False, return_tensors=\"pt\")\n",
    "            sample = model.generate(**inputs, max_new_tokens = 20)\n",
    "            text = tokenizer.decode(sample[0], skip_special_tokens=True)\n",
    "\n",
    "            # 注意：你可以删除这一行，看看会发生什么\n",
    "            text = clean_text(text)\n",
    "\n",
    "            print(prompt)\n",
    "            print(f'正確答案: {qa[\"answer_text\"]}')\n",
    "            print(f'模型輸出: {text}')\n",
    "            print()\n",
    "\n",
    "            print(f\"{idx},{qa['answer_text']},{text}\", file = f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "k3prlGCdjf38"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "ai",
   "language": "python",
   "name": "ai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "028ec097e62b4ee8a463708f4c3b8f24": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0386ad4eea2b4387b48cd58e95bcd4ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0440f9e26a0b420b86175bc62e212d01": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c9bf7b5808f74db9811f5b64e878dcd5",
       "IPY_MODEL_66a35d11743141b1b20904b13a4b86ea",
       "IPY_MODEL_c0145fad2cb843e9b75ac47d307cc346"
      ],
      "layout": "IPY_MODEL_d61e0c79febd45c8b00e3566296d7e8e"
     }
    },
    "04c0aa3d24944485b475601f786b56f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "052cccb5d69d476f9caac95848e0353a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0549ffe5a6a444dd9fbf8c4513ebe7f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_388d3d2566ff4a1cb68048ac78dd374f",
       "IPY_MODEL_6235eaf236d643c6a163dddb86d3664d",
       "IPY_MODEL_faf78b2739f841d8a213e4311926b598"
      ],
      "layout": "IPY_MODEL_91f03b6ed1c94cb68970f45ffa64695e"
     }
    },
    "06e4e53bd3e748ad90d14eeec117aac9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9c7a21dc260a4ad7b87f2123947058a2",
      "placeholder": "​",
      "style": "IPY_MODEL_7b7a0c46d9794d6fa66401f03684dec0",
      "value": "100%"
     }
    },
    "0dccf470d87f4585af65826582399466": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0f7a38ddfe7b42eaa5c8ffdb3c150641": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1023013dc30a4731b61383bd59f2dc91": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb4bdd1ff3cf4b0db273c53e32cad5de",
      "max": 335,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7c852afee0de49fe8dac74f3af82a725",
      "value": 335
     }
    },
    "1181117783fd4bca9d7f5f296507aff3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11dcde71ff984d829ec9612e7f43e3a3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "154be9baa03943d08c975dbc85c4edf8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1639410507da4b54996064d9fcc8621c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "16b30759d76a4c8f979212d1ae30c297": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1b2b45ecc27b4f28bc0194709fed0638": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1e20837bf23548249678da0a07745d58": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d26e50974c52452389c5abb001056546",
      "max": 29,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9eefbb580f0146658d18658f210d3a67",
      "value": 29
     }
    },
    "20cd68a7e03f4353aa99fcefc1f16d36": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "20e14c8de0f84979ac3f0d896263f49d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a2f5722362c4ec08ae8ed7d681dc7ba",
      "max": 9032074,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f51192579d014a52b132f99abf8901b2",
      "value": 9032074
     }
    },
    "23afbee21a644745b24b8ffefed6b286": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "24b96f20dd5f4a6d8cf9788e97a74f73": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "250451c12cee4aacb91a32d7da4ab861": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "28fc746a3c0548f4a4ff669b5dbb7320": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "29a7b6bb49004c3c96a6d48cd55e1ec3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_06e4e53bd3e748ad90d14eeec117aac9",
       "IPY_MODEL_6a47a3bc614a44e7a5b0dfef50cadaf7",
       "IPY_MODEL_fba6bb8f65904ae6ae4f5bca95e942e2"
      ],
      "layout": "IPY_MODEL_6d5ad2f51bb3459b85201f983a499ea2"
     }
    },
    "2a59b752f9414e69b8c1d24bd24159f2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c37badbe0504073b31abd67a8c0b808": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bfd7d35860c14dd3b8e5eea07032311d",
       "IPY_MODEL_db2c74f92f6d4a2c8bfdcb91d5bd3d64",
       "IPY_MODEL_b0af063597ea4de5a8c8bf4f370f782a"
      ],
      "layout": "IPY_MODEL_5b1ac1a0f04f4c29b3c1127ec3d95e81"
     }
    },
    "2c449a0e664c41a79b3a1de62e94b5b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d652cdca44e04070a420c5d1a24b4fd1",
      "placeholder": "​",
      "style": "IPY_MODEL_b5c71e0a94014278a22ba45a6f58b77d",
      "value": "Downloading (…)neration_config.json: 100%"
     }
    },
    "2d1a48d240b949ad9b465985e90557c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_98ce66c91c5c45968a3844fb0de50624",
      "placeholder": "​",
      "style": "IPY_MODEL_a24d04e7064048c2819ef88a50ce9093",
      "value": "Downloading (…)cial_tokens_map.json: 100%"
     }
    },
    "2fe4fa69d9544fd393a4d72c02245245": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "304609dbefa74735baa08ce7aadff92d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3366101d46a847dfa5700ffb652dcc86": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "33fa92f225be477d8cd176a0a4e8894e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "388d3d2566ff4a1cb68048ac78dd374f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9e312a0ec6a2478fb0f55691caf6d9ae",
      "placeholder": "​",
      "style": "IPY_MODEL_39d35e6e28264b5ca157745d0c1e1491",
      "value": "Downloading (…)tencepiece.bpe.model: 100%"
     }
    },
    "39d35e6e28264b5ca157745d0c1e1491": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3eaf82623d964c2abb92b4509ac5ca9d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33fa92f225be477d8cd176a0a4e8894e",
      "placeholder": "​",
      "style": "IPY_MODEL_b80166eeee984661a611a02d1eae4458",
      "value": "Downloading (…)solve/main/vocab.txt: 100%"
     }
    },
    "42dd646b18b24deda274c857d337c973": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a0f74d882c63403c8116c19745e011ba",
      "placeholder": "​",
      "style": "IPY_MODEL_55a7b680dae94cba8758288dacff2988",
      "value": " 3524/3524 [01:20&lt;00:00, 46.55it/s]"
     }
    },
    "447bb12155cf4dc193ffea3f64c6a8f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4d7c3ea23e9b42a2ba5b0b11ee190c69": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6912ca44b1864902a3e32d4a19a4972d",
      "placeholder": "​",
      "style": "IPY_MODEL_acbfe8b88e6945ab9a5959fbb5f294de",
      "value": " 9.03M/9.03M [00:00&lt;00:00, 35.9MB/s]"
     }
    },
    "5195ed241f09440eaf84a7e0b3dc8155": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "53a2b78a732c4522805de5c9aed92c41": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "53c1ce79c38b45fc8d1b675515642c54": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9370d0df1bca4b9a9c15e1d3e10142d9",
       "IPY_MODEL_611f0f0c70a5471b891bd6f6b8cb7863",
       "IPY_MODEL_986b627aef004704ac0806a9aa189938"
      ],
      "layout": "IPY_MODEL_f2b466617f0247638e0934c8fe95ed1d"
     }
    },
    "550e7aaac7fa4c3db37ceca6a5595ddb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "55a7b680dae94cba8758288dacff2988": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5a0f5ba8ad024f5b915e34e48f269437": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_154be9baa03943d08c975dbc85c4edf8",
      "max": 276,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2fe4fa69d9544fd393a4d72c02245245",
      "value": 276
     }
    },
    "5b1ac1a0f04f4c29b3c1127ec3d95e81": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5d9e4fd05f964fd7a11d66b964ea0873": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "611f0f0c70a5471b891bd6f6b8cb7863": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_78818b2d8cc748efbf1b5784a9ee2a85",
      "max": 3365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7c61a280288d4608b6ef341ce3adb7a8",
      "value": 3365
     }
    },
    "6235eaf236d643c6a163dddb86d3664d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c497cc2cfabb451aad71f5d7b19873e4",
      "max": 4920706,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7f7f2c3afc2647c086877a2f4eff6c12",
      "value": 4920706
     }
    },
    "641665cb9ec6433a81719d57eeaf8298": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ba9c1cd3cfe04add90aa2edf2aa87378",
       "IPY_MODEL_1e20837bf23548249678da0a07745d58",
       "IPY_MODEL_c1d24cb7607c44b989774113a2baf675"
      ],
      "layout": "IPY_MODEL_c9144705a33848a6b142f9887da6ac49"
     }
    },
    "642db3bee97a4bea8a4c1079e2b7526c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3366101d46a847dfa5700ffb652dcc86",
      "placeholder": "​",
      "style": "IPY_MODEL_a1602e58c3a7413490ce0a93f35a7ca9",
      "value": "100%"
     }
    },
    "667f153201d347a0a01e39b438fa736b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "66a35d11743141b1b20904b13a4b86ea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_16b30759d76a4c8f979212d1ae30c297",
      "max": 548,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1639410507da4b54996064d9fcc8621c",
      "value": 548
     }
    },
    "6912ca44b1864902a3e32d4a19a4972d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a2f5722362c4ec08ae8ed7d681dc7ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a47a3bc614a44e7a5b0dfef50cadaf7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_11dcde71ff984d829ec9612e7f43e3a3",
      "max": 2863,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a987cb79f4904973958a98f492c6443a",
      "value": 2863
     }
    },
    "6be63de77ed14eee9c37e93d3a75f743": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_304609dbefa74735baa08ce7aadff92d",
      "placeholder": "​",
      "style": "IPY_MODEL_250451c12cee4aacb91a32d7da4ab861",
      "value": " 110k/110k [00:00&lt;00:00, 1.53MB/s]"
     }
    },
    "6d5ad2f51bb3459b85201f983a499ea2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "715ad9f6bdaa46298317e049404f9755": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d41e81f720f44b52b9a914e4c619bfee",
      "placeholder": "​",
      "style": "IPY_MODEL_bee7c0220d4e476dbc035fbc5dc44fa3",
      "value": " 168/168 [00:00&lt;00:00, 4.67kB/s]"
     }
    },
    "745f78a378b5412eae007f029e8c321a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2c449a0e664c41a79b3a1de62e94b5b7",
       "IPY_MODEL_a9a16bf8795c402a90cf4504b1cd30a1",
       "IPY_MODEL_715ad9f6bdaa46298317e049404f9755"
      ],
      "layout": "IPY_MODEL_667f153201d347a0a01e39b438fa736b"
     }
    },
    "78818b2d8cc748efbf1b5784a9ee2a85": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7a7081fd9d5942af9148e29054b18f51": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7a729fb76a374da99334e7d4b4de102e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7b3c48ae548848c880439841c0b9cb6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb405f797a1441d280029e6e203befce",
      "placeholder": "​",
      "style": "IPY_MODEL_7a7081fd9d5942af9148e29054b18f51",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "7b7a0c46d9794d6fa66401f03684dec0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7c61a280288d4608b6ef341ce3adb7a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7c852afee0de49fe8dac74f3af82a725": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7dee4e5320d349159422709a69af61f4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_86dbdec7087f4312b945c3a3940d8965",
       "IPY_MODEL_cdbf6c3b5b3948229946fd51c227b51e",
       "IPY_MODEL_d132ff798f284c9cbe2a73d35d776592"
      ],
      "layout": "IPY_MODEL_fead01a1a86c40bcb1ba9133ef351d7c"
     }
    },
    "7f7f2c3afc2647c086877a2f4eff6c12": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7f928c5e620a49c1ae137868e34818a9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7fb466a306e6443287cb2fa2bea90f06": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "86dbdec7087f4312b945c3a3940d8965": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f19d9d34c978402691a89114849d2bb9",
      "placeholder": "​",
      "style": "IPY_MODEL_7a729fb76a374da99334e7d4b4de102e",
      "value": "Downloading pytorch_model.bin: 100%"
     }
    },
    "8710bb7686df4fb3a73e5af839be5305": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8b264499afca4694849b08e2091f6826": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3eaf82623d964c2abb92b4509ac5ca9d",
       "IPY_MODEL_aa3ff7831fbf47efa404805886ca4696",
       "IPY_MODEL_6be63de77ed14eee9c37e93d3a75f743"
      ],
      "layout": "IPY_MODEL_5195ed241f09440eaf84a7e0b3dc8155"
     }
    },
    "8d791392eed34a9b8e155da9a4b0a0d6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "901f4ebc70f144c188d03a4978920441": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91f03b6ed1c94cb68970f45ffa64695e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9370d0df1bca4b9a9c15e1d3e10142d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f9b28a5fbcd14568983ed36f9a1ca997",
      "placeholder": "​",
      "style": "IPY_MODEL_04c0aa3d24944485b475601f786b56f1",
      "value": "100%"
     }
    },
    "941a345ba82a44d5ad597a15c0af0d5c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "986b627aef004704ac0806a9aa189938": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8710bb7686df4fb3a73e5af839be5305",
      "placeholder": "​",
      "style": "IPY_MODEL_c7896f62a30f4eec95e3c84aef08e1b2",
      "value": " 3365/3365 [09:20&lt;00:00,  6.34it/s]"
     }
    },
    "98ce66c91c5c45968a3844fb0de50624": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9c7a21dc260a4ad7b87f2123947058a2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9c84cbc758db46e78d7fa361a385ccce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_642db3bee97a4bea8a4c1079e2b7526c",
       "IPY_MODEL_ed9f5d147e7844a187d8610130f1ecd9",
       "IPY_MODEL_42dd646b18b24deda274c857d337c973"
      ],
      "layout": "IPY_MODEL_9e2635394d5b4965b41a05a9bc37a4ec"
     }
    },
    "9e2635394d5b4965b41a05a9bc37a4ec": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9e312a0ec6a2478fb0f55691caf6d9ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9eefbb580f0146658d18658f210d3a67": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a075ee85c2d34ab3b2d011c63dc02286": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a0f74d882c63403c8116c19745e011ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a1602e58c3a7413490ce0a93f35a7ca9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a24d04e7064048c2819ef88a50ce9093": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a5bf4132f3dd4c7a807f27c8106fdd59": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a987cb79f4904973958a98f492c6443a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a9a16bf8795c402a90cf4504b1cd30a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7f928c5e620a49c1ae137868e34818a9",
      "max": 168,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d403f126d48746948c0f627dfdce5d65",
      "value": 168
     }
    },
    "aa3ff7831fbf47efa404805886ca4696": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_20cd68a7e03f4353aa99fcefc1f16d36",
      "max": 109540,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fb908d9c3e374983aa47478a95411ba7",
      "value": 109540
     }
    },
    "acbfe8b88e6945ab9a5959fbb5f294de": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b0af063597ea4de5a8c8bf4f370f782a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a075ee85c2d34ab3b2d011c63dc02286",
      "placeholder": "​",
      "style": "IPY_MODEL_052cccb5d69d476f9caac95848e0353a",
      "value": " 269k/269k [00:00&lt;00:00, 2.44MB/s]"
     }
    },
    "b1cd7728deff4d50a18cb22805351330": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b5c71e0a94014278a22ba45a6f58b77d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b80166eeee984661a611a02d1eae4458": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b91c8359a6444497b3665e66afbe237d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ba9c1cd3cfe04add90aa2edf2aa87378": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ebfa4ba9f18b4945947d326581d3b8e9",
      "placeholder": "​",
      "style": "IPY_MODEL_28fc746a3c0548f4a4ff669b5dbb7320",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "bab8f16e038b440a8016ef087ee12904": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bc4b72ff74d84c9e83b328e8823044b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c4da9fed45254ebdaa49312f16d9b159",
      "placeholder": "​",
      "style": "IPY_MODEL_53a2b78a732c4522805de5c9aed92c41",
      "value": " 276/276 [00:00&lt;00:00, 14.5kB/s]"
     }
    },
    "bee7c0220d4e476dbc035fbc5dc44fa3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bfd7d35860c14dd3b8e5eea07032311d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b91c8359a6444497b3665e66afbe237d",
      "placeholder": "​",
      "style": "IPY_MODEL_1181117783fd4bca9d7f5f296507aff3",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "c0145fad2cb843e9b75ac47d307cc346": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e515df89088042dbbe23594e5976e10d",
      "placeholder": "​",
      "style": "IPY_MODEL_0dccf470d87f4585af65826582399466",
      "value": " 548/548 [00:00&lt;00:00, 21.8kB/s]"
     }
    },
    "c1d24cb7607c44b989774113a2baf675": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c813a6f6ce09424582c06a3ac8a7d712",
      "placeholder": "​",
      "style": "IPY_MODEL_ff6cd33dca3c44ba9832b2d705fd2de1",
      "value": " 29.0/29.0 [00:00&lt;00:00, 1.06kB/s]"
     }
    },
    "c24b174369fb48cdbc3d4afaa4cceccb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c497cc2cfabb451aad71f5d7b19873e4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c4da9fed45254ebdaa49312f16d9b159": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c7896f62a30f4eec95e3c84aef08e1b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c813a6f6ce09424582c06a3ac8a7d712": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9144705a33848a6b142f9887da6ac49": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9bf7b5808f74db9811f5b64e878dcd5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2a59b752f9414e69b8c1d24bd24159f2",
      "placeholder": "​",
      "style": "IPY_MODEL_447bb12155cf4dc193ffea3f64c6a8f8",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "cbd776799c18488eb74a8f17cc27c6fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "cccbdce89ff545f6a66304352b320ad7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cdbf6c3b5b3948229946fd51c227b51e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cccbdce89ff545f6a66304352b320ad7",
      "max": 3465946427,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_cbd776799c18488eb74a8f17cc27c6fd",
      "value": 3465946427
     }
    },
    "d132ff798f284c9cbe2a73d35d776592": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b1cd7728deff4d50a18cb22805351330",
      "placeholder": "​",
      "style": "IPY_MODEL_550e7aaac7fa4c3db37ceca6a5595ddb",
      "value": " 3.47G/3.47G [00:19&lt;00:00, 179MB/s]"
     }
    },
    "d26e50974c52452389c5abb001056546": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d26f522a28d24494914df1dd8432af49": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d403f126d48746948c0f627dfdce5d65": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d41e81f720f44b52b9a914e4c619bfee": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d61e0c79febd45c8b00e3566296d7e8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d652cdca44e04070a420c5d1a24b4fd1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db2c74f92f6d4a2c8bfdcb91d5bd3d64": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_941a345ba82a44d5ad597a15c0af0d5c",
      "max": 268943,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_23afbee21a644745b24b8ffefed6b286",
      "value": 268943
     }
    },
    "ddd59aac5a5d48ceace9ede35bf8f11b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7fb466a306e6443287cb2fa2bea90f06",
      "placeholder": "​",
      "style": "IPY_MODEL_1b2b45ecc27b4f28bc0194709fed0638",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "e515df89088042dbbe23594e5976e10d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb405f797a1441d280029e6e203befce": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb4bdd1ff3cf4b0db273c53e32cad5de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ebfa4ba9f18b4945947d326581d3b8e9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed9f5d147e7844a187d8610130f1ecd9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d26f522a28d24494914df1dd8432af49",
      "max": 3524,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c24b174369fb48cdbc3d4afaa4cceccb",
      "value": 3524
     }
    },
    "ee0cb4c1f31346b5b180a7b826688eb4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_24b96f20dd5f4a6d8cf9788e97a74f73",
      "placeholder": "​",
      "style": "IPY_MODEL_a5bf4132f3dd4c7a807f27c8106fdd59",
      "value": " 335/335 [00:00&lt;00:00, 11.3kB/s]"
     }
    },
    "f19d9d34c978402691a89114849d2bb9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f2b466617f0247638e0934c8fe95ed1d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f51192579d014a52b132f99abf8901b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f607835a7acf4037821885704bde736d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ddd59aac5a5d48ceace9ede35bf8f11b",
       "IPY_MODEL_20e14c8de0f84979ac3f0d896263f49d",
       "IPY_MODEL_4d7c3ea23e9b42a2ba5b0b11ee190c69"
      ],
      "layout": "IPY_MODEL_5d9e4fd05f964fd7a11d66b964ea0873"
     }
    },
    "f9b28a5fbcd14568983ed36f9a1ca997": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "faf78b2739f841d8a213e4311926b598": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0386ad4eea2b4387b48cd58e95bcd4ba",
      "placeholder": "​",
      "style": "IPY_MODEL_028ec097e62b4ee8a463708f4c3b8f24",
      "value": " 4.92M/4.92M [00:00&lt;00:00, 41.4MB/s]"
     }
    },
    "fb908d9c3e374983aa47478a95411ba7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "fba6bb8f65904ae6ae4f5bca95e942e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d791392eed34a9b8e155da9a4b0a0d6",
      "placeholder": "​",
      "style": "IPY_MODEL_bab8f16e038b440a8016ef087ee12904",
      "value": " 2863/2863 [01:03&lt;00:00, 45.98it/s]"
     }
    },
    "fd1e236692ec43a18a1750a29515050d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7b3c48ae548848c880439841c0b9cb6b",
       "IPY_MODEL_1023013dc30a4731b61383bd59f2dc91",
       "IPY_MODEL_ee0cb4c1f31346b5b180a7b826688eb4"
      ],
      "layout": "IPY_MODEL_0f7a38ddfe7b42eaa5c8ffdb3c150641"
     }
    },
    "fda75dc7c2d24f7b827b2c78a6988586": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2d1a48d240b949ad9b465985e90557c5",
       "IPY_MODEL_5a0f5ba8ad024f5b915e34e48f269437",
       "IPY_MODEL_bc4b72ff74d84c9e83b328e8823044b1"
      ],
      "layout": "IPY_MODEL_901f4ebc70f144c188d03a4978920441"
     }
    },
    "fead01a1a86c40bcb1ba9133ef351d7c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ff6cd33dca3c44ba9832b2d705fd2de1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}