{ "cells": [ { "cell_type": "markdown", "id": "vem-019tN3GE", "metadata": { "id": "vem-019tN3GE" }, "source": [ "# Inseq：可视化解释LLM的输出\n", "\n", "> [HW7: Understand what Generative AI is thinking](https://colab.research.google.com/drive/1Xnz0GHC0yWO2Do0aAYBCq9zL45lbiRjM?usp=sharing#scrollTo=UFOUfh2k1jFN) 中文镜像版\n", ">\n", "> 指导文章：[12. Inseq 特征归因：可视化解释 LLM 的输出](https://github.com/Hoper-J/LLM-Guide-and-Demos-zh_CN/blob/master/Guide/12.%20Inseq%20特征归因：可视化解释%20LLM%20的输出.md)\n", "\n", "Feature attribution 特征归因，你可以将其当做对输出的解释，就像图像分类模型可以可视化关注区域一样，LLM 一样也可以。" ] }, { "cell_type": "code", "execution_count": 1, "id": "cnCnLzQYN3GL", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "cnCnLzQYN3GL", "outputId": "7ea18b41-94a4-4885-c84b-08692410c5b1", "scrolled": true, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[2mResolved \u001b[1m220 packages\u001b[0m \u001b[2min 1ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m205 packages\u001b[0m \u001b[2min 0.08ms\u001b[0m\u001b[0m\n", "\u001b[2mResolved \u001b[1m220 packages\u001b[0m \u001b[2min 1ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m205 packages\u001b[0m \u001b[2min 0.07ms\u001b[0m\u001b[0m\n", "\u001b[2mResolved \u001b[1m220 packages\u001b[0m \u001b[2min 1ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m205 packages\u001b[0m \u001b[2min 0.08ms\u001b[0m\u001b[0m\n", "\u001b[2mResolved \u001b[1m220 packages\u001b[0m \u001b[2min 1ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m205 packages\u001b[0m \u001b[2min 0.08ms\u001b[0m\u001b[0m\n", "\u001b[2mResolved \u001b[1m220 packages\u001b[0m \u001b[2min 1ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m205 packages\u001b[0m \u001b[2min 0.07ms\u001b[0m\u001b[0m\n" ] } ], "source": [ "%pip install inseq\n", "%pip install transformers\n", "%pip install accelerate\n", "%pip install sacremoses\n", "%pip install numpy" ] }, { "cell_type": "markdown", "id": "235184ca-2237-4812-b4fe-5164a7222b3f", "metadata": {}, "source": [ "## 设置模型下载镜像\n", "\n", "注意，需要在导入 inseq 前进行设置才能起效。" ] }, { "cell_type": "code", "execution_count": 2, "id": "1bcf948b-da1b-4372-b5e2-bf2343966bc6", "metadata": {}, "outputs": [], "source": [ "import os\n", "os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'" ] }, { "cell_type": "markdown", "id": "db496ea6-080a-4cdc-8934-eabf60b242f4", "metadata": {}, "source": [ "## 修复报错：ImportError: cannot import name 'display' from 'IPython.core.display'\n", "\n", "当前版本的 inseq 库存在一个导入路径问题。在 `inseq/data/viz.py` 文件中，库尝试从 `IPython.core.display` 导入 `display`，但在新版本的 IPython 中，应该从 `IPython.display` 导入。\n", "\n", "执行以下命令修复：" ] }, { "cell_type": "code", "execution_count": 3, "id": "e9f014b2-7c7e-41d1-a594-e55b5cbbf1b7", "metadata": {}, "outputs": [], "source": [ "import sys\n", "import IPython.display\n", "import IPython.core\n", "IPython.core.display = IPython.display\n", "sys.modules['IPython.core.display'] = IPython.display" ] }, { "cell_type": "markdown", "id": "SP6eYcWfN3GM", "metadata": { "id": "SP6eYcWfN3GM", "tags": [] }, "source": [ "## 可视化翻译任务" ] }, { "cell_type": "code", "execution_count": 4, "id": "5XV-BAoJW6RF", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "5XV-BAoJW6RF", "outputId": "4408ab15-1800-4a0f-cc7c-9062e72aaf31" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "使用 Helsinki-NLP/opus-mt-zh-en 模型\n", "\n", "======= 归因方法: saliency =======\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "Attributing with saliency...: 10%|█ | 1/10 [00:000th instance:
\n", "\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " Source Saliency Heatmap\n", "
\n", " x: Generated tokens, y: Attributed tokens\n", "
\n", " \n", "\n", " \n", "
Ilikemachinelearningandartificialintelligence.</s>
我喜欢0.240.440.0830.1130.1260.0480.0410.1160.203
机器0.1590.1160.4470.1780.1290.0560.0590.0860.114
学习0.0740.0760.1550.2830.1380.0420.0480.0540.07
和0.0660.0550.0410.0650.1790.0390.0330.0480.059
人工0.1050.0730.0810.1210.1340.3750.1560.1580.136
智能0.1460.1040.1140.1510.1710.3350.5240.4070.17
。0.140.0820.0350.0440.0790.0540.0640.0750.17
</s>0.070.0530.0450.0460.0440.0510.0750.0560.077
\n", "
\n", "\n", "
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "翻译结果: I like machine learning and artificial intelligence . \n", "\n", "======= 归因方法: attention =======\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "Attributing with attention...: 2it [00:00, 69.45it/s] \n" ] }, { "data": { "text/html": [ "
0th instance:
\n", "\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " Source Saliency Heatmap\n", "
\n", " x: Generated tokens, y: Attributed tokens\n", "
\n", " \n", "\n", " \n", "
Ilikemachinelearningandartificialintelligence.</s>
我喜欢0.210.5370.2330.0120.0260.0450.0150.0420.047
机器0.0870.0330.2170.1740.0430.030.020.0170.028
学习0.0380.0380.1230.2880.1420.0220.0210.0290.026
和0.0490.0390.0560.0380.1280.0930.0130.0320.028
人工0.0210.0230.0370.0430.0660.2540.1840.0360.032
智能0.0210.0230.0630.0570.0670.1130.3460.1360.027
。0.1350.0670.050.0290.060.0530.0430.1530.186
</s>0.4380.240.2210.3590.4680.3910.3580.5550.625
\n", "
\n", "\n", "
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "翻译结果: I like machine learning and artificial intelligence . \n" ] } ], "source": [ "import inseq\n", "import torch\n", "\n", "print(\"使用 Helsinki-NLP/opus-mt-zh-en 模型\")\n", "\n", "# 定义要使用的归因方法列表\n", "attribution_methods = ['saliency', 'attention']\n", "\n", "for method in attribution_methods:\n", " print(f\"\\n======= 归因方法: {method} =======\")\n", "\n", " # 直接用 inseq 加载模型\n", " inseq_model = inseq.load_model(\n", " \"Helsinki-NLP/opus-mt-zh-en\",\n", " attribution_method=method,\n", " model_kwargs={\n", " \"attn_implementation\": \"eager\" if method == \"attention\" else None\n", " }\n", " )\n", "\n", " # 准备输入文本\n", " input_text = \"我喜欢机器学习和人工智能。\"\n", "\n", " # 进行归因分析\n", " attribution_result = inseq_model.attribute(\n", " input_texts=input_text,\n", " show_progress=True\n", " )\n", "\n", " # 清理 tokenizer 中的特殊字符（可选）\n", " for attr in attribution_result.sequence_attributions:\n", " for item in attr.source:\n", " item.token = item.token.replace('▁', '')\n", " for item in attr.target:\n", " item.token = item.token.replace('▁', '')\n", "\n", " # 显示归因结果\n", " attribution_result.show()\n", "\n", " # 打印生成的翻译\n", " if attribution_result.sequence_attributions:\n", " # 获取生成的 tokens\n", " generated_tokens = attribution_result.sequence_attributions[0].target\n", " generated_text = \" \".join([token.token for token in generated_tokens])\n", " print(f\"翻译结果: {generated_text}\")\n", "\n", " # 清理内存\n", " del inseq_model\n", " torch.cuda.empty_cache()" ] }, { "cell_type": "markdown", "id": "H9RtJlsiN3GN", "metadata": { "id": "H9RtJlsiN3GN", "tags": [] }, "source": [ "## 可视化文本生成任务\n", "\n", "用小模型 Qwen3-0.6B 进行演示，如果你的显存充足，可以替换为更大模型。" ] }, { "cell_type": "code", "execution_count": 5, "id": "L1cHlkVPN3GP", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "L1cHlkVPN3GP", "outputId": "f613fdb0-1814-44c9-90e9-816cb0a32a3b" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "======= 归因方法: saliency =======\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Attributing with saliency...: 100%|██████████| 22/22 [00:02<00:00, 6.73it/s]\n" ] }, { "data": { "text/html": [ "
0th instance:
\n", "\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " Target Saliency Heatmap\n", "
\n", " x: Generated tokens, y: Attributed tokens\n", "
\n", " \n", "\n", " \n", "
!ThisisasimpleexamplethatshowshowtocreateaPythonscriptthatcanberunfromthe
Hello0.5760.4790.3420.3450.3340.360.2420.1930.190.1380.1750.1590.1850.240.1350.1630.0850.1160.0940.087
world0.4240.3240.2530.2510.2460.2450.1490.1420.1270.0930.1290.1140.1330.160.0780.1060.0580.0730.0550.06
!0.1960.1880.1380.1290.0830.0850.0890.1120.0810.0840.0950.050.0370.0490.0610.0320.0580.0310.046
This0.2160.1380.1130.0760.0830.0820.0590.0720.0470.0470.0430.0370.0450.0340.0290.0350.020.023
is0.1290.0910.0620.0660.0730.0480.0510.0390.0380.0360.0250.0360.0250.0220.0260.0170.02
a0.0880.0530.0650.0630.0540.0490.040.0360.0310.0240.0390.0270.0270.0260.020.023
simple0.1210.1130.0750.0760.0780.0470.0550.0520.030.0470.0340.020.0270.0210.023
example0.1970.1410.1040.1120.0620.0650.0630.040.0890.0490.0340.0440.030.035
that0.1420.0720.0850.0530.0560.0450.0290.0540.0360.0440.0410.0250.032
shows0.1590.1020.0770.0590.0540.0340.0490.0470.0350.0430.0250.028
how0.1390.1230.0830.0740.0410.070.0570.0550.0650.0360.046
to0.1240.0850.0850.0430.0660.0630.0510.0570.0340.048
create0.1080.0870.0540.0630.0430.0590.0450.0340.033
a0.0620.0450.0320.0370.0520.030.0340.032
Python0.1610.0610.0710.0590.060.0710.053
script0.0870.0840.0850.0890.1140.061
that0.0630.1010.0540.0450.035
can0.1490.0570.060.048
be0.0540.0690.038
run0.1660.095
from0.13
the
\n", "
\n", "\n", "
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "======= 归因方法: attention =======\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n", "Attributing with attention...: 3it [00:00, 10.94it/s]\n" ] }, { "data": { "text/html": [ "
0th instance:
\n", "\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " Target Saliency Heatmap\n", "
\n", " x: Generated tokens, y: Attributed tokens\n", "
\n", " \n", "\n", " \n", "
inC#ĊĊHelloworldisthesimplestandmostcommonexampleinprogramming,andit's
Hello0.8390.7450.730.680.6770.6120.6110.6070.620.5920.6140.5850.5930.5730.6360.5460.5560.552
world0.1610.1210.0450.040.0680.0530.0330.0220.0170.0140.0130.0140.0170.0120.0090.0150.0130.013
in0.1340.0780.040.0270.0350.0280.0150.0150.0110.0080.010.0120.0170.0070.0110.010.008
C0.1480.0720.0290.0310.0210.0120.0160.0090.0090.0130.0140.0240.0190.0110.0110.009
#ĊĊ0.1670.0780.0580.0510.0270.0230.020.0170.020.0270.030.0320.0290.0270.017
Hello0.1220.1080.0570.0380.0240.0180.0150.0160.0150.0120.010.0160.0170.017
world0.1030.0730.0620.0370.020.0220.0320.0290.0220.0130.0250.0140.034
is0.1250.0960.0610.0450.0340.0340.0350.030.0150.0340.0320.029
the0.120.0680.0580.0390.0350.0270.0230.0120.0240.020.019
simplest0.120.1060.080.060.0350.0250.0110.020.0180.018
and0.1070.0650.0440.0210.0190.0080.0210.0290.013
most0.0840.0420.0160.010.0030.0060.0070.005
common0.0960.0540.0380.0140.0210.0180.015
example0.1040.0730.0380.0440.0270.028
in0.0910.0570.0310.0230.015
programming0.1170.0440.0210.019
,0.1010.0650.045
and0.0940.06
it0.086
's
\n", "
\n", "\n", "
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import inseq\n", "\n", "# 定义要使用的归因方法列表\n", "attribution_methods = ['saliency', 'attention']\n", "\n", "for method in attribution_methods:\n", " print(f\"======= 归因方法: {method} =======\")\n", "\n", " # 直接用 inseq 加载模型\n", " inseq_model = inseq.load_model(\n", " \"Qwen/Qwen3-0.6B\",\n", " attribution_method=method,\n", " model_kwargs={\n", " \"device_map\": {\"\": 0}\n", " }\n", " )\n", "\n", " # 设置 padding token\n", " if inseq_model.tokenizer.pad_token is None:\n", " inseq_model.tokenizer.pad_token = inseq_model.tokenizer.eos_token\n", " inseq_model.tokenizer.pad_token_id = inseq_model.tokenizer.eos_token_id\n", "\n", " \n", " # 对输入文本进行归因分析\n", " attribution_result = inseq_model.attribute(\n", " input_texts=\"Hello world\",\n", " show_progress=True\n", " )\n", "\n", " # 清理 tokenizer 中的特殊字符（可选）\n", " for attr in attribution_result.sequence_attributions:\n", " for item in attr.source:\n", " item.token = item.token.replace('Ġ', '')\n", " for item in attr.target:\n", " item.token = item.token.replace('Ġ', '')\n", "\n", " # 显示归因结果\n", " attribution_result.show()\n", "\n", " # 清理内存\n", " del inseq_model\n", " torch.cuda.empty_cache()" ] }, { "cell_type": "code", "execution_count": null, "id": "c-1d5_by_coC", "metadata": { "id": "c-1d5_by_coC" }, "outputs": [], "source": [] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "ai", "language": "python", "name": "ai" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }