{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "provenance": [], "authorship_tag": "ABX9TyNITLrJEkH7cWkASvjXk2VT", "include_colab_link": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "source": [ "# LoRA&Multi-LoRA原理讲解\n", "\n", "Author: kaiyuan\n", "\n", "Email: kyxie@zju.edu.cn\n" ], "metadata": { "id": "QTnqJE5CHxOX" } }, { "cell_type": "markdown", "source": [ "# 1 矩阵的秩分解示例\n", "\n", "![svd](./svd.png)" ], "metadata": { "id": "kZlCbbmxxnTj" } }, { "cell_type": "markdown", "source": [ "1.1 创建一个W矩阵:" ], "metadata": { "id": "E_myG7kPzE2F" } }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "aahThPpXHqib", "outputId": "fca748d6-9489-432e-93a0-d65abd0f3a22" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Shape of W: torch.Size([10, 10])\n", "Rank of W: 2\n" ] } ], "source": [ "import torch\n", "import numpy as np\n", "# 固定随机种子,保证结果相同:\n", "torch.manual_seed(0)\n", "\n", "d, k = 10, 10\n", "\n", "# 创建一个非满秩矩阵(a rank-deficient matrix)\n", "W_rank = 2\n", "W = torch.randn(d,W_rank) @ torch.randn(W_rank,k)\n", "\n", "W_rank = np.linalg.matrix_rank(W)\n", "print(f'Shape of W: {W.shape}')\n", "print(f'Rank of W: {W_rank}')\n" ] }, { "cell_type": "markdown", "source": [ "1.2 对W矩阵进行抵制分解" ], "metadata": { "id": "XuDK0xQPzLp0" } }, { "cell_type": "code", "source": [ "# 对W进行SVD处理:(W = UxSxV^T)\n", "U, S, V = torch.svd(W)\n", "print(\"Singular Value Decomposition on W:\")\n", "print(f'Shape of U: {U.shape}')\n", "print(f'Shape of S: {S.shape}')\n", "print(f'Shape of V: {V.shape}')\n", "print(f'Value of S: {S}\\n')\n", "\n", "# 对于对角矩阵S保留前rank个数据即可,相应的U和V也只要保存前rank行的数据。\n", "U_r = U[:, :W_rank]\n", "S_r = torch.diag(S[:W_rank])\n", "V_r = V[:, :W_rank].t()\n", "\n", "# 定义: B = U_r * S_r;A = V_r\n", "B = U_r @ S_r\n", "A = V_r\n", "print(\"After decomposition, got A and B:\")\n", "print(f'Shape of B: {B.shape}')\n", "print(f'Shape of A: {A.shape}')" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "xKvz7Bf7qleY", "outputId": "813a50f1-cb39-4236-df58-37c77e6188c3" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Singular Value Decomposition on W:\n", "Shape of U: torch.Size([10, 10])\n", "Shape of S: torch.Size([10])\n", "Shape of V: torch.Size([10, 10])\n", "Value of S: tensor([1.1385e+01, 4.8439e+00, 3.7467e-07, 3.0064e-07, 2.3395e-07, 1.1759e-07,\n", " 7.7860e-08, 3.8146e-08, 1.4842e-08, 4.1472e-09])\n", "\n", "After decomposition, got A and B:\n", "Shape of B: torch.Size([10, 2])\n", "Shape of A: torch.Size([2, 10])\n" ] } ] }, { "cell_type": "markdown", "source": [ "1.3 对比结果差异" ], "metadata": { "id": "0syPmdETzRvA" } }, { "cell_type": "code", "source": [ "# 创建一个线性运算的输入, y = Wx + b\n", "\n", "bias = torch.randn(d)\n", "x = torch.randn(d)\n", "\n", "# 原始计算 y = Wx + bias\n", "y = W @ x + bias\n", "# 秩分解计算 y' = (B*A)x + bias\n", "y_prime = (B @ A) @ x + bias\n", "\n", "print(f\"The result is allclose: {torch.allclose(y, y_prime)}\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Awn2jLmUzWcz", "outputId": "dffe6d5d-96a8-47cf-a845-e686b4f4f666" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "The result is allclose: False\n" ] } ] }, { "cell_type": "markdown", "source": [ "# 2 LoRA训练的简单示例\n", "\n", "![lora](./lora.png)" ], "metadata": { "id": "IiDkeaUp07NU" } }, { "cell_type": "markdown", "source": [ "构建一个3层线性网络,训练其数字手写体MNIST(数字0-9)的识别能力:\n", "\n", "* step1:构建主模型并训练,训练数据集去掉数字‘1’;\n", "* step2:测试主模型的识别能力;\n", "* step3:创建LoRA层;\n", "* step4:主模型的参数冻结,用数字‘1’的数据进行微调;\n", "* step5:测试LoRA模型,观测数据‘1’识别度差异。" ], "metadata": { "id": "aKbc6KZhFHi3" } }, { "cell_type": "code", "source": [ "# 可视化库安装\n", "!pip install --upgrade torchvista" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Bnsk-J5WFJPs", "outputId": "0adfdea2-3cbd-4341-8571-216fc1cbefcd" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting torchvista\n", " Downloading torchvista-0.2.7-py3-none-any.whl.metadata (814 bytes)\n", "Requirement already satisfied: ipython>=7.0.0 in /usr/local/lib/python3.12/dist-packages (from torchvista) (7.34.0)\n", "Requirement already satisfied: numpy>=1.18.0 in /usr/local/lib/python3.12/dist-packages (from torchvista) (2.0.2)\n", "Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (75.2.0)\n", "Collecting jedi>=0.16 (from ipython>=7.0.0->torchvista)\n", " Downloading jedi-0.19.2-py2.py3-none-any.whl.metadata (22 kB)\n", "Requirement already satisfied: decorator in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (4.4.2)\n", "Requirement already satisfied: pickleshare in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (0.7.5)\n", "Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (5.7.1)\n", "Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (3.0.52)\n", "Requirement already satisfied: pygments in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (2.19.2)\n", "Requirement already satisfied: backcall in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (0.2.0)\n", "Requirement already satisfied: matplotlib-inline in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (0.2.1)\n", "Requirement already satisfied: pexpect>4.3 in /usr/local/lib/python3.12/dist-packages (from ipython>=7.0.0->torchvista) (4.9.0)\n", "Requirement already satisfied: parso<0.9.0,>=0.8.4 in /usr/local/lib/python3.12/dist-packages (from jedi>=0.16->ipython>=7.0.0->torchvista) (0.8.5)\n", "Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.12/dist-packages (from pexpect>4.3->ipython>=7.0.0->torchvista) (0.7.0)\n", "Requirement already satisfied: wcwidth in /usr/local/lib/python3.12/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=7.0.0->torchvista) (0.2.14)\n", "Downloading torchvista-0.2.7-py3-none-any.whl (1.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m16.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading jedi-0.19.2-py2.py3-none-any.whl (1.6 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m27.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hInstalling collected packages: jedi, torchvista\n", "Successfully installed jedi-0.19.2 torchvista-0.2.7\n" ] } ] }, { "cell_type": "markdown", "source": [ "## 2.1 手写体训练与测试" ], "metadata": { "id": "erM_nH6JXorK" } }, { "cell_type": "code", "source": [ "import torchvision.datasets as datasets\n", "import torchvision.transforms as transforms\n", "import torch.nn as nn\n", "import matplotlib.pyplot as plt\n", "from tqdm import tqdm\n", "\n", "# 设备选择:\n", "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n", "\n", "# 创建一个全连接的网络用于手写体识别:\n", "class MLP(nn.Module):\n", " def __init__(self, hidden_size_1=1000, hidden_size_2=2000):\n", " super(MLP,self).__init__()\n", " self.linear1 = nn.Linear(28*28, hidden_size_1)\n", " self.linear2 = nn.Linear(hidden_size_1, hidden_size_2)\n", " self.linear3 = nn.Linear(hidden_size_2, 10)\n", " self.relu = nn.ReLU()\n", "\n", " def forward(self, img):\n", " x = img.view(-1, 28*28)\n", " x = self.relu(self.linear1(x))\n", " x = self.relu(self.linear2(x))\n", " x = self.linear3(x)\n", " return x\n", "\n", "net = MLP().to(device)" ], "metadata": { "id": "O-C5nkca2cBt" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "# 打印层参数,保存原始参数总量用于后续对比\n", "total_parameters_original = 0\n", "for index, layer in enumerate([net.linear1, net.linear2, net.linear3]):\n", " total_parameters_original += layer.weight.nelement() + layer.bias.nelement()\n", " print(f'Layer {index+1}: W: {layer.weight.shape} + B: {layer.bias.shape}')\n", "print(f'Total number of parameters: {total_parameters_original:,}')" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "MWQOKhLoQrFH", "outputId": "23aa4493-c0a1-4242-9250-2d7502170833" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Layer 1: W: torch.Size([1000, 784]) + B: torch.Size([1000])\n", "Layer 2: W: torch.Size([2000, 1000]) + B: torch.Size([2000])\n", "Layer 3: W: torch.Size([10, 2000]) + B: torch.Size([10])\n", "Total number of parameters: 2,807,010\n" ] } ] }, { "cell_type": "code", "source": [ "from torchvista import trace_model\n", "\n", "# 可视化模型\n", "example_input = torch.randn(10, 28*28)\n", "trace_model(\n", " net,\n", " example_input,\n", " collapse_modules_after_depth=3,\n", " show_non_gradient_nodes=False,\n", " forced_module_tracing_depth=None,\n", " height=500\n", ")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 517 }, "id": "O76xBcRqFgDh", "outputId": "411270eb-9a8f-4ed7-a52f-c8f77ba8d05e" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "
\n", "
\n", " \n", "
\n", "
\n", "

\n", " \n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " forward() Parameters\n", "
\n", "
\n", "
\n", " Attributes\n", "
\n", "
\n", "
\n", "
\n", " Parameters\n", "
\n", "
\n", "

No further information available for this node.

\n", "
\n", "
\n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", "
\n" ] }, "metadata": {} } ] }, { "cell_type": "code", "source": [ "# 定义训练函数:\n", "def train(train_loader, net, epochs=5, total_iterations_limit=None):\n", " cross_el = nn.CrossEntropyLoss()\n", " optimizer = torch.optim.Adam(net.parameters(), lr=0.001)\n", "\n", " total_iterations = 0\n", "\n", " for epoch in range(epochs):\n", " net.train()\n", "\n", " loss_sum = 0\n", " num_iterations = 0\n", "\n", " data_iterator = tqdm(train_loader, desc=f'Epoch {epoch+1}')\n", " if total_iterations_limit is not None:\n", " data_iterator.total = total_iterations_limit\n", " for data in data_iterator:\n", " num_iterations += 1\n", " total_iterations += 1\n", " x, y = data\n", " x = x.to(device)\n", " y = y.to(device)\n", " optimizer.zero_grad()\n", " output = net(x.view(-1, 28*28))\n", " loss = cross_el(output, y)\n", " loss_sum += loss.item()\n", " avg_loss = loss_sum / num_iterations\n", " data_iterator.set_postfix(loss=avg_loss)\n", " loss.backward()\n", " optimizer.step()\n", "\n", " if total_iterations_limit is not None and total_iterations >= total_iterations_limit:\n", " return" ], "metadata": { "id": "QKLuOJe92iFF" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "# 下载MNIST手写体数字识别的数据\n", "transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])\n", "\n", "# 加载手写体数据:\n", "mnist_trainset = datasets.MNIST(root='./data', train=True, download=True, transform=transform) # 训练集\n", "train_loader = torch.utils.data.DataLoader(mnist_trainset, batch_size=10, shuffle=True)\n", "mnist_testset = datasets.MNIST(root='./data', train=False, download=True, transform=transform) # 测试集\n", "\n", "# 去掉数字‘1'的数据,模型对‘1'的识别率存在问题\n", "exclude_indices = torch.tensor([False if x == 1 else True for x in mnist_trainset.targets])\n", "mnist_trainset.data = mnist_trainset.data[exclude_indices]\n", "mnist_trainset.targets = mnist_trainset.targets[exclude_indices]\n", "\n", "# 训练模型:\n", "train(train_loader, net, epochs=1, total_iterations_limit=2000)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "WCgiS0a6-2E8", "outputId": "4f2dc110-4e5b-4104-b166-558ec5c2fb3f" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "100%|██████████| 9.91M/9.91M [00:00<00:00, 38.7MB/s]\n", "100%|██████████| 28.9k/28.9k [00:00<00:00, 1.18MB/s]\n", "100%|██████████| 1.65M/1.65M [00:00<00:00, 6.65MB/s]\n", "100%|██████████| 4.54k/4.54k [00:00<00:00, 3.36MB/s]\n", "Epoch 1: 100%|█████████▉| 1999/2000 [00:44<00:00, 45.14it/s, loss=0.327]\n" ] } ] }, { "cell_type": "code", "source": [ "# 加载测试数据,观测结果:\n", "test_loader = torch.utils.data.DataLoader(mnist_testset, batch_size=10, shuffle=True)\n", "def test(model=net):\n", " total = 0\n", " wrong_counts = [0 for i in range(10)]\n", " correct_counts = [0 for i in range(10)]\n", "\n", " with torch.no_grad():\n", " for data in tqdm(test_loader, desc='Testing'):\n", " x, y = data\n", " x = x.to(device)\n", " y = y.to(device)\n", " output = model(x.view(-1, 784))\n", " for idx, i in enumerate(output):\n", " if torch.argmax(i) == y[idx]:\n", " correct_counts[y[idx]] +=1\n", " else:\n", " wrong_counts[y[idx]] +=1\n", " total +=1\n", " result_str = \"\"\n", " for i in range(len(wrong_counts)):\n", " result_str += f'The wrong counts of digit {i}: {wrong_counts[i]}\\n'\n", " print(f'\\nAccuracy: {round(sum(correct_counts)/total, 3)}\\n{result_str}')\n", " return [x / (x+y) for x, y in zip(correct_counts, wrong_counts)]\n", "\n", "test()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Zm31BDpU2nMG", "outputId": "b9ad7bb2-2b69-4e7d-9d3c-a32dfb1f495f" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:03<00:00, 288.46it/s]" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.84\n", "The wrong counts of digit 0: 12\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 81\n", "The wrong counts of digit 3: 35\n", "The wrong counts of digit 4: 47\n", "The wrong counts of digit 5: 61\n", "The wrong counts of digit 6: 31\n", "The wrong counts of digit 7: 73\n", "The wrong counts of digit 8: 50\n", "The wrong counts of digit 9: 75\n", "\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ "[0.9877551020408163,\n", " 0.0,\n", " 0.9215116279069767,\n", " 0.9653465346534653,\n", " 0.9521384928716904,\n", " 0.9316143497757847,\n", " 0.9676409185803758,\n", " 0.9289883268482491,\n", " 0.9486652977412731,\n", " 0.9256689791873142]" ] }, "metadata": {}, "execution_count": 10 } ] }, { "cell_type": "markdown", "source": [ "## 2.2 LoRA微调" ], "metadata": { "id": "ZFKtrH8aXw9d" } }, { "cell_type": "code", "source": [ "# 定义LoRA对权重修改修改:\n", "class LoRAParametrization(nn.Module):\n", " def __init__(self, features_in, features_out, rank=1, alpha=1, device='cpu'):\n", " super().__init__()\n", " # 低秩矩阵的定义:\n", " self.lora_A = nn.Parameter(torch.zeros((rank,features_out)).to(device))\n", " self.lora_B = nn.Parameter(torch.zeros((features_in, rank)).to(device))\n", " nn.init.normal_(self.lora_A, mean=0, std=1)\n", "\n", " # 参考论文:https://arxiv.org/pdf/2106.09685 4.1节 设置一个比例系数:\n", " self.scale = alpha / rank\n", " # LoRA开关:\n", " self.enabled = True\n", "\n", " def forward(self, original_weights):\n", " if self.enabled:\n", " # Return W + (B*A)*scale\n", " return original_weights + torch.matmul(self.lora_B, self.lora_A).view(original_weights.shape) * self.scale\n", " else:\n", " return original_weights" ], "metadata": { "id": "_RdVOz-z9GTC" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "# 可视化LoRA层\n", "lora_param = LoRAParametrization(1000, 784)\n", "example_input = torch.randn(1000, 784)\n", "trace_model(\n", " lora_param,\n", " example_input,\n", " collapse_modules_after_depth=3,\n", " show_non_gradient_nodes=False,\n", " forced_module_tracing_depth=None,\n", " height=500\n", ")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 517 }, "id": "PGYoC3W6Lwsx", "outputId": "bd269da2-3951-485b-e0eb-688cf106fd83" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "
\n", "
\n", " \n", "
\n", "
\n", "

\n", " \n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " forward() Parameters\n", "
\n", "
\n", "
\n", " Attributes\n", "
\n", "
\n", "
\n", "
\n", " Parameters\n", "
\n", "
\n", "

No further information available for this node.

\n", "
\n", "
\n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", "
\n" ] }, "metadata": {} } ] }, { "cell_type": "code", "source": [ "import torch.nn.utils.parametrize as parametrize\n", "\n", "def linear_layer_parameterization(layer, device, rank=1, lora_alpha=1):\n", " # LoRA仅修改W,忽略bias修改。\n", " features_in, features_out = layer.weight.shape\n", " return LoRAParametrization(\n", " features_in, features_out, rank=rank, alpha=lora_alpha, device=device\n", " )\n", "\n", "# 保存一份原始权重数据,用于后续校验\n", "original_weights = {}\n", "for name, param in net.named_parameters():\n", " original_weights[name] = param.clone().detach()\n", "\n", "# 注册LoRA权重到原始层中:\n", "parametrize.register_parametrization(\n", " net.linear1, \"weight\", linear_layer_parameterization(net.linear1, device)\n", ")\n", "parametrize.register_parametrization(\n", " net.linear2, \"weight\", linear_layer_parameterization(net.linear2, device)\n", ")\n", "parametrize.register_parametrization(\n", " net.linear3, \"weight\", linear_layer_parameterization(net.linear3, device)\n", ")\n", "\n", "# 定义LoRA开关函数:\n", "def enable_disable_lora(enabled=True):\n", " for layer in [net.linear1, net.linear2, net.linear3]:\n", " layer.parametrizations[\"weight\"][0].enabled = enabled" ], "metadata": { "id": "ZYInB10D9OHE" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "total_parameters_lora = 0\n", "total_parameters_non_lora = 0\n", "for index, layer in enumerate([net.linear1, net.linear2, net.linear3]):\n", " total_parameters_lora += layer.parametrizations[\"weight\"][0].lora_A.nelement() + layer.parametrizations[\"weight\"][0].lora_B.nelement()\n", " total_parameters_non_lora += layer.weight.nelement() + layer.bias.nelement()\n", " print(\n", " f'Layer {index+1}: W: {layer.weight.shape} + B: {layer.bias.shape} + Lora_A: {layer.parametrizations[\"weight\"][0].lora_A.shape} + Lora_B: {layer.parametrizations[\"weight\"][0].lora_B.shape}'\n", " )\n", "# 模型原始参数量应当保持不变:\n", "assert total_parameters_non_lora == total_parameters_original\n", "# 打印添加LoRA后的参数对比:\n", "print(f'Total number of parameters (original): {total_parameters_non_lora:,}')\n", "print(f'Total number of parameters (original + LoRA): {total_parameters_lora + total_parameters_non_lora:,}')\n", "print(f'Parameters introduced by LoRA: {total_parameters_lora:,}')\n", "parameters_incremment = (total_parameters_lora / total_parameters_non_lora) * 100\n", "print(f'Parameters incremment: {parameters_incremment:.3f}%')" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "bCGjy22Q9Uv1", "outputId": "1196e09e-be00-459c-8090-628a717ded11" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Layer 1: W: torch.Size([1000, 784]) + B: torch.Size([1000]) + Lora_A: torch.Size([1, 784]) + Lora_B: torch.Size([1000, 1])\n", "Layer 2: W: torch.Size([2000, 1000]) + B: torch.Size([2000]) + Lora_A: torch.Size([1, 1000]) + Lora_B: torch.Size([2000, 1])\n", "Layer 3: W: torch.Size([10, 2000]) + B: torch.Size([10]) + Lora_A: torch.Size([1, 2000]) + Lora_B: torch.Size([10, 1])\n", "Total number of parameters (original): 2,807,010\n", "Total number of parameters (original + LoRA): 2,813,804\n", "Parameters introduced by LoRA: 6,794\n", "Parameters incremment: 0.242%\n" ] } ] }, { "cell_type": "code", "source": [ "# 将原始权重冻结:\n", "for name, param in net.named_parameters():\n", " if 'lora' not in name:\n", " print(f'Freezing non-LoRA parameter {name}')\n", " param.requires_grad = False\n", "\n", "# 过滤数据,仅保留‘1'的数据:\n", "mnist_trainset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n", "exclude_indices = torch.tensor([True if x == 1 else False for x in mnist_trainset.targets])\n", "mnist_trainset.data = mnist_trainset.data[exclude_indices]\n", "mnist_trainset.targets = mnist_trainset.targets[exclude_indices]\n", "train_loader = torch.utils.data.DataLoader(mnist_trainset, batch_size=10, shuffle=True)\n", "\n", "# 用数据‘1'训练带有LoRA的模型:\n", "train(train_loader, net, epochs=1, total_iterations_limit=100)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "UlfwGg5E9XB3", "outputId": "6aa30e7a-5f17-4438-e218-4ec1acbc4e77" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Freezing non-LoRA parameter linear1.bias\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.original\n", "Freezing non-LoRA parameter linear2.bias\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.original\n", "Freezing non-LoRA parameter linear3.bias\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.original\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Epoch 1: 99%|█████████▉| 99/100 [00:01<00:00, 63.21it/s, loss=5.25]\n" ] } ] }, { "cell_type": "markdown", "source": [ "# 2.3 参数与结果对比" ], "metadata": { "id": "palfTHbIX4CR" } }, { "cell_type": "code", "source": [ "# 微调不改变冻结权重,对比:\n", "assert torch.all(net.linear1.parametrizations.weight.original == original_weights['linear1.weight'])\n", "assert torch.all(net.linear2.parametrizations.weight.original == original_weights['linear2.weight'])\n", "assert torch.all(net.linear3.parametrizations.weight.original == original_weights['linear3.weight'])\n", "\n", "# 检测开启LoRA后的权重是否为满足公式: W + (B*A)*scale\n", "enable_disable_lora(enabled=True)\n", "# 使用了pytorch的权重变换函数,名称发生了改变\n", "# The original weights have been moved to net.linear1.parametrizations.weight.original\n", "# More info here: https://pytorch.org/tutorials/intermediate/parametrizations.html#inspecting-a-parametrized-module\n", "assert torch.equal(net.linear1.weight, net.linear1.parametrizations.weight.original + (net.linear1.parametrizations.weight[0].lora_B @ net.linear1.parametrizations.weight[0].lora_A) * net.linear1.parametrizations.weight[0].scale)\n", "\n", "# 关闭LoRA与原始权重相等:\n", "enable_disable_lora(enabled=False)\n", "assert torch.equal(net.linear1.weight, original_weights['linear1.weight'])" ], "metadata": { "id": "R4Jr08939sgB" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "# 测试有LoRA的情况:\n", "enable_disable_lora(enabled=True)\n", "lora_correct_rate = test()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "DjP_Dn-39v7Z", "outputId": "3f2ee19f-ca62-4eab-b495-c71b0d3ce6a8" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:07<00:00, 133.22it/s]" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.826\n", "The wrong counts of digit 0: 38\n", "The wrong counts of digit 1: 32\n", "The wrong counts of digit 2: 418\n", "The wrong counts of digit 3: 196\n", "The wrong counts of digit 4: 100\n", "The wrong counts of digit 5: 140\n", "The wrong counts of digit 6: 169\n", "The wrong counts of digit 7: 266\n", "The wrong counts of digit 8: 225\n", "The wrong counts of digit 9: 153\n", "\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\n" ] } ] }, { "cell_type": "code", "source": [ "# 关闭LoRA的对照组:\n", "enable_disable_lora(enabled=False)\n", "original_correct_rate = test()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "_Pn56zVK-jKK", "outputId": "f3d22f31-0a25-4970-ab66-525fdc48d6d5" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:03<00:00, 282.44it/s]" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.84\n", "The wrong counts of digit 0: 12\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 81\n", "The wrong counts of digit 3: 35\n", "The wrong counts of digit 4: 47\n", "The wrong counts of digit 5: 61\n", "The wrong counts of digit 6: 31\n", "The wrong counts of digit 7: 73\n", "The wrong counts of digit 8: 50\n", "The wrong counts of digit 9: 75\n", "\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\n" ] } ] }, { "cell_type": "markdown", "source": [ "定义一个绘图函数,绘制正确率的差异" ], "metadata": { "id": "fOV-Jco48oPK" } }, { "cell_type": "code", "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "# 绘制正确率对比的差异:\n", "def plot_correct_rate(original_data, lora_data, adapter=1):\n", "\n", " # Verify both arrays have the same length\n", " if len(original_data) != len(lora_data):\n", " raise ValueError(\"Arrays must have the same length\")\n", " array_length = len(original_data)\n", "\n", " # Generate labels for the bars on the x-axis\n", " x_labels = [f'digit {i}' for i in range(array_length)]\n", "\n", " # Set the positions of the bars on the x-axis\n", " x = np.arange(array_length)\n", "\n", " bar_width = 0.35\n", "\n", " pos1 = x - bar_width / 2\n", " pos2 = x + bar_width / 2\n", "\n", " fig, ax = plt.subplots(figsize=(12, 6))\n", " rects1 = ax.bar(pos1, original_data, bar_width,\n", " label='Original',\n", " color='steelblue',\n", " edgecolor='black',\n", " zorder=2)\n", "\n", " rects2 = ax.bar(pos2, lora_data, bar_width,\n", " label=f'LoRA finetune: {adapter}',\n", " color='lightcoral',\n", " edgecolor='black',\n", " zorder=2)\n", "\n", " # Set labels and title for the chart\n", " ax.set_xlabel('Nmbuer Category', fontsize=12, labelpad=12)\n", " ax.set_ylabel('correct rate(%)', fontsize=12, labelpad=12)\n", " ax.set_title('Correction Comparison', fontsize=14, pad=15)\n", "\n", " ax.set_xticks(x)\n", " ax.set_xticklabels(x_labels)\n", " ax.set_ylim(0, max(original_data + lora_data) * 1.25)\n", " ax.legend(loc='upper left', frameon=True, framealpha=0.9)\n", "\n", " ax.grid(True, which='major', axis='y', linestyle='--', alpha=0.7, zorder=1)\n", "\n", " # Function to attach a text label above each bar, displaying its height\n", " def autolabel(rects, values):\n", " for rect, value in zip(rects, values):\n", " height = rect.get_height()\n", " # Only add label if bar is tall enough (avoids clutter)\n", " if height > max(original_data + lora_data) * 0.05: # Only label bars >5% of max height\n", " ax.text(rect.get_x() + rect.get_width() / 2., height + 0.01,\n", " f'{value:.2f}', # Format to two decimal places\n", " ha='center', va='bottom', fontsize=9, rotation=0)\n", "\n", " autolabel(rects1, original_data)\n", " autolabel(rects2, lora_data)\n", "\n", " handles, labels = ax.get_legend_handles_labels()\n", " ax.legend(handles, labels, loc='upper left', frameon=True, framealpha=0.9)\n", "\n", " plt.tight_layout()\n", " plt.show()\n", "\n", "plot_correct_rate(original_correct_rate, lora_correct_rate)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 607 }, "id": "njV_w2zR5n3x", "outputId": "84912752-e61f-40d5-818b-0bd817bd3355" }, "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "
" ], "image/png": "iVBORw0KGgoAAAANSUhEUgAABKYAAAJOCAYAAACN2Q8zAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAnqFJREFUeJzs3Xl8Tmf+//H3nVXEkkSCbIQQ0ioJ2o7qQoeiHaraqm1KiuqopZapqr0NWh1bS6vUUjO1tJ0WXX7aoRSljCUUEbvYghCJBNnu+/eHyfnmdgdJJPedxOv5eHjIfZ3rnPtz3dc5933O55xzHZPFYrEIAAAAAAAAsDMnRwcAAAAAAACAexOJKQAAAAAAADgEiSkAAAAAAAA4BIkpAAAAAAAAOASJKQAAAAAAADgEiSkAAAAAAAA4BIkpAAAAAAAAOASJKQAAAAAAADgEiSkAAAAAAAA4BIkpAACAfOjVq5dMJpOOHz/u6FBQSC1atJDJZHJ0GAAAIBcSUwAAwMaOHTvUu3dv1a1bV56envLw8FBoaKj++te/6j//+Y+jwysWixYtkslk0qJFixwdSoFcvXpVM2fOVMuWLeXn5ydXV1f5+Pjo0Ucf1XvvvacLFy44OkQAAIBbMlksFoujgwAAACWD2WzW8OHDNX36dLm4uOjJJ59UgwYN5OrqqqNHj2rNmjVKSkrSO++8ozFjxjg63CK1aNEiRUVFaeHCherVq5fN9LNnzyo5OVmhoaFydXW1f4B52L17t5599lmdOHFCNWvW1J///GdVq1ZNKSkp+v3337Vjxw5VqlRJZ86ckaenp6PDdbj4+HhdvXpV9evXd3QoAADgf1wcHQAAACg5Ro8erenTpysiIkJff/21QkNDraZfu3ZNs2bN0sWLFx0UoeP4+/vL39/f0WEYTp06paeeekqJiYmaOnWqBg8eLGdnZ6s6u3bt0oABA5SZmemgKEuWGjVqODoEAABwE27lAwAAkqTDhw9rypQpqlKlilavXm2TlJIkDw8P/f3vf9eECROsyhMTE/XGG2+oVq1acnd3V9WqVdW5c2ft3bvXZhk5YzUdPXpUU6dO1X333Sd3d3fjKqWQkBCFhITo8uXLGjBggIKDg+Xi4mJ1i92ePXvUpUsX+fv7y83NTTVr1tTAgQNvmTDbvXu3unfvrqCgILm7u8vf319t27bVd999Z8QUFRUlSYqKipLJZDL+3Rx3XmNMLVy4UA8//LAqVKigChUq6OGHH87zlsD169fLZDJp/Pjx2r59u1q3bq2KFSuqcuXKeu655wo0ftWoUaN0/vx5vf322xo6dKhNUkqSIiMj9euvv6pSpUpW5d99951atmypypUry8PDQ40aNdK0adOUlZVlVe/48eMymUzq1auXYmNj9Ze//EVeXl7y9vZW165dlZiYKEnasmWL/vznP6tSpUry9vZWnz59lJaWdsu2b9q0SS1atFDFihXl5eWl559/XocPH7aJf926dXrllVdUr14947Nt2rSp5s6dm+dnYjKZ1KJFC50+fVovv/yyqlevLicnJ61fv15S3mNMmc1mffbZZ3rooYfk4+MjDw8PBQUFqX379sZ8uTmirwEAKMu4YgoAAEi6cStbdna2+vXrp2rVqt22rru7u/H3hQsX1KxZMx05ckQtWrRQly5ddOzYMX399df64Ycf9NNPP+nRRx+1WcbAgQP1+++/65lnnlH79u1VtWpVY1p6erqefPJJpaamqkOHDnJxcTFiWrVqlTp37iwnJyc9++yzCg4O1v79+zVr1iz99NNP2rp1q7y9vY1l/fvf/1a3bt1ksVjUvn171atXT+fPn9fWrVs1f/58tW/fXh07dtTly5e1cuVKPfvss4qIiMj35zZo0CB99NFHCgwMVO/evY33jIqK0q5duzRz5kybef773/9qypQpatmypfr166ddu3ZpxYoV+uOPP7R3716VK1futu959epVLVu2TB4eHho+fPht67q4WO/uTZs2TcOGDZOPj4+6desmT09PrVq1SsOGDdPGjRv1zTff2CRvjh07pkceeURNmzZVnz59tH37di1btkwnT57Ue++9p6eeekqtW7fWq6++qvXr12v+/Pkym81asGCBTTy///67Jk+erLZt22rgwIHat2+fvv32W23cuFG///67ateubdR9//33dfjwYf3pT3/Sc889p8uXL2v16tXq16+f4uLiNHXqVJvlX7x4Uc2aNZOPj4+6dOmi69ev2yTmchs5cqSmTJmi0NBQdevWTRUrVtTp06e1adMmrVmzRi1atDDqOqKvAQAo8ywAAAAWi6VFixYWSZY1a9YUaL6oqCiLJMvIkSOtyn/44QeLJEudOnUs2dnZRnnPnj0tkixBQUGWEydO2CyvZs2aFkmWNm3aWK5evWo1LTEx0VKpUiVLYGCg5fjx41bTli5dapFkGTBggFGWkJBg8fT0tHh6elp27txp814nT540/l64cKFFkmXhwoV5tjMn7mPHjhllv/76q0WSJTw83HL58mWj/NKlS5awsDCLJMuGDRuM8nXr1lkkWSRZli1bZrX8v/71rxZJlqVLl+b5/rmtX7/eIsny6KOP3rFubocPH7a4uLhYqlataomPjzfKr1+/bnn00UctkiyLFy82yo8dO2bEO2PGDKPcbDZbnn76aYski5eXl2XFihXGtIyMDEvDhg0tLi4uloSEhDzbPmfOHKu45syZY5Fk+ctf/mJVfvToUZs2ZGZmWlq3bm1xdna2WX9ylh8VFWXJysqymfeJJ56w3Lz76+PjYwkICLCkpaXZ1L948aLxt6P6GgCAso5b+QAAgCQpISFBkhQUFJTveTIyMrR06VJVqVJFo0ePtpr29NNPq3Xr1jp8+LB+++03m3n//ve/33bMnylTpsjDw8OqbPHixUpJSdHkyZNVs2ZNq2ldunRR48aNtWzZMqPs888/V1pamoYNG6bIyEib9yhIW/Py+eefS5LGjx+vypUrG+Xe3t4aN26cJOV5m9fjjz+ul156yarslVdekXTjCps7KUxfSdKSJUuUlZWlYcOGKTg42Ch3d3fX+++/f8t4Q0NDNWjQIOO1yWRSly5dJN24XfDZZ581prm6uuqFF15QVlaW9u/fb7OssLAw9e3b16qsb9++qlu3rn744QerpwjWqlXLZn4XFxe99tprys7O1rp162ymu7m5acqUKXne2ngrbm5uedb38fEx/nZUXwMAUNZxKx8AACi0AwcO6Pr162rZsqXKly9vM71ly5b6z3/+o5iYGD322GNW0x566KFbLrdcuXJ64IEHbMp///13SdLWrVt15MgRm+nXr19XYmKiEhMT5evrq23btkmSnnrqqQK1K7927dolSVa3e+Vo2bKlJCkmJsZmWpMmTWzKcpJMly9fLrL4bna7eJs1a6Zy5crlGW/Dhg1tbu/LGQg+r9sec6adOXPGZlrz5s3l5GR9btTJyUnNmzfXoUOHtHv3brVq1UqSdOXKFf3jH//QihUrdOTIEZtxq/Jafq1ateTr62tTfitdunTRxx9/rAYNGqhLly5q2bKlmjVrZpMULW19DQBAaUFiCgAASJKqV6+uAwcO6PTp06pXr16+5klJSZGkW45JlZOgyKmX2+3GsapatapNIkSSLl26JEmaPXv2beNKS0uTr6+vkpOTJUmBgYG3rV9YKSkpcnJykp+fn820atWqyWQy5dn2vMY8yhkLKjs7+47vW716dUnS6dOnCxxvTmw3M5lMqlatWp7LvF28t5uW19MAb9XvOeU5fZaRkaEWLVpo586dioyM1F//+ldVqVJFLi4uOn78uD7//HOlp6fne/m3MnPmTNWqVUsLFy5UdHS0oqOjVa5cOXXu3FlTp041klyO6msAAMo6buUDAACSblzJIklr167N9zw5B93nzp3Lc3rOLWd5HZznlXi607Sc5fzxxx+yWCy3/Jdzm5+Xl5ekgidw8qtSpUoym81Wt5/lOH/+vCwWy20H3i6sBx98UG5ubtq+fXueyZBbuV1/WSwWnTt3rljize1W60pOec5tcitXrtTOnTvVu3dv7dy5U5988omio6M1fvx4tW3b9pbLv916lRcXFxcNHz5c+/bt0+nTp7VkyRI99thjWrx4sbp3727Uc1RfAwBQ1pGYAgAAkqRevXrJ2dlZc+fOzfPgO7ecK1Xq16+vcuXK6b///a+uXr1qU2/9+vWS8r7dqzAefvhhSdKWLVvyVT/ndsGff/75jnVzxhgqyFUsOeNW5bQzt6Jue27ly5dXly5ddO3atTyfTJdbVlaWzGazpNvHu3XrVl2/fr1Y4s3tt99+M+LJYTabtXnzZplMJjVq1EiSjFs1c49flWPjxo3FEltAQIC6du2q1atXq06dOlqzZo2uXbsmyXF9DQBAWUdiCgAASJLq1KmjN998U4mJiWrXrp2OHTtmU+f69euaNm2axo8fL+nGoNFdu3ZVYmKiJk+ebFV39erV+umnn1SnTh3jaqy7FRUVpYoVK2rUqFHat2+fzfSrV68a41BJUs+ePVWhQgVNnTo1z/F/cl9JlTPQ9cmTJ/MdT8+ePSVJEyZMsLpyKTk5WRMmTLCqU9QmTpwoPz8/TZw4UR9++KFNskeS9uzZoxYtWhixdevWTS4uLpo2bZrV+EwZGRkaMWKEpBsJyuJ08OBBzZs3z6ps3rx5OnjwoJ555hnjVrmcq942bdpkVffXX3+1mb+w0tPTtXnzZpvytLQ0paamytXV1RgPy5F9DQBAWcYYUwAAwBAdHa3r169r+vTpqlevnp588kk1aNBArq6uOnbsmNasWaOLFy8qOjramOf999/Xr7/+qujoaG3evFkPP/ywjh8/rq+++krly5fXwoULbQa7Liw/Pz8tXbpUL774oho1aqS2bduqfv36Sk9P1/Hjx/Xrr7/qkUce0erVqyXdGKtq8eLF6tKlix566CF16NBB9erVU2JiorZu3aqQkBCtWLFCkowBr2fMmKGkpCQjQXLz0wZze/zxxzVw4EB99NFHatCggZ5//nlZLBb9+9//1qlTpzRo0CA9/vjjRdL2mwUFBennn39Wx44dNXjwYE2fPl1//vOfVa1aNaWkpGjbtm3673//q0qVKsnV1VXSjafrvf/++xo2bJgaNmyozp07y9PTU999953i4uL07LPPqkePHsUSb442bdpo0KBB+vHHH3X//fdr3759+u677+Tr66uZM2ca9dq3b6+QkBBNmTJFe/fuVYMGDRQXF6fvv/9ezz33nL7++uu7juXatWtq3ry5wsLC1KRJE9WoUUOpqan6/vvvlZCQoOHDh8vd3V2SY/saAICyjMQUAAAwODk5adq0aerWrZs++eQTbdiwQRs2bJDZbJa/v7/atGmjqKgo46lp0o1k0datW/Xuu+9q5cqV2rhxoypXrqyOHTtq3LhxatCgQZHG+Mwzz2jXrl364IMPtGbNGv3nP/+Rp6engoKCFBUVZZNYee6557R161ZNnjxZv/76q1atWiVfX19FRESob9++Rj0fHx99/fXXGj9+vObNm2fcwnW7xJQkffjhh4qMjNQnn3yiuXPnSpLuv/9+vfPOO4qKiirStt8sIiJC+/fv17x58/Ttt99q5cqVunz5sipUqKDw8HBFR0erX79+8vT0NOYZOnSo6tSpo2nTpulf//qXMjIyFBYWpqlTp2rQoEEFHqOpoP70pz9p9OjRGj16tD788EM5OzurY8eOmjJlimrXrm3Uq1Chgn755Rf9/e9/14YNG7R+/Xrdf//9+uKLL1StWrUiSUx5enrq/fff19q1a7Vx40adP39e3t7eqlevniZPnqwuXbpY1XdkXwMAUFaZLBaLxdFBAAAAoGxbv369WrZsqXHjxhm3ggIAADDGFAAAAAAAAByCxBQAAAAAAAAcgsQUAAAAAAAAHIIxpgAAAAAAAOAQXDEFAAAAAAAAhyAxBQAAAAAAAIcgMQUAAAAAAACHIDEFAAAAAAAAhyAxBQAAAAAAAIcgMQUAAAAAAACHIDEFAAAAAAAAhyAxBQAAAAAAAIcgMQUAAAAAAACHIDEFAAAAAAAAhyAxBQAAAAAAAIcgMQUAAAAAAACHIDEFAAAAAAAAhyAxBQAAAAAAAIcgMQUAAAAAAACHcHF0AKWJ2WzWmTNnVLFiRZlMJkeHAwAAAAAAUOJYLBZduXJFAQEBcnK6/TVRJKYK4MyZMwoODnZ0GAAAAAAAACXeyZMnFRQUdNs6JKYKoGLFipJufLCVKlVycDQAAAAAAAAlT0pKioKDg408yu2QmCqAnNv3KlWqRGIKAAAAAADgNvIzDBKDnwMAAAAAAMAhSEwBAAAAAADAIUhMAQAAAAAAwCEYY6oYZGdnKyMjw9Fh4B7m5uYmZ2dnR4cBAAAAAMBtkZgqQhaLRadPn9alS5ccHQogHx8fBQYG5muwOQAAAAAAHIHEVBHKSUpVr15dnp6ecnLiTknYn9lsVlpamhISEiRJQUFBDo4IAAAAAIC8ldrE1IYNG/TBBx9ox44dOnv2rL799lt17NjxlvW/+eYbffLJJ4qJiVF6erruv/9+jR8/Xm3atCmSeLKzs42kVNWqVYtkmUBheXp6SpISEhLk7+/PbX0AAAAAgBKp1F7Sk5aWpkaNGmn27Nn5qr9hwwa1bt1aP/74o3bs2KGWLVuqffv22rVrV5HEkzOmVE5CAHC0nHWR8c4AAAAAACVVqb1iql27dmrXrl2+68+YMcPq9aRJk7Ry5Up99913ioyMLLK4uH0PJQXrIgAAAACgpCu1iam7ZTabdeXKFfn4+NyyTnp6utLT043XKSkpkqSsrCxlZWVJunHw7+TkJLPZLIvFYvyTJJPJZPydW0HLC6Ko3rMg5cePH1ft2rW1c+dORURE5Gs5ixYt0pAhQ5SUlHSnJuU7lpw4du3apUaNGt1VmwpTXhD26ieLxWKsrznranZ2ttU8zs7OMplMxjqdu1y6cZtqfspdXFxksVisyk0mk5ydnWU2m2U2m+9Ynnt7yqv85thpE22iTbSJNtEm2kSbaBNtok20iTaVvDbdvKzbuWcTU//4xz+Umpqqzp0737LO5MmTNWHCBJvyXbt2GbdJ+fn5KTQ0VKdPn5bZbFZWVpYyMzPl7OwsZ2dnZWVl6cSJE7p48aKkGytCTvnNK4KTk5MyMzOt3utWK0heK7Ekubq6ysfHRwEBAUaZyWSSq6urzGazzUqfu/zkyZN699139fPPPysxMVH+/v5q3769Ro0apSpVqli1KXfsgYGBOnv2rCpXrmwVv4uLi0wmk02bXFxc9NJLL6l169ZW01xdXY1ESm5ubm425Xm1Kfeybm6rk5OTXFxclJ2dbbXB3qpNtyq/XZvyKr/bNt2uPD9tMpvN2rdvnySpdu3aqlq1qvbu3atr164Z9evXry8vLy/t2rXLavkNGzaUm5ubtm/fbhV706ZNlZGRoT179li954MPPqjk5GQdOHDAKPfw8FCjRo2UmJioo0ePGuWVK1dWeHi4zpw5o1OnThnlOdvTsWPHdOHCBaM8KChIQUFBOnjwoJKTk41y2kSbaBNtok20iTbRJtpEm2gTbaJNJa9NaWlpyi+T5W4v/SgBTCbTHQc/z23JkiXq27evVq5cqVatWt2yXl5XTAUHB+vixYuqVKmSpP/LOKalpenw4cOqW7euPDw8jLhOnDih+uH36fq1q4VvYAGV8yivA7H7VaNGDaPsTlfWHD16VI888ojCwsL07rvvqnbt2tq7d6/efPNNZWRkaMuWLfLx8bFZTkZGhtzc3ErE1UVcMWVdfvXqVR06dEi1a9eWh4cHGX/aRJtoE22iTbSJNtEm2kSbaBNtok12aVNKSoqqVKmi5ORkI39yK/dcYmrZsmV65ZVX9NVXX+mZZ54p0PukpKSocuXKeX6w165d06FDh6wSU5K0c+dONWnSRA88P0yefsEFer/CSLtwUn/8e6p27Nihxo0b53u+du3aae/evTp48KBV/AkJCQoNDdXLL7+sTz75RCEhIerdu7cOHTqkFStWqFOnTho/frxq1aqlXbt2GbfyrVq1SsOGDdPJkyfVrFkz9erVS7169VJSUpK8vLy0aNEivfHGG7p8+bIkafz48VqxYoWGDRumMWPGKCkpSe3atdO8efNUsWJFSdLq1asVHR2tvXv3ytnZWc2aNdPMmTMVGhoq6UZi6uY47mW3WicBAAAAAChOt8uf3OyeupVv6dKleuWVV7Rs2bICJ6XulqdfsCoF1LHre+bXpUuX9NNPP2nixIk2CYzq1aure/fuWr58uT7++GNJN26DHDt2rMaNG5fn8o4dO6YXXnhBgwcPVp8+fbRr1y4NHz78jnEcOXJEK1as0Pfff6+kpCR17txZ7733niZOnCjpxpMYhw4dqoYNGyo1NVVjx47Vc889p5iYGDk5MdA3AAAAAAClTalNTKWmpurw4cPG62PHjikmJkY+Pj6qUaOGRo4cqdOnT2vx4sWSbty+17NnT82cOVMPP/ywEhISJN24h7Ny5coOaUNJcejQIVksFoWHh+c5PTw8XElJScY9q08++aSGDRtmTD9+/LhV/U8//VT16tXTBx98IEmqV6+e9u7daySYbsVsNmvRokXGFVJ//etftXbtWmO+559/3qr+ggUL5Ofnp/3796tBgwb5bzAAAAAAACgRSu1lJtu3b1dkZKQiIyMlSUOHDlVkZKTGjh0rSTp79qzi4+ON+nPnzlVWVpZef/11+fv7G/8GDx7skPhLovze1dm0adPbTo+Li9ODDz5oVfbQQw/dcbkhISFGUkqS/P39df78eeP1oUOH1LVrV9WuXVuVKlVSSEiIJFn1MwAAAAAAKD1K7RVTLVq0uG0iZdGiRVav169fX7wBlWJ16tSRyWRSbGysnnvuOZvpsbGx8vb2lp+fnyQZTyQsaq6urlavTSaT1QBu7du3V82aNTVv3jwFBATIbDarQYMGysjIKJZ4AAAAAABA8Sq1V0yh6FSpUkWtW7fWxx9/bPXISOnG4OdffPGFXnrpJZlMpnwtr169ejaPk/zvf/97VzFevHhRcXFxGj16tP785z8btxcCAAAAAIDSi8QUJEmzZs1Senq62rRpow0bNujkyZNavXq1WrdurcDAwDuOD5Vbv379dODAAY0YMUIHDx7Ul19+aVzBlt/k1s28vb1VpUoVzZ07V4cPH9Yvv/yioUOHFmpZAAAAAACgZCi1t/KVNmkXTpbo96lbt662b9+ucePGqXPnzrp06ZKqV6+ujh07aty4cfLx8cn3smrVqqWvv/5aw4YN08yZM9WsWTONGjVKf/vb3+Tu7l6o+JycnLRs2TINGjRIDRo0UL169fThhx+qRYsWhVoeAAAAAABwPJMlvyNeQykpKapcubKSk5NVqVIlq2nXrl3ToUOHVLduXXl4eBjl8fHxqlc/XNevXbVbnOU8yivuQKxq1Khht/e8k4kTJ2rOnDk6edI+CTrcep0EAAAAAKA43S5/cjOumCpmNWrUUNyBWCUmJtrtPX19fR2elPr444/14IMPqkqVKvrtt9/0wQcfaMCAAQ6NCQAAAAAAlCwkpuygRo0aDk8U2duhQ4cUHR2tS5cuqUaNGho2bJhGjhzp6LAAAAAAAEAJQmIKxWL69OmaPn26o8MAAAAAAAAlGE/lAwAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYQolmsVj06quvysfHRyaTSTExMWrRooXeeOMNR4cGAAAAAADukoujA7gXxMfHKzEx0W7v5+vrqxo1auS7fq9evXT58mWtWLGiUO/Xq1cvff7555IkFxcXBQUF6cUXX9Q777yjcuXKWdU9deqUateurbCwMO3du/eOy169erUWLVqk9evXq3bt2vL19dU333wjV1fXQsV6uzbczWdgDxMnTtQPP/ygmJgYubm56fLly44OCQAAAACAu0JiqpjFx8crvH59Xb12zW7vWd7DQ7EHDhQoOXW32rZtq4ULFyozM1M7duxQz549ZTKZ9P7771vVW7RokTp37qwNGzZo69atevjhh2+73CNHjsjf31+PPPKIUebj41MsbSjpMjIy9OKLL6pZs2aaP3++o8MBAAAAAOCukZgqZomJibp67ZrmduqkMF/fYn+/g4mJevWbb5SYmFhkialff/1Vf//737V79275+PioZ8+eio6OlovL/60+7u7uql69uiQpODhYrVq10n/+8x+rxJTFYtHChQv18ccfKygoSPPnz79tYir3lVgmk0k1a9bU8ePH1aJFC0VERGjGjBmSpJCQEL366qs6fPiwvvrqK3l7e2v06NF69dVXjWWdPHlSw4YN088//ywnJyc99thjmjlzpkJCQjR+/Hir95GkdevWSZJatmyppKQkeXl5SZJiYmIUGRmpY8eOKSQkRIsWLdIbb7yh5cuX64033tDJkyf16KOPauHChfL39zfe/7PPPtPUqVON+QYNGqT+/fsXqB8mTJgg6UZyDwAAAACAsoDElJ2E+foqIiDA0WEU2OnTp/X000+rV69eWrx4sQ4cOKC+ffuqXLlyGj9+fJ7z7N27V5s3b1bNmjWtytetW6erV6+qVatWCgwM1COPPKLp06fL09Mzz+XMnDlToaGhmjt3rv773//K2dn5lnFOnTpV7777rt5++219/fXX+tvf/qYnnnhC9erVU2Zmptq0aaNmzZpp48aNcnFxUXR0tNq2bas9e/Zo+PDhio2NVUpKihYuXCjpxlVZmzdvztdndPXqVf3jH//QP//5Tzk5OalHjx4aPny4vvjiC0nSF198obFjx2rWrFmKjIzUrl271LdvX3l6eqpnz56SpBYtWhiJLgAAAAAA7hUkpnBbH3/8sYKDgzVr1iyZTCbVr19fZ86c0YgRIzR27Fg5Od0YP//7779XhQoVlJWVpfT0dDk5OWnWrFlWy5o/f766dOkiZ2dnNWjQQLVr19ZXX32lXr165fnelStXVsWKFeXs7GxcjXUrTz/9tHEF0ogRIzR9+nStW7dO9erV0/Lly2U2m/XZZ58ZV0QtXLhQXl5eWr9+vZ566il5eHgoPT39ju+Tl8zMTM2ZM0ehoaGSpAEDBuidd94xpo8bN05Tp05Vp06dJEm1atXS/v379emnnxqJqRo1alhdYQUAAAAAwL2AxBRuKzY2Vs2aNTMSOpLUvHlzpaam6tSpU8btgi1bttQnn3yitLQ0TZ8+XS4uLnr++eeNeS5fvqxvvvlGmzZtMsp69Oih+fPn3zIxVRANGzY0/jaZTKpevbrOnz8vSdq9e7cOHz6sihUrWs1z/fp1HTly5K7fu3z58kZSSpL8/f2N905LS9ORI0fUu3dv9e3b16iTlZWlypUrG68XL15813EAAAAAAFDakJhCkfD09FSdOnUkSQsWLFCjRo00f/589e7dW5K0ZMkSXb9+3WpMKYvFIrPZrIMHDyosLOyu3v/mp/SZTCaZzWZJUmpqqpo0aWLcWpebn5/fLZeZczWYxWIxyjIzM/P13jnzpKamSpLmzZtnM57W7W5NBAAAAADgXuDk6ABQsoWHh2vLli1WyZnffvtNFStWVFBQUJ7zODk56e2339bo0aN17X9PI5w/f76GDRummJgY49/u3bv12GOPacGCBcXahsaNG+vQoUOqWrWq6tSpY/Uv56olNzc3ZWdnW82Xk7Q6e/asURYTE1Og965WrZoCAgJ09OhRm/euVavW3TUMAAAAAIBSjsQUJEnJyclWSaOYmBidPHlS/fv318mTJzVw4EAdOHBAK1eu1Lhx4zR06FDjiqK8vPjii3J2dtbs2bMVExOjnTt3qk+fPmrQoIHVv65du+rzzz9XVlZWsbWte/fu8vX11bPPPquNGzfq2LFjWr9+vQYNGqRTp05JuvFkvz179iguLk6JiYnKzMxUnTp1FBwcrPHjx+vQoUP64YcfNHXq1AK//4QJEzR58mR9+OGHOnjwoP744w8tXLhQ06ZNM+q8/PLLGjly5G2XEx8fr5iYGMXHxys7O9vop5yrsgAAAAAAKG24lc9ODiYmluj3Wb9+vSIjI63Kevfurc8++0w//vij/v73v6tRo0by8fFR7969NXr06Nsuz8XFRQMGDNCUKVMUFxen++67T/Xr17ep99xzz2nAgAH68ccf1aFDh0LFfifly5fXhg0bNGLECHXq1ElXrlxRYGCg/vznP6tSpUqSpL59+2r9+vVq2rSpUlNTtW7dOrVo0UJLly7V3/72NzVs2FAPPvigoqOj9eKLLxbo/fv06aPy5cvrgw8+0N///nd5enrqgQce0BtvvGHUiY+Pv22iT5LGjh2rzz//3Hid0185sQIAAAAAUNqYLLnv0cJtpaSkqHLlykpOTjYSGjmuXbumQ4cOqW7duvLw8DDK4+PjFV6/vq7+75Y2eyjv4aHYAweMgclxb7rVOgkAAAAAQHG6Xf7kZlwxVcxq1Kih2AMHlGinK6YkydfXl6QUAAAAAAAo8UhM2UGNGjVIFAEAAAAAANyEwc8BAAAAAADgECSmAAAAAAAA4BAkpgAAAAAAAOAQJKaKmNlsdnQIgCTWRQAAAABAyUdiqoi4ublJktLS0hwcCXBDzrqYs24CAAAAAFDS8FS+IuLs7CwfHx8lJCRIkjw9PeXkRN4P9mc2m5WWlqaEhAT5+PjI2dnZ0SEBAAAAAJAnElNFKDAwUJKM5BTgSD4+PsY6CQAAAABASURiqgiZTCYFBQXJ399fGRkZjg4H9zA3NzeulAIAAAAAlHgkpoqBs7OzPDw8HB0GAAAAAABAicYgSAAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTAAAAAAAAcAgSUwAAAAAAAHAIElMAAAAAAABwCBJTsJGZmakBAwbI29tbPj4+GjhwoLKysvKse+TIEbVr107e3t4KDAzUlClTrKbv2LFDjz76qCpVqqTatWtr8eLF9mgCAAAAAAAoBUhMwUZ0dLQ2bdqk/fv3a9++fdq4caMmTZpkUy87O1sdOnRQ48aNdf78ef3yyy+aNWuWlixZIkm6fPmynn76afXo0UNJSUlaunSpBg4cqE2bNtm7SQAAAAAAoAQyWSwWi6ODKC1SUlJUuXJlJScnq1KlSo4Op9gEBwdr+vTpeuGFFyRJX331lYYPH64TJ05Y1du/f78aNmyoq1evys3NTZI0YcIErVu3TuvXr9ePP/6o1157TfHx8cY8UVFRslgsWrRokd3aAwAAAAAA7Kcg+ROumIKVpKQknTp1ShEREUZZRESE4uPjlZycbFXXbDZLknLnNs1ms/bs2WP8fXPeM/d0AAAAAABwbyMxBSupqamSJC8vL6Ms5+8rV65Y1a1Xr55CQkI0duxYpaena9++fVqwYIFSUlIkSc2aNVNaWppmzZqlzMxM/fbbb/r222+N6QAAAAAA4N5GYgpWKlSoIElWV0fl/F2xYkWruq6urlq5cqV27dqlwMBAde/eXVFRUapSpYokqUqVKvruu++0ZMkSVa9eXW+99ZbVdAAAAAAAcG8jMQUr3t7eCgoKUkxMjFEWExOj4OBgVa5c2ab+/fffr59//lmJiYmKiYlRenq6nnjiCWN68+bNtXnzZl28eFEbN25UQkKC1XSUbkX1BMf4+HhVqFDB6p+Li4s6dOhgr6YAAAAAAByg1CamNmzYoPbt2ysgIEAmk0krVqy44zzr169X48aN5e7urjp16jAA9y1ERUVp4sSJSkhIUEJCgiZNmqQ+ffrkWXfPnj1KS0tTRkaGvvnmGy1YsECjR482pu/atUvp6em6du2a5s2bp/Xr1+uNN96wU0tQ3IrqCY41atRQamqq8e/SpUvy8vJSly5d7N0kAAAAAIAdldrEVFpamho1aqTZs2fnq/6xY8f0zDPPqGXLloqJidEbb7yhPn366KeffirmSEufMWPGqFmzZgoPD1d4eLiaN2+ut99+W5L02muv6bXXXjPqfvnll6pRo4a8vb31j3/8QytWrFDDhg2N6R9++KGqVasmPz8/ffXVV/rll18UEBBg9zaheOQkIv39/eXv769Ro0Zp/vz5NvXi4uIUFxencePGydXVVfXq1VPv3r01d+7cPJe7YsUKmc1mderUqbibAABAqcRVywCAssJkufmxaaWQyWTSt99+q44dO96yzogRI/TDDz9o7969RlmXLl10+fJlrV69Ol/vU5DHHQJlXVJSknx8fHTo0CHVqVNHknTo0CGFhYXp8uXLVrd+7t27VxEREUpLS5O7u7skady4cfroo4906dIlm2W3adNGYWFh+uijj+zTGJR5mZmZGjJkiL744guZTCZ1795d06dPl4uLi03dI0eOaMCAAfr9999Vvnx5DR48WG+++aakGwdw9913n1X969ev6+mnn9aqVavs0hYAkG78jq5cuVL/7//9P0lSu3bt1KlTJ40dO9aqXnZ2tho2bKiOHTtq/PjxOnr0qFq3bq333ntP3bp1s1luRkaGAgIC9OGHH+Y5HQCA/ChI/sR2j7yM2rJli1q1amVV1qZNm9veVpaenq709HTjdc7T5LKysowzUk5OTnJycpLZbJbZbDbq5pRnZ2crd+7vVuXOzs4ymUw2Z7qcnZ0l3dipyE+5i4uLLBaLVbnJZJKzs7NNjLcqL2ltslgseuONN7RkyRKZTCZ17dpV06ZNk7u7u02MR48e1eDBg40DyoEDB2r48OFWbZo3b56mTZumU6dOyc/PTzNnztRf/vIX+qmAbbp8+bKkGwPmZ2VlycXFxUhGJSUlydPT04i9bt26CgkJ0ejRozV+/HgdOXLEeIJj7vd1cnLSyZMntWbNGk2aNMlmO2N7ok2FbdM777yjjRs36o8//pCzs7PatWun6Oho49bjnBgzMzPVoUMHdejQQd98842OHz+uNm3aKCAgQF26dFFAQIAuX75s1L927ZqCg4P14osvKisri36iTbSJNtmtTQsWLNA//vEP+fn5SZJGjhypN998U6NHj7aKMTY2VnFxcRozZoxMJpNCQ0MVFRWluXPnqlu3bjZt+uabb2Q2m/Xss8/a/EbTT7SJNtEm2kSb8tumW13Fm5d7JjGVkJCgatWqWZVVq1ZNKSkpunbtmjw8PGzmmTx5siZMmGBTvmvXLnl6ekqS/Pz8FBoaqmPHjunChQtGnaCgIAUFBengwYNWT7irXbu2qlatqr179+ratWtGef369eXl5aVdu3ZZdW7Dhg3l5uam7du3W8XQtGlTZWRkaM+ePUaZs7OzHnzwQSUnJ+vAgQNGuYeHhxo1aqTExEQdPXrUKK9cubLCw8N15swZnTp1yigvaW36/vvvtWHDBv3zn/+UJA0dOlRDhgzRxx9/bNWm7Oxs9ezZUy+99JI+/vhjbdu2TYMHD1Z6erp69Oih0NBQTZ48WfPmzdOECRMUFhamcuXKqXLlyvRTIdp06NAhSdKmTZtUs2ZNPfjgg0Z8hw8fVkJCgtGm5ORkvfPOO5o5c6YCAwNVvXp1RUVF6ZNPPrFavp+fn/75z3/qvvvuU2ZmpjGN7Yk23W2b5s6dq8GDB+vKlSsKDw/Xa6+9pgkTJqht27ZWbVq7dq3i4uL09NNPa/fu3QoKClLv3r314YcfGlcG5m7TRx99pIyMDNWoUUPbt2+nn2gTbaJNdmlTnTp1dOrUKTk5ORnzeHh4KD4+XkePHlViYqJR//z585Kk06dPG7GfOnXKeNDNzW365JNP1L17d8XHx9NPtIk20SbaRJsK3aa0tDTl1z1zK19YWJiioqI0cuRIo+zHH3/UM888o6tXr+aZmMrriqng4GBdvHjRuBStJGcoc38+pTnrWqtWLU2bNk3PPfecJOnrr7/WiBEjdOLECasY9+/fr8aNG+vq1atycXGR2WzWu+++q/Xr12vdunWyWCwKDAzUwoUL1bp1a4e2qaz0U61atTR16lR16tRJLi4u+uqrrzRs2DDjS/R2bXr77bd19OhRLV261Gr5oaGhGjFihF599VWHtCm3stJP93qbLl68qKpVqyo2NlZ169aVs7Oz4uLiVL9+fSUmJqpy5cpGjHv27FHjxo2VnJwsd3d3OTk5acKECfroo4+Mg7vcbXrqqadUt25dzZw5065tKov9RJtoE23Kf5vOnj2rGjVq6OzZs/L19ZUkJSYmyt/fXydOnLAazzMrK0sNGjRQp06dNG7cOB0+fFh/+ctfdPbsWWVlZVm16cSJEwoLC9POnTvVoEED+ok20SbaRJtoU6HblJKSoipVqnArX27Vq1fXuXPnrMrOnTunSpUq5ZmUkiR3d3djPJzcXFxcbMYlyengm+V0WH7L8xrvpKDlJpMpz/JbxVjQcnu2KSkpSadOnVJkZKRR3qRJE+MsXs4BZU680o1b/3LH/scff8jJyUn79+/XuXPntHv3bv3tb39TVlaW2rVrp6lTp95yQ6Gfbl8eFRWl9957T48//rikG1cZ9unTJ8/tY+/evQoNDZWrq6u+//57LViwQGvXrrWq+9NPPykxMVHdu3fP833ZnmjTrcpv16br169Lknx9fY0YfHx8JEnXrl1TlSpVjPrh4eEKCQnRO++8o3feeUdxcXHGbac3L//EiRNau3atPvjgA5tp9BNtok206Xbld9umChUqSLrxMKDq1atLklJTUyXdOKueu76Li4tWrlypIUOGqGbNmgoKClJUVJQ+/fRTmzb985//VGRkpBo1apRnHMXZJqns9ZNEmwpTTpto063KaVPpatOt5slLqX0qX0E1a9ZMa9eutSr7z3/+o2bNmjkoIuRHzk6Wl5eXUZbz95UrV6zq1qtXTyEhIRo7dqzS09O1b98+44BSkjHI9po1a7R9+3bFxMTo2LFjGjJkSPE3pIwqyic4StL8+fP1wgsvWA2cDtytnAO43Jcx5/xdsWJFq7qurq5auXKldu3apcDAQHXv3l1RUVFWyascCxcuvOMBHO6sqJ4sJkktWrSQu7u71dPFzpw5Y49mAHbl7e2toKAg43Y8SYqJiVFwcHCev6H333+/fv75ZyUmJiomJkbp6el64oknrOqYzWYtXLhQffr0Ke7wAQCwUmqvmEpNTdXhw4eN18eOHVNMTIx8fHxUo0YNjRw5UqdPn9bixYsl3ThInjVrlt5880298sor+uWXX/Tll1/qhx9+cFQTHC4+Pt5qDAJH8/X1VY0aNazKch9Q5lyqfqcDyiFDhigwMNDmjGDOskaOHGksa+TIkeratWvxNaqMc3V11ezZszV79mybaXPmzLF6HR0drejo6Nsu78svvyzS+ADJ+gAuNDRUUv4O4HKMGDHilgdwuW8PR+FER0dr06ZN2r9/v6QbTxabNGlSnk8W69Chgzp27KhVq1YZTxYLCgqyenLY+++/f9sHmwBlRVRUlCZOnKjmzZtLkiZNmnTLpNKePXvyvGo5t//85z9KTExkvwhlXlE9qVe6cUJky5YtcnV1NcoOHjxodTstgHywlFLr1q2zSLL517NnT4vFYrH07NnT8sQTT9jMExERYXFzc7PUrl3bsnDhwgK9Z3JyskWSJTk5uWga4UAnTpywlPMon+dn6Kh/5TzKW06cOGETa1BQkOXrr782Xn/11VeW4ODgfLXzzTfftLz44osWi8ViuXr1qqVcuXKWNWvWGNPXrFlj8fPzu8tPE0BJN2bMGEtkZKTl7NmzlrNnz1oiIyMtEyZMyLPu7t27LampqZb09HTLv//9b4uvr69l9+7dVnVWr15tKV++vOXy5cv2CL9MCwoKsnz11VfG6y+//NJSo0YNm3r79u2zODs7W9LT042y8ePHW/3WP/HEE5bp06cXZ7hAiZGRkWHp37+/xcvLy+Ll5WUZMGCAJTMz02KxWCz9+vWz9OvXz6g7atQoi4+Pj6V8+fKWZs2aWTZt2mSzvBdffNHy8ssv2y1+wFHGjh1radSokeXMmTOWM2fOWBo1apTnPkFWVpblvvvus7z99tuWjIwMy4EDByzBwcGWL774wqjD7w5wawXJn5SJwc/tJSUlRZUrV87X4F0l3c6dO9WkSRM98PwwefoFOzocpV04qT/+PVU7duxQ48aNraaNHTtW33//vX788UdJ0tNPP62OHTvanE2XbM8I9uvXT2vXrjVuGevbt69OnDih5cuXy2QyqXPnzqpZs6bmzZtX/I0E4DCZmZl64403tGTJEklSjx49jLOjObec5lzlN3r0aH3yySe6fv26GjVqpA8++MC4IiFH586d5eHhoc8//9y+DSljkpKS5OPjo0OHDhlPPTx06JDCwsJ0+fJlqyva9u7dq4iICKWlpRnjP44bN04fffSRcat2ixYttHfvXpnNZtWsWVNDhgzRyy+/bP+GAQBKrODgYE2fPl0vvPCCJOmrr77S8OHDdeLECat6+/fvV8OGDXX16lW5ublJkiZMmKB169Zp/fr1km787nTs2JErdYE8FCR/Umpv5UPR8PQLVqWAOneu6EBjxozRxYsXFR4eLunGAWXucYyk/zug/PLLL60OKG8ex2jGjBl6/fXXVatWLbm7u6tDhw6aNm2anVsEwN647bRkutM4grkTU7nHEXznnXd0+PBhq3EEpRsPYLjvvvtUvnx5/fLLL+rcubMqVqxoPNUVAHBvy3mwUkREhFEWERFh9WClHDlPOMt9HYfZbNaePXuslhkdHa133nmHEyLAXeCKqQIoi1dM/em1GSUiMZVy5rB+n/NGnldMAQDKppwrpg4fPmyM/3X48GHVrVvX5oopSdq3b5+GDBminTt3KigoSB06dNCnn35q89TdHG+++abi4+O1bNmyYm8LAKDkO3nypGrUqKELFy4YY85euHBBVatW1cmTJxUUFGTUzczMVHh4uJ5//nnjhEjbtm119uxZ4yEdW7ZssTkhsmjRIk6IACpY/uSeeSofAAAoWYrjyWK55fXYZADAvauon9TbrFkzVa5cWa6urmrTpo369eun5cuX26ElQNnCrXwA7qg0PMERQOlUVE8Wu3z5sjZv3qwWLVrI3d1d69ev15w5cxhDEABgKI4n9ebGCRGgcEhMAbit+Ph4hdevr6vXrjk6FEN5Dw/FHjhAcgooA4pqHMHMzExNmDBBXbp0kSSFhIRo2rRpevHFF+3dpDKlKB+r/sILL+i3335TWlqaqlSpot69e2v06NH2bA4AcEIEKIFITAG4rcTERF29dk1zO3VS2P/uxXekg4mJevWbb5SYmEhiCigDimpgej8/P23durVYYryXRUdHa9OmTdq/f78kqV27dpo0aZLNk3Gzs7PVoUMHdezYUatWrdLRo0fVunVrBQUFqVu3bpJuPEUxLCxM7u7uio+PV9u2bRUSEqIePXrYvV2lCVctl04kdUsuTogAJQ+JKQD5Eubrq4iAAEeHAdwWB3BA0VqwYIGmT58uf39/SdKoUaM0fPhwm8RUXFyc4uLiNG7cOLm6uqpevXrq3bu35s6daySmHnjgAaO+yWSSk5OTDh06ZL/GlELx8fGqVz9c169ddXQohnIe5RV3IJbvtjsgqVtycUKkZCOpe28iMYUSJTY21tEhGDigBEoXDuCAolUcj1Xv37+/Fi1apGvXrqlmzZrq1atXsbahtEtMTNT1a1f1wPPD5OkX7OhwlHbhpP7491SuWs4HkrpA4ZDUvTeRmEKJkH4lSU4mU4n6kmAcIxRUUZ3hOX/+vIYMGaJff/1VKSkpCg0N1YQJE9ShQwd7N6lU4QAOKFqpqamSJC8vL6Ms5+8rV65YJabq1aunkJAQjR071nis+oIFC5SSkmK1zI8//lizZs3Szp07tWrVKnl7exd7O8oCT79gVQqo4+gwkE8kdYHCI6l7byIxhRIh63qqzBYL4xihVCuqMzypqamKjIzU+++/r4CAAP3www/q0qWL/vvf/+q+++5zRNNKFQ7ggKKR+7Hqvv/7bb7TY9WHDBmiwMBABQUFKSoqSp9++qnNcp2cnNS0aVOtW7dOw4cP12effVbMLQHsi6QuUDgkde9dJKZQojCOEUqzojrDU7t2bQ0fPtyo3759e9WrV0+///47iSkAdlPcj1XPzMzkzDXKJJK6QOGQ1L13kZgCgCJQHGd4cpw/f16xsbHGU2CA0oiB6Uunonqs+okTJ7R9+3a1adNG5cuX1++//64PP/xQgwYNsltbAHshqQsUDkndexeJKQAoAsVxhkeSMjIy1KVLF3Xu3FlNmzYt1jYAxYWB6UuvonqsuiTNmDFDvXv3ltlsVkBAgAYOHKi33nrLzi0C7IOkruNxQqT0Ial77yIxBQBFoDjO8GRkZOiFF15Q+fLlNW/ePDu0AigeDExfehXVY9Vr1qypjRs3FkuMQElEUtexOCFSepHUvTeRmAKAIlDUZ3gyMjL04osvKiMjQytXrpSbm1vxNwIoZgxMD+BeQVLXsTghUnqR1L03kZgCgCJSVGd4MjMz1blzZ6Wlpen777+Xu7u73doAAABQVnBCpPQhqXtvIjEFAEWkqM7wbN68WStXrlS5cuWM2wIl6e233zaWBwAAAABlgZOjAwCAsiLnDE9SUpKSkpL00UcfycXlRv5/zpw5Vmd5oqOjdfHiRaWlpWnz5s3GVVaS9MQTT8hisejatWtKTU01/pGUAuwjMzNTAwYMkLe3t3x8fDRw4EBlZWXlWffIkSNq166dvL29FRgYqClTplhNT0lJUbdu3VSpUiVVq1ZN7777rj2aAAAAUGqQmAIAAMglOjpamzZt0v79+7Vv3z5t3LhRkyZNsqmXnZ2tDh06qHHjxjp//rx++eUXzZo1S0uWLDHqDBw4UJcuXVJ8fLw2btyoefPmafHixfZsDgAAQInGrXwAAAC5LFiwQNOnT5e/v78kadSoURo+fLjGjh1rVS8uLk5xcXEaN26cXF1dVa9ePfXu3Vtz585Vt27ddPXqVS1btky//fabvLy85OXlpYEDB2r+/Pl6+eWXHdG0W+Kx6gAAwFFITAEAAPxPUlKSTp06pYiICKMsIiJC8fHxSk5OtnrKptlsliRZLBarsj179ki6kbjKyMiwWVZeV185Eo9VBwqPpC4A3D0SUwAAAP+TmpoqSfLy8jLKcv6+cuWKVWKqXr16CgkJ0dixY/XOO+/o8OHDWrBggVJSUoxleXp6GmPN5SzrypUrxd+QAuCx6kDhkNQFCqekJXQlkrqORmIKAADgfypUqCBJSk5ONp6KmZycLEmqWLGiVV1XV1etXLlSQ4YMUWBgoIKCghQVFaVPP/3UWNbVq1eVlZVlJKeSk5NtllNS8Fh1oGBI6gIFVxITuhJJXUcjMQUABcAZHqBs8/b2VlBQkGJiYhQaGipJiomJUXBwsNXVUjnuv/9+/fzzz8brESNG6IknnpB044oqV1dX7d69W02aNDGW9cADD9ihJQDshaQukH8lLaErkdQtCUhMAUA+cYYHuDdERUVp4sSJat68uSRp0qRJ6tOnT5519+zZo9DQULm6uur777/XggULtHbtWklS+fLl9dJLL2nMmDFaunSpzp8/r48++kjvvvuu3doCAEBJREIXuZGYAoB84gwPcG8YM2aMLl68qPDwcElSjx499Pbbb0uSXnvtNUnSnDlzJElffvmlPvnkE12/fl2NGjXSihUr1LBhQ2NZs2bNUr9+/RQUFCQPDw8NGDCgxD2RDwAAwJFITAFAAXGGByjbXF1dNXv2bM2ePdtmWk5CKkd0dLSio6NvuaxKlSpp6dKlRR4jAABAWeHk6AAAAAAAAABwbyIxBQAAAAAAAIcgMQUAAAAAAACHIDEFAAAAAAAAhyAxBQAAAAAAAIfgqXwAAOCeFBsb6+gQrPj6+qpGjRqODgMAAMCuSEwBAIB7SvqVJDmZTOrRo4ejQ7FS3sNDsQcOkJwCAAD3FBJTAADgnpJ1PVVmi0VzO3VSmK+vo8ORJB1MTNSr33yjxMREElMAAOCeQmIKAADck8J8fRUREODoMAAAAO5pDH4OAAAAAABwk8zMTA0YMEDe3t7y8fHRwIEDlZWVlWfd06dPq2PHjqpSpYp8fX3VuXNnXbhwwZjeq1cvubm5qUKFCsa/LVu22KspJRqJKQAAAAAAgJtER0dr06ZN2r9/v/bt26eNGzdq0qRJedZ9/fXXJUknTpzQsWPHdP36dQ0aNMiqTv/+/ZWammr8a9asWbG3oTQgMQUAAIBSoSjPXOe4du2a6tSpIy8vr2KOHgBQ2ixYsECjR4+Wv7+//P39NWrUKM2fPz/PukePHlXnzp1VoUIFVaxYUS+99JL++OMPO0dcOpGYAgAAQKlQ1GeuJWns2LGqWbNmscYNACh9kpKSdOrUKUVERBhlERERio+PV3Jysk39oUOH6quvvlJycrIuX76spUuXqn379lZ1Fi9eLB8fH91///2aOnWqzGZzcTejVCAxBQAAgFKhqM9c79ixQ6tXr9aIESPsET4AoBRJTU2VJKsranP+vnLlik395s2b6/z588ZVvUlJSRo5cqQxfdCgQYqLi9OFCxc0f/58zZw5UzNnzizWNpQWJKYAAABQ4hX1meusrCz17dtXs2fPlpubmz2aAAAoRSpUqCBJVr8xOX9XrFjRqq7ZbFbr1q3VvHlzY/yo5s2b66mnnjLqNG7cWH5+fnJ2dtaf/vQnvfXWW1q+fLkdWlLykZgCAABAiVfUZ64/+OADRUZG6vHHHy/WuAEApZO3t7eCgoIUExNjlMXExCg4OFiVK1e2qnvp0iWdOHFCgwYNUvny5VW+fHkNHDhQW7duVWJiYp7Ld3IiHZODTwIAAAAlXlGeuT58+LDmzJmjDz74wE7RAwBKo6ioKE2cOFEJCQlKSEjQpEmT1KdPH5t6vr6+qlOnjmbPnq3r16/r+vXrmj17toKCguTr6ytJ+vLLL5WSkiKLxaLt27frvffe0/PPP2/vJpVIJKYAAABQ4hXlmetNmzbp3LlzCgsLk6+vr5599lmlpKTI19dXW7dutXPLAAAl1ZgxY9SsWTOFh4crPDxczZs319tvvy1Jeu211/Taa68ZdVeuXKmdO3cqMDBQ/v7+2rZtm1atWmVMnzVrlmrUqKGKFSuqe/fu6t+/v4YNG2b3NpVEJKYAAABQKhTVmevOnTvr8OHDiomJUUxMjD777DNVrFhRMTExioyMdEDLSr/MzEwNGDDAuHVy4MCBysrKyrPu6dOn1bFjR1WpUsXojwsXLkiS0tPT1bdvX9WqVUsVK1ZU/fr1tWDBAns2BQAMrq6umj17tpKSkpSUlKSPPvpILi4ukqQ5c+Zozpw5Rt377rtPP/30ky5evKikpCT98ssvVr8pGzZs0OXLl5Wamqq4uDi9+eab3M73P3wKAAAAKBWK6sx1+fLlFRQUZPzz8/OTyWRSUFAQA6EXUnR0tDZt2qT9+/dr37592rhxoyZNmpRn3ddff12SdOLECR07dkzXr1/XoEGDJN0YlN7f319r1qxRSkqKFi1apGHDhunnn3+2W1sAAPbl4ugAAAAAgPzIOXM9e/Zsm2m5z1pL/3fmOj9atGihy5cvF0WI96wFCxZo+vTp8vf3lySNGjVKw4cP19ixY23qHj16VG+99ZYxbthLL72kyZMnS5I8PT31zjvvGHX/9Kc/qWXLltq0aZPV060AAGUHV0wBAAAAKLSkpCSdOnVKERERRllERITi4+OtBqvPMXToUH311VdKTk7W5cuXtXTpUrVv3z7PZV+/fl3btm1Tw4YNiyt8AICDkZgCAAAAUGipqamSJC8vL6Ms5+8rV67Y1G/evLnOnz9vjEeVlJSkkSNH2tSzWCzq06eP6tatq06dOhVL7AAAxyMxBQAAAKDQcm7Jy311VM7fFStWtKprNpvVunVrNW/eXKmpqUpNTVXz5s1tbtOzWCzq37+/4uLitGLFCgYIBoAyjDGmAAAAABSat7e3goKCFBMTo9DQUElSTEyMgoODVblyZau6ly5d0okTJzRo0CCVL19ekjRw4EB98MEHSkxMlK+vrywWi15//XVt3bpVa9eutVkGABSH2NhYR4dg8PX1VY0aNRwdht2QmAIAwM4yMzM1ZMgQffHFFzKZTOrevbumT59uPH44t9OnT+v111/Xxo0bZTKZ9OSTT2r27Nny8/OTJM2aNUuLFi3SH3/8oXbt2mnFihV2bg0ASFFRUZo4caKaN28uSZo0aZL69OljU8/X11d16tTR7NmzNW7cOEnS7NmzFRQUJF9fX0nSgAED9Ntvv+mXX36Rt7e3/RoB4J6UfiVJTiaTevTo4ehQDOU9PBR74MA9k5wiMQUAgJ3lfqy6JLVr106TJk3K8+lVuR+rbrFY1L17dw0aNEhLly6VJAUEBGj06NFas2aNTp06Zb9GAMWMM9ely5gxY3Tx4kWFh4dLknr06KG3335bkvTaa69J+r8nJ65cuVJDhgxRYGCgzGazIiMjtWrVKkk3vus+/vhjubu7q2bNmsbye/ToYfPkRQAoClnXU2W2WDS3UyeF/S9B7kgHExP16jffKDEx8Z757SExBQCAnRXVY9UlGQMCx8TEkJhCmcCZ69LJ1dVVs2fP1uzZs22m3ZxQuu+++/TTTz/luZyaNWvKYrEUS4wAcDthvr6KCAhwdBj3JBJTAADY0Z0eq37zWCo5j1V/5plnZLFYbvtYdaAs4Mw1AAD3FhJTAADY0Z0eq35zYqp58+aaN2+eMc5Ks2bN8nysOlDWcOYaAIB7A89dBQDAjorjseoAAABAaUViCgAAO8r9WPUc+X2sevny5TVw4EBt3bpViYmJdo4cAAAAKHokpgAAsLOcx6onJCQoISEhX49Vv379uq5fv27zWPWsrCxdv35dWVlZMpvNun79ujIyMuzdJAAAAKBQGGMKAAA7K6rHqktSdHS0JkyYYLz28PDQE088ofXr19upNQDuNbGxsY4OwYqvry8D0wNAKUZiCgAAOyuqx6pL0vjx4zV+/PiiDhEAbKRfSZKTyaQePXo4OhQr5T08FHvgAMkpACilSEwBAAAAuKOs66kyWyya26mTwv53O7GjHUxM1KvffKPExEQSUwBQSpGYAgAAAJBvYb6+iggIcHQYAIAygsHPAQAAAAAA4BAkpgAAAAAAAOAQpT4xNXv2bIWEhKhcuXJ6+OGHtW3bttvWnzFjhurVqycPDw8FBwdryJAhun79up2iBQAAAADghszMTA0YMEDe3t7y8fHRwIEDlZWVlWfdChUqWP1zdXVVw4YN8z0dKKlK9RhTy5cv19ChQzVnzhw9/PDDmjFjhtq0aaO4uDhVrVrVpv6SJUv01ltvacGCBXrkkUd08OBB9erVSyaTSdOmTXNACwAAZV1Jeqw6j1QHAKBkiY6O1qZNm7R//35JUrt27TRp0iSNHTvWpm5qaqrV64YNG6pLly75ng6UVKU6MTVt2jT17dtXUVFRkm48YvuHH37QggUL9NZbb9nU37x5s5o3b65u3bpJkkJCQtS1a1dt3brVrnEDAMq+kvhYdR6pDgBAybJgwQJNnz5d/v7+kqRRo0Zp+PDheSamctu2bZv279+vXr16FWo6UJKU2sRURkaGduzYoZEjRxplTk5OatWqlbZs2ZLnPI888oj+9a9/adu2bXrooYd09OhR/fjjj/rrX/+aZ/309HSlp6cbr1NSUiRJWVlZxuWVTk5OcnJyktlsltlstorFyclJ2dnZslgsdyx3dnaWyWSyuWzT2dlZkpSdnZ2vchcXF1ksFqtyk8kkZ2dnqxhvdXkorGVnZ1t9Vvbup9uV22vdK6nrSlZWlsxms123p5L6WUjW30tS2Vj3CtpPJa1/Stpj1XMeqX7+/HnVqFHDrv1U0vqmJMvZlu25PdE/+Wfv773c8+L2cu8X2KufSvK24+hjjduV22M/oqRuOzd/xycmJurUqVNq0KCBsrKy5OzsrIiICMXHx+vixYuqXLmypLz7Y968eWrXrp38/f1tPgMXFxd99tlnatu2rapWraqsrKwS008lebspaXLv35e2/fKC7l+U2sRUYmKisrOzVa1aNavyatWq6cCBA3nO061bNyUmJurRRx81DmJee+01vf3223nWnzx5siZMmGBTvmvXLnl6ekqS/Pz8FBoaqmPHjunChQtGnaCgIAUFBengwYNKTk42ymvXrq2qVatq7969unbtmlFev359eXl5adeuXVad27BhQ7m5uWn79u1WMTRt2lQZGRnas2ePUebs7KwHH3xQycnJVp+Bh4eHGjVqpMTERB09elSSFBcXl2ebYe3IkSNW/WHvfpKkypUrKzw8XGfOnNGpU6eMcnuteyXpNqTcYmNjFRISYtftqSRvN7GxsVY/RmVh3StoP5XUdbWkPVY9p8/s2U+HDh0qruaUOTnbsj23p5L83VbS2Pt7z8mp1A8HazexsbGqUqWKXX+fSvK24+hjDcmx+xE3HyyXFDnf8Tlt2rFjhyTp+PHjunz5stEmSfrtt9+M4Wlu7qdr165p6dKlWrx4sa5du2bTT/fdd5+WLVum0aNHG/OUlH4qydtNSZN7/7607ZcfOHBAaWlp+W5rqU1MFcb69es1adIkffzxx3r44Yd1+PBhDR48WO+++67GjBljU3/kyJEaOnSo8TolJUXBwcGKjIxUpUqVJP3fDkOtWrVUs2ZNo25OeVhYmE2GUpIaNGhgk6GUpMjISKsYcsqbNm1qU+7h4WFTLt1YaXOXm0wmSTfGFvHx8bGKA7cXGhpq1Sf27qfc5QEBAapevbpRbq91Lzw83Cb2kiA8PFy+/7sKxV7bU0nebsLDw9W4cWPjdVlY9wraTyV1XS1p/Pz8JNm3n0rytlPS5GzL9tye6J/8s/f33u7du4sw+rItPDxctWrVkmS/firJ246jjzVylztiPyJnWklz83d8kyZNJN34LEJDQ+Xs7KwjR45Ikpo3b25zxVTO5/7555+rYsWKat++vVxcXGz66V//+pc8PT01aNAgubjcOOQvKf1Ukrebkib3/n1p2y9v2rSpccdZfpTaxJSvr6+cnZ117tw5q/Jz585ZdVRuY8aM0V//+lf16dNHkvTAAw8oLS1Nr776qkaNGmWzkbi7u8vd3d1mOS4uLsYGniPnkrib3epL8VblNy+3MOUmkynP8twx3mp5sObs7JznZ2Wvfrqb8uJe9xzNxcXFaLe9tqeS+llIeX8v5ZTfrLSvewVpE2zdabspjn6ib/Lv5m3ZHv1E/+Sfvb/3cg5EcGe59wvs1U8ledtx9LHG3ZQXRT+V1G3n5u9cX19fBQUFae/evapXr54kKSYmRsHBwapSpUqe80vSwoUL1bNnT7m6ulqV5/jss8/Us2dPlStXzmYZju6nkrzdlDQFyTuUxP3ygvR1qU1Xurm5qUmTJlq7dq1RZjabtXbtWjVr1izPea5evWrTWTkdUlLvQwYAAAAAlE1RUVGaOHGiEhISlJCQoEmTJhkXUuQlLi5OmzdvVu/evQs1HSiJSnW6cujQoerZs6eaNm2qhx56SDNmzFBaWprxlL6XX35ZgYGBmjx5siSpffv2mjZtmiIjI41b+caMGaP27duX2Ms9AQAAAABl05gxY3Tx4kVjSIIePXoYYyC/9tprkm48fT7H/Pnz9dhjj6lu3bp5Lu9O04GSqFQnpl566SVduHBBY8eOVUJCgiIiIrR69WpjQPT4+HirK6RGjx4tk8mk0aNH6/Tp0/Lz81P79u01ceJERzUBAAAAAHCPcnV11ezZszV79mybabkTUjmmTJly2+XdaTpQEpXqxJQkDRgwQAMGDMhz2vr1661eu7i4aNy4cRo3bpwdIgMAAAAAAMDtlNoxpgAAAAAAd5aZmakBAwbI29tbPj4+GjhwoLKysvKsW6FCBat/rq6uatiwoU29a9euqU6dOvLy8irm6AGUdSSmAAAAAKAMi46O1qZNm7R//37t27dPGzdu1KRJk/Ksm5qaavUvPDxcXbp0sak3duxYq8fTA0BhkZgCAAAAgDJswYIFGj16tPz9/eXv769Ro0Zp/vz5d5xv27Zt2r9/v3r16mVVvmPHDq1evVojRowopogB3EsKPcZUamqqDhw4oMTERJlMJvn6+iosLEwVK1YsyvgAAAAAAIWUlJSkU6dOKSIiwiiLiIhQfHy8kpOTVbly5VvOO3/+fLVr104BAQFGWVZWlvr27avZs2fLbDYXZ+gA7hEFSkwdO3ZMn3/+uVauXKm9e/fafBE5OTnp/vvvV8eOHfXyyy+rdu3aRRosAAAAACD/UlNTJclqLKicv69cuXLLxFRaWpqWLVumxYsXW5V/8MEHioyM1OOPP27zsCncXmxsrKNDMPj6+qpGjRqODgOQlM/E1P79+zV27Fh9++238vLyUosWLfTiiy+qdu3a8vb2lsViUVJSko4dO6YdO3Zo1qxZevfdd/Xcc8/p3XffVXh4eHG3AwAAAABwkwoVKkiSkpOT5evra/wt6bZ3u3z11VcqX768nnnmGaPs8OHDmjNnjnbt2lWMEZc96VeS5GQyqUePHo4OxVDew0OxBw6QnEKJkK/EVKNGjfTMM8/ohx9+UKtWreTicvvZsrKytGbNGs2ZM0eNGjVSRkZGkQQLAAAAAMg/b29vBQUFKSYmRqGhoZKkmJgYBQcH3/Y2vs8++0w9e/a0OvbbtGmTzp07p7CwMEk3nvZ35coV+fr66ocfftDDDz9cvI0ppbKup8pssWhup04K+19y0JEOJibq1W++UWJiIokplAj5Skzt2bOnQFc9ubi4qG3btmrbtq0OHDhQ6OAAAAAAAHcnKipKEydOVPPmzSVJkyZNUp8+fW5ZPy4uTps3b9bChQutyjt37qxWrVoZr7ds2aI+ffooJiZGVatWLZ7gy5AwX19F5BqvC8AN+UpM3c2tePXr1y/0vAAAAACAuzNmzBhdvHjROK7r0aOH3n77bUnSa6+9JkmaM2eOUX/+/Pl67LHHVLduXavllC9fXuXLlzde+/n5yWQyKSgoqLibAKAMK/RT+XIzm836/fffdfr0aVWvXl3NmjW74+1+AAAAAIDi5+rqqtmzZ2v27Nk203InpHJMmTIlX8tt0aKFLl++fLfhAbjH3XX26MCBA2rfvr1OnTolb29vXbhwQYGBgVqxYoXVI0kBAAAAAACA3JzudgH9+/dXu3btlJSUpDNnzujs2bMKDQ3Vq6++WhTxAQAAAAAAoIzKd2Lqtdde06VLl2zKDx48qF69eqlcuXKSJF9fX3Xq1EkHDx4suigBAAAAAABQ5uQ7MXXmzBnVqVNHM2fOVHZ2tlHeokULDRs2TBs3btThw4f1/fffa9q0aWrRokVxxAsAAAAAAIAyIt+JqVWrVmnp0qWaO3euGjRooNWrV0uSPv74YwUGBqpVq1YKCwtTp06d1LhxY82bN6/YggYAAAAAAEDpV6DBz9u0aaM9e/boo48+Urdu3dSsWTPNmDFD//rXv7R48WIlJiaqSpUqcnZ2Lq54AQAAAAC3EBsb6+gQDL6+vqpRo4ajwwBQwhX4qXzOzs5644031L17d40aNUqNGjXS3/72N40fP15Vq1YtjhgBAAAAALeRfiVJTiaTevTo4ehQDOU9PBR74ADJKQC3VeDEVEZGhq5duyY/Pz/NnTtX/fv31+DBg1WnTh1FR0erT58+MplMxRErAAAAACAPWddTZbZYNLdTJ4X5+jo6HB1MTNSr33yjxMREElMAbivfiamzZ8/qlVde0X/+8x9ZLBbVqVNH8+bN0+OPP65ff/1Vy5cv15tvvqmPP/5YM2fO1OOPP16ccQMAAAAAbhLm66uIgABHhwEA+Zbvwc/79eun48ePa+3atdq1a5ciIiL0/PPP6+rVq5Kkl156SQcOHFCHDh3Url07de7cudiCBgAAAAAAQOmX78TUhg0b9MYbb+iJJ55Qw4YN9f777+vixYvav3+/UcfDw0MTJkxQbGwst/MBAAAAAADgtvKdmPL399fvv/9uvP79999lMplUvXp1m7o1atTQ8uXLiyZCAAAAAAAAlEn5HmNq8uTJ6tKlizZt2iQvLy/t3LlTgwYNUlBQUHHGBwAAAAAAgDIq34mpjh07KjY2Vj///LOuXbumGTNmqHnz5sUZGwAAAAAAAMqwfCemJKlWrVrq169fccUCAAAAAACAe0i+xpg6efJkod/gbuYFAAAAAABA2ZWvxFSdOnX0yiuvaNu2bfle8ObNm/Xyyy+rbt26hQ4OAAAAAAAAZVe+buXbuHGjRo8erT/96U+qWbOmnnzySTVu3Fi1atWSt7e3LBaLkpKSdOzYMW3fvl2//PKLTp8+rZYtW2rDhg3F3QYAAAAAAACUQvlKTD300EP6+eefFRMTo4ULF2rlypVauHChJMlkMkmSLBaLJCk4OFgdO3bUK6+8ooiIiOKJGgAAAAAAAKVegQY/j4iI0MyZMzVz5kydOXNGBw4c0MWLFyVJVapUUf369RUQEFAsgQIAAAAAAKBsKVBiKreAgACSUAAAAAAAACi0QiemJCk9PV07d+7U+fPn1bx5c/n6+hZVXAAAAAAAACjj8vVUvrx8+OGH8vf316OPPqpOnTppz549kqTExET5+vpqwYIFRRYkAAAAAAAAyp5CJaYWLlyoN954Q23bttX8+fONgc8lydfXV08++aSWLVtWZEECAAAAAACg7ClUYmrq1Kl69tlntWTJErVv395mepMmTbRv3767Dg4AAAAAAABlV6ESU4cPH1a7du1uOd3Hx8d4Wh8AAAAAAACQl0Ilpry8vJSYmHjL6fv371f16tULHRQAAAAAAADKvkIlpp5++mnNnTtXly9ftpm2b98+zZs3Tx06dLjb2AAAAAAAAFCGFSoxFR0drezsbDVo0ECjR4+WyWTS559/rh49eqhp06aqWrWqxo4dW9SxAgAAAAAAoAwpVGIqICBAO3bsUNu2bbV8+XJZLBb985//1HfffaeuXbvq999/l6+vb1HHCgAAAAAAgDLEpbAzVq1aVZ999pk+++wzXbhwQWazWX5+fnJyKlSuCwAAAAAAAPeYQmWRXnnlFW3dutV47efnp2rVqhlJqW3btumVV14pmggBAAAAAABQJhUqMbVo0SIdOXLkltOPHTumzz//vNBBAQAAAAAAoOwrlvvuzpw5Iw8Pj+JYNAAAAAAAAMqIfI8xtXLlSq1cudJ4PXfuXK1Zs8am3uXLl7VmzRo9+OCDRRMhAAAAAAAAyqR8J6b279+vr776SpJkMpm0detW7dixw6qOyWSSp6enHn/8cU2bNq1oIwUAAAAAAECZku/E1MiRIzVy5EhJkpOTk+bPn69u3boVW2AAAAAAAAAo2/KdmMrNbDYXdRwAAAAAAAC4xxTL4OcAAAAAAADAnRQ6MfX//t//U+vWrVWlShW5uLjI2dnZ5h8AAAAAAABwK4VKTP373//WX/7yF507d05dunSR2WxW165d1aVLF3l4eKhhw4YaO3ZsUccKAAAAAACAMqRQianJkyfroYce0q5duzRhwgRJ0iuvvKIvvvhCe/fu1dmzZ1WrVq0iDRQAAAAAAABlS6ESU/v371eXLl3k7OwsF5cb46dnZmZKkkJCQtS/f3+9//77RRclAAAAAAAAypxCJabKly8vNzc3SZKXl5fc3d119uxZY3q1atV07NixookQAAAAAAAAZVKhElP16tXT/v37jdcRERH65z//qaysLF2/fl1LlixRjRo1iixIAAAAAAAAlD2FSkw999xzWrlypdLT0yVJo0aN0vr16+Xl5SU/Pz9t3LhRb731VpEGCgAAAAAAgLKlUImp4cOHKz4+Xu7u7pKkv/zlL1q/fr369u2rfv36ae3aterVq1dRxgkAKKDMzEwNGDBA3t7e8vHx0cCBA5WVlZVn3V69esnNzU0VKlQw/m3ZssWYfuTIEbVr107e3t4KDAzUlClT7NUMAAAAAGWYS0FnSE9P108//aSQkBA1bNjQKH/sscf02GOPFWlwAIDCi46O1qZNm4xbr9u1a6dJkyZp7Nixedbv37+/ZsyYYVOenZ2tDh06qGPHjlq1apWOHj2q1q1bKygoSN26dSvOJgAAAAAo4wp8xZSbm5tefPFFbd68uTjiAQAUkQULFmj06NHy9/eXv7+/Ro0apfnz5xd4OXFxcYqLi9O4cePk6uqqevXqqXfv3po7d24xRA0AAADgXlLgxJTJZFLdunWVmJhYHPEAAIpAUlKSTp06pYiICKMsIiJC8fHxSk5OznOexYsXy8fHR/fff7+mTp0qs9ksScb/FovFqGs2m7Vnz57iawAAAACAe0Khxph6++23NWvWLMXFxRV1PACAIpCamipJ8vLyMspy/r5y5YpN/UGDBikuLk4XLlzQ/PnzNXPmTM2cOVPSjSexhoSEaOzYsUpPT9e+ffu0YMECpaSkFHs7AAAAAJRtBR5jSpJ+//13ValSRQ0aNFCLFi0UEhIiDw8Pqzomk8k4qAEA2FeFChUkScnJyfL19TX+lqSKFSva1G/cuLHx95/+9Ce99dZbWrx4sYYMGSJXV1etXLlSQ4YMUWBgoIKCghQVFaVPP/3UDi0BAAAAUJYVKjE1a9Ys4++1a9fmWYfEFAA4jre3t4KCghQTE6PQ0FBJUkxMjIKDg1W5cuU7zu/kZH1B7f3336+ff/7ZeD1ixAg98cQTRRs0AAAAgHtOoW7lM5vNd/yXnZ1d1LECAAogKipKEydOVEJCghISEjRp0iT16dMnz7pffvmlUlJSZLFYtH37dr333nt6/vnnjel79uxRWlqaMjIy9M033xgDqwMAAADA3SjUFVMAgJJvzJgxunjxosLDwyVJPXr00Ntvvy1Jeu211yRJc+bMkXTjSthXX31VWVlZCgwMVP/+/TVs2DBjWV9++aU++eQTXb9+XY0aNdKKFSvUsGFDO7cIAAAAQFlDYgoAyihXV1fNnj1bs2fPtpmWk5DKsWHDhtsuKzo6WtHR0UUaHwAAAAAU6la+kmT27NkKCQlRuXLl9PDDD2vbtm23rX/58mW9/vrr8vf3l7u7u8LCwvTjjz/aKVoAAAAAAADkKNVXTC1fvlxDhw7VnDlz9PDDD2vGjBlq06aN4uLiVLVqVZv6GRkZat26tapWraqvv/5agYGBOnHihNXj1AEAAAAAAGAfpToxNW3aNPXt21dRUVGSbtya8sMPP2jBggV66623bOovWLBAly5d0ubNm+Xq6ipJCgkJsWfIAAAAAAAA+J9Sm5jKyMjQjh07NHLkSKPMyclJrVq10pYtW/KcZ9WqVWrWrJlef/11rVy5Un5+furWrZtGjBghZ2dnm/rp6elKT083XqekpEiSsrKylJWVZbynk5OT8TTC3LE4OTkpOztbFovljuXOzs4ymUzGcnOXS7J5yuGtyl1cXGSxWKzKTSaTnJ2drWK8+X2Qt+zsbKvPyt79dLtye617JXVdycrKktlstuv2VFI/C8n6e0kqG+teQfupJPdPSZLTZ/bsJ/om/3K2ZXtuT/RP/tn7ey/3vLi93PsF9uontp38MZvNVp+VPfYj2HbyJysrS9nZ2Xbd32O7yb/c+/elbb+8oPsXhUpM1a5dWzNmzFCHDh3ynP79999r0KBBOnr0aGEWny+JiYnKzs5WtWrVrMqrVaumAwcO5DnP0aNH9csvv6h79+768ccfdfjwYfXv31+ZmZkaN26cTf3JkydrwoQJNuW7du2Sp6enJMnPz0+hoaE6duyYLly4YNQJCgpSUFCQDh48qOTkZKO8du3aqlq1qvbu3atr164Z5fXr15eXl5d27dpl1bkNGzaUm5ubtm/fbhVD06ZNlZGRoT179hhlzs7OevDBB5WcnGz1GXh4eKhRo0ZKTEw0+iQuLi7PzwjWjhw5YtUf9u4nSapcubLCw8N15swZnTp1yii317oXGxtb4M/NHmJjYxUSEmLX7akkbzexsbFWP0ZlYd0raD+V1HW1pMnpM3v206FDh4qrOWVOzrZsz+2pJH+3lTT2/t5zcir1w8HaTWxsrKpUqWLX3ye2nfy5cOGC1bptj/2Imw+WkbfY2FhVrFjRrvt7bDf5l3v/vrTtlx84cEBpaWn5bmuhElPHjx9XamrqLaenpqbqxIkThVl0sTKbzapatarmzp0rZ2dnNWnSRKdPn9YHH3yQZ2Jq5MiRGjp0qPE6JSVFwcHBioyMVKVKlST93w5DrVq1VLNmTaNuTnlYWJhNhlKSGjRoYJOhlKTIyEirGHLKmzZtalPu4eFhUy7dWGlzl5tMJkmSr6+vfHx8rOLA7YWGhlr1ib37KXd5QECAqlevbpTba90LDw+3ib0kCA8Pl6+vryT7bU8lebsxmUxW8e3evVuSdcwWi0U7d+6UxWKxKs/IyNDOnTuNM805UlNT8yxPSkrKszwhIUHnz5+Xt7e36tata5Tb63uvpK6rJY2fn58k+/4+leRtp6QJDw9X48aN7bofQf/kn73393K+y3Fn4eHhqlWrliT79RPbTv74+fkpIiLCeG2Pfdi87oaBrfDwcIWFhUmy37EG203+5ewTSI4/JizM8VPOHWf5Uehb+XI+mLz897//LfYBxX19feXs7Kxz585ZlZ87d86qo3Lz9/eXq6ur1RdVeHi4EhISlJGRITc3N6v67u7ucnd3t1mOi4uLXFysP7qcS+JudqsvxVuV37zcwpSbTKY8y3PHeKvlwZqzs3Oen5W9+uluyot73XM0FxcXo9322p5K4meRfiVJTiaTevbs6ehQDOU9PBR74IBq1KhhVe6I7z3YutN2Uxz9RN/k3837GPboJ/on/+z9m3u7/W1Yy71fYK9+YtvJHycnpwLt8xZFP7Ht5I+Li4vxudrrWIPtJv8KkncoifmIgvR1vmvOnDlTM2fONN7ojTfe0KhRo2zqJScn6/Lly+rWrVu+gygMNzc3NWnSRGvXrlXHjh0l3bgiau3atRowYECe8zRv3lxLliyxOst/8OBB+fv72ySlAKA0yLqeKrPFormdOinsf1eQOdLBxES9+s03SkxMtElMAQAAAMDN8p2Yqlq1qu6//35JN27lCwwMVGBgoFUdk8kkT09PNWnSRP379y/aSPMwdOhQ9ezZU02bNtVDDz2kGTNmKC0tzXhK38svv6zAwEBNnjxZkvS3v/1Ns2bN0uDBgzVw4EAdOnRIkyZN0qBBg4o9VgAoTmG+vooICHB0GAAAAABQIPlOTHXt2lVdu3aVJLVs2VKjR4/Wn//852ILLD9eeuklXbhwQWPHjlVCQoIiIiK0evVqY0D0+Ph4q8vZgoOD9dNPP2nIkCFq2LChAgMDNXjwYI0YMcJRTQAAAAAAALhnFeoGz3Xr1hV1HIU2YMCAW966t379epuyZs2a6ffffy/mqAAAAAAAAHAnhRoSf9myZerVq9ctp0dFRenLL78sbEwAAAAAAAC4BxQqMTVt2rQ8n1aXw8PDQ9OnTy90UAAAAAAAACj7CpWYiouLU2Rk5C2nN2rUSAcOHCh0UAAAAAAAACj7CpWYslgsunz58i2nJyUlKTMzs7AxAQAAAAAA4B5QqMRUZGSkli5dqoyMDJtp6enpWrJkyW2vqAIAAAAAAAAKlZh66623tHfvXrVs2VLfffedjh49qqNHj2rVqlVq0aKF9u3bp7feequoYwUAAAAAAEAZ4lKYmdq1a6f58+dr8ODB6tixo1FusVhUsWJFzZs3T88880xRxQgAAAAAAIAyqFCJKUnq1auXOnXqpJ9//llHjx6VJIWGhuqpp55SxYoViyxAAAAAAAAAlE2FTkxJUqVKlfTCCy8UVSwAAAAAAAC4hxRqjClJys7O1rJly9SvXz8999xz+uOPPyRJycnJ+uabb3Tu3LkiCxIAAAAAAABlT6ESU5cvX1bz5s3VrVs3LV26VKtWrdKFCxckSRUqVNCgQYM0c+bMIg0UAAAAAAAAZUuhn8q3b98+/fTTTzp69KgsFosxzdnZWS+88IJ+/PHHIgsSAAAAAAAAZU+hElMrVqzQwIED1bp1a5lMJpvpYWFhOn78+N3GBgAAAAAAgDKsUImp5ORk1apV65bTMzMzlZWVVeigAAAAAAAAUPYVKjEVGhqqnTt33nL6zz//rPvuu6/QQQEAAAAAAKDsK1Riqk+fPlqwYIGWL19ujC9lMpmUnp6uUaNGafXq1erXr1+RBgoAAAAAAICyxaUwMw0ePFj79u1T165d5eXlJUnq1q2bLl68qKysLPXr10+9e/cuyjgBAAAAAABQxhQqMWUymTRv3jz17NlTX3/9tQ4dOiSz2azQ0FB17txZjz/+eFHHCQAAAAAAgDKmwImpq1evqkePHnr++efVvXt3Pfroo8URFwAAAAAAAMq4Ao8xVb58ea1Zs0ZXr14tjngAAAAAAABwjyjU4OePPvqotmzZUtSxAAAAAAAA4B5SqMTUrFmztHHjRo0ePVqnTp0q6pgAAAAAAABwDyhUYqpRo0Y6deqUJk+erJo1a8rd3V2VKlWy+le5cuWijhUAAAAAAABlSKGeyvf888/LZDIVdSwAAAAAAAC4hxQ4MWWxWPThhx/K1dVVHh4exRETAAAAAAAA7gEFvpUvIyNDPj4+mjVrVnHEAwAAAAAAgHtEgRNT7u7uql69utzc3IojHgAAAAAAANwjCjX4ea9evbR48WJlZGQUdTwAAAAAAAC4RxRq8PMHHnhAK1as0P33369evXopJCQkz/GmOnXqdNcBAgAAAAAAoGwqVGKqa9euxt9jxozJs47JZFJ2dnbhogIAAAAAAECZV6jE1Lp164o6DgAAAAAAANxjCpWYeuKJJ4o6DgAAAAAAANxjCpWYym3//v06ceKEJKlmzZq677777jooAAAAAAAAlH2FTkytXLlSQ4cO1fHjx63Ka9WqpWnTpqlDhw53GxsAAAAAAADKMKfCzPTjjz/q+eeflyRNmjRJ3377rb799ltNmjRJFotFnTp10urVq4s0UAAAAAAAAJQthbpi6t1331XDhg21ceNGeXp6GuUdOnTQgAED9Oijj2rChAlq27ZtkQUKAAAAAACAsqVQV0zt2bNHPXv2tEpK5fD09FSvXr20Z8+euw4OAAAAAAAAZVehElPlypXTpUuXbjn90qVLKleuXKGDAgAAAAAAQNlXqMTUk08+qZkzZ2rLli0207Zu3aoPP/xQrVq1uuvgAAAAAAAAUHYVaoypKVOmqFmzZnr00Uf10EMPqV69epKkuLg4bdu2TVWrVtX7779fpIECAAAAAACgbCnUFVO1atXSnj17NGjQICUlJWn58uVavny5kpKSNHjwYO3evVshISFFHCoAAAAAAADKkkJdMSVJVatW1fTp0zV9+vSijAcAAAAAAAD3iEJdMZWVlaWUlJRbTk9JSVFWVlahgwIAAAAAAEDZV6jE1KBBg/TII4/ccnrz5s01bNiwQgcFAAAAAACAsq9QianVq1frhRdeuOX0F154QT/++GOhgwIAAAAAAEDZV6jE1JkzZxQYGHjL6QEBATp9+nShgwIAAAAAAEDZV6jEVJUqVRQXF3fL6bGxsapUqVKhgwIAAAAAAEDZV6jEVNu2bfXpp59q165dNtN27typuXPnql27dncdHAAAAAAAAMoul8LM9O6772r16tV66KGH1KFDB91///2SpL179+q7775T1apV9e677xZpoAAAAAAAAChbCpWYCggI0Pbt2/XWW29p5cqV+vbbbyVJlSpVUvfu3TVp0iQFBAQUaaAAAAAAAAAoWwqVmJIkf39/ff7557JYLLpw4YIkyc/PTyaTqciCAwAAAAAAQNlV6MRUDpPJpKpVqxZFLAAAAAAAALiHFGrwcwAAAAAAAOBukZgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5T6xNTs2bMVEhKicuXK6eGHH9a2bdvyNd+yZctkMpnUsWPH4g0QAAAAAAAAeSrVianly5dr6NChGjdunHbu3KlGjRqpTZs2On/+/G3nO378uIYPH67HHnvMTpECAAAAAADgZqU6MTVt2jT17dtXUVFRuu+++zRnzhyVL19eCxYsuOU82dnZ6t69uyZMmKDatWvbMVoAAAAAAADk5uLoAAorIyNDO3bs0MiRI40yJycntWrVSlu2bLnlfO+8846qVq2q3r17a+PGjbd9j/T0dKWnpxuvU1JSJElZWVnKysoy3tPJyUlms1lms9kqFicnJ2VnZ8tisdyx3NnZWSaTyVhu7nLpRkItP+UuLi6yWCxW5SaTSc7OzlYx3vw+yFt2drbVZ2Xvfrpdub3WvZK6rmRlZclsNtt1eyqpn0VJlNf3ZHH3E/2TPznfF/b8faJv8i9n27HnfgT9k3/23t/LPS9uL/d+gb36iW0nf8xms9VnZY99WLad/MnKylJ2drZdjzXYbvIv9/60o48JC3P8VJC+LrWJqcTERGVnZ6tatWpW5dWqVdOBAwfynGfTpk2aP3++YmJi8vUekydP1oQJE2zKd+3aJU9PT0mSn5+fQkNDdezYMV24cMGoExQUpKCgIB08eFDJyclGee3atVW1alXt3btX165dM8rr168vLy8v7dq1y6pzGzZsKDc3N23fvt0qhqZNmyojI0N79uwxypydnfXggw8qOTnZ6jPw8PBQo0aNlJiYqKNHj0qS4uLi8vUZ3OuOHDli1R/27idJqly5ssLDw3XmzBmdOnXKKLfXuhcbG1vgz80eYmNjFRISYtftie0m/2JjY40fR3t975XUdbWkyfm+sOfv06FDh4qrOWVOzrZjz/0Ivtvyz977e05OpfrmBruKjY1VlSpV7LpfzraTPxcuXLBat+2xD3vzwTLyFhsbq4oVK9r1WIPtJv9y7087+piwMMdPaWlp+W5rqU1MFdSVK1f017/+VfPmzZOvr2++5hk5cqSGDh1qvE5JSVFwcLAiIyNVqVIlSf+3w1CrVi3VrFnTqJtTHhYWZpOhlKQGDRrYZCglKTIy0iqGnPKmTZvalHt4eNiUSzdW2tzlJpNJkuTr6ysfHx+rOHB7oaGhVn1i737KXR4QEKDq1asb5fZa98LDw21iLwnCw8ONbdle2xPbTf6Fh4ercePGkuz3vVdS19WSxs/PT5J9f5/YdvIvZ9ux534E/ZN/9t7f2717dxFGX7aFh4erVq1akuzXT2w7+ePn56eIiAjjtT32YXOm4fbCw8MVFhYmyX7HGmw3+Zd7f9rRx4SFOX7KueMsP0ptYsrX11fOzs46d+6cVfm5c+esOirHkSNHdPz4cbVv394oy8k+uri4KC4uTqGhoVbzuLu7y93d3WZZLi4ucnGx/uhyLom72a2+FG9VfvNyC1NuMpnyLM8d462WB2vOzs55flb26qe7KS/udc/RXFxcjHbba3sqqZ9FSZTX96Qjvvdg607bTXH0E32TfzdvO/boJ/on/+z9m5tzIII7y71fYK9+YtvJHycnpwLt8xZFP7Ht5I+Li4vxudrrWIPtJv8KkncoifmIgvR1qU1Xurm5qUmTJlq7dq1RZjabtXbtWjVr1symfv369fXHH38oJibG+NehQwe1bNlSMTExCg4Otmf4AAAAAAAA97xSna4cOnSoevbsqaZNm+qhhx7SjBkzlJaWpqioKEnSyy+/rMDAQE2ePFnlypVTgwYNrOb38vKSJJtyAAAAAAAAFL9SnZh66aWXdOHCBY0dO1YJCQmKiIjQ6tWrjQHR4+PjuYcVAAAAAACghCrViSlJGjBggAYMGJDntPXr19923kWLFhV9QAAAAAAAAMgXLicCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDkJgCAAAAAACAQ5CYAgAAAAAAgEOQmAIAAAAAAIBDlPrE1OzZsxUSEqJy5crp4Ycf1rZt225Zd968eXrsscfk7e0tb29vtWrV6rb1AQAAAAAAUHxKdWJq+fLlGjp0qMaNG6edO3eqUaNGatOmjc6fP59n/fXr16tr165at26dtmzZouDgYD311FM6ffq0nSMHAAAAAABAqU5MTZs2TX379lVUVJTuu+8+zZkzR+XLl9eCBQvyrP/FF1+of//+ioiIUP369fXZZ5/JbDZr7dq1do4cAAAAAAAApTYxlZGRoR07dqhVq1ZGmZOTk1q1aqUtW7bkaxlXr15VZmamfHx8iitMAAAAAAAA3IKLowMorMTERGVnZ6tatWpW5dWqVdOBAwfytYwRI0YoICDAKrmVW3p6utLT043XKSkpkqSsrCxlZWVJupEMc3JyktlsltlsNurmlGdnZ8tisdyx3NnZWSaTyVhu7nJJys7Ozle5i4uLLBaLVbnJZJKzs7NVjDe/D/KWnZ1t9VnZu59uV26vda+kritZWVkym8123Z5K6mdREuX1PVnc/UT/5E/O94U9f5/om/zL2XbsuR9B/+Sfvff3cs+L28u9X2CvfmLbyR+z2Wz1WdljH5ZtJ3+ysrKUnZ1t12MNtpv8y70/7ehjwsIcPxWkr0ttYupuvffee1q2bJnWr1+vcuXK5Vln8uTJmjBhgk35rl275OnpKUny8/NTaGiojh07pgsXLhh1goKCFBQUpIMHDyo5Odkor127tqpWraq9e/fq2rVrRnn9+vXl5eWlXbt2WXVuw4YN5ebmpu3bt1vF0LRpU2VkZGjPnj1GmbOzsx588EElJydbJec8PDzUqFEjJSYm6ujRo5KkuLi4fH1O97ojR45Y9Ye9+0mSKleurPDwcJ05c0anTp0yyu217sXGxhb4c7OH2NhYhYSE2HV7YrvJv9jYWOPH0V7feyV1XS1pcr4v7Pn7dOjQoeJqTpmTs+3Ycz+C77b8s/f+npNTqb25we5iY2NVpUoVu+6Xs+3kz4ULF6zWbXvsw958sIy8xcbGqmLFinY91mC7yb/c+9OOPiYszPFTWlpavttaahNTvr6+cnZ21rlz56zKz507p+rVq9923n/84x967733tGbNGjVs2PCW9UaOHKmhQ4car1NSUhQcHKzIyEhVqlRJ0v/tMNSqVUs1a9Y06uaUh4WF2WQoJalBgwY2GUpJioyMtIohp7xp06Y25R4eHjbl0o2VNne5yWSSdOMzy7ltkR2d/AkNDbXqE3v3U+7ygIAAq3XbXuteeHi4TewlQXh4uHx9fSXZb3tiu8m/8PBwNW7cWJL9vvdK6rpa0vj5+Umy7+8T207+5Ww79tyPoH/yz977e7t37y7C6Mu28PBw1apVS5L9+oltJ3/8/PwUERFhvLbHPmzONNxeeHi4wsLCJNnvWIPtJv9y7087+piwMMdPOXec5UepTUy5ubmpSZMmWrt2rTp27ChJxkDmAwYMuOV8U6ZM0cSJE/XTTz/l+SHm5u7uLnd3d5tyFxcXubhYf3Q5l8Td7FZfircqv3m5hSk3mUx5lueO8VbLgzVnZ+c8Pyt79dPdlBf3uudoLi4uRrvttT2V1M+iJMrre9IR33uwdaftpjj6ib7Jv5u3HXv0E/2Tf/b+zc05EMGd5d4vsFc/se3kj5OTU4H2eYuin9h28sfFxcX4XO11rMF2k38FyTuUxHxEQfq6VK8VQ4cOVc+ePdW0aVM99NBDmjFjhtLS0hQVFSVJevnllxUYGKjJkydLkt5//32NHTtWS5YsUUhIiBISEiRJFSpUUIUKFRzWDgAAAAAAgHtRqU5MvfTSS7pw4YLGjh2rhIQERUREaPXq1caA6PHx8VZZw08++UQZGRl64YUXrJYzbtw4jR8/3p6hAwAAAAAA3PNKdWJKkgYMGHDLW/fWr19v9fr48ePFHxAAAAAAAADyhZHHAAAAAAAA4BAkpgAAAAAAAOAQJKYAAAAAAADgECSmAAAAAAAA4BAkpgAAAAAAAOAQJKYAAAAAAADgECSmAAAAAAAA4BAkpgAAAAAAAOAQJKYAAAAAAADgECSmAAAAAAAA4BAkpgAAAAAAAOAQJKYAAAAAAADgECSmAAAAAAAA4BD/v717j4qq3P84/hlAQECUFPESoZB3O6GcvOU1MDWTKMvSEFAX6vGYGqWJubKjZmqatrqJlpaCpp7Sysq0BDVvx1S6qqWpmYV3REwhYP/+aDE/R/ACInuPvF9r7bWcZ579zPPMx43D1733UJgCAAAAAACAKShMAQAAAAAAwBQUpgAAAAAAAGAKClMAAAAAAAAwBYUpAAAAAAAAmILCFAAAAAAAAExBYQoAAAAAAACmoDAFAAAAAAAAU1CYAgAAAAAAgCkoTAEAAAAAAMAUFKYAAAAAAABgCgpTAAAAAAAAMAWFKQAAAAAAAJiCwhQAAAAAAABMQWEKAAAAAAAApqAwBQAAAAAAAFNQmAIAAAAAAIApKEwBAAAAAADAFBSmAAAAAAAAYAoKUwAAAAAAADAFhSkAAAAAAACYgsIUAAAAAAAATEFhCgAAAAAAAKagMAUAAAAAAABTUJgCAAAAAACAKShMAQAAAAAAwBQUpgAAAAAAAGAKClMAAAAAAAAwBYUpAAAAAAAAmILCFAAAAAAAAExBYQoAAAAAAACmoDAFAAAAAAAAU1CYAgAAAAAAgCkoTAEAAAAAAMAUFKYAAAAAAABgCgpTAAAAAAAAMAWFKQAAAAAAAJiCwhQAAAAAAABMQWEKAAAAAAAApqAwBQAAAAAAAFNQmAIAAAAAAIApKEwBAAAAAADAFBSmAAAAAAAAYAoKUwAAAAAAADAFhSkAAAAAAACYgsIUAAAAAAAATEFhCgAAAAAAAKagMAUAAAAAAABTUJgCAAAAAACAKShMAQAAAAAAwBQUpgAAAAAAAGAKClMAAAAAAAAwBYUpAAAAAAAAmMLpC1Ovv/666tWrJ09PT7Vu3Vr/+9//rth/+fLlaty4sTw9PXXHHXfo008/LaeZAgAAAAAA4GJOXZhaunSpEhISNGHCBO3cuVN33nmnunXrpmPHjhXbf/Pmzerbt68GDRqkXbt2KSoqSlFRUfr+++/LeeYAAAAAAABw6sLUyy+/rPj4eA0YMEBNmzbVnDlz5OXlpfnz5xfb/5VXXlH37t01evRoNWnSRJMmTVLLli312muvlfPMAQAAAAAA4LSFqdzcXO3YsUMRERH2NhcXF0VERGjLli3F7rNlyxaH/pLUrVu3y/YHAAAAAADAjeNm9gRK68SJE8rPz1dAQIBDe0BAgPbs2VPsPhkZGcX2z8jIKLZ/Tk6OcnJy7I/PnDkjSTp16pTy8vIk/V0Mc3FxUUFBgQoKCux9C9vz8/NlGMZV211dXWWz2ezjXtwuSfn5+dfU7ubmJsMwHNptNptcXV0d5piZmSlJyvp9n/JyLxS7/vJ07vhhSdI3f/yhc7m5Js9G2nfypKS/Mz916pS9vbxy+v333+1/L4vLTyq/v3uFx5PVssnMzFRmZma5Hk9WO24k6x47mZmZ9mOnvH7uWS0fq2aTlZWlrKyscv33iWyu7tJjpzw/R5DPlRVmk52drczMzHL9vHf27FlJZHMlxX0uKK/P5Rw7V3bxvzsXf54uj8+wHDtXdvFxc/r06XL9XcNqx41k7XwKjx2zfycsze9PWVlZkuTwOpdlOKkjR44YkozNmzc7tI8ePdpo1apVsftUqlTJWLx4sUPb66+/btSsWbPY/hMmTDAksbGxsbGxsbGxsbGxsbGxsbGVcDt8+PBV6ztOe8ZUjRo15OrqqqNHjzq0Hz16VLVq1Sp2n1q1apWof2JiohISEuyPCwoKdOrUKVWvXl02m+06V4CLZWVlKTAwUIcPH5avr6/Z08FFyMbayMe6yMa6yMbayMe6yMbayMe6yMbayOfGMAxDZ8+eVZ06da7a12kLU+7u7goLC9OXX36pqKgoSX8Xjr788ksNHz682H3atm2rL7/8UqNGjbK3rV27Vm3bti22v4eHhzw8PBzaqlWrVhbTx2X4+vryw8CiyMbayMe6yMa6yMbayMe6yMbayMe6yMbayKfsVa1a9Zr6OW1hSpISEhIUGxurf/7zn2rVqpVmz56tc+fOacCAAZKkmJgY1a1bVy+++KIkaeTIkerUqZNmzpypnj176r333tPXX3+tuXPnmrkMAAAAAACACsmpC1OPPvqojh8/rueee04ZGRkKDQ3V6tWr7Tc4//XXX+Xi8v9fPNiuXTstXrxY48eP17hx49SgQQOtXLlSzZs3N2sJAAAAAAAAFZZTF6Ykafjw4Ze9dC8tLa1I2yOPPKJHHnnkBs8KJeXh4aEJEyYUuXQS5iMbayMf6yIb6yIbayMf6yIbayMf6yIbayMf89kM41q+uw8AAAAAAAAoWy5X7wIAAAAAAACUPQpTAAAAAAAAMAWFKZSZzp07a9SoUfbH9erV0+zZs695/4MHD8pmsyk9Pb3M51bRkY21kY91kY11kY21kY91kY21kY91kY21kY9zozCFG2b79u0aPHjwNfcPDAzUH3/8Yf+WxLS0NNlsNmVmZl5132+//VYdOnSQp6enAgMDNX369NJOu0Ior2wuXLiguLg43XHHHXJzc1NUVNR1zLriKK980tLS9MADD6h27dry9vZWaGioUlJSrmfqN73yymbv3r3q0qWLAgIC5OnpqeDgYI0fP15//fXX9Uz/plae/+YU2rdvn6pUqaJq1aqVcLYVT3nlU/iLxaXb1q1br2f6N7XyPHYMw9CMGTPUsGFDeXh4qG7dunrhhRdKO/UKobzyef7554s9dry9va9n+je18jx2Pv/8c7Vp00ZVqlSRv7+/evfurYMHD5Zy5hVDeeazbNkyhYaGysvLS0FBQXrppZdKO+0Ky+m/lQ/W5e/vX6L+rq6uqlWrVolfJysrS/fee68iIiI0Z84cfffddxo4cKCqVatWoh9GFUl5ZZOfn6/KlStrxIgRev/990u8f0VVXvls3rxZ//jHP/TMM88oICBAq1atUkxMjKpWrar777+/xONVBOWVTaVKlRQTE6OWLVuqWrVq+uabbxQfH6+CggJNmTKlxONVBOWVTaG//vpLffv2VYcOHbR58+ZSj1NRlHc+X3zxhZo1a2Z/XL169VKPdbMrz2xGjhypNWvWaMaMGbrjjjt06tQpnTp1qlRjVRTllc/TTz+toUOHOrSFh4frrrvuKvFYFUV5ZXPgwAE98MADSkhIUEpKis6cOaMnn3xSDz30kHbu3Fni8SqK8srns88+0+OPP65XX31V9957r3bv3q34+HhVrlxZw4cPL/F4FRVnTKFUzp07p5iYGPn4+Kh27dqaOXNmkT6Xnj65Z88etW/fXp6enmratKm++OIL2Ww2rVy5UpLj6ZMHDx5Uly5dJEl+fn6y2WyKi4srdi4pKSnKzc3V/Pnz1axZMz322GMaMWKEXn755bJetlOwUjbe3t568803FR8ff12/YNxMrJTPuHHjNGnSJLVr104hISEaOXKkunfvrg8++KCsl+0UrJRNcHCwBgwYoDvvvFNBQUGKjIzU448/ro0bN5b1sp2ClbIpNH78eDVu3Fh9+vQpq2U6LSvmU716ddWqVcu+VapUqayW61SslM3u3bv15ptv6sMPP1RkZKTq16+vsLAwde3atayX7TSslI+Pj4/DMXP06FH9+OOPGjRoUFkv2ylYKZsdO3YoPz9fkydPVkhIiFq2bKmnn35a6enpFfZMaivls2jRIkVFRWno0KEKDg5Wz549lZiYqGnTpskwjLJe+k2LM6ZQKqNHj9b69ev14YcfqmbNmho3bpx27typ0NDQYvvn5+crKipKt912m7Zt26azZ8/qqaeeuuz4gYGBev/999W7d2/t3btXvr6+qly5crF9t2zZoo4dO8rd3d3e1q1bN02bNk2nT5+Wn5/fda3V2VgpGxRl9XzOnDmjJk2alHRZNwUrZ7Nv3z6tXr1aDz30UGmW5vSsls26deu0fPlypaenV9hC7sWslo8kRUZG6sKFC2rYsKHGjBmjyMjI61mi07JSNh9//LGCg4O1atUqde/eXYZhKCIiQtOnT9ctt9xSFst1OlbK51JvvfWWGjZsqA4dOpRmaU7PStmEhYXJxcVFCxYsUFxcnLKzs7Vo0SJFRERU2KK7lfLJycmRl5eXQ1vlypX122+/6dChQ6pXr15pl1mhUJhCiWVnZ+vtt99WcnKywsPDJUnvvvuubr311svus3btWu3fv19paWn2M2deeOGFy/4vmaurq/1DSs2aNa94/46MjAzVr1/foS0gIMD+XEUqTFktGziyej7Lli3T9u3blZSUdM373Cysmk27du20c+dO5eTkaPDgwZo4cWIJV+b8rJbNyZMnFRcXp+TkZPn6+pZyVTcPq+Xj4+OjmTNn6u6775aLi4vef/99RUVFaeXKlRWuOGW1bH755RcdOnRIy5cv18KFC5Wfn68nn3xSDz/8sNatW1fKVTovq+VzsQsXLiglJUVjx44twYpuHlbLpn79+lqzZo369OmjIUOGKD8/X23bttWnn35ayhU6N6vl061bNz355JOKi4tTly5dtG/fPvsZXH/88QeFqWvEpXwosf379ys3N1etW7e2t91yyy1q1KjRZffZu3evAgMDHS7natWq1Q2dZ0VENtZm5XxSU1M1YMAAzZs3z+G+LBWFVbNZunSpdu7cqcWLF+uTTz7RjBkzynR8Z2C1bOLj49WvXz917NixTMZzdlbLp0aNGkpISFDr1q111113aerUqYqOjq6QN6K1WjYFBQXKycnRwoUL1aFDB3Xu3Flvv/22UlNTtXfv3jJ5DWditXwutmLFCp09e1axsbFlPrYzsFo2GRkZio+PV2xsrLZv367169fL3d1dDz/8cIW8VMxq+cTHx2v48OG6//775e7urjZt2uixxx6TJLm4UG65VrxTcHqF18FfrPAx9zUCrm79+vXq1auXZs2apZiYGLOng4sEBgaqadOm6tu3r6ZOnarnn39e+fn5Zk+rQlu3bp1mzJghNzc3ubm5adCgQTpz5ozc3Nw0f/58s6eHYrRu3Vr79u0zexoVXu3ateXm5qaGDRva2wovHf/111/NmhaK8dZbb+n++++3X4EAc73++uuqWrWqpk+frhYtWqhjx45KTk7Wl19+qW3btpk9vQrPZrNp2rRpys7O1qFDh5SRkWEvegUHB5s8O+dBYQolFhISokqVKjn8IDx9+rR++umny+7TqFEjHT582KGAtH379iu+TuE9o672S1jbtm21YcMGh5v/rV27Vo0aNapQl/FJ1ssGjqyYT1pamnr27Klp06ZV6G+xtGI2lyooKNBff/2lgoKCEu/rzKyWzZYtW5Senm7fJk6cqCpVqig9PV0PPvjgtSzppmK1fIqTnp6u2rVrl3g/Z2e1bO6++27l5eVp//799rbCuQQFBV1x35uR1fIpdODAAaWmplbYm55L1svmzz//LHLmjaurqyRVuM8EkvXyKeTq6qq6devK3d1dS5YsUdu2bUv8zYAVGYUplJiPj48GDRqk0aNHa926dfr+++8VFxd3xVMVu3btqpCQEMXGxurbb7/Vpk2bNH78eEl/V5mLExQUJJvNplWrVun48ePKzs4utl+/fv3k7u6uQYMG6YcfftDSpUv1yiuvKCEh4foX62Sslo0k/fjjj0pPT9epU6d05swZ+y9zFZHV8klNTVXPnj01YsQI9e7dWxkZGcrIyKiQX91ttWxSUlK0bNky7d69W7/88ouWLVumxMREPfrooxXuRqdWy6ZJkyZq3ry5fatbt65cXFzUvHnzCvefIZL18nn33Xe1ZMkS7dmzR3v27NGUKVM0f/58PfHEE9e/WCdjtWwiIiLUsmVLDRw4ULt27dKOHTs0ZMgQde3a1eEsqorCavkUmj9/vmrXrq0ePXqUfnFOzmrZ9OzZU9u3b9fEiRP1888/a+fOnRowYICCgoLUokWL61+wk7FaPidOnNCcOXO0Z88epaena+TIkVq+fLnDNwLiGhhAKZw9e9aIjo42vLy8jICAAGP69OlGp06djJEjR9r7BAUFGbNmzbI/3r17t3H33Xcb7u7uRuPGjY2PP/7YkGSsXr3aMAzDOHDggCHJ2LVrl32fiRMnGrVq1TJsNpsRGxt72fl88803Rvv27Q0PDw+jbt26xtSpU8t4xc7DatkEBQUZkopsFZWV8omNjS02m06dOpX9wp2AlbJ57733jJYtWxo+Pj6Gt7e30bRpU2PKlCnG+fPnb8DKrc9K2VxqwYIFRtWqVa9/kU7MSvm88847RpMmTQwvLy/D19fXaNWqlbF8+fIbsGrnYKVsDMMwjhw5Yjz00EOGj4+PERAQYMTFxRknT54s41U7D6vlk5+fb9x6663GuHHjynilzsdq2SxZssRo0aKF4e3tbfj7+xuRkZHG7t27y3jVzsNK+Rw/ftxo06aN4e3tbXh5eRnh4eHG1q1bb8Cqb242w6iAd0yDJWzatEnt27fXvn37FBISYvZ0cBGysTbysS6ysS6ysTbysS6ysTbysS6ysTbysRYKUyg3K1askI+Pjxo0aKB9+/Zp5MiR8vPz01dffWX21Co8srE28rEusrEusrE28rEusrE28rEusrE28rE2N7MngIrj7NmzeuaZZ/Trr7+qRo0aioiI0MyZM82eFkQ2Vkc+1kU21kU21kY+1kU21kY+1kU21kY+1sYZUwAAAAAAADAF38oHAAAAAAAAU1CYAgAAAAAAgCkoTAEAAAAAAMAUFKYAAAAAAABgCgpTAAAAAAAAMAWFKQAAAAAAAJiCwhQAAAAAAABMQWEKAAAAAAAApqAwBQAAAAAAAFNQmAIAAAAAAIApKEwBAAAAAADAFBSmAAAAAAAAYAoKUwAAAAAAADAFhSkAAAAAAACYgsIUAAAAAAAATEFhCgAAAAAAAKagMAUAAHCJzp07q3nz5mZPAwAA4KZHYQoAAFjaO++8I5vNJk9PTx05cqTI8xSR/nb06FE9/fTTaty4sby8vOTt7a2wsDBNnjxZmZmZJR5v8eLFmj17dpnPEwAA4GJuZk8AAADgWuTk5Gjq1Kl69dVXzZ6K5Wzfvl333XefsrOzFR0drbCwMEnS119/ralTp2rDhg1as2ZNicZcvHixvv/+e40aNeoGzBgAAOBvFKYAAIBTCA0N1bx585SYmKg6deqYPZ1yde7cOXl7exf7XGZmph588EG5urpq165daty4scPzL7zwgubNm1ce0zTFld4bAABgfVzKBwAAnMK4ceOUn5+vqVOnXrWvzWbT8OHDtXz5cjVt2lSVK1dW27Zt9d1330mSkpKSdPvtt8vT01OdO3fWwYMHix1nx44dateunSpXrqz69etrzpw5Ds8XXmZ46f5paWmy2WxKS0tzaN+2bZu6d++uqlWrysvLS506ddKmTZsc+jz//POy2Wz68ccf1a9fP/n5+al9+/aXXWtSUpKOHDmil19+uUhRSpICAgI0fvx4++MPP/xQPXv2VJ06deTh4aGQkBBNmjRJ+fn59j6dO3fWJ598okOHDslms8lms6levXr253NycjRhwgTdfvvt8vDwUGBgoMaMGaOcnByH1z5//rxGjBihGjVqqEqVKoqMjNSRI0dks9n0/PPPO/TdtWuXevToIV9fX/n4+Cg8PFxbt2516FP4fq9fv17Dhg1TzZo1deuttyo1NVU2m00rVqwosv7FixfLZrNpy5Ytl30PAQCAeThjCgAAOIX69esrJiZG8+bN09ixY6961tTGjRv10Ucf6d///rck6cUXX9T999+vMWPG6I033tCwYcN0+vRpTZ8+XQMHDtS6desc9j99+rTuu+8+9enTR3379tWyZcv0r3/9S+7u7ho4cGCJ579u3Tr16NFDYWFhmjBhglxcXLRgwQLdc8892rhxo1q1auXQ/5FHHlGDBg00ZcoUGYZx2XE/+ugjVa5cWQ8//PA1zeOdd96Rj4+PEhIS5OPjo3Xr1um5555TVlaWXnrpJUnSs88+qzNnzui3337TrFmzJEk+Pj6SpIKCAkVGRuqrr77S4MGD1aRJE3333XeaNWuWfvrpJ61cudL+WnFxcVq2bJn69++vNm3aaP369erZs2eROf3www/q0KGDfH19NWbMGFWqVElJSUnq3Lmz1q9fr9atWzv0HzZsmPz9/fXcc8/p3Llz6ty5swIDA5WSkqIHH3zQoW9KSopCQkLUtm3ba3p/AABAOTMAAAAsbMGCBYYkY/v27cb+/fsNNzc3Y8SIEfbnO3XqZDRr1sxhH0mGh4eHceDAAXtbUlKSIcmoVauWkZWVZW9PTEw0JDn07dSpkyHJmDlzpr0tJyfHCA0NNWrWrGnk5uY6zO3ifQ3DMFJTUw1JRmpqqmEYhlFQUGA0aNDA6Natm1FQUGDv9+effxr169c3unbtam+bMGGCIcno27fvNb0/fn5+xp133nlNfQtf81JDhgwxvLy8jAsXLtjbevbsaQQFBRXpu2jRIsPFxcXYuHGjQ/ucOXMMScamTZsMwzCMHTt2GJKMUaNGOfSLi4szJBkTJkywt0VFRRnu7u7G/v377W2///67UaVKFaNjx472tsL3u3379kZeXp7DuImJiYaHh4eRmZlpbzt27Jjh5ubm8FoAAMBauJQPAAA4jeDgYPXv319z587VH3/8ccW+4eHhDpefFZ5107t3b1WpUqVI+y+//OKwv5ubm4YMGWJ/7O7uriFDhujYsWPasWNHieadnp6un3/+Wf369dPJkyd14sQJnThxQufOnVN4eLg2bNiggoICh32GDh16TWNnZWU5rOdqKleubP/z2bNndeLECXXo0EF//vmn9uzZc9X9ly9friZNmqhx48b2dZw4cUL33HOPJCk1NVWStHr1akl/n910sSeeeMLhcX5+vtasWaOoqCgFBwfb22vXrq1+/frpq6++UlZWlsM+8fHxcnV1dWiLiYlRTk6O/vvf/9rbli5dqry8PEVHR191XQAAwBwUpgAAgFMZP3688vLyrnqvqdtuu83hcdWqVSVJgYGBxbafPn3aob1OnTpFbqrdsGFDSbrsPaku5+eff5YkxcbGyt/f32F76623lJOTozNnzjjsU79+/Wsa29fXV2fPnr3mufzwww968MEHVbVqVfn6+srf399euLl0Dpdbyw8//FBkHYXvzbFjxyRJhw4dkouLS5F13H777Q6Pjx8/rj///FONGjUq8lpNmjRRQUGBDh8+7NBe3HvTuHFj3XXXXUpJSbG3paSkqE2bNkVeEwAAWAf3mAIAAE4lODhY0dHRmjt3rsaOHXvZfpeeUXO1duMK93G6HJvNVmz7xTcSl2Q/G+qll15SaGhosfsU3sOp0MVnNl1J48aNlZ6ertzcXLm7u1+xb2Zmpjp16iRfX19NnDhRISEh8vT01M6dO/XMM88UOWurOAUFBbrjjjv08ssvF/v8pYW/G+Fy701MTIxGjhyp3377TTk5Odq6datee+21Gz4fAABQehSmAACA0xk/frySk5M1bdq0G/Yav//+u86dO+dw1tRPP/0kSfZLBP38/CT9XfC52KFDhxweh4SESPr77KaIiIgynWevXr20ZcsWvf/+++rbt+8V+6alpenkyZP64IMP1LFjR3v7gQMHivS9XNEtJCRE33zzjcLDwy/bR5KCgoJUUFCgAwcOqEGDBvb2ffv2OfTz9/eXl5eX9u7dW2SMPXv2yMXF5ZqLXY899pgSEhK0ZMkSnT9/XpUqVdKjjz56TfsCAABzcCkfAABwOiEhIYqOjlZSUpIyMjJuyGvk5eUpKSnJ/jg3N1dJSUny9/dXWFiYfR6StGHDBnu//Px8zZ0712GssLAwhYSEaMaMGcrOzi7yWsePHy/1PIcOHaratWvrqaeeshfOLnbs2DFNnjxZ0v+fLXbx2WG5ubl64403iuzn7e1d7KV9ffr00ZEjRzRv3rwiz50/f17nzp2TJHXr1k2Sioz96quvOjx2dXXVvffeqw8//NDhEsmjR49q8eLFat++vXx9fYtd+6Vq1KihHj16KDk5WSkpKerevbtq1KhxTfsCAABzcMYUAABwSs8++6wWLVqkvXv3qlmzZmU+fp06dTRt2jQdPHhQDRs21NKlS5Wenq65c+eqUqVKkqRmzZqpTZs2SkxM1KlTp3TLLbfovffeU15ensNYLi4ueuutt9SjRw81a9ZMAwYMUN26dXXkyBGlpqbK19dXH3/8canm6efnpxUrVui+++5TaGiooqOj7YWznTt3asmSJWrbtq0kqV27dvLz81NsbKxGjBghm82mRYsWFXsZY1hYmJYuXaqEhATddddd8vHxUa9evdS/f38tW7ZMQ4cOVWpqqu6++27l5+drz549WrZsmT7//HP985//VFhYmHr37q3Zs2fr5MmTatOmjdavX28vnl18ttXkyZO1du1atW/fXsOGDZObm5uSkpKUk5Oj6dOnl+j9iImJ0cMPPyxJmjRpUqneUwAAUH4oTAEAAKd0++23Kzo6Wu++++4NGd/Pz0/vvvuunnjiCc2bN08BAQF67bXXFB8f79AvJSVFQ4YM0dSpU1WtWjUNGjRIXbp0UdeuXR36de7cWVu2bNGkSZP02muvKTs7W7Vq1VLr1q0dvv2vNFq3bq3vv/9eL730kj755BMtWrRILi4uatKkicaOHavhw4dLkqpXr65Vq1bpqaee0vjx4+Xn56fo6GiFh4fbz3AqNGzYMKWnp2vBggWaNWuWgoKC1KtXL7m4uGjlypWaNWuWFi5cqBUrVsjLy0vBwcEaOXKk/SbokrRw4ULVqlVLS5Ys0YoVKxQREaGlS5eqUaNG8vT0tPdr1qyZNm7cqMTERL344osqKChQ69atlZycbP/WxGvVq1cv+fn5qaCgQJGRkdfxrgIAgPJgM0pzp08AAACgFNLT09WiRQslJyfr8ccfL/Px8/LyVKdOHfXq1Utvv/12mY8PAADKFveYAgAAwA1x/vz5Im2zZ8+Wi4uLw83Xy9LKlSt1/PhxxcTE3JDxAQBA2eJSPgAAANwQ06dP144dO9SlSxe5ubnps88+02effabBgwdf8zftXatt27bp22+/1aRJk9SiRQt16tSpTMcHAAA3BpfyAQAA4IZYu3at/vOf/+jHH39Udna2brvtNvXv31/PPvus3NzK9v9H4+LilJycrNDQUL3zzjtq3rx5mY4PAABuDApTAAAAAAAAMAX3mAIAAAAAAIApKEwBAAAAAADAFBSmAAAAAAAAYAoKUwAAAAAAADAFhSkAAAAAAACYgsIUAAAAAAAATEFhCgAAAAAAAKagMAUAAAAAAABTUJgCAAAAAACAKf4PLZNRgsFoGuoAAAAASUVORK5CYII=\n" }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "# 3 MultiLoRA的示例\n", "\n", "![multi_lora](./multi_lora.png)\n", "\n", "* step1:构建主模型并训练,训练数据集去掉数字‘0’、‘1’、‘2’;\n", "* step2:测试主模型的识别能力;\n", "* step3:创建Multi-LoRA层;\n", "* step4:主模型的参数冻结,分别用数字‘‘0’、‘1’、‘2’数据对不同适配器进行微调;\n", "* step5:测试LoRA模型,观测数据‘0’、‘1’、‘2’识别度差异。\n", "\n" ], "metadata": { "id": "Zt_qodp3rCKD" } }, { "cell_type": "code", "source": [ "net_for_multi = MLP().to(device)\n", "\n", "# 过滤数据,去掉‘0'、‘1'、‘2'的数据:\n", "mnist_trainset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n", "exclude_indices = torch.tensor([False if x in [0, 1, 2] else True for x in mnist_trainset.targets])\n", "mnist_trainset.data = mnist_trainset.data[exclude_indices]\n", "mnist_trainset.targets = mnist_trainset.targets[exclude_indices]\n", "train_loader = torch.utils.data.DataLoader(mnist_trainset, batch_size=10, shuffle=True)\n", "\n", "train(train_loader, net_for_multi, epochs=1, total_iterations_limit=2000)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Q75Fkebaux2q", "outputId": "f43b47f2-6b20-44a0-92fc-f6357f622504" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Epoch 1: 100%|█████████▉| 1999/2000 [00:41<00:00, 48.09it/s, loss=0.294]\n" ] } ] }, { "cell_type": "code", "source": [ "test(model=net_for_multi)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "CZogke8Jv_Tc", "outputId": "26cc8d39-7947-4302-cb4f-6a21444739c6" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:03<00:00, 270.36it/s]" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.643\n", "The wrong counts of digit 0: 980\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 1032\n", "The wrong counts of digit 3: 46\n", "The wrong counts of digit 4: 51\n", "The wrong counts of digit 5: 75\n", "The wrong counts of digit 6: 16\n", "The wrong counts of digit 7: 74\n", "The wrong counts of digit 8: 59\n", "The wrong counts of digit 9: 105\n", "\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ "[0.0,\n", " 0.0,\n", " 0.0,\n", " 0.9544554455445544,\n", " 0.9480651731160896,\n", " 0.9159192825112108,\n", " 0.9832985386221295,\n", " 0.9280155642023347,\n", " 0.9394250513347022,\n", " 0.8959365708622399]" ] }, "metadata": {}, "execution_count": 21 } ] }, { "cell_type": "code", "source": [ "# 定义LoRA对权重修改修改:\n", "class MultiLoRAParametrization(nn.Module):\n", " def __init__(self, features_in, features_out, rank=1, alpha=1, device='cpu'):\n", " super().__init__()\n", " # 低秩矩阵的定义:\n", " self.digit_0_lora_A = nn.Parameter(torch.zeros((rank,features_out)).to(device))\n", " self.digit_0_lora_B = nn.Parameter(torch.zeros((features_in, rank)).to(device))\n", " self.digit_1_lora_A = nn.Parameter(torch.zeros((rank,features_out)).to(device))\n", " self.digit_1_lora_B = nn.Parameter(torch.zeros((features_in, rank)).to(device))\n", " self.digit_2_lora_A = nn.Parameter(torch.zeros((rank,features_out)).to(device))\n", " self.digit_2_lora_B = nn.Parameter(torch.zeros((features_in, rank)).to(device))\n", "\n", " for w in [self.digit_0_lora_A , self.digit_1_lora_A, self.digit_2_lora_A]:\n", " nn.init.normal_(w, mean=0, std=1)\n", "\n", " # 参考论文:https://arxiv.org/pdf/2106.09685 4.1节 设置一个比例系数:\n", " self.scale = alpha / rank\n", " # LoRA开关&选择\n", " self.enabled = True\n", " self.digit_value = 0\n", "\n", " def forward(self, original_weights):\n", " if self.enabled:\n", " # Return W + (B*A)*scale\n", " # 选择适配器\n", " if self.digit_value == 0:\n", " return original_weights + torch.matmul(self.digit_0_lora_B, self.digit_0_lora_A).view(original_weights.shape) * self.scale\n", " elif self.digit_value == 1:\n", " return original_weights + torch.matmul(self.digit_1_lora_B, self.digit_1_lora_A).view(original_weights.shape) * self.scale\n", " elif self.digit_value == 2:\n", " return original_weights + torch.matmul(self.digit_2_lora_B, self.digit_2_lora_A).view(original_weights.shape) * self.scale\n", " else:\n", " raise ValueError(f\"The LoRA adapter of digit: {self.digit_value} is out of range.\")\n", " else:\n", " return original_weights" ], "metadata": { "id": "YNudvQlgrHkP" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "# 重新定义参数更新函数\n", "def linear_layer_parameterization(layer, device, rank=1, lora_alpha=1):\n", " # LoRA仅修改W,忽略bias修改。\n", " features_in, features_out = layer.weight.shape\n", " return MultiLoRAParametrization(\n", " features_in, features_out, rank=rank, alpha=lora_alpha, device=device\n", " )\n", "\n", "\n", "# 注册LoRA权重到原始层中:\n", "parametrize.register_parametrization(\n", " net_for_multi.linear1, \"weight\", linear_layer_parameterization(net_for_multi.linear1, device)\n", ")\n", "parametrize.register_parametrization(\n", " net_for_multi.linear2, \"weight\", linear_layer_parameterization(net_for_multi.linear2, device)\n", ")\n", "parametrize.register_parametrization(\n", " net_for_multi.linear3, \"weight\", linear_layer_parameterization(net_for_multi.linear3, device)\n", ")\n", "\n", "# 定义LoRA开关函数:\n", "def select_lora(enabled=True, digit_value=0):\n", " for layer in [net_for_multi.linear1, net_for_multi.linear2, net_for_multi.linear3]:\n", " layer.parametrizations[\"weight\"][0].enabled = enabled\n", " layer.parametrizations[\"weight\"][0].digit_value = digit_value" ], "metadata": { "id": "1vekZk-HwY0D" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "for name, param in net_for_multi.named_parameters():\n", " print(f'Freezing non-LoRA parameter {name}')" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "6MIRdt8NyHBT", "outputId": "ea0e8d03-e6d7-4239-c5f9-d875afeda6b9" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Freezing non-LoRA parameter linear1.bias\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.original\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_0_lora_A\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_0_lora_B\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_1_lora_A\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_1_lora_B\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_2_lora_A\n", "Freezing non-LoRA parameter linear1.parametrizations.weight.0.digit_2_lora_B\n", "Freezing non-LoRA parameter linear2.bias\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.original\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_0_lora_A\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_0_lora_B\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_1_lora_A\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_1_lora_B\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_2_lora_A\n", "Freezing non-LoRA parameter linear2.parametrizations.weight.0.digit_2_lora_B\n", "Freezing non-LoRA parameter linear3.bias\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.original\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_0_lora_A\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_0_lora_B\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_1_lora_A\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_1_lora_B\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_2_lora_A\n", "Freezing non-LoRA parameter linear3.parametrizations.weight.0.digit_2_lora_B\n" ] } ] }, { "cell_type": "code", "source": [ "# 将原始权重冻结:\n", "for name, param in net_for_multi.named_parameters():\n", " param.requires_grad = False\n", "\n", "# 对每个LoRA适配器进行微调:\n", "for digit in [0 , 1, 2]:\n", "\n", " print(f\"Training the LoRA adapter: {digit}\")\n", " select_lora(enabled=True, digit_value=digit)\n", " for name, param in net_for_multi.named_parameters():\n", " if f'digit_{digit}' in name:\n", " param.requires_grad = True\n", " # 过滤数据,仅保留选定数字的数据:\n", " mnist_trainset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n", " exclude_indices = torch.tensor([True if x == digit else False for x in mnist_trainset.targets])\n", " mnist_trainset.data = mnist_trainset.data[exclude_indices]\n", " mnist_trainset.targets = mnist_trainset.targets[exclude_indices]\n", " train_loader = torch.utils.data.DataLoader(mnist_trainset, batch_size=10, shuffle=True)\n", "\n", " # 训练对应的LoRA适配器:\n", " train(train_loader, net_for_multi, epochs=1, total_iterations_limit=100)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "XZf4fnMmwcR4", "outputId": "c401b8a9-6a11-41cb-d606-4563bf8ff09c" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Training the LoRA adapter: 0\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Epoch 1: 99%|█████████▉| 99/100 [00:01<00:00, 73.86it/s, loss=4.36]\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "Training the LoRA adapter: 1\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Epoch 1: 99%|█████████▉| 99/100 [00:01<00:00, 73.89it/s, loss=2.55]\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "Training the LoRA adapter: 2\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Epoch 1: 99%|█████████▉| 99/100 [00:01<00:00, 71.20it/s, loss=5.25]\n" ] } ] }, { "cell_type": "code", "source": [ "# 打印未开LoRA的测试输出结果:\n", "select_lora(enabled=False)\n", "print(\"The original model test result:\")\n", "test(model=net_for_multi)\n", "\n", "# 分别开启不同的LoRA适配器观测结果:\n", "for digit in [0 , 1, 2]:\n", " select_lora(enabled=True, digit_value=digit)\n", " print(f\"The LoRA_adapter_digit_{digit} test result:\")\n", " test(model=net_for_multi)\n" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "QAmF-YLWzHYa", "outputId": "6035ea84-4466-4202-e9f6-1976bca87b07" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "The original model test result:\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:03<00:00, 277.23it/s]\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.643\n", "The wrong counts of digit 0: 980\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 1032\n", "The wrong counts of digit 3: 46\n", "The wrong counts of digit 4: 51\n", "The wrong counts of digit 5: 75\n", "The wrong counts of digit 6: 16\n", "The wrong counts of digit 7: 74\n", "The wrong counts of digit 8: 59\n", "The wrong counts of digit 9: 105\n", "\n", "The LoRA_adapter_digit_0 test result:\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:06<00:00, 151.66it/s]\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.544\n", "The wrong counts of digit 0: 37\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 1032\n", "The wrong counts of digit 3: 130\n", "The wrong counts of digit 4: 473\n", "The wrong counts of digit 5: 490\n", "The wrong counts of digit 6: 409\n", "The wrong counts of digit 7: 383\n", "The wrong counts of digit 8: 203\n", "The wrong counts of digit 9: 268\n", "\n", "The LoRA_adapter_digit_1 test result:\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:06<00:00, 160.67it/s]\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.369\n", "The wrong counts of digit 0: 980\n", "The wrong counts of digit 1: 11\n", "The wrong counts of digit 2: 1032\n", "The wrong counts of digit 3: 728\n", "The wrong counts of digit 4: 540\n", "The wrong counts of digit 5: 605\n", "The wrong counts of digit 6: 252\n", "The wrong counts of digit 7: 522\n", "The wrong counts of digit 8: 811\n", "The wrong counts of digit 9: 831\n", "\n", "The LoRA_adapter_digit_2 test result:\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "Testing: 100%|██████████| 1000/1000 [00:06<00:00, 152.41it/s]" ] }, { "output_type": "stream", "name": "stdout", "text": [ "\n", "Accuracy: 0.216\n", "The wrong counts of digit 0: 980\n", "The wrong counts of digit 1: 1135\n", "The wrong counts of digit 2: 41\n", "The wrong counts of digit 3: 701\n", "The wrong counts of digit 4: 706\n", "The wrong counts of digit 5: 699\n", "The wrong counts of digit 6: 899\n", "The wrong counts of digit 7: 823\n", "The wrong counts of digit 8: 963\n", "The wrong counts of digit 9: 888\n", "\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\n" ] } ] }, { "cell_type": "markdown", "source": [ "# 附1: 打印MNIST的手写体图片" ], "metadata": { "id": "qGfyzmZ1G92n" } }, { "cell_type": "code", "source": [ "import torch\n", "import torchvision\n", "from torchvision import datasets, transforms\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "# 数据预处理\n", "transform = transforms.Compose([\n", " transforms.ToTensor(),\n", " transforms.Lambda(lambda x: x.squeeze())\n", "])\n", "\n", "# 加载数据集\n", "train_dataset = datasets.MNIST(\n", " root='./data',\n", " train=True,\n", " download=True,\n", " transform=transform\n", ")\n", "\n", "print(f\"Training set size: {len(train_dataset)}\")\n", "\n", "def show_digits(dataset, title=\"MNIST Digits (0-9)\"):\n", " \"\"\"显示每个数字的一个示例\"\"\"\n", " digit_examples = {}\n", "\n", " # 使用DataLoader批量处理数据\n", " dataloader = torch.utils.data.DataLoader(dataset, batch_size=64, shuffle=False)\n", "\n", " for images, labels in dataloader:\n", " for i in range(len(labels)):\n", " label = labels[i].item()\n", " if label not in digit_examples:\n", " digit_examples[label] = images[i].numpy()\n", "\n", " if len(digit_examples) == 10:\n", " break\n", "\n", " # 创建图形\n", " fig, axes = plt.subplots(2, 5, figsize=(12, 6))\n", " fig.suptitle(title, fontsize=16)\n", "\n", " for digit in range(10):\n", " row, col = divmod(digit, 5)\n", " axes[row, col].imshow(digit_examples[digit], cmap='gray')\n", " axes[row, col].set_title(f'Digit {digit}')\n", " axes[row, col].axis('off')\n", "\n", " plt.tight_layout()\n", " plt.show()\n", "\n", "# 显示数字\n", "show_digits(train_dataset)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 602 }, "id": "3XtnhZ6wG8qA", "outputId": "11bf463a-0329-41df-fc67-58984ed5cf3e" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Training set size: 60000\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "
" ], "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAAI3CAYAAABKw+g5AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVjJJREFUeJzt3XeYVOXd+P/PwEoviqCCSFHsBFEsaFQQQQ0mii32lkQTjaI+aDRKFGvsEQsmNlDUqIkKaEQlIthiIrbYOyggBhGQJm3n94c/+Kqw9+4yu2d34PW6Lq7nybznnHPvys0uH87M5vL5fD4AAAAAIEN1anoBAAAAAKx5DKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAKAIdejQIXK5XORyuTjttNOSz73qqquWP7ekpGSF3rNnz+X96quvLvM8v/rVryKXy8WgQYO+9/i4ceOWH78yn3/+eZxzzjnRtWvXaNq0adSrVy/atGkT2267bZxwwgkxbNiwWLp06QprqcyviljZuRs3bhytW7eOH//4x3HqqafG2LFjI5/Pl3mO4447LnK5XAwbNqxC1yzPss9dz549q+R8hSotLY3tt98+Nthgg5g3b95Kn/O3v/0tevbsGeuss040btw4ttlmm7jyyitj8eLFq3zdJUuWxJAhQ6J79+7RrFmzaNSoUfzoRz+Kiy++OBYsWLDSYy655JLI5XLx2GOPrfJ1AYCateJ3pgBAUbnnnnviqquuinr16q2033HHHRU+1x//+Mf41a9+FWuvvXaVrO2FF16IfffdN2bNmhVNmjSJHXfcMdZff/2YO3duvPHGG3HbbbfFbbfdFgcffHA0adIk9tlnn+jQocMK57nzzjsjImLvvfeODTbYoKA1bbPNNtG1a9eIiFi0aFHMmDEjXn/99XjhhRfixhtvjC5dusSwYcNi2223Leg6hVo2bEsNyara7bffHi+//HLceOON0bhx4xX66aefHoMHD46SkpLo1atXNGnSJMaOHRtnn312PPLII/Hkk09Gw4YNK3XNhQsXxk9/+tP45z//GfXr118+mPr3v/8d559/fjz44IMxbty4FX5PnnHGGXHjjTfGGWecEX369Im11lqrkA8dAKgJeQCg6LRv3z4fEfntt98+HxH5Bx54YKXPe/755/MRkd9hhx3yEZGvW7fuCs/p0aNHPiLyjRo1ykdE/uyzz17puX75y1/mIyJ/wQUXfO/xp59+Oh8R+R9+W/HNN9/kN9xww3xE5I844oj87NmzVzjnO++8kz/rrLPy8+fPT368y87/9NNPJ5+Xsuzj/OH6l3nmmWfyO+644/LPxUsvvbTCc6ZOnZp/55138rNmzVrldXzXvHnz8u+8805+0qRJK7SVfU6r0/z58/OtWrXKt2nTJr9o0aIV+sMPP5yPiHyTJk3yL7/88vLHp0+fnv/Rj36Uj4j8gAEDKn3ds846Kx8R+Q033DD/xhtvLH/866+/zu+7777Lf/+szNVXX52PiPzgwYMrfV0AoOZ5+R4AFLFf/OIXEVH23VC33377956Xcuqpp0adOnXi+uuvj6lTpxa8tueeey6mTJkSJSUlccstt0SzZs1WeM4WW2wRV155ZaXvrqkOu+22Wzz77LOx6667xvz58+OII45Y/rLCZVq3bh1bbLFFNG/evEqu2ahRo9hiiy2iXbt2VXK+Qtx9990xffr0OOaYY1Z619Fll10WERHnnHNObLfddssfb9myZQwZMiQiIm688caYPXt2ha+5ePHiuPnmmyPi25fjde7ceXlr2rRp3HbbbdGwYcP461//Gh9++OEKxy9b6/XXX5/pHWUAQNUwlAKAIvajH/0ott9++3jyySdjypQp32tz586NBx54INq2bRt77bVXuefq3LlzHH300bFgwYK44IILCl7bF198ERERTZo0WelLwWqjevXqxZ///OeIiPjggw9ixIgR3+up95RasmRJXHPNNdG5c+do0KBBrLfeenHIIYfE22+/HcOGDYtcLhfHHXfc945Z2XtKDRo06Hvvk/XD98GaOHHi8va3v/0tevfuHeuuu26stdZase6668ZWW20VJ5xwQvz3v/+t1Md+4403Lv8Yf2jKlCnx0ksvRUTEEUccsULfddddY6ONNoqFCxdW6j2e3nnnnZg7d25ERPTu3XuFvsEGG0Tnzp0jn8/Hgw8+uEJv1apV9O3bNz766KN4/PHHK3xdAKB2MJQCgCL3i1/8IkpLS1cYlDzwwAMxd+7cOPbYY6NOnYp9yb/ooouifv36MXTo0Hj33XcLWteyu39mzZpVZW8MnoWtt956+ftJjRkzpkLHlJaWxgEHHBBnnnlmfPDBB9GjR4/o1atXvPLKK7HDDjvEyy+/XOHrd+3aNY499tjl//vYY4/93q8mTZpExLf/rX7+85/H+PHjo3PnznHIIYdE9+7do27dunH77bfH2LFjK3zNTz75JP773/9G27ZtY/PNN1+hv/rqqxER0aJFi+jYseNKz7H99tt/77kVsWwgFRGx7rrrrvQ5LVu2jIgo83PYp0+fiIgVBogAQO3njc4BoMgdccQRMWDAgBg2bFicd955yx+/4447IpfLVeile8u0a9cufvvb38a1114b5557bjz00EOrvK5ddtkltt1223j11Vfj+OOPjyFDhsRee+0V22+/fWy//fbRtm3bVT53devWrVu8+uqr8dZbb1Xo+TfeeGM8+uij0bp163j66aeXD3aWLl0aAwYMiMGDB1f42v369Yt+/fotf3P3lQ30Fi5cGJdffnk0adIkJkyYsMIgadKkSWX+1LqVWTbA2nnnnVfaP/nkk4iI5MsMN9poo+89tyLWW2+95f//xx9/HFtvvfUKz/n444+T591ll10iIuKpp56q8HUBgNrBnVIAUOSaN28eBx54YHz44Ycxfvz4iIh477334vnnn48ePXrExhtvXKnznXfeedG8efN4+OGH48UXX1zlddWpUyf+8Y9/xE9+8pOIiHjppZfi0ksvjQMOOCA22mij2HzzzeOKK66o1PAkK8vuzpkxY0aFnr9s6DRo0KDvDYjq1q0bV155ZWy44YZVur6vv/46FixYEBtvvPFK72xq3759bLHFFhU+37K7m7bccsuV9jlz5kREJF+GuewOrq+//rrC1+3UqdPyQdett966Qh83bly89957yfMuG2R99NFHlbo2AFDzDKUAYDXwwzc8X/Z/K3OX1DItWrSIs88+OyJi+f9dVa1bt47HHnss3nzzzbjsssviZz/72fIBzfvvvx/nnHNO7LzzzjFr1qyCrlPVSktLIyK+995OZZk8efLyu3lW9n5L9erVi4MPPrhK19eqVavo0KFD/Pe//40BAwbE22+/XdD5lr3/V1kvoatOy96/7IYbbojzzz8/Jk2aFDNnzoy///3vceihhy5/0/WyXoJar1695QOxZR8HAFAcDKUAYDWwxx57RMeOHePvf/97zJw5M+66665o1qzZKg9DTj/99GjTpk0888wz8eijjxa8vq233jp+//vfx6hRo2Ly5Mnx9ttvxymnnBK5XC5ef/31773ssDb48ssvI+LbAV15Jk+eHBHf3l21bDjyQx06dKiytS1z1113xXrrrRfXXnttbL311rHuuutG3759409/+tPy9VfUsp+Yt7KfkBjx7U/Ci4iYN29emedY9v5Q3z3HbbfdFscdd9wKv777fmW/+MUv4sILL4xcLhcXX3xxdOjQIVq0aBGHHHJIrLfeenHWWWdFRPq/xbJrzpw5syIfLgBQSxhKAcBqYNlPdps/f34ce+yxMW3atDjssMOiYcOGq3S+hg0bLr+D5dxzz11+51BV2XLLLeOGG26IU089NSJq35tUv/LKKxHx7U83rKjUXVUVueOqsnbbbbeYOHFi/O1vf4tTTjklOnToEE888UT83//9X2y88caVeo+ltddeOyLKfoncsqHaZ599VuY5lrXvDuCee+65uPPOO1f4NW3atO8de/7558f7778fV111VfzmN7+J/v37x7333hsvv/xy5PP5iEj/t1g2VFtnnXWSHycAULsYSgHAauK4446LOnXqxCOPPBIRq/bSve/65S9/GZtvvnm88cYbMXz48KpY4gr22muviIhK39lTnd5666147bXXIuL/rS9l2csRp0+fXuadRBMnTqyq5X1Pw4YN4+CDD44bbrghXn755Zg2bVqceOKJMWfOnEr991/2huNlvYfWsp9GOGPGjDLfcHzChAkREbHddtstf2zYsGGRz+dX+NWzZ88Vjt94443jzDPPjJtvvjkGDx4chx9+eNSrVy+effbZiPh/P2XvhxYuXLj8877++utX4KMFAGoLQykAWE20a9cu9t9//1h33XWje/fusdNOOxV0vrp168Zll10WEd/eybJw4cJKHb/sDpeUTz/9NCKi1vwkvkWLFsVvfvObiIjYYostYr/99iv3mI022mj53UF//etfV3rOBx98sNJrWfZeSkuWLKnwMa1atYorr7wyIr793Fb05WzLBkllvTdV27ZtY4cddoiIiHvvvXeF/txzz8Vnn30W9evXj759+1Z4veV58cUX47nnnouNNtoo9t9//5U+580334yIb980vayXHwIAtZOhFACsRh566KH48ssv41//+leVnO/AAw+MnXbaKT799NN46KGHKnXsI488Ev369YsxY8bE0qVLV+jjxo2LQYMGRUTEYYcdVhXLLcjzzz8fu+22Wzz33HPRpEmTuOeee8p8c+0f6t+/f0R8+6bd77///vLHS0tL4/e//33yZW9lWTaoe+utt1ZokyZNittuu22lL7dbdqfcOuusU+EhzR577BERkfx9c+6550ZExOWXX7785Y0R3949dfLJJ0dExCmnnBLNmzev0DWXmTlz5vKfsPddL774Yhx00EGRy+XilltuiZKSkpUe/8ILL0RERK9evSp1XQCg5q38qzsAwP/viiuuiJ49e8b8+fMrdVxpaWmMHDkyRo4cGc2bN4/tttsuNthgg5g3b168//77y9/sunfv3pm+0fmIESOWv5xu8eLF8dVXX8Vrr722/H2Ottlmmxg2bFh07dq1wufs379/jBkzJkaPHh1dunSJPfbYI9Zee+146aWXYurUqXHyySfHkCFDol69ehU+50EHHRRXX3119O7dO3r16rX8zcavuOKKmDlzZpxwwglx8sknR9euXaNjx44REfHBBx/Eq6++GrlcLq666qqoW7duha7VsWPH6NKlS/z3v/+Nd955J7bccssVntOvX7/o379/XH/99dG9e/fYc889o3HjxvHUU0/FrFmz4sc//nFcfPHFFf74lpk0aVJsu+22seWWW0anTp2icePG8d5778Wrr74aa621Vtx+++2xzz77lHn8P//5z+XrAwCKi6EUAJDUo0eP6Nu3bzz22GOVOm6fffaJJ554Ip566ql4/vnn4+OPP15+J856660X/fr1i8MPPzwOOeSQankj8LK8/vrr8frrr0fEt+/J1Lx58+jYsWMcfPDBccABB8Qee+xR6fXUrVs3Ro4cGdddd10MGzYsnn766WjatGnstttuMWLEiHj44Ycj4tuf0FdRF198cdSpUyceeuihGDFiRCxatCgiIgYOHBibbLJJXHfddTF+/Ph4880347HHHot8Ph8bbrhhHHPMMdG/f//o1q1bpT6GU045JU488cQYNmxYXHHFFSt9zuDBg+PHP/5x3HTTTfHCCy/E4sWLY5NNNolzzjknzjjjjEoN3ZbZcMMN49e//nU899xzMX78+Fi4cGG0adMmTjjhhBgwYEBsvvnmZR47ffr0GD16dGyyySbJwRUAUDvl8hV5wwcAAFZZr1694umnn44HH3wwDjzwwJpezkrNnz8/OnToECUlJTFx4sRVGjBl7ZprrokzzzwzBg8evPwllABA8fCeUgAAVeC1115bfjfTMosWLYpBgwbF008/Heutt16Vvgl4VWvUqFFceuml8fnnn8ctt9xS08sp17x58+LKK6+MzTbbLE466aSaXg4AsArcKQUAUAV69uwZr732WmyzzTbRunXrmDlzZrzxxhvx+eefR4MGDeLhhx+u9S8xKy0tjR133DEmT54cH330UTRu3Liml1SmSy65JP7whz/EP/7xj1o97AMAymYoBQBQBe65556455574r///W/MmDEj8vl8tGnTJvbYY48YMGBAbLXVVjW9RACAWsVQCgAAAIDMeU8pAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QqMoMGDYpcLrdKxw4bNixyuVxMnDixahcFrJT9CsXHvoXiYb9C8bFv+SFDqRq0bFMt+9WgQYNo06ZN7L333nH99dfHnDlzqn0NQ4YMiWHDhlXqmFGjRsV2220XDRo0iHbt2sUFF1wQS5YsqZ4FQi1RjPv1/vvvj6OOOio23XTTyOVy0bNnz2pbG9RGxbZvZ8yYEVdddVXsvvvu0apVq1h77bWje/fucf/991fvIqEWKLb9GhFxxhlnxHbbbRctWrSIRo0axZZbbhmDBg2KuXPnVt8ioRYpxn37XR999FE0aNAgcrlcTJgwoWoXRoXl8vl8vqYXsaYaNmxYHH/88XHRRRdFx44dY/HixTFt2rQYN25cjBkzJtq1axejRo2KLl26LD9myZIlsWTJkmjQoEGlr7d06dJYvHhx1K9ff/l0unPnztGyZcsYN25chc4xevTo2HfffaNnz55x+OGHxxtvvBE33XRTnHjiiXHzzTdXek1QLIpxv/bs2TNefvnl2GGHHeK1116LLl26VPhYWB0U27599NFH48ADD4y+ffvGHnvsESUlJfHggw/G008/Heeff35ceOGFlV4TFIti268REbvuumt069YtOnXqFA0aNIhXX3017rjjjth+++3jmWeeiTp1/Ps/q7di3Lfftd9++8XYsWNj3rx58dJLL8X2229f6XNQBfLUmKFDh+YjIv/SSy+t0J566ql8w4YN8+3bt8/Pnz+/2taw9dZb53v06FHh52+11Vb5bbbZJr948eLlj5133nn5XC6Xf+edd6phhVA7FON+/fTTT/NLly5dpWNhdVBs+/bjjz/OT5w48XuPlZaW5nv16pWvX79+fu7cudWwQqgdim2/luXqq6/OR0T+X//6V9UsCmqxYt63jz/+eL5evXr5gQMHlvkxkA3j+1qqV69e8Yc//CEmTZoUd9999/LHV/Ya3AULFkT//v2jZcuW0bRp09hvv/1iypQpkcvlYtCgQcuf98PX4Hbo0CHeeuutGD9+/PJbLlMv73n77bfj7bffjhNPPDFKSkqWP37yySdHPp+Pv//971XysUOxqY37NSJio4028q+0UIbauG87duwY7du3/95juVwu+vXrFwsXLoyPP/644I8bilFt3K9l6dChQ0REzJo1q9LHwuqkNu/bxYsXx2mnnRannXZabLLJJlXx4VIAf1upxY4++uiIiHjyySeTzzvuuOPihhtuiL59+8YVV1wRDRs2jH333bfc81933XXRtm3b2GKLLWL48OExfPjwOO+888p8/quvvhoRscJtjW3atIm2bdsu77Amqm37FShfsezbadOmRUREy5YtK30srC5q635dsmRJfPnllzF16tR48sknY+DAgdG0adPYcccdK/aBwWqstu7b6667LmbOnBkDBw6s2AdCtSop/ynUlLZt20bz5s3jo48+KvM5r7zySjzwwANx+umnx5/+9KeI+PbOpeOPPz5ef/315Pn79esXAwcOjJYtW8ZRRx1V7no+//zziIho3br1Cq1169YxderUcs8Bq6vatl+B8hXDvv3qq6/itttui912222lX39hTVFb9+uECRNi5513Xv6/N9988xg1alS0aNGiwueA1VVt3LfTpk2Liy++OK6++upo1qxZxT8Yqo07pWq5Jk2aJH9qweOPPx4R327c7zr11FOrfC0LFiyIiIj69euv0Bo0aLC8w5qqNu1XoGJq874tLS2NI488MmbNmhU33HBDtV8ParvauF+32mqrGDNmTIwYMSJ+97vfRePGjf30PfiO2rZvzz777Nh4443jV7/6VbWcn8pzp1QtN3fu3FhvvfXK7JMmTYo6depEx44dv/d4p06dqnwtDRs2jIiIhQsXrtC++eab5R3WVLVpvwIVU5v37amnnhqPP/543HXXXbHNNttU+/WgtquN+7VZs2bRu3fviIjYf//949577439998/XnnlFfsWonbt2xdffDGGDx8eTz31lPddrUX8l6jFJk+eHLNnz641f2Fd9rKBZS/j+67PP/882rRpk/WSoNaobfsVKF9t3rcXXnhhDBkyJC6//PLl78kBa7LavF+/68ADD4yIiPvuu6+GVwI1r7bt29/97nex2267RceOHWPixIkxceLE+PLLLyPi27/PfvrppzW8wjWToVQtNnz48IiI2Hvvvct8Tvv27aO0tDQ++eST7z3+4YcfVugaP/zJByldu3aNiG9fO/9dU6dOjcmTJy/vsCaqbfsVKF9t3bc33XRTDBo0KE4//fQ4++yzK308rI5q6379oYULF0ZpaWnMnj274HNBsatt+/bTTz+NZ555Jjp27Lj811lnnRUREfvtt1906dKlwuei6hhK1VJjx46Niy++ODp27BhHHnlkmc9btsGHDBnyvccr+t4TjRs3rvCPrN16661jiy22iFtuuSWWLl26/PGbb745crlcHHzwwRU6D6xuauN+BdJq6769//77o3///nHkkUfGtddeW+HjYHVWG/frrFmzYvHixSs8ftttt0XEij+tGtY0tXHf3nLLLfHwww9/79ey9666+uqr45577qnQeaha3lOqFhg9enS8++67sWTJkvjiiy9i7NixMWbMmGjfvn2MGjUqGjRoUOax3bp1i4MOOiiuu+66mDFjRnTv3j3Gjx8f77//fkSUPznu1q1b3HzzzXHJJZdEp06dYr311otevXqV+fyrrroq9ttvv9hrr73isMMOizfffDNuvPHG+NWvfhVbbrnlqn0CoIgU03595pln4plnnomIiOnTp8e8efPikksuiYiI3XffPXbffffKfvhQlIpl3/7nP/+JY445JtZdd93Yc889V/jmeJdddomNN964kh89FJdi2a/jxo2L/v37x8EHHxybbrppLFq0KJ599tl46KGHYvvtt/eTclmjFMu+3WuvvVZ4bNlAq0ePHobJNSVPjRk6dGg+Ipb/qlevXn6DDTbI9+nTJz948OD8119/vcIxF1xwQf6H/9nmzZuX/+1vf5tv0aJFvkmTJvl+/frl33vvvXxE5C+//PIVrvfJJ58sf2zatGn5fffdN9+0adN8ROR79OhR7roffvjhfNeuXfP169fPt23bNj9w4MD8okWLVvnzAMWgGPfrsuuv7NcFF1xQyKcDikKx7dsfrveHv4YOHVropwRqrWLbrx9++GH+mGOOyW+88cb5hg0b5hs0aJDfeuut8xdccEF+7ty5BX8+oBgU275NfQwvvfRSpY6j6uTy+Xy+yidd1LjXXnsttt1227j77ruTt0sCNc9+heJj30LxsF+h+Ni3aw7vKbUaWLBgwQqPXXfddVGnTh0vz4Faxn6F4mPfQvGwX6H42LdrNu8ptRq48sor4+WXX4499tgjSkpKYvTo0TF69Og48cQTY6ONNqrp5QHfYb9C8bFvoXjYr1B87Ns1m5fvrQbGjBkTF154Ybz99tsxd+7caNeuXRx99NFx3nnnRUmJuSPUJvYrFB/7FoqH/QrFx75dsxlKAQAAAJA57ykFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgcxV+K/tcLled6wBWopCfQ2DPQvbsWSgu9iwUF3sWiktF9qw7pQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMldS0wug5nXr1i3ZTznllGQ/5phjkv2uu+5K9htuuCHZX3nllWQHAAAAio87pQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMpfL5/P5Cj0xl6vutVBNunbtmuxjx45N9mbNmlXhalY0e/bsZF933XWr9fq1WQW350rZs6yqgQMHJvuFF16Y7HXqlP3vHT179kweO378+GSv7exZVqZp06bJ3qRJk2Tfd999k71Vq1bJfu211yb7woULk311Zs/WTptttlmyr7XWWsm+++67J/uQIUOSvbS0NNlr2siRI8tshx12WPLYRYsWVfVyMmXPUoz23HPPMts999yTPLZHjx7J/t57763SmrJSkT3rTikAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMyV1PQCKNyOO+6Y7A8++GCyN2/ePNnz+Xyyz5kzJ9kXLVqU7Ouuu26yd+/evcz2yiuvFHRtWBMdd9xxyX722Wcne2lp6Spfu7w/T6A26tChQ7KXt2d23nnnZO/cuXNll1QprVu3Tvb+/ftX6/VZ82y99dbJXt7XoUMOOSTZ69RJ/7t6mzZtkr28r2O1/WvVfvvtV2b785//nDz29NNPT/avv/56VZZEBey+++7JXt7fiR5++OGqXA4Z2mGHHcpsL730UoYrqZ3cKQUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMldT0Aoho1KhRsm+33XbJfvfddyd7eT8KulAffPBBsl955ZXJft999yX7888/X2YbOHBg8tg//vGPyQ5rovbt2yd7gwYNMloJZGOLLbZI9vJ+RPqRRx6Z7A0bNkz2XC6X7J999lmyz5kzJ9m33HLLZP/5z3+e7EOGDCmzvfvuu8ljYWXK+/6rb9++Ga1kzXPMMcck++23357sqe+7KUzPnj2TfdNNN032hx9+uApXQ1WqUyd9r0/Hjh3LbOV9X17e9xCrA3dKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkrqSmF0DEX/7yl2Q//PDDM1rJqtluu+2SvUmTJsk+fvz4ZO/Zs2eZrUuXLsljYU3Uu3fvZD/11FMLOv+7776b7D/96U/LbF988UVB12bN1Lx582S/4oorkv3QQw9N9qZNm1Z6TZXxwQcfJPvee++d7GuttVayl7cnW7ZsWVCHyhozZkyy9+3bt6Dz/+9//0v222+/Pdnr1En/u3xpaWml1/Rdu+yyS7L36NGjoPNTnI455phk/9e//pXRSqhqrVu3TvYTTjihzHb33Xcnjy3va/zqwJ1SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZK6npBawJunXrluz77rtvsudyuYKuP378+GR/5JFHkv3qq69O9qlTpyb7q6++muwzZ85M9l69epXZCv3cQDHaddddk33o0KHJ3rx584Kuf9VVVyX7pEmTCjo//NABBxyQ7L/61a8yWsnKffTRR8nep0+fZP/ss8+SvVOnTpVeE9Skm2++OdlHjBhR0PkXL16c7NOmTSvo/IVq1qxZsr/55pvJ3qZNm1W+dnmf2wkTJqzyuSlMnTruB1ld3Xbbbat87AcffFCFKylOdgYAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOZKanoBq4OuXbsm+5gxY5K9WbNmyZ7P55N99OjRyX744Ycne48ePZJ94MCByX7bbbcl+/Tp05P99ddfT/bS0tIy27777ps8drvttkv2V155JdmhNjr22GOTvU2bNgWdf9y4ccl+1113FXR+qKxDDjmkWs8/ceLEZH/ppZeS/eyzz072zz77rLJL+p4tt9yyoOMha0uWLEn2QvdEbbf33nsn+zrrrFNt1548eXKyL1y4sNquvabr0qVLsq+//voZrYSsNW/efJWPLW9WsCZwpxQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOZKanoBxWCzzTZL9rPOOivZmzdvnuxffvllsn/++efJfueddyb73Llzk/0f//hHQb0mNWzYMNkHDBiQ7EceeWRVLgeqRMuWLZP9F7/4RbKXlpYm+6xZs5L9kksuSXbI2gknnJDsJ554YrI/+eSTyf7hhx8m+//+979kr27rr79+jV4f+L7DDjss2cv7M6u8718Lcf7551fbuUnr27dvslfnf3eqV3lfhzt27LjK554yZcoqH7u6cKcUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmSmp6AbVB/fr1k/3qq69O9r59+yb7nDlzkv2YY45J9gkTJiR7w4YNk31N1q5du5peAqygQ4cOyf7ggw9W6/VvuOGGZH/66aer9fpQWVOnTk32QYMGZbOQGrLzzjvX9BJgtXLkkUcm+znnnJPsnTp1Sva11lqr0muqjNdee63Mtnjx4mq9NmXbfPPNCzr+rbfeqqKVUNXKmwesv/76yf7++++X2cqbFawJ3CkFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJC5kppeQG2w7bbbJnvfvn0LOv/++++f7OPHjy/o/EBx2WeffZK9S5cuBZ3/qaeeSvbBgwcXdH5Y0/Tv3z/ZGzduXK3X/9GPflTQ8S+88EKy/+tf/yro/PBDHTp0SPajjz462Xv37l2Fq1nRrrvumuz5fL5ar//1118n+znnnJPsjz32WJltwYIFq7Qmat5LL71U00soWs2aNUv28r73Puqoo5J9r732qvSavuviiy8us82aNaugc68O3CkFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJC5kppeQG1w7bXXJnsul0v28ePHF9RJq1On7NlpaWlphiuBiunXr1+yX3755QWd/7nnnkv2Y489Ntlnz55d0PWhtmnUqFGyb7XVVsl+wQUXJHvfvn0rvabvSn0diyj8a9nUqVOT/fjjj0/2pUuXFnR91jydO3dO9lGjRiV7u3btqnI5RefZZ59N9ltuuSWjlVCbtGjRosauvc022yR7eX8f7t27d7K3bds22evVq5fsRx55ZLKX93V2wYIFyf7vf/872RcuXJjsJSXpscrLL7+c7Gs6d0oBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGSupKYXkJWf/vSnZbauXbsmj83n88k+atSoVVkSFVRaWlpmK++/zWuvvVbFq4GIDh06JPuDDz5Yrdf/+OOPk/2LL76o1utDVVtrrbWSfdttt0328vZc69atk33BggXJPnXq1GT/17/+lez77LNPsjdq1CjZy1NSkv527sADD0z2wYMHl9kWLVq0SmtizZbL5Qrq1a1OnfS/y6e+96wKqb+XRET85Cc/SfbRo0dX5XKoIuV9LSnv7y1//vOfk/3cc8+t9JoqqkuXLsle3p5dsmRJss+fPz/Z33777WS/4447kn3ChAnJPn78+GQv73vnyZMnJ3vDhg2T/d133032NZ07pQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMldS0wvISsOGDcts9erVSx77v//9L9nvv//+VVrTmqJ+/frJPmjQoFU+99ixY5P997///SqfG8py9tlnJ3tpaWm1Xv/yyy+v1vNDVSvv6+w+++yT7A899FBB17/wwguTvbyvJc8//3yyt2jRoqDzd+7cOdnL06pVq2T/4x//mOyffvppmW3EiBHJYxcuXJjsrJ7efPPNZO/Zs2eyH3XUUcn+xBNPJPs333yT7NXtl7/8ZbKfeuqpGa2E2uTkk09O9kmTJiX7LrvsUpXLqZTU14GI8r8WvPPOO8n+4osvVnZJmTrxxBOTvbyvsx9//HFVLmeN404pAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMldT0AorBwoULk/3zzz/PaCW1U/369ZN94MCByX7WWWcl++TJk8ts11xzTfLYuXPnJjusTNeuXZN9r732qtbrjxw5Mtnfe++9ar0+rIq11lqrzHbhhRcmjy3v60B5Ro8enew33HBDss+aNSvZW7VqleyPPfZYsv/oRz9K9kWLFiX7lVdemeydO3dO9v333z/Z77nnnjLbP//5z+SxV1xxRbLPnDkz2cvz2muvFXQ8NWPSpEnJfumll2a0kuoxaNCgZD/11FOzWQhFpbw/L6k5e+65Z0HHP/jgg1W0kjWTO6UAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJXUtMLKAajRo2q6SXUqK5duyb7WWedleyHHnposo8cOTLZDzrooGSHqvbkk08m+zrrrFPQ+V988cVkP+644wo6P1SHunXrJvvFF19cZjvzzDOTx86bNy/ZzznnnGS/7777kn3WrFnJvv322yf7jTfemOzbbrttsn/wwQfJftJJJyX7008/nezNmjVL9l122SXZjzzyyDLbfvvtlzx2zJgxyV6ezz77LNk7duxY0PmhOuy99941vQSgFnn44YdreglFzZ1SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZK6npBWQll8utUouI6NevX7Kfdtppq7KkWuOMM85I9j/84Q/J3rx582S/5557kv2YY45Jdsjauuuum+ylpaUFnX/IkCHJPnfu3ILOD9XhxBNPTPYzzzyzzDZ//vzksb/+9a+T/cknn0z27t27J/vxxx+f7D/5yU+SvWHDhsl+0UUXJfvQoUOT/bPPPkv28nz99dfJ/vjjj69yP/zww5PHHnHEEclenvK+B6H6rLXWWmW2vfbaK3ns2LFjk33BggWrtKbaorw/MwYPHpzRSgBWf+6UAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADIXElNLyAr+Xx+lVpExAYbbJDs119/fbLfcccdyT5jxoxk7969e7IfffTRyb7NNtske9u2bZP9008/TfYnnngi2YcMGZLskLWhQ4cme5061Tuvf+GFF6r1/FAdzj///FU+tm7dusl+1llnJfugQYOSvVOnTpVdUqWUd/0//vGPyb506dIqXE22/vrXvxbUqTm77rprsp933nlltj59+iSP7dixY7J/9tlnyV7dWrRokex9+/ZN9muvvTbZGzVqVOk1fdeCBQuS/Ztvvino/EDVyuVyyb7ZZpsl+4svvliVy1ntuFMKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmSup6QUUg/J+lPXJJ5+c7AcddFCyf/3118m+6aabJnuhyvvx9E8//XSyF/JjwqE6dO3aNdl79+6d7KWlpcm+aNGiZL/pppuS/Ysvvkh2qI2mTZuW7K1atSqz1a9fP3nsNttss0prWuaxxx5L9meeeSbZR4wYkewTJ05M9qVLlyY71IQbb7wx2Tt37rzK5/7d736X7HPmzFnlc1eFPn36JPt2222X7Pl8vqDrjxs3LtlvvvnmZC/ve28gW+X9mVCnjnt9CuGzBwAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQuZKaXkBW/vWvf5XZXnrppeSxO+ywQ0HX3mCDDZJ9/fXXL+j8M2bMSPb77rsv2U877bSCrg+1zdprr53s5e3J8kyZMiXZzzzzzILOD7XR7rvvnuz9+vUrs2233XbJY//3v/8l+x133JHsM2fOTPZFixYlO1A5J510Uk0voVqV92fSI488kuzlfW/9zTffVHpNQO218847J/uwYcOyWUiRcqcUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmSmp6AVmZPHlyme3AAw9MHvvrX/862QcOHLhKa6qowYMHJ/vNN9+c7B9++GFVLgeANdCcOXOSffjw4avUgOpx3HHHJfupp55aZjv22GOreDVV66OPPkr2+fPnJ/uzzz6b7Lfcckuyv/nmm8kOrF5yuVxNL2G15k4pAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDM5fL5fL5CT8zlqnstwA9UcHuu1Jq8ZzfYYINkv//++5N91113TfZPPvkk2Tt16pTsrL7sWSgua/KerV+/fpntuOOOSx57ySWXJPs666yT7CNGjEj2MWPGJPvIkSOTfdq0aclO8VqT9yzVp7w/8+64445kv/XWW5P917/+dWWXtNqoyJ51pxQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOZy+Xw+X6En5nLVvRbgByq4PVfKnoXs2bNQXOxZKC72LBSXiuxZd0oBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZC6Xz+fzNb0IAAAAANYs7pQCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlCoygwYNilwut0rHDhs2LHK5XEycOLFqFwWUyZ6F4mLPQnGxZ6G42LP8kKFUDVq2qZb9atCgQbRp0yb23nvvuP7662POnDnVvoYhQ4bEsGHDKvz8Dh06fG/Ny3795je/qb5FQi1RjHs2ImLOnDnxu9/9Ljp27Bj169ePDTfcMA4++OCYP39+9SwSaoli27Pjxo1b6dfYZb8uvfTS6l0s1LBi27MREd9880388Y9/jK222ioaNWoUG264YRxyyCHx1ltvVd8ioZYoxj07d+7cOP3006Nt27ZRv3792HLLLePmm2+uvgVSrlw+n8/X9CLWVMOGDYvjjz8+LrrooujYsWMsXrw4pk2bFuPGjYsxY8ZEu3btYtSoUdGlS5flxyxZsiSWLFkSDRo0qPT1li5dGosXL4769esvn0537tw5WrZsGePGjavQOTp06BDrrLNODBgw4HuPb7bZZrHjjjtWek1QTIpxz86ePTt69OgRkydPjhNPPDE6deoU06dPj2effTaGDx8e66yzTqXXBcWi2PbsF198EWPGjFnh8eHDh8eTTz4Z//nPf2KHHXao9LqgWBTbno2IOOigg2LUqFFxwgknxHbbbRdTp06Nm266KRYsWBBvvPFGtG/fvtLrgmJRbHt26dKlsfvuu8eECRPit7/9bWy66abxxBNPxMiRI+PSSy+Nc889t9JrogrkqTFDhw7NR0T+pZdeWqE99dRT+YYNG+bbt2+fnz9/frWtYeutt8736NGjws9v3759ft9996229UBtVox79qSTTsqvvfba+Y8//rja1gS1VTHu2ZXp1KlTftNNN62aBUEtVmx7dvLkyfmIyJ955pnfe3zs2LH5iMhfe+211bBCqD2Kbc8+8MAD+YjI33777d97/KCDDso3aNAg/8UXX1TDCimPl+/VUr169Yo//OEPMWnSpLj77ruXP76y1+AuWLAg+vfvHy1btoymTZvGfvvtF1OmTIlcLheDBg1a/rwfvga3Q4cO8dZbb8X48eOX33LZs2fPCq1v0aJFMW/evEI/TFht1MY9O2vWrBg6dGiceOKJ0bFjx1i0aFEsXLiwKj9sKFq1cc+uzH/+85/48MMP48gjj1zVDxVWC7Vxzy57adL666//vcdbt24dERENGzYs4COG4lYb9+yzzz4bERGHHXbY9x4/7LDD4ptvvomRI0cW9kGzSgylarGjjz46IiKefPLJ5POOO+64uOGGG6Jv375xxRVXRMOGDWPfffct9/zXXXddtG3bNrbYYosYPnx4DB8+PM4777xyjxs7dmw0atQomjRpEh06dIjBgwdX7AOC1Vxt27PPPfdcfPPNN9GpU6c4+OCDo1GjRtGwYcP48Y9/HK+99lqlPjZYHdW2Pbsy99xzT0SEoRRE7duzm2yySbRt2zauueaaeOSRR2Ly5Mnxn//8J37zm99Ex44dV/iLL6xpatueXbhwYdStWzfq1av3vccbNWoUEREvv/xyudek6pXU9AIoW9u2baN58+bx0UcflfmcV155JR544IE4/fTT409/+lNERJx88slx/PHHx+uvv548f79+/WLgwIHRsmXLOOqooyq0pi5dusSuu+4am2++ecyYMSOGDRsWp59+ekydOjWuuOKKin9wsBqqbXv2gw8+iIiI3//+97HJJpvEXXfdFbNnz44LL7wwevXqFW+99dbyf82FNVFt27M/tHTp0rj//vtjxx13jE6dOlX6eFjd1LY9u9Zaa8WDDz4YRxxxROy3337LH+/WrVu88MILsfbaa1fsA4PVVG3bs5tvvnksXbo0Xnzxxdh1112XP77sDqopU6ZU5MOiirlTqpZr0qRJ8qcWPP744xHx7cb9rlNPPbVa1jNq1Kj43e9+F/vvv3/84he/iPHjx8fee+8d1157bUyePLlargnFpDbt2blz50ZERC6Xi6eeeiqOOOKIOOmkk2LEiBExc+bMuOmmm6r8mlBsatOe/aGnnnoqvvjiC3dJwXfUtj27zjrrRNeuXeOcc86JESNGxNVXXx0TJ06MQw45JL755ptquSYUk9q0Z4844oho3rx5/OIXv4gxY8bExIkT45ZbbokhQ4ZExLcvIyR7hlK13Ny5c6Np06Zl9kmTJkWdOnWiY8eO33s8q39RzeVyccYZZ8SSJUsq/FNKYHVWm/bssvey+NnPfhZNmjRZ/nj37t2jY8eO8cILL1T5NaHY1KY9+0P33HNP1K1bNw499NBqvxYUi9q0Z2fPnh277bZb7LzzzvHHP/4x9t9//xgwYEA8+OCD8dxzz8XQoUOr/JpQbGrTnt1ggw1i1KhRsXDhwthrr72iY8eOcdZZZ8UNN9wQEfG975fJjqFULTZ58uSYPXt2rb9lf6ONNoqIiK+++qqGVwI1q7bt2TZt2kTEim/AGhGx3nrrxcyZM7NeEtQqtW3PfteCBQvi4Ycfjt69e690D8OaqLbt2QcffDC++OKL7710LyKiR48e0axZs3j++edraGVQO9S2PRsRsfvuu8fHH38cr776ajz33HMxZcqU6N69e0REbLbZZjW8ujWToVQtNnz48IiI2Hvvvct8Tvv27aO0tDQ++eST7z3+4YcfVugaP/zJB6vi448/joiIVq1aFXwuKGa1bc9269YtIlb++vipU6fas6zxatue/a5Ro0bFnDlzvHQPvqO27dkvvvgiIr59/7fvyufzsXTp0liyZEmFzwWro9q2Z5epW7dudO3aNX784x9HkyZN4p///GdERPTu3bvS56JwhlK11NixY+Piiy+Ojh07Jr8hXbbBl70OdplltyCWp3HjxjFr1qwKPferr75a4Yvu4sWL4/LLL4969erFHnvsUaHzwOqoNu7ZzTffPLbZZpsYOXJkfPnll8sff/LJJ+Ozzz6LPn36VOg8sDqqjXv2u+69995o1KhRHHDAAZU+FlZHtXHPLrur4r777vve46NGjYp58+bFtttuW6HzwOqoNu7ZlZk+fXpcccUV0aVLF0OpGuKn79UCo0ePjnfffTeWLFkSX3zxRYwdOzbGjBkT7du3j1GjRkWDBg3KPLZbt25x0EEHxXXXXRczZsyI7t27x/jx4+P999+PiPInx926dYubb745LrnkkujUqVOst9560atXr5U+d9SoUXHJJZfEwQcfHB07doyvvvoq7r333njzzTfjsssuiw022GDVPwlQRIplz0ZE/OlPf4o+ffrErrvuGr/+9a9j9uzZce2118Zmm20WJ5100qp9AqDIFNOejfj2H4FGjx4dBx10kPe3YI1ULHv2Zz/7WWy99dZx0UUXxaRJk6J79+7x4Ycfxo033hitW7eOX/7yl6v+SYAiUix7NuLbl9fuvPPO0alTp5g2bVrccsstMXfu3Hj00UejTh337NSIPDVm6NCh+YhY/qtevXr5DTbYIN+nT5/84MGD819//fUKx1xwwQX5H/5nmzdvXv63v/1tvkWLFvkmTZrk+/Xrl3/vvffyEZG//PLLV7jeJ598svyxadOm5ffdd99806ZN8xGR79GjR5nrnTBhQv5nP/tZfsMNN8zXq1cv36RJk/yuu+6af+CBBwr+XEAxKLY9u8yYMWPy3bt3zzdo0CDfokWL/NFHH53//PPPV/nzAMWiWPfsn//853xE5EeNGrXKHzsUo2Lcs1999VX+jDPOyG+22Wb5+vXr51u2bJk/7LDD8h9//HFBnwsoBsW4Z88444z8xhtvnK9fv36+VatW+SOOOCL/0UcfFfR5oDC5fD6fr96xFzXhtddei2233Tbuvvtu70cBRcCeheJiz0JxsWehuNizaw73p60GFixYsMJj1113XdSpUyd23333GlgRkGLPQnGxZ6G42LNQXOzZNZv3lFoNXHnllfHyyy/HHnvsESUlJTF69OgYPXp0nHjiibHRRhvV9PKAH7BnobjYs1Bc7FkoLvbsms3L91YDY8aMiQsvvDDefvvtmDt3brRr1y6OPvroOO+886KkxNwRaht7FoqLPQvFxZ6F4mLPrtkMpQAAAADInPeUAgAAACBzhlIAAAAAZM5QCgAAAIDMVfhdw3K5XHWuA1iJQt7yzZ6F7NmzUFzsWSgu9iwUl4rsWXdKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZK6kphcAQNXabLPNkv3xxx9P9rp16yZ7+/btK70mAACAH3KnFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5kpqegEAVM4NN9yQ7Iceemiyt2jRItkfffTRSq8JAACgstwpBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQuVw+n89X6Im5XHWvBfiBCm7PlbJna6/1118/2R966KFk7969e7KX9/vmzTffTPY999wz2WfMmJHsazJ7FoqLPQvFxZ6F4lKRPetOKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzJXU9AIoX926dZO9efPm1Xr9U045JdkbNWqU7Jtvvnmy//a3v032q6++usx2+OGHJ4/95ptvkv3yyy9P9gsvvDDZYWU222yzZE/9no6I2GmnnQq6/u9///tknzBhQrLPmDGjoOsDAKuucePGyT5u3LgyW5s2bZLH/vjHP072iRMnJjtAVXOnFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5kpqegHFoF27dsler169ZN9ll12Sfdddd032tddeO9kPOuigZK9pkydPTvbrr78+2Q844IAy25w5c5LHvv7668k+fvz4ZIdV0aJFi2Tv27dvtV6/vD339NNPV+v1AaCYtWnTJtlbtWpV0PlnzpyZ7HvssUeyd+vWrcz23nvvJY+dMWNGsgNkzZ1SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyFxJTS+gNujatWuyjx07NtmbN29ehaspPqWlpck+cODAZJ87d26y33PPPWW2zz//PHlseT9yt7wfmwsrs9lmmyX7vffem+y5XK6g6x944IHJPnLkyILOD1StAQMGJHu9evWSfcstt0z2I488stJr+q533323zLb11lsXdG5YFZ07d072/v37J3v79u0Lun55X+fbtWtX0Pkvv/zyZN9qq62SPfV9xJQpU5LHlvfnDVSHnXbaKdmPOuqoZO/Ro0eyF/q16swzz0z2qVOnJvuuu+6a7HfffXeZ7d///nfy2DWBO6UAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJXUtMLqA0+/fTTZJ8xY0ayN2/evCqXU+X+/e9/J/usWbOSfY899kj2RYsWJfvw4cOTHYrN0Ucfnezt2rVL9sceeyzZf/Ob3yT7lClTkh2onB49eiR7586dCzr+gAMOSPZcLpfs5cnn8wUdv+mmm5bZ3n777eSxW221VUHXhpXp1atXsv/yl7+s1usvXLgw2e++++5kL2/955xzTqXX9F2pPT9s2LDkseX9vQZWxaGHHprsgwcPTvaWLVsme3lfJ8eNG5fsrVq1Svarrroq2ctT3vpS1z/ssMMKuvbqwJ1SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZK6npBdQGX331VbKfddZZyf7Tn/402V999dVkv/7665O9PK+99lqy9+nTJ9nnzZuX7FtvvXWyn3baackOxeaFF15I9q5duyb7xIkTk/2MM85I9ilTpiQ7rG5at26d7H/961+TfeONNy7o+s2bN0/2xo0bJ3sul0v2l19+Odm32267ZK9udeqU/W+U5X3ssCoGDRqU7OV9712eO++8M9mnT5+e7FdffXVBx5f3fcITTzyR7C1btlzl6//9739PHgsrU1KSHgtsv/32yX7rrbcme6NGjZL9mWeeSfaLL7442Z977rlkr1+/frI/8MADyb7XXnsle3kmTJhQ0PGrO3dKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkrqSmF1AMRowYkexjx45N9jlz5iT7Nttsk+y//OUvk/3qq69O9nnz5iV7ed56661kP/HEEws6P2Rt//33T/addtop2fP5fLL/7W9/S/Zvvvkm2WF107t372S/9dZbk32jjTaqyuVUua222irZv/zyy2Rv2bJlsrdp0ybZhw4dmuxt27ZN9pS33357lY+FsjRu3DjZGzZsmOyTJk1K9vPOOy/ZP//882QvT6dOnZL93HPPTfZWrVole3nfuw8aNKjM5nsMVsVRRx2V7LfddltB5x8zZkyyH3roocn+9ddfF3T98s6/1157FXT+yZMnJ/udd95Z0PlXd+6UAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADIXElNL2B18PXXXxd0/OzZsws6/oQTTkj2+++/P9lLS0sLuj7UNmuvvXay77bbbtV6/ZkzZyb75MmTq/X65TnttNOSfaONNiro/GeeeWZBx7P6+d3vfpfshf6eK8/ChQuT/eyzz072F198Mdnfe++9Sq/pu2bMmJHs5e3Ztm3bFnT9iRMnltmOPvrogs4NK/P3v/892ffZZ59k32qrrZL98ssvT/aTTz452Zs3b57s1157bbLvu+++yf7VV18l+6WXXprsN998c7LDD1188cXJfu655yZ7Pp9P9iFDhiT7wIEDk73Qv0+X57zzzqvW8/fv3z/Zp0+fXq3XL3bulAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyFxJTS+AiEGDBiV7t27dkr1Hjx7J3rt372R/8sknkx2KzdKlS5O9vD1Vp056Xl9aWprszzzzTLIX6owzzijo+FNPPTXZ27dvX9D5BwwYUGZr27Zt8tgpU6YUdG1qzl577VVm6969e7Ve+9NPP032o48+Otmff/75qlxOlStv3xRq5MiRZbYvv/yyWq/Nmum1115L9hdffDHZt9pqq2Tv1atXsvfp0yfZ//SnPyV7u3btkr08F154YbLfcMMNBZ2fNc/555+f7Oeee26yL1q0KNmfeOKJZD/77LOTfcGCBclengYNGiR76nuQiPL3bC6XS/ZLLrkk2VNfRymfO6UAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJXUtMLIGLevHnJfsIJJyT7K6+8kuy33nprsj/99NPJPmHChGS/6aabkj2fzyc7VLUePXok+2677ZbspaWlyf7pp58m+5dffpns5enatWuyl7f+/fbbr6Drl/dn0uTJk5N98803L7P9/e9/Tx572GGHJfukSZOSnZozYMCAMlujRo0KOvcLL7yQ7BdeeGGyP//88wVdv1DrrLNOsu+zzz7Jvvvuuxd0/fI+f4899lhB54fKWrhwYbJ//fXXBZ2/TZs2yf7ggw8mey6XS/byvre9/fbbk33EiBHJDiuz9tprl9lOPvnk5LHl/Z594oknkr1fv37JXqhOnTol+z333JPs3bp1K+j65X1/euWVVxZ0ftLcKQUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkLmSml4A5fvoo4+S/bjjjkv2oUOHJvvRRx9dUG/cuHGy33XXXcn++eefJzv8UNOmTZO9Y8eOBZ1/6tSpyT58+PBk//DDD5N9s802S/azzjor2ffff/9k//LLL5P9ySefTPZrrrkm2Zs3b57sY8eOXeVjKV633HJLma1ly5bJY2fPnp3sRxxxRLJPmzYt2Wvab37zm2S/+OKLCzr/W2+9lew///nPk722f/5Y80yaNKmml5D02GOPJfvVV1+d7J999llVLoc1RL169cps5X2dLU///v2Tfb311kv2448/Ptn322+/ZO/cuXOyN2nSJNnz+XxB/e677072efPmJTuFcacUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmcvl8Pl+hJ+Zy1b0Wqknnzp2T/dprr032Pffcs6Dr/+Uvf0n2Sy+9NNmnTJlS0PWLWQW350qtznv2Jz/5SbI/8sgjBZ3/oosuKqivv/76yX7rrbcme9++fZN97ty5yT58+PBkP/PMM5N90003Tfa//e1vyd66desyW3lrO/XUU5O9trNn10w/+9nPkv2BBx5I9rXWWivZlyxZkuxnnHFGst98883JviazZ2tG3bp1k/2+++5L9oMOOqgql7OCf/zjH8le3p6n+qzJe3bttdcus73zzjvJY1u1apXs5X1uCvm8V8TUqVOTvbz1pb73jIiYPn16Qcez6irye8edUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMicoRQAAAAAmSup6QVQ/d58881k//nPf57sP/vZz5J96NChyf7rX/862TfddNNk79OnT7Kz5unSpUu1nv+iiy4q6PiHHnoo2XfaaaeCzr///vsn+/jx45O9e/fuyf7cc89Vek3fdd1115XZzjzzzILODbXRiBEjkj2fzxd0/v79+yf7LbfcUtD5IWv33Xdfsh944IHJXuieKk91nx9WxaxZs8ps/fr1Sx776KOPJnuLFi2S/aOPPkr2kSNHJvuwYcOS/auvvkr28v7MaN26dUHHU7PcKQUAAABA5gylAAAAAMicoRQAAAAAmTOUAgAAACBzhlIAAAAAZM5QCgAAAIDMGUoBAAAAkLmSml4ANW/WrFnJPnz48GS/7bbbkr2kJP3bbPfdd0/2nj17ltnGjRuXPJbV09prr53suVwu2UeOHFnQ9bt27ZrsHTp0SPby1jdgwIBkHz9+fLJvttlmyX7vvfcme6Hru+6665Idis1ll12W7HXqpP+Nr7S0tKDrl7fnIWtt2rRJ9uOPPz7ZDzrooGTP5/PJ/sorryT766+/nuzlrW+99dZLdqht/v3vfyd7q1atMlrJqinv74M9evRI9vK+zn788ceVXhPZcacUAAAAAJkzlAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmSmp6AVS/Ll26JPvBBx+c7DvssEOyl5QU9tvo7bffTvZnnnmmoPOz5snn8wX1QpWWlhZ0/fL27KeffprsDRo0SPZPPvkk2Xfbbbdknz17drJDsalXr16yb7vttsle6J4/7bTTkv2DDz5IdsjannvumewXXXRRQecfOHBgst94443J3q9fv2Q//vjjk728702BqtWwYcNkL/Tr7H333VfpNZEdd0oBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADInKEUAAAAAJkzlAIAAAAgc4ZSAAAAAGSupKYXQPk233zzZD/llFOS/cADD0z2DTbYoNJrqoylS5cm++eff57spaWlVbkcVgMjR45M9rPOOivZ999//2Tv3r17snft2jXZmzZtmuzlOeaYY5I9l8sl+5dffpnsgwYNSvYpU6YkOxSbRo0aJftRRx2V7H369Cno+n/961+T/Z577kl2XwfJWs+ePZP9+uuvL+j8++23X7L/85//TPbyvnc9//zzK72m75o4cWJBxwOV88QTT9T0EqhB7pQCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMhcSU0vYE2wwQYbJPvhhx+e7Kecckqyd+jQobJLqlITJkxI9ksvvTTZR40aVZXLYQ2wePHiZJ8/f36yN2rUKNmff/75ZM/n88le3ebMmZPsDzzwQLKPHj26KpcDNa5p06bJfuuttyb7wQcfXND1zzjjjGS/8cYbk720tLSg60NV69OnT7I3b9482cePH5/sjz76aLKvtdZayf7Tn/402ctbXy6XS/bp06cnO1C19t5775peAjXInVIAAAAAZM5QCgAAAIDMGUoBAAAAkDlDKQAAAAAyZygFAAAAQOYMpQAAAADIXElNL6AYrL/++sm+1VZbJXt5Pwp6iy22qPSaqtK///3vZL/qqquSfeTIkcnuR11T1V5++eVkP/zww5P9//7v/5K9Z8+elV1Spdx5553J/sYbbyT7q6++muzl/ShuWN1suOGGyX7wwQcXdP6PPvoo2a+//vqCzg+1TXnfu+Xz+YL6Wmutlez9+vVL9sGDByf7zJkzk/22225L9ptvvjnZgaq18cYb1/QSqEHulAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyFxJTS8gKy1atCiz/eUvf0ke27Vr12TfeOONV2VJVeaFF15I9muuuSbZn3jiiWRfsGBBpdcENekf//hHQR2oXbbYYotkHzBgQEHnf//995P9Jz/5SUHnh2Kz3nrrFXT89OnTk33MmDHJvttuuxV0/eOPPz7ZH3nkkYLOD1StZ599Ntnr1EnfS1NaWlqVyyFj7pQCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADJnKAUAAABA5gylAAAAAMhcSU0voKJ22mmnZD/rrLOSfccddyyzbbjhhqu0pqoyf/78ZL/++uuT/bLLLkv2efPmVXpNAFBb/OEPf0j2Qw89tKDz33DDDck+adKkgs4Pxeadd94p6PiDDz442XO5XLJ/9dVXyX7TTTcl+z//+c9kB2qXN998M9k/+OCDZN94442TfZNNNkn26dOnJzvVy51SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZK6npBVTUAQccUFAvxNtvv53sjz76aLIvWbIk2a+55ppknzVrVrIDQDHbeuutk71Zs2YFnf+WW25J9rFjxxZ0fljd3Hnnncler169ZP/DH/6Q7BMmTEj2UaNGJfuf/vSnZAdWL5dddlmy33bbbcl+6aWXJvupp56a7OXNAyiMO6UAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMwZSgEAAACQOUMpAAAAADKXy+fz+Qo9MZer7rUAP1DB7blS9ixkz55dNVdccUWyDxgwINknTZqU7H379k329957L9lZfdmzUFzs2TVTs2bNkv2BBx5I9t69eyf7Qw89lOzHH398ss+bNy/Z12QV2bPulAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyFwun8/nK/TEXK661wL8QAW350rZs5A9e3bV7Lnnnsn+xBNPJPtBBx2U7CNHjqz0mlgz2LNQXOxZVqZZs2bJfumllyb7SSedlOxdunRJ9rfffjvZ12QV2bPulAIAAAAgc4ZSAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyFwun8/nK/TEXK661wL8QAW350rZs5A9exaKiz0LxcWeheJSkT3rTikAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABkzlAKAAAAgMzl8vl8vqYXAQAAAMCaxZ1SAAAAAGTOUAoAAACAzBlKAQAAAJA5QykAAAAAMmcoBQAAAEDmDKUAAAAAyJyhFAAAAACZM5QCAAAAIHOGUgAAAABk7v8DXN0LdF0Z5IEAAAAASUVORK5CYII=\n" }, "metadata": {} } ] } ] }