{ "cells": [ { "cell_type": "markdown", "id": "f8728e4d-0905-400b-a07f-c73f476311a3", "metadata": {}, "source": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "Supplementary code for the Build a Reasoning Model (From Scratch) book by Sebastian Raschka
\n", "
Code repository: https://github.com/rasbt/reasoning-from-scratch\n", "\n", "		\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "7f5813d3-dc84-4fe0-9b57-2fb5d01f0a04", "metadata": {}, "source": [ "# Chapter 4: Improving Reasoning with Inference-Time Scaling" ] }, { "cell_type": "markdown", "id": "780b0404-b939-4983-b5ab-57f994d5559b", "metadata": {}, "source": [ "Packages that are being used in this notebook:" ] }, { "cell_type": "code", "execution_count": 1, "id": "d9221d09-75c3-4c73-8cfc-c20d111089e8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "matplotlib version: 3.10.8\n", "reasoning_from_scratch version: 0.1.13\n", "torch version: 2.9.1\n", "tokenizers version: 0.22.2\n" ] } ], "source": [ "from importlib.metadata import version\n", "\n", "used_libraries = [\n", " \"matplotlib\",\n", " \"reasoning_from_scratch\",\n", " \"torch\",\n", " \"tokenizers\" # Used by reasoning_from_scratch\n", "]\n", "\n", "for lib in used_libraries:\n", " print(f\"{lib} version: {version(lib)}\")" ] }, { "cell_type": "markdown", "id": "8ef62f83-9bf5-4941-b10d-b6d3f7444e6b", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "3488ce72-5305-4a9b-ba3a-30c201ceeae1", "metadata": {}, "source": [ " \n", "## 4.1 Introduction to inference-time scaling" ] }, { "cell_type": "markdown", "id": "71b91060-82bb-4f2c-b12a-d73b67714381", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "1d8101c3-49e1-4f47-8a8f-cc4880b975f4", "metadata": {}, "source": [ "- Plot above inspired by OpenAI o1 blog post, https://openai.com/index/learning-to-reason-with-llms/" ] }, { "cell_type": "markdown", "id": "dd1c5224-0fb2-4e0c-accb-7dd6ea72336f", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "66733262-b936-4a04-9c4a-d72c353f13f3", "metadata": {}, "source": [ "- All 3 methods more than double the accuracy of the base model on MATH-500 (the benchmark from the prev. chapter)\n", "- Methods 1 and 2 are covered in this chapter; method 3 is covered in the next chapter" ] }, { "cell_type": "markdown", "id": "669b2667-149b-4cf7-bf16-40756291ab1f", "metadata": {}, "source": [ " \n", "## 4.2 Loading a pre-trained model" ] }, { "cell_type": "markdown", "id": "f3b1de64-f400-440c-9b0c-7d06421fe9fe", "metadata": {}, "source": [ "- Model loading code similar to chapters 2 and 3" ] }, { "cell_type": "code", "execution_count": 2, "id": "07a828f1-1592-46c7-a89e-201682017449", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using Apple Silicon GPU (MPS)\n", "✓ qwen3/qwen3-0.6B-base.pth already up-to-date\n" ] } ], "source": [ "import torch\n", "from reasoning_from_scratch.ch02 import get_device\n", "from reasoning_from_scratch.ch03 import (\n", " load_model_and_tokenizer\n", ")\n", "\n", "device = get_device()\n", "\n", "# Use CPU for the first run of this chapter\n", "device = torch.device(\"cpu\")\n", "\n", "model, tokenizer = load_model_and_tokenizer(\n", " which_model=\"base\",\n", " device=device,\n", " use_compile=False\n", ")" ] }, { "cell_type": "markdown", "id": "40ca069d-bdb0-4f39-aa69-c705ab9e5042", "metadata": {}, "source": [ "- Let's try the model on a prompt from the MATH-500 dataset, which we worked with in the previous chapter:" ] }, { "cell_type": "code", "execution_count": 3, "id": "90809902-3568-48de-b283-bf5a12da4699", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "You are a helpful math assistant.\n", "Answer the question and write the final result on a new line as:\n", "\\boxed{ANSWER}\n", "\n", "Question:\n", "Half the value of $3x-9$ is $x+37$. What is the value of $x$?\n", "\n", "Answer:\n" ] } ], "source": [ "from reasoning_from_scratch.ch03 import render_prompt\n", "\n", "raw_prompt = (\n", " \"Half the value of $3x-9$ is $x+37$. \"\n", " \"What is the value of $x$?\"\n", ")\n", "prompt = render_prompt(raw_prompt)\n", "\n", "print(prompt)" ] }, { "cell_type": "markdown", "id": "c77fc106-1e57-4211-9eb3-fd7ae3c1261e", "metadata": {}, "source": [ "- The following function is based on `generate_text_stream_concat` from the previous chapter, but we modify it slightly to be able to pass in other `generate_*` functions later" ] }, { "cell_type": "code", "execution_count": 4, "id": "e80d7351-7ae5-4f16-b224-df4146735943", "metadata": {}, "outputs": [], "source": [ "from reasoning_from_scratch.ch02 import generate_text_basic_stream_cache\n", "\n", "\n", "def generate_text_stream_concat_flex(\n", " model, tokenizer, prompt, device, max_new_tokens,\n", " verbose=False, \n", " generate_func=None, # New\n", " **generate_kwargs # New\n", "):\n", "\n", " if generate_func is None: # New\n", " generate_func = generate_text_basic_stream_cache\n", " \n", " input_ids = torch.tensor(\n", " tokenizer.encode(prompt), device=device\n", " ).unsqueeze(0)\n", "\n", " generated_ids = []\n", " for token in generate_func( # New\n", " model=model,\n", " token_ids=input_ids,\n", " max_new_tokens=max_new_tokens,\n", " eos_token_id=tokenizer.eos_token_id,\n", " **generate_kwargs, # New\n", " ):\n", " next_token_id = token.squeeze(0)\n", " generated_ids.append(next_token_id.item())\n", "\n", " if verbose:\n", " print(\n", " tokenizer.decode(next_token_id.tolist()),\n", " end=\"\",\n", " flush=True\n", " )\n", " return tokenizer.decode(generated_ids)" ] }, { "cell_type": "code", "execution_count": 5, "id": "7e68cac3-fbc6-42c1-82c3-34410d2178c7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \\boxed{20}" ] } ], "source": [ "response = generate_text_stream_concat_flex(\n", " model, tokenizer, prompt, device,\n", " max_new_tokens=2048, verbose=True,\n", " generate_func=generate_text_basic_stream_cache # NEW\n", ")" ] }, { "cell_type": "markdown", "id": "7ef29231-2357-4235-a12d-6a9dbfb47036", "metadata": {}, "source": [ "- By the way, the correct answer is 83" ] }, { "cell_type": "markdown", "id": "ba3626af-a322-438f-a739-09841679b054", "metadata": {}, "source": [ " \n", "## 4.3 Generating better responses with chain-of-thought prompting" ] }, { "cell_type": "markdown", "id": "72b4e62f-1627-46eb-96ba-92bb5781170f", "metadata": {}, "source": [ "- A classic way to improve model performance is to modify the prompt to encourage the model to think or explain step by step" ] }, { "cell_type": "markdown", "id": "2d1b6b82-c3a8-4abf-8c77-408068b83dd1", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 6, "id": "50c541fe-8583-4a61-bb52-329f8bd60ba3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " To solve the problem, we need to find the value of \\( x \\) such that half the value of \\( 3x - 9 \\) is equal to \\( x + 37 \\).\n", "\n", "### Step 1: Set up the equation\n", "We are given that half the value of \\( 3x - 9 \\) is equal to \\( x + 37 \\). This can be written as:\n", "\\[\n", "\\frac{1}{2}(3x - 9) = x + 37\n", "\\]\n", "\n", "### Step 2: Eliminate the fraction\n", "To eliminate the fraction, multiply both sides of the equation by 2:\n", "\\[\n", "2 \\cdot \\frac{1}{2}(3x - 9) = 2(x + 37)\n", "\\]\n", "Simplifying both sides:\n", "\\[\n", "3x - 9 = 2x + 74\n", "\\]\n", "\n", "### Step 3: Solve for \\( x \\)\n", "Subtract \\( 2x \\) from both sides to isolate \\( x \\):\n", "\\[\n", "3x - 2x - 9 = 74\n", "\\]\n", "Simplify:\n", "\\[\n", "x - 9 = 74\n", "\\]\n", "Add 9 to both sides to solve for \\( x \\):\n", "\\[\n", "x = 74 + 9\n", "\\]\n", "\\[\n", "x = 83\n", "\\]\n", "\n", "### Final Answer:\n", "\\[\n", "\\boxed{83}\n", "\\]" ] } ], "source": [ "prompt_cot = prompt + \" \\n\\nExplain step by step.\"\n", "\n", "response_cot = generate_text_stream_concat_flex(\n", " model, tokenizer, prompt_cot, device,\n", " max_new_tokens=2048, verbose=True,\n", ")" ] }, { "cell_type": "markdown", "id": "c2c5cf96-7a18-46c9-9a78-49de5be2212a", "metadata": {}, "source": [ " \n", "## 4.4 Controlling output diversity with temperature scaling" ] }, { "cell_type": "markdown", "id": "257b0984-b918-4ec6-95aa-ff0f48357d6e", "metadata": {}, "source": [ "- So far, we have used deterministic decoding; this section introduces stochastic decoding\n", "- We start with temperature scaling, and later in section 4.5, we add top-p to balance randomness and coherence better" ] }, { "cell_type": "markdown", "id": "e89ef54d-ee68-46cd-a96f-211e9ca8cb01", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "139b28c2-1980-4ebe-8dae-41b72027e0d7", "metadata": {}, "source": [ " \n", "### 4.4.1 Understanding the process of selecting the next token" ] }, { "cell_type": "code", "execution_count": 7, "id": "da89ad46-19a1-41d8-badf-3bb65ab9703b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Berlin" ] } ], "source": [ "ex_prompt = \"The capital of Germany is\"\n", "\n", "response = generate_text_stream_concat_flex(\n", " model, tokenizer, ex_prompt, device,\n", " max_new_tokens=1, verbose=True\n", ")" ] }, { "cell_type": "markdown", "id": "f0050687-ee3b-461b-86a2-e648603ed02e", "metadata": {}, "source": [ "- What happens under the hood:" ] }, { "cell_type": "markdown", "id": "20f74976-4fd7-479c-b6b2-1411ed1ba336", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 8, "id": "97800c88-921e-4286-9fe5-e0db6efc819b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[ 785, 6722, 315, 9856, 374]])\n" ] } ], "source": [ "# 1) Convert input text into token IDs\n", "\n", "input_token_ids = torch.tensor(\n", " tokenizer.encode(ex_prompt), device=device\n", ").unsqueeze(0)\n", "\n", "print(input_token_ids)" ] }, { "cell_type": "code", "execution_count": 9, "id": "8d3e69ae-44b6-4e7b-9fd0-7e63aa2f1099", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "torch.Size([1, 151936])\n" ] } ], "source": [ "# 2) Get scores for next token\n", "\n", "with torch.inference_mode():\n", " next_token_logits = model(input_token_ids)[:, -1]\n", "print(next_token_logits.shape) # Shape: [1, vocab_size]" ] }, { "cell_type": "code", "execution_count": 10, "id": "e96fd349-1995-4aa1-a9bd-faf227b4837c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Token ID: 19846\n", "Decoded token: ' Berlin'\n" ] } ], "source": [ "# 3) Find vocabulary index with highest score\n", "\n", "max_token_id = torch.argmax(next_token_logits)\n", "print(f\"Token ID: {max_token_id}\")\n", "print(f\"Decoded token: '{tokenizer.decode([max_token_id])}'\")" ] }, { "cell_type": "markdown", "id": "45e4e8c0-9c25-4042-a5be-c22f52c95780", "metadata": {}, "source": [ "- The LLM has a vocabulary size of 151,936 tokens with token indices 0 to 151,935; we only focus on a subset of 100 tokens between indices 19,800 and 19,900 (easier to visualize)" ] }, { "cell_type": "code", "execution_count": 11, "id": "4cd653a7-f0ba-4f7a-a09e-271d55072284", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnMAAAHWCAYAAAAciQ/OAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANuZJREFUeJzt3QeYFFW6//F3hjDEIc4QhyQCIhl2EQUFZQkiwsVrAhFYhN2rroGMIEFlQVhBXUERFNCrgq6CrLB4kVWRIIgCipKTRGEVmAEd0tT/ec9u9797kt0zHepMfz/P0zBdXV1VXW9X9a9P1amOcxzHEQAAAFgpPtoLAAAAgLwjzAEAAFiMMAcAAGAxwhwAAIDFCHMAAAAWI8wBAABYjDAHAABgMcIcAACAxQpLDMnIyJCjR49K6dKlJS4uLtqLAwAAkCP9XYe0tDSpWrWqxMfn3P4WU2FOg1xKSkq0FwMAACBghw4dkurVq+f4eEyFOW2R86yUxMTEsLcCnjx5UpKSknJN04gs6uI+1MSdqIs7UZfYqklqaqpphPLkl5zEVJjzHFrVIBeJMJeenm7mQ5hzD+riPtTEnaiLO1GX2KxJ3K+cGkaTEQAAgMUIcwAAABYjzAEAAFiMMAcAAGAxwhwAAIDFCHMAAAAWI8wBAABYjDAHAABgMcIcAACAxVwR5iZPniy/+c1vzM9VJCcnS8+ePWXnzp1+4+jVlR944AGpUKGClCpVSm677Tb54YcforbMAAAAbuCKMPfpp5+aoPb555/LypUr5eLFi9KpUyc5d+6cd5xHH31U/v73v8s777xjxj969Kj06tUrqssNAAAQba74bdYVK1b43Z8/f75pofvyyy/l+uuvlzNnzsgrr7wib775ptx4441mnHnz5slVV11lAuA111wTpSUHAACILle0zGWm4U2VL1/e/K+hTlvrOnbs6B2nQYMGUqNGDVm/fn3UlhMAACDaXNEy5ysjI0MeeeQRue6666RRo0Zm2PHjx6Vo0aJStmxZv3ErVapkHsvJ+fPnzc0jNTXVOw+9hZNO33GcsM8HwaEuBb8mOp3Lly9LkSJFQjK9WMW24k7UJbZqkhHgNF0X5vTcuW3btsmaNWtC0rFi4sSJWYafPHnSdKgIJy2AtjBqgePjXdkAGpOoS8GuiW7X+kXwp59+MqdftGvXztyuvvpqtsMo1gWhQ11iqyZpaWn2hbkHH3xQPvjgA1m9erVUr17dO7xy5cpy4cIFOX36tF/rnPZm1cdyMnr0aBkyZIhfy1xKSookJSVJYmJi2IsbFxdn5sWO0D2oS8GuibbI6T5BO0h98skn5qZ0e69du7b0799fHnroITOs7mPL/Z67588352veBQ3bijtRl9iqSbFixewJc5pm//SnP8nixYvNzld3ur5atmxpDpmsWrXKXJJE6aVLvv/+e2nTpk2O001ISDC3zHRlRyJgaXEjNS8EjroU3JroubUvvviiLF261PSM/+677+Ts2bPmi9zWrVtl+PDh5jQOlSFxfs9lOw1fXRBa1CV2ahIf4PQKu+XQqvZUff/998215jznwZUpU0aKFy9u/h84cKBpZdNOEfotW8OfBjl6sgKxSVvh9Evdxo0b5YsvvjA3DWzaip/TTtET5ACgIHFFmNNv0qp9+/Z+w/XyI3pYRM2YMcPsjLVlTjs1dO7cWWbNmhWV5QUQ+db7gwcP+gU37eWurW6Z6Rc+vQi5funT0zZ++eUXqVatmrz77rvSunVrSgegwCnslh11IMeNZ86caW4ACrYTJ06YwOYb3v71r39lGa9kyZLSokULE95++9vfmv/1NI1p06bJyJEjzTg33HCDLFq0yPR+B4CCyBVhDkDs0vPZ1q5dK7t375ZNmzaZ4Kbnw2am5802adLEL7jphcMLFSqUZdznnnvO+8sxTz/9NJcpAVCgEeYARIxeOkTPa/NtddPz3jK3zuvJxHphcA1snvCmQS7Qnl3Lly83h2D1MiUAUNAR5gCErYOC9ib1HCbV8PbNN9+YHqeZ6aWI9Hw2T4ub9mDPz+WDmjZtms+lBwB7EOYA5Ju2rO3bt88b3DwdFH7++ecs4+q1mDwtbp7gpvT3mLkEBgAEjzAHIGjHjh3zC256019dyKxUqVLSqlUrv/BWs2ZNcxjV94Kb2uEBAJA3hDkAudJfXvF0TPDcDh8+nGU8/f1kPbzpOVSqt/r162fbQQEAEDqEOQBeek22LVu2+F0SZNeuXVnWkLasNWzY0K9naePGjbP9xRUAQHgR5oAYdenSJfn222/9epZu27bNDM9Mr93m27NUr+2mh1ABANFHmANipIPCnj17/HqWbt682bTEZaYdEXwPleqtYsWKUVluAMCvI8wBBdCRI0eydFDQc98y099C1g4KvuEtJSXFr4MCAMDdCHOA5bQXaeYOCkePHs0ynp7P1qxZM7/z3OrVq8flQADAcoQ5IAJq1aoljzzyiLkpbflavHix9OzZM0/TS0tLk+HDh8uqVavM4dPM9HptV199td95bo0aNTI9TgEABQthDviP/v37y4IFC7zro3z58iYITZ061fyUVKiv01auXLk8P1/Pd5s9e7b3/hVXXOF3jpt2UNAfoQcAFHyEOcBHly5dZN68eebv48ePy9ixY+WWW27J9offA3HhwoVsW8MqV66cr/Xerl0707JXvHhxc85bhQoV8jU9AIC94qO9AICb6HllGrT0pueXjRo1Sg4dOiQnT540j+vfd9xxh5QtW9a03PXo0UMOHDjg17qnh04nTZokVatWNRfNzY4eZl2yZIn5W5+v99977z3p0KGDlChRwlx8d/369Tkup46v8+ncuTNBDgBiHGEOyMHZs2flf//3f6Vu3bomMOkPxGt40h6gn332maxdu9Zca01b87QFzkPPY9u5c6esXLlSPvjgg4DX75gxY2TYsGHmor3aMeHuu+/O9ppvAAD44jAr4EPDl+diuOfOnZMqVaqYYdqh4M033zS/Izp37lzvpTv0kKy20n3yySfSqVMnM0zPVdNxgu1soEGuW7du5u+JEyeaDgzauaFBgwbUCACQI1rmAB96mFNbxjw/aaUtcV27dpWDBw/K1q1bTbjSljkNfHrTQ63p6emyd+9e7zT0Z63y0mvUt5OFhkjFD9ADAH4NLXOAD21V08OqHtrCVqZMGZkzZ4457NqyZUt54403sqyzpKQkv2nkRZEiRbx/e1r+tCUQAIDcEOaAXGio0kOs+rNXermPRYsWmZ+7SkxMZL0BAFyBw6yAj/Pnz5tLkuht+/bt8qc//cm0yHXv3l369OljfqNUe7BqB4j9+/ebc+UeeughOXz4MOsRABAVtMwBPlasWOE9X03PjdPOB++88460b9/eDFu9erWMHDlSevXqZX6FoVq1anLTTTfRUgcAiBrCHPAf8+fPN7fc6PXnfH8lIrOcnu97LTrlOI7fT3353lfaQzbzMAAAssNhVgAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYq4Jc6tXr5bu3btL1apVJS4uTpYsWeL3eP/+/c1w31uXLl2itrwAAABu4Jowd+7cOWnatKnMnDkzx3E0vB07dsx7e+uttyK6jAAAAG5TWFyia9eu5pabhIQEqVy5csSWCQAAwO1cE+YC8cknn0hycrKUK1dObrzxRnnqqaekQoUKOY5//vx5c/NITU01/2dkZJhbOOn0HccJ+3wQHOriPtGqSbw4WZYD0a8LckddYqsmGQFO05owp4dYe/XqJbVr15a9e/fKY489Zlry1q9fL4UKFcr2OZMnT5aJEydmGX7y5ElJT08P6/JqAc6cOWMKHB/vmqPZMY+6uE+0anJVOf8wd+LEiYjN2wZsK+5EXWKrJmlpaQUrzN11113evxs3bixNmjSRK664wrTW3XTTTdk+Z/To0TJkyBC/lrmUlBRJSkqSxMTEsBdXO2novAhz7kFd3CdaNdl+Ks7vvrb6I/p1Qe6oS2zVpFixYgUrzGVWp04dqVixouzZsyfHMKfn2OktM13Zkdg5aXEjNS8Ejrq4TzRqkiH+YY7t1B11wa+jLrFTk/gAp2ftFnr48GH58ccfpUqVKtFeFAAAgKhxTcvc2bNnTSubx/79+2XLli1Svnx5c9Nz32677TbTm1XPmRsxYoTUrVtXOnfuHNXlBgAAiCbXhLlNmzZJhw4dvPc957r169dPXnzxRfn6669lwYIFcvr0aXNh4U6dOsmTTz6Z7WFUAACAWOGaMNe+fXvTEyQnH374YUSXBwAAwAbWnjMHAAAAwhwAAIDVaJkDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoWjvQAAUFDVGrXM7/6BKd2itiwACi5a5gAAACxGmAMAALCYa8Lc6tWrpXv37lK1alWJi4uTJUuW+D3uOI6MGzdOqlSpIsWLF5eOHTvK7t27o7a8AAAAbuCaMHfu3Dlp2rSpzJw5M9vHp06dKs8//7y89NJLsmHDBilZsqR07txZ0tPTI76sQG7nSPneAACImQ4QXbt2NbfsaKvcs88+K2PHjpUePXqYYa+99ppUqlTJtODdddddEV5aAAAAd3BNmMvN/v375fjx4+bQqkeZMmWkdevWsn79+hzD3Pnz583NIzU11fyfkZFhbuGk09cQGu75wF11iRcny/wQ3ZpEs1Y2vx/Yh7kTdYmtmmQEOE0rwpwGOaUtcb70vuex7EyePFkmTpyYZfjJkyfDfnhWC3DmzBlT4Ph41xzNjnnhrstV5fw/vE+cOBHz69yt20okamXz+4F9mDtRl9iqSVpaWsEJc3k1evRoGTJkiF/LXEpKiiQlJUliYmLYi6sdOXRehDn3CHddtp+K87ufnJwc8nkUNNHaViJRK5vfD+zD3Im6xFZNihUrVnDCXOXKlc3/P/zwg+nN6qH3mzVrluPzEhISzC0zXdmR+NDQ4kZqXnBHXTLE/8Ob2ke/JtGsle3vB/Zh7kRdYqcm8QFOz4o9S+3atU2gW7VqlV8rm/ZqbdOmTVSXDQAAIJpc0zJ39uxZ2bNnj1+nhy1btkj58uWlRo0a8sgjj8hTTz0lV155pQl3jz/+uLkmXc+ePaO63AAAANHkmjC3adMm6dChg/e+51y3fv36yfz582XEiBHmWnSDBw+W06dPS9u2bWXFihUBH08GAAAoiFwT5tq3b296guR2PPqJJ54wNwAAAFh0zhwAAACyR5gDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsJhrLk0CAEB2ao1a5nf/wJRurCjABy1zAAAAFiPMAQAAWIwwBwAAYDHCHAAAgMXoAAEgXzg5HQAsbJl7/fXX5brrrpOqVavKwYMHzbBnn31W3n///VAvHwAAAEIZ5l588UUZMmSI3HzzzXL69Gm5fPmyGV62bFkT6AAAAODiMPfXv/5V5syZI2PGjJFChQp5h7dq1Uq++eabUC8fAAAAQhnm9u/fL82bN88yPCEhQc6dOxfs5AAAABDJMFe7dm3ZsmVLluErVqyQq666Kj/LAgAAgHD3ZtXz5R544AFJT08Xx3Fk48aN8tZbb8nkyZNl7ty5wU4OcB16ZwIACnSYu++++6R48eIyduxY+fnnn6V3796mV+tzzz0nd911V3iWEgAAH3zpAvJ5nbk+ffqYm4a5s2fPSnJycl4mU6B3MvHiyFXlHNl+Kk72Tbkl2osEAAAKqHxdNLhEiRLmBsAetGgAQIyHOe0AERcXl+Pj+/bty+8yAQAAIFxh7pFHHvG7f/HiRdm8ebPpzTp8+PBgJwcAAIBIhrmHH3442+EzZ86UTZs25WdZgLDiXEYAQEGUp99mzU7Xrl3l3XffDdXkAAAAEMkw97e//U3Kly8fqskBAAAgHIdZ9ae8fDtA6IWDjx8/LidPnpRZs2YFOzkAAABEMsz17NnT7358fLwkJSVJ+/btpUGDBvlZFgAAAIQ7zI0fPz7YpwAAACCaYS41NTXgCSYmJuZneQAAABDqMFe2bNlcLxTsOXdOx7l8+XIw8wcAAEC4w9zHH3+cn3kAAAAgmmHuhhtuCNf8AQAAEMkOEB4///yzfP/993LhwgW/4U2aNMnP8gAAACCcYU6vJzdgwAD5xz/+ke3jnDMHAADg4l+AeOSRR+T06dOyYcMGKV68uKxYsUIWLFggV155pSxdujQ8SwkAAIDQtMz985//lPfff19atWplLhhcs2ZN+d3vfmcuSTJ58mTp1q1bsJMEAABApMLcuXPnJDk52fxdrlw5c9i1Xr160rhxY/nqq6/yuhwAALherVHL/O4fmEIDBiw8zFq/fn3ZuXOn+btp06Yye/ZsOXLkiLz00ktSpUqVcCwjAAAAQtUy9/DDD8uxY8e8P+3VpUsXeeONN6Ro0aIyf/78YCcHwCK0SgBAAQhz99xzj/fvli1bysGDB2XHjh1So0YNqVixYqiXDwAAAKEMc2vWrJG2bdt675coUUJatGgR7GQAoECh1RKANefM3XjjjVK7dm157LHH5LvvvgvPUgEAACA8Ye7o0aMydOhQ+fTTT6VRo0bSrFkzmTZtmhw+fDjYSQEAACDSYU7Pi3vwwQdl7dq1snfvXrn99tvNRYNr1aplWu0A5P0wne8NAICw/jar0sOto0aNMpcoefzxx01rHaIrFs/bicXXDABAnlvmPLRl7v777zfXluvdu7c55LpsGa0JAAAArm6ZGz16tCxcuNCcO6c/4/Xcc89Jjx49TK9WAAAAuDzMrV69WoYPHy533HEH15UDAsBhYABuxL4phsOcHl4FAABAAegAAQCAW9DShFhFmAMAlyKcAAgEYQ4FEh+CAIBYQZgDAMBSfHFFnsJcnTp15IsvvpAKFSr4DT99+rS0aNFC9u3bF5Y1O2HCBJk4caLfsPr168uOHTvCMj8AiJTsfvEjmhe/1uWJF0euKufI9lNxsm/KLVFbFkQW4TBGwtyBAwfk8uXLWYafP39ejhw5IuF09dVXy0cffeS9X7gwDYsAACC2BZyGli5d6v37ww8/lDJlynjva7hbtWqV+X3WcNLwVrly5bDOAwAKKlpdgIIp4DDXs2dP839cXJz069fP77EiRYqYIPfMM89IOO3evVuqVq0qxYoVkzZt2sjkyZOlRo0aYZ0nkBM+GAH3YztFLAg4zGVkZJj/a9eubc6Zq1ixokRS69atZf78+eY8uWPHjpnz59q1ayfbtm2T0qVLZ/scPfSrN4/U1FTva/G8nnDQc030Fmf+///rLhJ0vr4iOe9oye415zQskLoEug6DmW9epxcO+ZlPfl5fTnR8x3Ei/l4N9foOZe2DfX5+ltFt+7Ccliev21q01mG4ZN5e3LY9x6KMMO7DAp1m0Ced7d+/X6Kha9eu3r+bNGliwl3NmjXl7bffloEDB2b7HG25y9xpQp08eVLS09PDtqx60rDuAKuXEokTkRMnToRtXtnN21ck5x0t2b3mnIYFUpdA12Ew883r9MIhP/PJz+vLbWd15swZszOMj9cKRUao13coax/s87MzcMEXfvdf6feboJYnWvuwnJYnr9taJOocSZm3F7dtz7EoI4z7sLS0tNCFueeff14GDx5sDm/q37l56KGHJBLKli0r9erVkz179uQ4zujRo2XIkCF+LXMpKSmSlJQkiYmJYVs27f2l3250k9hxSiQ5OTls88pu3r4iOe9oye415zQskLoEug6DmW9epxcO+ZlPfl5fbjtCPX1Dt8tIhrlQr+9Q1j7Y54djeaK1D8tpefK6rUWizpGUeXtx2/YcizLCuA/T3BWyMDdjxgzp06ePmaj+nRN9MZEKc2fPnpW9e/dK3759cxwnISHB3DLTlR3OD40M811WzI5Q/47kB5Rn3h6RnHe0ZPeacxv2a3UJdB0GO9+8TC8c8jOf/Ly+3Oi+I9zbZbjXdyhrH+zzw7U80diH5bY8ednWIlHnSPPdXty4PceiuDDtwwKdXuFgD61G6zDrsGHDpHv37ubQ6tGjR2X8+PFSqFAhufvuu6OyPAAAAG5gzYXaDh8+bILbjz/+aJoy27ZtK59//rn5GwAAIFYFHeZ8z0HL3MSoh2Hr1q0rPXr0kPLly0soLVy4MKTTAwAAiMkwt3nzZvnqq6/MhYL1MiFq165d5pBngwYNZNasWTJ06FBZs2aNNGzYMBzLDAAAgLyGOU+r27x587w9QrVL7n333WcOfQ4aNEh69+4tjz76qPmlCPwbF64EAACuCHPTpk2TlStX+l3aQ3/aa8KECdKpUyd5+OGHZdy4ceZvxDYCLAAA4Rd0H1pthcvuIoJ6IV7PLyzoNeAuXLgQmiUEAABA6MKcHmb9/e9/L4sXLzY9TPWmf+uvMHh+v3Xjxo3mgr4AAAAIr6APs86ePducD3fXXXfJpUuX/j2RwoWlX79+3gsKa0eIuXPnhn5pAQAAkL8wV6pUKZkzZ44Jbvv27TPD6tSpY4Z7NGvWLNjJIp/no6kDU7qxHuEKnC8JABZcNFjDm+dacr5BDgAAFBx8OSuA58zpD8o+8cQTpger/rSW3rTDw5NPPmkeAwAAgItb5saMGSOvvPKKTJkyRa677jozTC8QrJcmSU9Pl0mTJoVjOQEAABCKMLdgwQLTueHWW2/1DmvSpIlUq1ZN7r//fsIcAACAmw+z/vTTT6a3amY6TB8DAACAi1vmmjZtKi+88II8//zzfsN1mD4Ge3GSKwAAMRDmpk6dKt26dZOPPvpI2rRpY4atX79eDh06JMuXLw/HMgIAACBUYe6GG26QXbt2ycyZM2XHjh1mWK9evcz5clWrVg12cgAAABFRK9M1WgvK9VnzdJ05DW2Ze63qz3oNHjxYXn755VAtGwAAAMJ10eDMfvzxR3PJEsIcAAAIhYLakhb13qwAAAAogC1zAOBWfLsHUJAR5gDgPwh9AAp0mNMeq7k5ffp0KJYHYcAHFAAABVfAYa5MmTK/+vi9994bimUCAABAqMPcvHnzAh0VKHBo3QQAuBW9WQEAACxGBwgAEUHrJgCEBy1zAAAAFqNlDgCtZoDL0JKNYBDmAADIBoEqfFi3ocVhVgAAAIvRMoeI4tsYAEQe+96CjZY5AAAAi9EyBwAAYHFLJmEOiALbdhQAAPcizMGacJJ5HuGaDwAANiHMAQBcg1ZrIHh0gAAAALAYYQ4AAMBiHGZFTOOQDgD2I7AdYQ5Atgi6AGAHDrMCAABYjDAHAABgMcIcAACAxQhzAAAAFiPMAQAAWIwwBwAAYDEuTQIAQeKyLQDchJY5AAAAixHmAAAALMZhVuSKw0kAALgbLXMAAAAWI8wBAABYjDAHAABgMcIcAACAxQhzAAAAFqM3KwAr0LMaAApIy9zMmTOlVq1aUqxYMWndurVs3Lgx2osEAAAQNVaFuUWLFsmQIUNk/Pjx8tVXX0nTpk2lc+fOcuLEiWgvGgAAQFRYFeamT58ugwYNkgEDBkjDhg3lpZdekhIlSsirr74a7UUDAACICmvOmbtw4YJ8+eWXMnr0aO+w+Ph46dixo6xfvz7b55w/f97cPFJTU83/GRkZ5hYu8eKYW5z5/9/z0/u+8jv/zNOTXOYT6Lwj8dz8rIf8vuZA6+KmYflV97Hlfvf3/Plm170+/d9xnF99vaF870RqW8nr9IJ9fn6WMafnZt5WsnsvhYObtqtobruBbi+ReM9G8vXldbxozjfQfVheBDrNOEeXwAJHjx6VatWqybp166RNmzbe4SNGjJBPP/1UNmzYkOU5EyZMkIkTJ2YZvmvXLildunRYl1cLcObMGSlTpowJndkZuOALv/uv9PtNwMPyKz/TDPXyROo1B1oXN8m8HiL9Pgk3XWatQvVSjhw+Gydzsnkdub2WSKwHt00v1K85p+eGex8WTJ1DLVrvm2C250C3l0DnHWrhqHMw6yEv834lDPMN9+dKWlqa1KtXz0w/MTHR/pa5vNBWPD3HzrdlLiUlRZKSknJdKaGgxY2LizPzyqm420/F+d1PTk4OeFh+5WeaoV6eSL3mQOviJpnXQ6TfJ+Gmy6zfgPUb5Y5T2b+O3F5LJNaD26YX6tec03PDvQ8Lps6hFq33TTDbc6DbS6DzDrVw1DmY9ZCXeSeHYb7h/lzRzp6BsCbMVaxYUQoVKiQ//PCD33C9X7ly5Wyfk5CQYG6Z6cqOxAe5Fje3eWWI/xtDxwt0WH7lZ5qhXp5IveZA6+ImmddDpN8n4bZvyi1mR6idmHTHmN3rUPndhvLDbdML9WvO7bnh3IcFU+dQi9b7JpjtObdpOv/5O5i6hGPbDWS++d2eAx0vIx81DcV44fpcCXR67t/b/0fRokWlZcuWsmrVKu8w/RDQ+76HXQEAAGKJNS1zSg+Z9uvXT1q1aiW//e1v5dlnn5Vz586Z3q0AAKDgOzClW7QXwXWsCnN33nmnnDx5UsaNGyfHjx+XZs2ayYoVK6RSpUrRXjQAiDl8qALuYFWYUw8++KC5AQAAwMIwBwDwRwsZENsIcwAQwwiCgP0IcwAAABZ/8bHm0iQAAADIijAHAABgMQ6zAgAQ44fpYDfCHKKOHRwAAHlHmAMARBxf4oDQIcwBQC4IHbGBOsNmhDmXYYcCAACCQW9WAAAAi9EyB7gYLbUAgF9DyxwAAIDFaJkDUKDQmgneI4i1fQYtcwAAABYjzAEAAFiMMAcAAGAxzpkDAIsUhPN7AIQWYc4C7LwBAEBOOMwKAABgMcIcAACAxQhzAAAAFuOcOQAAUOAciKHOQoQ5AAAKkFgKMfg3DrMCAABYjDAHAABgMQ6zAgCAsOCQb2QQ5qKINzkAAMgvwhwARBBf4gCEGufMAQAAWIwwBwAAYDHCHAAAgMU4Zw4AEDKcEwhEHmEOAABLgnJGRoacOHFCkpOTo704cBEOswIAAFiMljkAAJBvHGKPHlrmAAAALEaYAwAAsBhhDgAAwGKcMwcAAKKOc+7yjjAHwFrs/AGAw6wAAABWo2UOrkSLCwAAgaEDBAAAgMVomQPgKrTKAkBwaJkDAACwGC1zCBotJwAAuActcwAAABYjzAEAAFiMw6wAAIQYp6NE1oEp3SSW0TIHAABgMcIcAACAxQhzAAAAFiPMAQAAWMyaMFerVi2Ji4vzu02ZMiXaiwUAABBVVvVmfeKJJ2TQoEHe+6VLl47q8gAAAESbVWFOw1vlypWjvRgAAACuYc1hVqWHVStUqCDNmzeXadOmyaVLl6K9SAAAAFFlTcvcQw89JC1atJDy5cvLunXrZPTo0XLs2DGZPn16js85f/68uXmkpqaa/zMyMswtnHT6juOEfT55FS+O3323Lmes1SVQ+/7c1e++vh5ba1pQalLQhLIutr43IyWY9cP24j4ZYdyHBTrNqIa5UaNGydNPP53rONu3b5cGDRrIkCFDvMOaNGkiRYsWlT/84Q8yefJkSUhIyPa5+tjEiROzDD958qSkp6dLOGkBzpw5YwocH+++BtCryvnvPE6cOCGxwO11icWaFuSa2CyUdbH1vRkpwawfthf3yQjjPiwtLc39YW7o0KHSv3//XMepU6dOtsNbt25tDrMeOHBA6tevn+042nrnGwK1ZS4lJUWSkpIkMTFRwl1c7XGr83LjB9T2U3F+95OTkyUWuL0usVjTglwTm4WyLra+NyMlmPXD9uI+GWHchxUrVsz9YU5fuN7yYsuWLWal5fam1xa77Frt9HmR+NDQ4kZqXsHKEP+dhxuXMRbrEqs1Lag1sV2o6mLzezMSgl0/bC/uExemfVig07PinLn169fLhg0bpEOHDqZHq95/9NFH5Z577pFy5cpFe/EAAACixoowp61rCxculAkTJpgODbVr1zZhzvcQKgAANjowpVu0FwGWsyLMaS/Wzz//PNqLAQAA4DqcuAAAAGAxwhwAAIDFCHMAAAAWI8wBAABYjDAHAABgMcIcAACAxQhzAAAAFrPiOnMIPS5SCQBAwUDLHAAAgMUIcwAAABYjzAEAAFiMMAcAAGAxOkAAAMKKDldAeNEyBwAAYDHCHAAAgMUIcwAAABYjzAEAAFiMMAcAAGAxwhwAAIDFCHMAAAAWI8wBAABYjDAHAABgMcIcAACAxQhzAAAAFiPMAQAAWIwwBwAAYDHCHAAAgMUIcwAAABYjzAEAAFiMMAcAAGAxwhwAAIDFCHMAAAAWI8wBAABYjDAHAABgMcIcAACAxQhzAAAAFiPMAQAAWKxwtBcAQGgcmNKNVQkAMYiWOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAADAYoQ5AAAAixHmAAAALEaYAwAAsBhhDgAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLFZYY4jiO+T81NTXs88rIyJC0tDQpVqyYxMeTmd2CurgPNXEn6uJO1CW2apL6n7ziyS85iakwpytbpaSkRHtRAAAAAs4vZcqUyfHxOOfX4l4BS89Hjx6V0qVLS1xcXFjnpWlaQ+OhQ4ckMTExrPNC4KiL+1ATd6Iu7kRdYqsmjuOYIFe1atVcW/1iqmVOV0T16tUjOk8tLGHOfaiL+1ATd6Iu7kRdYqcmZXJpkfPgZC4AAACLEeYAAAAsRpgLk4SEBBk/frz5H+5BXdyHmrgTdXEn6uI+CS74vI+pDhAAAAAFDS1zAAAAFiPMAQAAWIwwBwAAYDHCnI/Vq1dL9+7dzcX59KLCS5Ys8VtZP/zwg/Tv3988XqJECenSpYvs3r3bb5zjx49L3759pXLlylKyZElp0aKFvPvuu37j/PTTT9KnTx9zPZqyZcvKwIED5ezZs37jfP3119KuXTvz8yB6McKpU6dKLIpETQ4cOGBqULt2bSlevLhcccUV5mTWCxcu+E2HmkS2Lr7Onz8vzZo1M/PasmULdYni9uKxbNkyad26tdlmypUrJz179vR7/Pvvv5du3bqZ+SQnJ8vw4cPl0qVLEmsiVZNdu3ZJjx49pGLFiuazpW3btvLxxx/7jUNNQluXvXv3yn/9139JUlKSWed33HGHeV40Pu8Jcz7OnTsnTZs2lZkzZ2ZZUdpPRHdW+/btk/fff182b94sNWvWlI4dO5rnedx7772yc+dOWbp0qXzzzTfSq1cvU2Ad30ML++2338rKlSvlgw8+MG+qwYMH+11NulOnTmb6X375pUybNk0mTJggL7/8ssSaSNRkx44d5tdBZs+ebeoyY8YMeemll+Sxxx7zToOaRL4uvkaMGGF2qplRl+jURYOEhosBAwbI1q1bZe3atdK7d2/v45cvXzZBTr8QrVu3ThYsWCDz58+XcePGSayJVE1uueUWE5b/+c9/ms8NnacO0yCoqElo66L/6+e0BkFd57oN6PtdA6J+nkT88157syIrXTWLFy/23t+5c6cZtm3bNu+wy5cvO0lJSc6cOXO8w0qWLOm89tprftMqX768d5zvvvvOTOeLL77wPv6Pf/zDiYuLc44cOWLuz5o1yylXrpxz/vx57zgjR4506tevH9OlCldNsjN16lSndu3a3vvUJHp1Wb58udOgQQPn22+/NdPdvHkzdYliXS5evOhUq1bNmTt3bo7z1prFx8c7x48f9w578cUXncTERL/9WqwJV01OnjxpprN69Wrv46mpqWbYypUrzX1qEtq6fPjhh+Y9fubMGe84p0+fNp/lnnUeyc97WuYCpId5lDaD+v48mF5XZs2aNd5h1157rSxatMg0rWo6X7hwoaSnp0v79u3N4+vXrzdNra1atfI+R9O+TmvDhg3eca6//nopWrSod5zOnTubb2anTp0KLq0XYKGqSXbOnDkj5cuX996nJtGpix6yGDRokLz++uvmUEdm1CXydfnqq6/kyJEj5rnNmzeXKlWqSNeuXWXbtm1+dWncuLFUqlTJbx+mrRDaSoHQ1qRChQpSv359ee2110yLkbbQ6ZEGPbzdsmVLahKGuug42irne205HV/H84wTyc97wlyAGjRoIDVq1JDRo0ebFazNqU8//bQcPnxYjh075h3v7bfflosXL5qNS4v8hz/8QRYvXix169Y1j2uTt25gvgoXLmyCg6c5XP/33Qkqz33POAhdTTLbs2eP/PWvfzXjeVCTyNdFvzDrOSt//OMf/XaGvqhL5Ouih56UHgoaO3asOXSk58xpsNCwQV0iXxMNFR999JE5HFi6dGkTKqZPny4rVqwwtaEmoa/LNddcY85fHDlypPz8888mRA8bNswczvaME8nPe8JcgIoUKSLvvfeeOclUC6GtBHpyqX4j1ZTt8fjjj8vp06fNhrVp0yYZMmSIObdBz3OA+2uiLQ56ouvtt99uWoQQvbpooE5LSzM7VLinLp7zgcaMGSO33XabafmZN2+eCRTvvPMOpYpCTfSLzwMPPGCCw2effSYbN24053zp+Vu+oRChq4t2etD3+9///ncpVaqUlClTxtRIO6f41i5SCkd8jhbTnZb2pNNDcJrUtZjam8vTaqA9W1544QVzuOHqq682w/QES9249CRLPaleeyOdOHHCb7raJK7faPUxpf9n7hHjue8ZB6GricfRo0elQ4cO5pBG5pNPqUnktxU9qVgPQWT+iRydhp5UrCfVU5fI10UPq6qGDRt6p6s1qlOnjukt6dleNFD4Yh8W3m1FW0i1FUl7TapZs2aZk+51Oxk1ahQ1CXFdlHZc0Pr861//Mi1uekhV3/u6LahIft7TMpcHmsC1sNpNWb8laXdwpU2tZqVmSuWFChXyfptt06aNSe/aa8VDN0R9XN8onnG0x4s2q3voRqnnRHiazBG6mnha5PQwkaeVIfP41CTydXn++edNT0ndoept+fLlZrieOzRp0iTqEqW66Dai4U3P6fHQfZVe4kd75Hm2F2018v0g032YBg3fEIjQ1CSncfS+72cPNQldXXzp5WA0yOlnub7nb7311sh/3gfVXaKAS0tLMz3l9KarZvr06ebvgwcPmsfffvtt5+OPP3b27t3rLFmyxKlZs6bTq1cv7/MvXLjg1K1b12nXrp2zYcMGZ8+ePc5f/vIX03Nl2bJl3vG6dOniNG/e3IyzZs0a58orr3Tuvvtuvx4xlSpVcvr27Wt60yxcuNApUaKEM3v2bCfWRKImhw8fNuPcdNNN5u9jx455bx7UJPJ1yWz//v1ZerNSl+jU5eGHHzY9WrVH344dO5yBAwc6ycnJzk8//WQev3TpktOoUSOnU6dOzpYtW5wVK1aYnoCjR492Yk0kaqK9WStUqGCep+tbe2MOGzbMKVKkiLmvqElo66JeffVVZ/369aYmr7/+uulhPGTIEL9xIvV5T5jzoYXToma+9evXzzz+3HPPOdWrVzcbSI0aNZyxY8dm6Wa/a9cuU3DdsWlBmjRpkqVL+Y8//miKWapUKdNVf8CAAeaN5Wvr1q1O27ZtnYSEBLPTnDJlihOLIlGTefPmZTuPzN91qElk6xJImKMukd9ePAFj6NChZpzSpUs7HTt29LuMgzpw4IDTtWtXp3jx4k7FihXN+HpZk1gTqZro5S80PGug0Jpcc8015nIkvqhJaOuilxDRIKbjaEh75plnnIyMjKh83sfpP3loeQQAAIALcM4cAACAxQhzAAAAFiPMAQAAWIwwBwAAYDHCHAAAgMUIcwAAABYjzAEAAFiMMAcAAGAxwhyAAq1///7Ss2fPfE8nLi5OlixZIpGmv3mq89bfp82PCRMmSLNmzUK2XADcgzAHICy6d+8uXbp0yfaxzz77zASUr7/+mrX/K1JSUuTYsWPSqFEj1hWAbBHmAITFwIEDZeXKlXL48OEsj82bN09atWolTZo0iZm1f+HChTw9r1ChQlK5cmUpXLhwyJcJQMFAmAMQFrfccoskJSXJ/Pnz/YafPXtW3nnnHRP21LvvvitXX321JCQkSK1ateSZZ57xG//8+fMycuRI00Kl49StW1deeeUV89jly5fNdGrXri3FixeX+vXry3PPPZft8kycONEsT2Jiovzxj3/0C1c632effdZvfD0kqYcmc6LLVK9ePSlRooTUqVNHHn/8cbl48WKWw5pz5841y1esWDF57bXXpEKFCuY1+dLDwH379g3oMOsnn3xi7q9atcoEYp3/tddeKzt37vR73pQpU6RSpUpSunRps47S09OzTFuX7aqrrjLL1qBBA5k1a5b3sd///vcmbHuWVddX8+bN5d57781xnQCIDsIcgLDQliT94Ncw5ziOd7gGOQ1hd999t3z55Zdyxx13yF133SXffPONCUAainwDoE7jrbfekueff162b98us2fPllKlSpnHMjIypHr16maa3333nYwbN04ee+wxefvtt/2WRYOPPleDkE7rvffeM+EuPzQk6XLqfDVAzpkzR2bMmOE3zp49e0xY1flpGLv99tvNa1+6dKl3nBMnTsiyZctMeArGmDFjTPDdtGmTWde+z9fXr+vyz3/+s3m8SpUqfkFNvfHGG2Z9TZo0yawbHVfX/YIFC8zjur7PnTsno0aN8s7v9OnT8sILL+RpfQEIIwcAwmT79u2a4pyPP/7YO6xdu3bOPffcY/7u3bu387vf/c7vOcOHD3caNmxo/t65c6d5/sqVKwOe5wMPPODcdttt3vv9+vVzypcv75w7d8477MUXX3RKlSrlXL582dyvWbOmM2PGDL/pNG3a1Bk/frz3vi7H4sWLc5zvtGnTnJYtW3rv63OLFCninDhxwm+8//mf/3G6du3qvf/MM884derUcTIyMrKd7v79+828N2/ebO7rutT7H330kXecZcuWmWG//PKLud+mTRvn/vvv95tO69atzWvyuOKKK5w333zTb5wnn3zSPNdj3bp15jU8/vjjTuHChZ3PPvssx9cPIHpomQMQNnroTg8Bvvrqq96WKu384DnEqi1C1113nd9z9P7u3btNC5a2Zuk5YzfccEOO85g5c6a0bNnSHELVFruXX35Zvv/+e79xmjZtag5HerRp08Yc7j106FCeX9uiRYvMsur5bDrfsWPHZplvzZo1zXL5GjRokPzf//2fHDlyxNzX1j3tcauHToPhe76htrx5Wvk867V169Z+4+tr9tAWt71795o66LJ7bk899ZQZ7vucYcOGyZNPPilDhw6Vtm3bBrWMACKDMAcgrDQw6KHGtLQ00/HhiiuuyDWc+dLz4HKzcOFCEzZ0HhqQNPwNGDAg6M4G8fHxfoeCle/5b5mtX79e+vTpIzfffLN88MEHsnnzZnMYMvN8S5YsmeW5et6Zhks9f04PM3/77bcmzAWrSJEi3r89QVAPOwdCg6zSQ8O6zjy3bdu2yeeff+4dT6e3du1aE6g1iANwJ8IcgLDSc+I0LL355psmwOi5XZ7woSffa1jwpfe1Y4EGiMaNG5tA8emnn2Y7bR1XW/7uv/9+E5K0c4Rvy5LH1q1b5ZdffvHe18CiLVHaqUJp65le/sMjNTVV9u/fn+NrWrdunWl10wCnnRCuvPJKOXjwYMDr5L777jMtchpuO3bs6F2OUNH1umHDBr9hviFNO0ZUrVpV9u3bZ9aZ7007a3hMmzZNduzYYdb/ihUrzPICcB/CHICw0tB05513yujRo01g8m2F0kN32jlBD+Pt2rXLnHyvJ9hra5unl2m/fv1MANQL9mrA0k4Mng4OGqL0BP8PP/zQPF9P4P/iiy+yLIO2mGnrnXZWWL58uYwfP14efPBBEzLVjTfeKK+//ro5BKwdMXSeGiZzovPVQ6raMqjhUTsLLF68OOB10rt3b3PJFm0ZC7bjQyAefvhhc2hbw5euF3292gLoSzuATJ482Sy7jqOvW8efPn26eVxbG7WDhPZ41cPJOlynqwEQgMtE8Xw9ADFCT6TX3c3NN9+c5bG//e1vpsODnmhfo0YN05HAl57U/+ijjzpVqlRxihYt6tStW9d59dVXzWPp6elO//79nTJlyjhly5Y1nQtGjRrld6K/doDo0aOHM27cOKdChQqm48OgQYPMcz3OnDnj3HnnnU5iYqKTkpLizJ8//1c7QGhHDc/09LnagUKXw0Of67scmfXt29d0zPBdjmA6QJw6dco7jj6mw3Rcj0mTJjkVK1Y0y6frYMSIEVmW54033nCaNWtm1mu5cuWc66+/3nnvvffMOteaDB482G/8W2+91bn22mudS5cu5brMACIrTv+JdqAEgFhz0003mevracsYAOQHYQ4AIujUqVPmUPF///d/m8O+eqFjAMgPfh8GACJIO2pooHv66acJcgBCgpY5AAAAi9GbFQAAwGKEOQAAAIsR5gAAACxGmAMAALAYYQ4AAMBihDkAAACLEeYAAAAsRpgDAACwGGEOAABA7PX/AJ5arFH+KK3SAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "def plot_scores_bar(\n", " next_token_logits, start=19_800, end=19_900,\n", " arrow=True, ylabel=\"Logit value\"\n", "):\n", " # Select vocabulary subsection\n", " x = torch.arange(start, end)\n", "\n", " # .cpu() is a shortcut for to(torch.device(\"cpu\"))\n", " logits_section = next_token_logits[0, start:end].float().cpu()\n", "\n", " # Plot logits\n", " plt.bar(x, logits_section)\n", " plt.xlabel(\"Vocabulary index\")\n", " plt.ylabel(ylabel)\n", "\n", " # Highlight max logit\n", " if arrow:\n", " max_idx = torch.argmax(logits_section)\n", " plt.annotate(\n", " \"Berlin\",\n", " xy=(x[max_idx], logits_section[max_idx]),\n", " xytext=(x[max_idx] - 25, logits_section[max_idx] - 2),\n", " arrowprops={\n", " \"facecolor\": \"black\", \"arrowstyle\": \"->\", \"lw\": 1.5\n", " },\n", " fontsize=10,\n", " )\n", "\n", " plt.grid(alpha=0.3)\n", " plt.tight_layout()\n", " plt.show()\n", "\n", "plot_scores_bar(next_token_logits)" ] }, { "cell_type": "markdown", "id": "29c1cc3d-5416-416f-a2d7-8fb29d01b08d", "metadata": {}, "source": [ " \n", "### 4.4.2 Rescaling token scores (logits) via a temperature parameter" ] }, { "cell_type": "markdown", "id": "2aae0b2e-aa6a-45b6-80b9-8d97b706cbb6", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 12, "id": "cf7ed951-90ee-4f10-a866-776900a86dbf", "metadata": {}, "outputs": [], "source": [ "def scale_logits_by_temperature(logits, temperature):\n", " if temperature <= 0:\n", " raise ValueError(\"Temperature must be positive\")\n", " return logits / temperature" ] }, { "cell_type": "markdown", "id": "6db3ae54-aa8e-47e4-b5bd-21a548bd702e", "metadata": {}, "source": [ "- Technically, it's better to use a bar plot, but lines make it easier to see the difference:" ] }, { "cell_type": "code", "execution_count": 13, "id": "7b2b0e37-c500-46db-b083-3c3268b9c0de", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnQAAAHWCAYAAAD+VRS3AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzsnQV4E/f/x9+pO6VIcXeHMQG2wZjBxjbmY+4uzOU3d/nPlSlzd9hgGww2hgwY7g6lUHdvk//z/l4uuaTxpm3afl7Pk6dpmiaXu8vd+z7y/pgsFosFgiAIgiAIQpMlrLEXQBAEQRAEQagbIugEQRAEQRCaOCLoBEEQBEEQmjgi6ARBEARBEJo4IugEQRAEQRCaOCLoBEEQBEEQmjgi6ARBEARBEJo4IugEQRAEQRCaOBGNvQChhtlsRnp6OhITE2EymRp7cQRBEARBaGFYLBYUFRWhU6dOCAvzLfYmgs4JirmuXbvWx/YRBEEQBEHwmX379qFLly4+PVcEnROMzOkrMSkpCfURAczKykK7du18Vt1C/SLbJLSQ7RF6yDYJPWSbNO/tUVhYqIJLuibxBRF0TuhpVoq5+hJ05eXl6rVF0IUGsk1CC9keoYdsk9BDtknL2B4mP0q/JEQkCIIgCILQxBFBJwiCIAiC0MQRQScIgiAIgtDEkRo6QRAEoUVTU1ODqqoqNLWaLS4z67akHrvx8Xd7REZGIjw8PKjLIIJOEARBaLFeXwcPHkR+fj6a4rJTRNCrTDxTG59AtkdycjI6dOgQtO0ngk4QBEFokehirn379oiLi2tSwogCorq6GhEREU1quZsrFj+2B59bWlqKzMxM9XvHjh2Dsgwi6ARBEIQWmWbVxVybNm3Q1BBB17S3R2xsrPpJUcd9MBjpV2mKEARBEFoces0cI3OC0Bjo+16w6jdF0AmCIAgtFklXCs1l32uygu7pp59WK2P69Om2x9hdcsMNN6jweUJCAs4880xkZGQ06nIKgiAIgiDUN01S0C1fvhwzZszAsGHDHB6/9dZb8fPPP+Prr7/GwoULkZ6ejjPOOKPRllMQBEEQQo3du3ergMjq1at9/p+ZM2eqrsyGXo4ePXrgpZdeCur7XnrppZg6dSqaG01O0BUXF+OCCy7AO++8g9atW9seLygowHvvvYcXXngBEydOxCGHHIIPPvgAixcvxtKlSxt1mQVBEAQhmOzbtw9XXXUVOnfujKioKHTv3h233HILcnJyvP4vh74fOHAAQ4YM8fn9zj33XGzduhXNgZdfflkJVJ0JEyY4ZPuaKk1O0DGlevLJJ+O4445zeHzlypWqsND4+IABA9CtWzcsWbKkEZZUEARBEILPzp07ceihh2L79u347LPP1M+33noL8+bNw5gxY5Cbm+v2fysrK1VHJf3P2JHpK+zKZDdmc6BVq1ZBjzaGAk1K0H3xxRf477//8NRTT7n0E+JVivNGSk1NVX9zR0VFBQoLCx1uhAaB9XXTDQjlFjrrQLZJ428D2R6Nv95b2jbhZ2qKNwY2eL6bPXs2xo8fryJukyZNwu+//479+/fjvvvusz2XKctHH30UF198MZKSknD11Vdj165dKtW5atUq2/N+/PFH9O3bFzExMTjmmGNUBIvPycvLU39nxovnV/35Dz30EEaMGIGPPvpIvQdF0nnnnafOofpzfv31Vxx55JHq/1jbPmXKFCU+jZ+FePqszn/fs2cPTjvtNFUnz89zzjnnqHO8xfCcxx57TInPxMREXHHFFbj77rvVsup/11Ou+n2WaDFqx8/LG9cPRTGzge3atVNiluvm/fff97qc3j6Pq5u7/bPZ+tAxvMxwMndY7nDBguLwkUceqfV4VlaWarIINtxITA9zI8q4ltBAtkloIdsj9GiO24QZHX4ueofx1lSg0Jg7d64SaRR1/Bx6t2Tbtm0xbdo0fPXVV3jllVdsjz///PP43//+p4Qe0T+v/tkpYM4++2zcdNNNuOyyy1RN2z333OPwHF1g6P/L33fs2IHvv/9e3ejpd/755+PJJ59UgopQ3N18880YOnSoKpfiufb000/HihUr1H7kvBzu0LcTf+pijtFIPsbXZzr4jz/+UM9lxJLL8Oqrr6poJdcFa/AoOo3Lrr/m//3f/2HLli0YPHiwEqmEIu62227Dhg0bVF0+xSg/a1lZmdvl5HeD3ob+dK/qn4lpco4CM8KJE81W0DGlSgO+UaNG2R7jyvvrr7/w2muvqR2coWTuVMYoHbtcGVp2x7333qs2nA53QF7tcINS/QcbbjxubL5+czkwNnVkm4QWsj1Cj+a4TXjBzpMm047G1OOpry1CVlFlgy9Pu8Qo/HTjkV6fR/FF8UABwtSpsxAYNGiQqidnZE1PkbKu/M4773RoRiD6Z+fz+/fvr8QN4Wtv2rRJCSP9Ofp219cVf+d+8eGHH6pIGLnwwguxYMEC23MYPTPCKB+XibV4rN/Tn+e8DZzhe/HvDOisX79epZx5niaMEPK1Vq1apdLQb775Ji6//HIVmSMPP/ywEn8UlMZl11+TYi06Ohrx8fHo0qWL7T3T0tIwcuRIHH744er3Pn36wBect4cn9PXKZXAOVAUSuGoygu7YY4/FunXrHB7jlQTr5BhO5cbliuSGo10Joereu3evUunu4IbkzRl9g9cHPDDW5+sL/iPbJLRolO1RkgO8OxE45RWg1/iGe98mQnP7jvBz6Ck2Y0SFYu5gYfCzM77gS2RHfw5FnX7f+H/Gx/T7o0eP9vgcCiyKIeNzdCHjvI6MPxn1MgY+OnXqpAIv+nO2bduGBx98EMuWLUN2drYtyseMG6N2rpbV3WfmbfPmzepcz9p4HYpPBnE2b96Mww47TJ33r7/+eofX4+Pz58+v9R7O68T4+3XXXae0BIXiCSecoFK0Y8eOdbuM7raHJ/T3dPW9CuR71mQEHa8AnDtyqKipbPXHqcgZbUtJSVE7GcPHFHNHHHFEIy21IAhNhqoSIG83sOoTEXQtmHaJ0SH9vowUUQQwgnbKKafU+jsfpwMEo6nGc2V94ByN4nIZa7+4fOy+pSsFxR7/xvM1s2mhzuTJk1W93i+//KIigwwqsXZRj2KGIk1G0PnCiy++qFQtVTWbHU488US88cYbjb1YgiA0Bcxa/Qss1p9Ci+Tnm7ynPRsTBjGOP/54lVpk0EJPdxI2B3z66aeqAcKfKQRMt1K4OPu91gXWhTFaRjF31FFHqccWLVpUp9ccOHCgiu7xpqdcN27cqEqtBg0aZPssXHauA18/C2sR9fo3IxTFl1xyibrxMzBtHcqCrknHzpmrNxoOMuf8+uuvq6LRkpISfPfddx7r5wRBEGyYm05hvNCyYd04gxa08GIdOQXOnDlzlNCjL90TTzzh1+tdc801KmXJ8iWmX9lIoPu0BTqeilFCis+3335bdbYy5WmsVw8E2pIxVcvuUzpe/Pvvv0q4sdN39OjR6jkUuawJZG0fU76PP/441q5d6/FzMHXMtDBrC/XUMFPF7PzlsrM5YtasWUpQhjJNWtAJgiAEjZrQTwMJAqGFBqNOPXv2VB2evXv3VnYktBuh7yrLjvyBr/PNN9+oIAgnMDH6x65Y4qrG3BeYLaPVGBsamWblJKfnnnuuThuQoowii2Lx6KOPVgKvV69e+PLLL23Podhjs+Mdd9yhmijZREJrEk9NBnwuG0wY5WNUjrX3jNrxdbg++F78Oz9PKGOy6MYpgq3LlX46bNGvry5XFo2y06e5FBc3dWSbhBaNtj0qioCnugBH3gYcp9kXCI28Teq5y5Une4qZYFphNRQ8ddP2gp2SwR7yThjlo1kxo39NneOPP15l6z7++OOQ2h6e9sFAtEizqqETBEEImOhE4NpFQEKqrEShxcF6c3a6Mk36zz//qGjajTfeiKZGaWmpEqKsoWdU7fPPP1cedWxsaO6IoBMEQSDscN00Czj8GlkfQotDrzdjDTptQW6//XaVcmxqmEwm1eDBCCMjYGyS+Pbbb2uNC22OiKATBEEguTuBhU8DMUnAmBtknQgtCrpE8NbUiY2NtU2NaGk0j2IIQRCEulJj7XLdv1LWpSAITQ4RdIIgCES6XAVBaMKIoBMEQSAi6ARBaMKIoBMEQSCt9MHcwbeAEARBqG+kKUIQBIF0PQyY/BwQ21rWhyAITQ6J0AmCIJCyPKDvccCws2V9CEI9ctFFF+HJJ5/06bkTJkzA9OnTm9z2eOutt3DKKac06HuKoBMEQSBrvwJeGQlkbpL1IYS0zxpvnNjB8VT8qT/G28MPPxzQ63799dcYMGCAmljAean0cvM2S934vvrt4MGDHv9vzZo16rVvvvlmNGcuv/xyNW/277//brD3FEEnCIJgbIr47X5ZH0LIcuDAAXVLT0/H888/r8ZC6Y/xxrmk/rJ48WJMmzYNV1xxBVatWoWpU6eq2/r1673+75YtWxzen+PhPPHqq6/i7LPPRkJCAhqLmpoaNc6uPkeAUWyff/75eOWVV9BQiKATBEEwCjrpdhVCGM4k1W+c9cmomPGxQITSyy+/jEmTJuHOO+/EwIED8dhjj6nB9q+99prX/6WAM76/p1m/FFLffPNNrVQkx4717dtXRQdTU1Nx1llnOfyd4uuuu+5CSkqKeg/nKOQLL7ygoorx8fHo2rUrrr/+ehQXF9v+PnPmTCQnJ+Onn37CoEGDEB0djb179+LSSy9VwvWRRx5Bu3btlDi+9tprUVlZ6fDeTz31lJq3StPi4cOHq8/gHKn89ddfcfjhh6vPsGjRIvU3fk6+Z1lZGRoCEXSCIAikpsrxpyA0YSjsPN0oXHSWLFlSazQWZ6HycW+MGDECHTt2xPHHH69mwHpi7dq1atj86NGjbY+tWLFCpV8fffRRFe2bM2cOjj76aIf/+/DDD5VYW7ZsGZ599ln1XONsVorIV155BRs2bFDPnT9/vhKAzjNen3nmGbz77rvqeXokcd68edi0aZMSZpz7+t133ymBp0Mx99FHH6maOP7frbfeigsvvBALFy50eH2OSeO4sY0bN2LYsGHqMX5ORuu43A2BdLkKgiAQs3VShAg6oRmwevVqj39nNEqHdW+MjBnh757q4SjiKHIoWioqKpRQYgMDxQuje67Ys2cPwsPDHdKyjJRRrE2ZMgWJiYno3r07Ro4c6fB/FEgPPfSQus9IHiOHFGIUkcTYNNGjRw81k5aClZE/naqqKvU7I2xGmBp9//33ERcXh8GDByuxyEglo5T8HzZvcJTYmDFj1PN79eqlInAzZszA+PHjba9DEUhRHBERoSJ2hK/JKCo/d0Mggk4QBIFMvB+oLgcOrpP10dIpOqjdjMQmA617AFXlQNbm2v/TaYT2M3sbUFni+LfkbkBcClCSDRSkOf4tOhFo0zvYnwB9+vRBfcKh97zpjB07Fjt27FDzYD/++GOX/8PUI9OduuAhFGUUcRRKTPvydvrppysxpKNHvIxiMjMz0/Y7BddTTz2FzZs3o7CwUEXFysvLVVROfx0KN+fXIRR4xveicGO6dt++feonX0MXjjpMyTqLTmPU0QjTtHyNhkAEnSAIgs4Jj8u6EIAVHwALn3ZcE0PPAc58ByjcD7xtj8zYeLhA+/nDdUDacse/nf42MPxcYMP3wC9OTQu9JwIXfR/0te6tlo5pQ0bYCOvSMjIyHP7O3/m4Pxx22GG2+jFXtG3bVokbCiIKLMKoHLtBmfL87bff8OCDD6oaueXLl6u6NxIZGenwOhSEelPD7t27VXTvuuuuUylP1tlxGdjgwffRxRqFlVFI+oJehzd79mx07tzZ4W8UpkYYZXRFbm6uqs9rCETQCYIgkL9fAHK2A1PtaRqhhTL6MqD/5NoROpLUGbjasX7Kgalvuo7QkcGnA10OrR2ha+SUK6NSTGEaU5esUdPTjP68J6NnnurtCOvM9PuEaUqmK3ljapVCjnVwZ5xxhtf3XLlypRJ3zz//vK0h46uvvvJ5mWmjwsghBR9ZunSpEsNsrqA41BsojOlVX2HEkpFC52hefSGCThAEgeTuBFZ/qom6K36TddKSSeyg3VwRGWNPr7qibV/3f4tvq90aAH9SrrfccosSLBRFJ598Mr744gvVrPD22287FP3v379fNQiQl156SXV+su6MooU1dBRhjLK5g5Eq1tcxgqYLulmzZmHnzp2qEaJ169bKo44CzZjO9fY5Wev26quvqq5SNmbokUdfYBSP0bz7779fRfsoKG+88UYlDhk9pA0MGyG4TEceeaRq6uB7UBBfcsklHl+bHnRMJffuHfyUuiuky1UQBMHYDJHXMAXMghAqsP7ts88+UwJOt+X44YcfMGTIENtz6DHHSJVRCN1+++3KLoRikJEu1rIde+yxHt/ryiuvxKeffmr7ndE4dpZOnDhRWaZQjLHblELRF7i8tC155pln1PLytVlP5ytcXjZaUFCee+65OPXUUx1sUdgc8cADD6jX5PKxxo8pWIpZb/BzXHXVVWgoTBa64Ak2WFDJrhSqcGNIOlhQ5bOYk10+nvx6hIZDtklo0Wjb4+vLgA3fAbEpwN27Gu59mwDN8TvCqNKuXbvUiZneYU0N3cDW2FXZFGB6k9G3L7/80u+UbrC59NJLkZ+fr8RrsLcHLU4oUrdu3ao0hb/7YCBapHl8MwVBEIJmLCw+dIJQX7BWjWnb7OzsZr2SDxw4oD6nOzFXH0gNnSAIAjnierbPAVulfk4Q6hP61TV3jnMyam4IRNAJgiCQHuO0bsRDr5T1IQgtgJkzZ6I5IYJOEASBbPgBiGkF9D5G1ocgCE0OEXSCIAhkyeuAuQrYvQgYfxcQ4WgcKgiCEMpIU4QgCILeFJG3G/j7/4CqMlkngiA0KUTQCYIg6N2tkdbxPeZqWSeCIDQpRNAJgiDoEbqoeEcLE0EQhCaCCDpBEATS7Qigw1BtXYigEwShiSFNEYIgCOS014CsrUB4JBChDeoWBEFoKkiEThAEgZTmAq27A6e/BSSmyjoRhHrioosuwpNPPmn7vbS0FGeeeaYaccWxWRzH1aNHD7z00kshtQ2ys7PV+Lu0tDSEIiLoBEEQyCsjgH9eAbK3SZerELJQ8PDGmbpRUVHqp/4Yb8bB8v4Y7Bpfgzdf5tsuWLAAo0aNQnR0NPr06eOTUe+aNWvwyy+/4Oabb7Y99uGHH+Lvv//G4sWL1cisQMZlzZw5E8nJybUeD6YwbNu2LS6++GI89NBDCEVE0AmCIOhdrvl7gNdGAwfXyzoRQhIKHt7S09Px/PPPq6iW/hhvd9xxR0Cv6/w6e/bs8fh8DpU/+eSTccwxx2D16tWYPn06rrzySsydO9fj/7366qs4++yzkZCQYHtsx44dGDhwIIYMGYIOHTooQRlqVFZqjVKXXXYZPv30U+Tm5iLUEEEnCIJg63K1nmSkKaJFk1ue63Cr9mBj4/zcKppTuyGvPM/xubyI8BMKHv3GSBbFj/Exo1DyB+fXSU31XHbw1ltvoWfPnkpUUozdeOONOOuss/Diiy+6/Z+amhp88803OOWUUxzmuvI1/vrrL7UM7ua8vvDCCxg6dCji4+PRtWtXXH/99SguLrZFCi+77DIUFBQ4RCr5WhSmt956q+1xnUWLFuGoo45CbGysej1GDEtKShwie4899piKyFHsXn311erxwYMHo1OnTvj+++8RaoigEwRBsFg07zmxLREAjP9yvMNtd8Fut+tl0reTHJ67JXeL2+ee9sNpDs9dnbW63tY3hZ2n27XXXuvwfIqj7t27K3Fz2mmnYcOGDR5ff8mSJbUG0J944onqcXesXbtWia7Ro0fbHvvuu+9w1VVXYcyYMSoyyN9dwdTyK6+8opaLKdr58+fjrrvuUn8bO3asSqsao4yMVPK1unTpgkcffdT2uB4RnDRpkqrb4zJ9+eWXSuBRlBr5v//7PwwfPhyrVq3CAw88YHv8sMMOUyniUEO6XAVBEPRIiU3Q+R85EYRQgmlQT1D86PTv3x/vv/8+hg0bpgQXhQxFEsUTBZErDh48WCuKx98LCwtRVlamIl/OMFoWHh6uGgt0UlJSEBcXp+oBGRl0B1O6xujZ448/rkTpG2+8of63lSFaaYTvl5iY6PD4U089hQsuuMD2mn379lVicfz48XjzzTdt9YMTJ07E7bffXmtZGKGjyAs1RNAJgiDQquSuXUBVKTDvUZkUITR52KTgK4yO8aZDMcc06owZM1TaMVhQ6LGBIpAauT/++EMJsc2bNyvRWF1djfLyctUhS0HoD2zMYGSOtXA6FosFZrNZ1QbysxNjJNEIxSrfN9QQQScIgsATTFwKYGkNPJwv60No8nirpbvwwgtVHZwrIiMjMXLkSGzfvt3t/zPilZGR4fAYf2fkz1V0Tu8SpRBigwGjar6ye/duTJkyBddddx2eeOIJFdVjivSKK65Qr+WvoCsuLsY111zj0Gmr061bN9t91uu5gg0R7dq1Q6ghgk4QBIEedN9dDRxzL9D5EFkfLZyF5y50+D0pyp6edGbOmXMcfk+MSnT73B+n/ggLLPbnRrp/bkOmXF01L6xbtw4nnXSS2+cwokf7ESO///67Q6TPmREjRqifGzdutN33hZUrV6roGZsnWEtHvvrqK4fnREVFqeV2xtXjtFrhMvgTxTSyfv16t80bjYk0RQiCIFSWANt/B0rzgLePATbPlnXSgkmJSXG4RYRF+PzcyLBIt89tHdPa8blM9dcTFCuebsY6NjYN/Pbbb9i5cyf+++8/Fb1jvRttSHTuvfde1fGpw/o1Pp+NCUyDspaNIosdpe5gVItiitE1fz9LVVWVsjzhe3788ce1oos9evRQkbd58+YpA2A9JcrH2UG7f/9+9Ti5++67lecdmyAofLdt24Yff/yxVlOEK/i6FJgnnHACQg0RdIIgCLpNSUQ0cGA1UOyYShKE5kxeXp7qNGXtGKNyrFGj4Bk0aJDtOewQ3bt3r+13WpbMnj1bReXYCcro2bvvvqs6XT1BkWisXfMFvj5tS5555hnlVcf/Zz2dkbFjxyqRee655yrh+Oyzz9rEKlO2vXv3tqVJ2fyxcOFCbN26VVmXML384IMPqmYHb1D4MS3L/ws1TBZWAgo2uCOzW4adPp5C0oHCsHFmZqa6OtJDx0LjItsktGiU7ZG1BXj9MODy34CPTgWOfxQ4/JqGee8mQHP8jrCgngXwFCa+TEUINXjqZmNARERESBrxemqMYFctrUI8pWdDlSOOOELV3p1//vl13h6e9sFAtEjz+GYKgiAEI0LHFFh4lBgLC0I9wYaJjz76yJb+bEpkZ2fjjDPOwLRp0xCKSFOEIAhCQgdg0tNAq64A66VkUoQg1Buh2FDgC+zS1c2MQxERdIIgCAntgCOu09bDKS8DbfvJOhEEoUkhgk4QBKHoILB7EdB/MjDoVFkfgiA0OaSGThAEIWM98O0VQFkesOZLYPc/sk5aCNIXKDSXfU8EnSAIQk21tg7YELHoRWDTz7JOmjmchkBCcYST0DIote57+r5YVyTlKgiC4NDlGilNES0ADm1PTk5WdiyE46Oakv1HU7Utaa5Y/NgefC7FHPc97oPcF4OBCDpBEASboIsSQdeC4DxSoou6poQ+TJ6+gCLoGp9AtgfFnL4PBgMRdIIgCDGtgI4jrIKOPnRVsk5aADzxduzYURkmc7RUU4LiIScnB23atGk2Zs9NGbOf24Np1mBF5nRE0AmCIPQ9HupGuh4OxKXIOmlB8MQa7JNrQwgIigJOGBBB1/iEwvYQQScIgsBuMz1Ncvwjsj4EQWhyNJk47ZtvvqkG6nKmGW+cAffrr786zES74YYbVLgzISEBZ555JjIyZMC2IAg+sPRN4Kmu2v3KEqC8QFabIAhNiiYj6Lp06YKnn34aK1euxIoVKzBx4kScdtpp2LBhg/r7rbfeip9//hlff/01Fi5ciPT0dDVzTRAEwSvmKsBkPRz+cB3w9aWy0gRBaFI0mZTrKaec4vD7E088oaJ2S5cuVWLvvffew2effaaEHvnggw8wcOBA9fcjjjiikZZaEIQm0+XKZggiTRGCIDRBmoygM1JTU6MicSUlJSr1yqgdO5SOO+4423MGDBiAbt26YcmSJR4FXUVFhbrpFBYW2goceQs2fE29vVkIDWSbhBaNsT1M1RR0kbCYzTCFRQDVFeq+0HjbRPCMbJPmvT3MAbxOkxJ069atUwKO9XKsk/v+++8xaNAgrF69GlFRUcrTxUhqaioOHjzo8TWfeuopPPJI7SLorKws9T7BhhupoKBAbXjpTAoNZJuEFo2xPRKK8hGDcGRnZiKpsgaRFaXIaYLeZPWFfEdCD9kmzXt7FBUVNW9B179/fyXeuNK++eYbXHLJJaperi7ce++9uO222xwidF27dkW7du1U80V9bHR6H/H1RdCFBrJNQotG2R7H3gWMuxbtU9rDlJAE5EJ5kwmNuE0Ej8g2ad7bIyYmpnkLOkbh+vTpo+4fcsghWL58OV5++WWce+65qKysRH5+vkOUjl2u3lyYo6Oj1c0ZbpD6OnBxo9fn6wv+I9ukhW+PhHbajRz/KGBh6lW+n426TQSvyDZpvtsjLIDXCGvqipj1bxR3NPSbN2+e7W9btmzB3r17VYpWEATBI/99DCx4RrsfFQ9EJ8oKEwShSdFkInRMjU6ePFk1OjC3zI7WBQsWYO7cuWjVqhWuuOIKlTpNSUlRqdKbbrpJiTnpcBUEwSt7lwI524AJdwNrvwK2/wGc8basOEEQmgxNRtBxePLFF1+MAwcOKAFHk2GKueOP18b1vPjiiypESUNhRu1OPPFEvPHGG4292IIgNDXbkrzdwI4/G3uJBEEQmqego8+ctwLC119/Xd0EQRD8F3SR2n3+5O+CIAhNiCZdQycIghAUaqqAMF3QRQHmalmxgiA0KZpMhE4QBKHeGDwVsFgMkyIkQicIQtNCBJ0gCMKwc+zroPtY4MQnZZ0IgtCkEEEnCIKQtgKISgDaDwBSB2s3QRCEJoTU0AmCIMy5B1jymrYe8vdp1iWsqxMEQWgiiKATBEEw2pbsXwl8dxVQWSzrRRCEJoMIOkEQBEbjbLYlVmFXI52ugiA0HUTQCYIgOPjQ6YJOOl0FQWg6iKATBEFI6ADEtdXWgy7sRNAJgtCEkC5XQRCEy2bb10FsMtB5NBAWLutFEIQmgwg6QRAEIx2HA1fNk3UiCEKTQlKugiAILw8HlhvmRXNqhD45QhAEoQkggk4QBKHooH1+a+Zm4JFkYN+/sl4EQWgyiKATBEFgA0RYhGNThFmMhQVBaDqIoBMEoWVjrgEsZrtdiXS5CoLQBBFBJwhCy0Yf8WUTdLoPnUToBEFoOkiXqyAILRtG5K74HWjd0/q7GAsLgtD0EEEnCELLhn5zXQ+z/x7TCrjpPyAhtTGXShAEwS8k5SoIQsumLB+Ycx+QtdUu8Nr0BqITGnvJBEEQfEYEnSAILZvyAmDp60BhmvY7/ee+uwbY9VdjL5kgCILPiKATBKFl49wUYTIB674Gsrc16mIJgiD4gwg6QRBaNmYnQaffly5XQRCaECLoBEFo2dBU2Og/ZxN01scFQRCaACLoBEFo2cS2BkZfAcS3tz8WHiGCThCEJoXYlgiC0LJJ7gZMecHxsXG3AJ0PaawlEgRB8BsRdIIgtGzY5Zq3G2g3EIiIsgs6QRCEJoSkXAVBaNns+xeYcTRQmm1/bO8yIGtLYy6VIAiCX4igEwShZWNrijB0uc6aDqx4v9EWSRAEwV9E0AmC0LKx+dAZu1wjpSlCEIQmhQg6QRBaNs7Gwvp9sS0RBKEJIYJOEAQhKgEIM0ToeF+MhQVBaEJIl6sgCC2bYWdrNyOtugCxKY21RIIgCH4jgk4QBMGZM9+RdSIIQpNCUq6CEGL06NEDl156qe33BQsWwGQyqZ9CPbDqU+CdibJqBUFo0oigEwQvzJw5Uwkq4619+/Y45phj8Ouvv8r6a+qUZAK5uxwf++lm4MuLGmuJBEEQ/EZSroLgI48++ih69uwJi8WCjIwMJfROOukk/Pzzz5gyZUq9rcejjz4aZWVliIoydGEKwYPND8YOV1JVBpTmyloWBKHJIIJOEHxk8uTJGD16tO33K664Aqmpqfj888/rLOgoEsvLyxEbG1vrb2FhYYiJiZHtVF/QnsRZ0IkPnSAITQxJuQpCgCQnJysBFhFhvy4ym8146aWXMHjwYCXCKPiuueYa5OXl1aqTowicO3euEol8nRkzZrh8H1c1dBMmTMCQIUOwceNGlfqNi4tD586d8eyzz8r2DEjQGSxLiAg6QRCaGCLoBMFHCgoKkJ2djaysLGzYsAHXXXcdiouLceGFF9qeQ/F25513Yty4cXj55Zdx2WWX4dNPP8WJJ56Iqiqrga2VLVu2YNq0aTj++OPVc0eMGOHXtqBInDRpEoYPH47nn38eAwYMwN133y11ff4y/HxgyguOjzFiZ672+6UEQRAaC0m5CoKPHHfccQ6/R0dH4/3331eCjCxatAjvvvuuEnDnn3++7XmMoFF4ff311w6Pb9++HXPmzFFiLxDS09Px0Ucf4aKLLrKlgLt374733ntPpYcFH2k/QLsZGXMDMOoSWYWCIDQZRNAJgo+8/vrr6Nevn7rPpohPPvkEV155JRITE3HGGWcowdaqVSsl8BjJ0znkkEOQkJCAP//800HQscEiUDFH+JrG6CCbJg477DDs3LlTtqk/bJkDlOUBI6bZH2vdQ9ahIAhNChF0guAjFEvGpgimS0eOHIkbb7xR1cNt27ZNpWVpaeKKzMxMh98p6OpCly5dVG2dkdatW2Pt2rV1et0Wx8YfgbxdjoJu+zxg3zLgmPsac8kEQRB8RgSdIAQIu0+ZTmX9G8UcGyIo5phydUW7du0cfnfV0eoP4eHhbjtmhTo2Rez/D1jxvgg6QRCaDCLoBKEOVFdrhfNsjujduzf++OMP1RBRV7EmNCBiWyIIQjNAulwFIUDYtfrbb7+p2rWBAwfinHPOQU1NDR577DGXwi8/P1/WdVMxFubvfFwQBKGJIBE6QfARjvnavHmzrR7us88+U6nWe+65B0lJSRg/fryyLXnqqaewevVqnHDCCYiMjFTPYcMEU7NnnXVWo65vpmOd6+5aPF0PA8IiXPjQiaATBKHpIIJOEHzkwQcftN2naTB93958800l4nTeeust1dVKk+D77rtPmQ7TRJjdqEzFNiZ79+7FoYcequ4PGzYMQ4cOVT95GzRoUMudRnHUbbUfSx0CjL6MChgQASwIQhPAZJEKagcKCwuV9QS7FRl1CTYsnGd0h8XzLKoXGp+Wsk327NmDUaNGITe39oxSfm5asjgLPfraNXREr8G3R1EGEBEFxLau//dqorSU70hTQrZJ894ehQFoEYnQCUILgeJs3759asoFrU3WrVunfvKWk5Oj0sm8ffXVV7b/oceeLvCMP3mgaTZ8MU2LyJ36iv2xsnzNyiR1KBAuh0lBEEIfOVIJQguCM1+ZdtVTr4RB+gMHDjgIPN7nnNiioiIsXrxY3Yx069bNFsXThR4jfMa5tk26y3XXQuCri4G7dgFxKY21ZIIgCD7TBI++giAEE6ZUO3XqpG7GyRXs4uW8WWehxygf6/F4mzVrlsMoNHb7Ogu91NTU0G7EcNflqv9NEAShCSCCThAEl7BDd8iQIerGqRg6eXl5StgZhd769euVFx+7e3lzNlQ21uXx/uDBg0PHq09F6Fx0uRJzPQi64kwgJlmr2xMEQQgSIugEQfALjhc7+uij1c1YELx79+5atXm0bMnKysL8+fPVTYdFw3379nUQenoTRmhF6CqD/37/1xcYcQEw9Y3gv7YgCC0WEXSCINQZCrRevXqp29SpU22Pl5aWqlo8o9Bbs2aNasJgOpe3b775xvb8hIQEJfL69OmjZucOHz5c/Z6cnBzQcpWUlOCdd97Bscceq17HJTczoug0Li0iBohuRaWKeiHM9dg2QRCEQBFBJwhCvTZhjB49Wt2MTRgZGRkOdXn8SeHHtO2SJUvU7eOPP7b9T9euXV02YTAt7AlO8rj11lvVNI8XXngB119/fe16PlddrDQbvncv6oXkbkC841xfQRCEFiPo6L7/3XffKVsF1t6MHTsWzzzzDPr37297Tnl5OW6//XZ88cUXqKioUAXeb7zxhirKFoRG5dd7gKzNwMU/tPgNQUHVoUMHdeM0DWMTBlO0rMFbunQpdu7cqcQemy/YiMHb7Nmzbc/XR645Cz2+ri7aJk+ejFNOOQU///wzbrzxRixYsEBF7Bwifh+fDoy6BBhsjyzWG7k7gfy9QMH+Fr8fCILQQgXdwoULccMNNyi7Bc7FpAs/Twa8qo+Pj1fP4ZU4D/gcs0SfLB7AzzjjDPzzzz+NvfhCS6emAshY39hLEdIw2saJFZzAMXHiRJtBJ2fgsunCOaLHaB7Tt7wZadu2rUNt3v33348jjzxS/WR6d+XKlfjyyy/t1i27/wH6n+S4MLm7gM/OBc6YAXQaGbwPmbVF+ykmxoIgtFRBN2fOHIffZ86cqQ74PDizOJtuyu+9956ar8mTAfnggw/UFTyv9o844ohGWnJBANCqC2CukVURAIymUZDxZmzC4OQLZ0uVrVu3Ijs7G3/++ae66TBix7Qt/7Zr1y4V4ecM3kcfeQQmNj44z3K1mIHsLUBFcfA7XGECTng8uK8rCEKLp8kIOmco4EhKimb6SWHHlM1xxx1new6v9GmAynocd4KOqVnejOM29BMGb8GGr8kaovp4bSF0t0nYvEe196qqsFtiCHXaHuyI5W3KlCm2x8rKylTUXrdV0QUfO22ZutVhlP/xxx/HhvXr8N1wC8xhkcgpLMMPq9Mxrk9bDIiNAIf3mKsrgtsYUZQBU1wbWExh9ddwUQ/IcSv0kG3SvLeHOYDXaZKCjh90+vTpatg5PbLIwYMHVU2Nczcc6+f4N0+1eY888kitx3kCYE1efSw7xSg3vMxEDA0aYpt0sP7M3rsF5vj29fIezYW6bg9G4ng76aSTVLqWdXMsxWDZBidfGImPi1M/C0vK8PAPazB7Yw5SEyPx43kd1DYryM1CRSKjasEhMXsP4kuzUf7ZRSg47nk0FeS4FXrINmne26PI6VjVbAUda+lYU7No0aI6v9a9996L2267zSFCx5MBzVB9HYjr70Zn+oevL4IuNKj3bWLRLDEsUYlom9oRiGsT/PdoRgRre7AR4uyzz1aRex1e8LFZimJv0qRJaNs6CeZ1XyOp+zjsW5OunpNRVIW4ZE10t0qIA9oHUYD3Gw+s/wQxYdWIDubr1jNy3Ao9ZJs07+0RExPT/AUdGx04buivv/5Cly5dbI+zs62yslJdkRujdLRH4N/cwXFFvDnDDVJfgosbvT5fXwixbVKulQeYTn0ZpgSxq2io7cGLM4o5NlqcfPLJKjXL2rla82ZHXaR+lFbusz1UFhaPpGlfIIwNEcHcJ4adA2yeBVN5IUxN7Psvx63QQ7ZJ890eYQG8RpMRdAxj3nTTTfj+++9VCqVnz54Ofz/kkENUl9y8efNw5plnqsdoWsq6mTFjxjTSUgsClUKuvcMxfx+Q3FVWSwNw0UUX4bzzzvPsVVdRBKz7Buh3Ikorqm0Pl9aEAf0nB3+h0lZoAl9mxAqCEGQimlKalR2sP/74IxITE211cbQnoS8df15xxRUqfcpGCaZLKQAp5qTDVWhUYloBJz8PzL4diE4Cxt4oG6SB8GY8jJJsYNZ04OKfUFpl70Iuobj7+3mg97FApxHBWyB63lUUAp0PCd5rCoIgNCVB9+abb6qfEyZMcHic1iSXXnqpuv/iiy+qMCUjdEZjYUFoVOJSgEOvBBa/CpRkycYIJfRIWXgkSitKbQ+XVtYAfz6pifFgCbqqMk3MTXwAGHJGcF5TEAShKaZcfSkifP3119VNEEKGrK1A+n9AXFstIiSEDvSgo42JKQKVNXabgJLKaiA8KripUeVBB82oOKVX8F5XEASBdXeyFgShntkxD/h5OpCQCpQEzwJDCAJmTbCVmcMdHi5jhI5+gVbBFxT06Oz+/4DfHwze6wqCIIigE4QGaopg2jWlJxCp+Z4JIQK3R4+jUGbSxgc61NCpCF0QBV1liRalLc4A1nwZvNcVBEFoSilXQWiylOUCsSnAiU809pIIzrTrD1w6C4WZHPG1w7GGbsAUoE3f4K2zXuOBu3YAi14CqoNvWi4IQstGBJ0g1DelOUBca1nPoUhNtUq7llY41sopQXfKS/XznhHRwY38CYIgSMpVEBqA1j2AzqOBLXOAp7rZjIaFEKlvfKIDqgoOODxcyqaIogy7h2Aw+O1+4LNztVQuI3Q+NHoJgiD4ijRFCEJ9c9zDwHEPAVFxQEWBdLqGEtZIWWmNyeHhkooazTNuwdPBe6+83dr7dRwBjL1ZBF2g5OzQzKAFQXBAUq6CUN9UFANR8UB8O3u3Y5vest5DAastSXGVU5drVXXwu1yLs7TGmC6HaDchMGbdCuxaCAw4GYiMlbUoCFYkQicI9c1zfYBlbzkKOiG0BF21iwhd0H3oMrR9gGncXX8D1VJHF3AJQ7sBIuYEwQkRdIJQn3A6QHWZ1uUa2xowhYmgCyWsEbgSJ0Gnaujqw4eOXoT7VwIfTtGaZQT/4Tbhd4lzkdnUIgiCQgSdINQnelE9fejCwoEr/wAGTZV1HioMORO4bRNKnAJxqss1mIKODRAX/wgMPl2L/BGxLgmMiiLte/XyMGDrr8HZPoLQDJAaOkGobw86wggdaSlD2XnCXTYDGH83EBbC141sVImKQ0nFZoeHSyjoLv8GMDnW1gWMyQR0Ga3dL9in/RTrksAYPk37+c9LwPL3gIGnBGcbCUITJ4SPtILQDNDTaozQkRUfaEKnufPXc8DCp0M/rbjpZ+C7a7SInIFSNSkiMnhiNHs7MOc+oCRH86EjEqELjIFTtNvoK4Cdf2pdr4IgBCboPv74Y4wbNw6dOnXCnj171GMvvfQSfvzxR1mlgmCk+zjgts1Aq67a73v+ATb+1PzXEZsJ2vYHEqyNIKFK1hZg++9azZwBJfBoWfLr3UF6n03A0teZewUi44HEjsF53ZbI5l+AzE3A4KlATDKw8oPGXiJBaJqC7s0338Rtt92Gk046Cfn5+aip0a5sk5OTlagTBMEAozxJHYFwa3VDfPuW0RRxcB0Q1wYoL0RIY2YkLlpLsRpQAi93p/Y5gkFxppa+Zeq9XT/g9s1Ax+HBee2Wxqzp2kURLUsOvUIaIwQhUEH36quv4p133sH//vc/hIfb60tGjx6NdeuCdPAThOYCh7D/dLP99/i2QEkmmjVsADiwGti7GNgU4tFI1rGFR2opVgMlwW6KoKCjZUko1xM2JV/H6ETt/rEPApODaP4sCE0Yv48uu3btwsiRI2s9Hh0djZKSkmAtlyA0D9JXAfuW2X/nSb0sL7j+ZqEGRdCRt2n3S7IR+oIuqlYNXWW1GWZTRPAEHUW8nn5mw8gLg4FtfwTntVsS5hqgqsQu6EhlKbB5dmMulSA0TUHXs2dPrF69utbjc+bMwcCBA4O1XIIQesy+A/jlLv+7XPUOV9JphCZ2mrOgY9H/hLuBlN6hn17uewIw7uZago5UmSKDt526jQWGn6/dD4sACtOAihBPR4eqZQkxCrrtvwNfnA8cXN9oiyUITdK2hPVzN9xwA8rLy2GxWPDvv//i888/x1NPPYV33323fpZSEEKB5e9oP0961vf/YTRG73AlHYZqt+ZM2grAYrZORQjxLteeR6tbyZ8Lav2puO/piO4zITjvM+xs+/2IGO1ndUVwXrslQYHdaaRm0KzT/yQgoQOw4n1gyguNuXSC0LQE3ZVXXonY2Fjcf//9KC0txfnnn6+6XV9++WWcd9559bOUgtBUoaBJHex4QmIKtk1fINFwUmpO0B+svECrF2R6OZTZ/5+a5lHKUV9O5LcZjjbtEoLzPnuXaSOruM1Zm0dqRND5DdPWVzuJb67PEedr3a4nP695/glCCySgCt0LLrgA27ZtQ3FxMQ4ePIi0tDRcccUVwV+65kjmRqT8eCGQv7exl0Twl2Pu16ID/jDmBmDoWY6CbubJwM7aEaFmA1NfqUOBs94Hpn2BkIaegH8+Ucu2hFjSVwOrPw/O+3x8OrD+G+0+BUd4tMxyDbThhjdnOo/SLh7YfCIILZQ6tVzFxcWhffv2wVualoApHFEHltvd4oWmw9F31I4OeINirtcEp8kECaFXW7bhB+Bza41XXWuc8nYBHYZotXShHi2pqYSFXa4uauiid88Hfrs/OF2ZLOSnZY3ORd9p5riCf2yfBzzWFihMd3ycUfABU7S5yYLQQokIpCnC5OEgvXPnzrouU/MmqZP2s+hAYy+J4A+MCjzRAZj8DHDIpb79DweHr/sa6HkU0KqLk3VJiAm6nO2O3biBQsNXkjoE2DQL+Odl4MrfEbLUVMIcFolqc+2oT4U5PDhNEbpNTYJB0PU4su6v2xJhIwm9A6PiHR9P6QWc92ljLZUgNE1BN336dIffq6qqsGrVKtXleueddwZz2Zon0Ykw0ym+UARdk6KqTBvV9PMtQL9JQGIH7//DFNAP1wLnfe4k6ELQXHj330BpNpC9DWjbN/DXYaE/59W2669NYUj7F6gsqX0CDhVqqlDDblYXVFiCZFtS7ELQUei2Hwz0Pa7ur9+SqCzWfjLK7er7RiPr1t0bfLEEoUkKultuucXl46+//jpWrFgRjGVq9pjjUxEuEbrQoSwPYcUZgKfyAaPFBDtXfRF0znNcdSiY9E7HUPL30j9bXWA08qr52v34NtpPitdQFXStOqPC7FrQlZvDAHNVcC4GKOKNKdeVHwIDThJBF0hKn2IuzG5qb+PHGzXBd7GMoBRaJkGzLZ88eTK+/fbbYL1cs6bo8NthGWIolBcaj8oShD3XC+0+m+j5ecYRVvSW8wX9eUYfOjL1jdCzVwiP8u+zuYORZ7NZu0/bEsKB9KHKlBdx8PAHbL9GhtvLSfIi2muzeF0V4ftD72OAO7fZBS6hoK8OkmlxSxN0Rg86IyoqvLWhl0gQmp+g++abb5CS4nTiElxS0fM4rStLaHyyNqsfJtbleII2HDq+WnHo0S7ONA11jrhe+1kXmxEKuVcPsQ6h5+duq/0MtfSyE8Y5rm0Tom33tyaPAy75qX4aOyKixLYkEA6/BrjUzVSItv2BovTQnx8sCKGScuXYL2NTBM2FaV2SlZWFN954I9jL1yyJyN4M7P0ZOOyqxl4UwVrEXxPbFh5P2zQDvn4p8MYRvqclGYXpNAqITXZ8fPVnwG8PAHduD50uUNZysbazLoKO3a3s5mw/yN78ccY7QMdhCFneOxHt4/sDOFH92i4xGgcKytX9svJKrUOV6eK6bKc59wK5u4Dzv3CK0IkPnd/EttZurtBrP1kH2uWQADeWILQgQTd16lSH38PCwtCuXTtMmDABAwYMCOayNVsiD66EafGTwOgrZFh3iAg6rxG6yBig/UCtPqfdAN9Fkquid9p5sAGB6aOYJIQEnIV57APA4DMCf40M6+glfRIGDV+HnYOQprIY1dFmlxG6Lpl/Ak/dB9y1q3YdpD/k7dE6M430nwzEOAl9wTtLXtemkIy9qfbf2vbThLLUJwstFL8F3UMPPVQ/S9LCmiKUgOBJ3dj5JjQ8mRvVD1NViYo2u2XHn5ox7Kmv+R6tYQSGtWnOz7fVlmWFjqCb/4RmpVGX6RU0FOZnM+7T677RIip9jkVIUlOJStgL7NsmWGsJGaGrsVak1LXTlbYlTAcaGee6uUzwwo75QGQsABeCLjoBuC/ddcOEILQAfBJ0hYW+1yQkJYXICSqEqeHcQVK4XwRdY3PsQzCPvxeZkV3Q3pNQO7gO2PgT0OUwLQVnnP7gjlm3ATnbgCt+cyPosoE2vRESVBYB678FUgf57rPnTP4ezX/OyL/vACk9Q1rQKXsSK0y56pTUhAdH0LGD2tl3jsa4FPxcN4LvMKpt7BZ2RsSc0ILxSdAlJyd7NBMmjG7wOTU1tR3XhdoROltHoL+jpITg0mmEVsyf6WVkEG1LopOAjT9q0x58EXTsGI1pVftxY4QuVKBXHG1WODEiUEF3xttAZanjY8pEORshS00VKmhP4iLlWlJlPebVxVyYUd/iLMdh8oQ1lBR6l84K/LVbIqxpdNflqqdkN3wPXPlHQy6VIDQdQffnn3/W/5K0IMyxbWDpewJMng5MQv2TsUGb5DDgFKT8dCtw1jtAe6fUmA4755geZS2Vr6bQbJ5wFYGhjcmV8+pm4Fsfgo7UpSmCUOw6CzrORA1VLvoBq1bkAJtyawm6YlvKtY5edJfNBhKtE2J0pCki+LYl+nrd/59mCcNOYkFoQfgk6MaPH1//S9KSMIXBMu1LmMKC5hojBMKexcDi14AhZyMqcw3MakRTf88ROtaDZWh1d15hxItTE5zhdu8yOnS2GSOUnUdrEcVAfej2LgN+vhm48FvHqRi0LtENlkORdv2QBVrXaJ+7TXyUKnlkYG2DpRdw+xa7/Uog8MVc7QMUG5w8IvhvW9JxuPu/04vOUgPk7gTaS5Oe0LLwuylCp7S0FHv37kVlpWN9ybBhIWxREGpXmhwk7cvEAaH+GiLYGccokrPXnDPsSuQ2y9/rexSL4ijOjcXCP69o/nQjL0CjQ4HJKBKX6a/nAnuNA2uAnB2165t48mWtaKgy+w6k5rErV9sH4qMjEB8VgeKKahRUhdX9+8nxZys+AMbf5dgpy0hSMMaKtTTG3uj573rzSfYWEXRCi8NvQUe/ucsuuwy//vqry79LDZ1vmBjNKM+XMTWNbVlCKxK9zs2TIemg07Sf2/7Q6nh84bol7tM+2//Qon2hIOgYoeOIK0YNR12shaf89V3LWKfZuTh/3kGnardQZeVMJKXeZBB04YiNCleCLr4iE/jiAmDi/dp+Eug+tuxNYMLdjo+rTs0Q8SBsKjD1zS5X1h27cwfgxRm/V9kyMUJoefid85s+fTry8/OxbNkyxMbGYs6cOfjwww/Rt29f/PTTT/WzlM2RxI5ap5vQOFC0MELHE3V4FMwRsZrAdsfuRUD2ds1XbtKTvr0HLUDcmaDyhBQqzQI8+T3eXpUC4MQnAjPRpWVJB6cOV1JTDRSkhaaJLvcBc5XdnoTljVGM0GndrTWVZcDmWUCxl4YZT/B/aV3j7Dl37IPADUsDf92WCCPjn50D7F/p/jncdy/4BhgVYGOPILQkQTd//ny88MILGD16tDIV7t69Oy688EI8++yzeOqpp+pnKZshFiXofCyuF+rnav/Qq4BeE9SvhROeBHp7mOf64w3Aqo+1Lk6mFzlw3RMF+7XoDl3rXcFOV1WzF0INEaZwYN+/WmrZH8w1WiTK2bKEZG0CXhys2b6EGtZmh9Jq+2GQYi4uSktcFFZahS2jl4HCbcw0dKhMBGnK6Pult2YyRpoTrJ3kDbEP7VzYMO8lCMEWdCUlJWjfXgt3t27dWqVgydChQ/Hff//5+3Itl6SOmveXzB1sHJga5GQEa3NCeZ+TtHo6d+hdrozqzThaqxfzBN3qGd1xF5lSdh4hYlvC/ZDQ6Pq9431v+jDC+ZqDT6/9uNFzL9Sw1rCVGiJ0FHNx1ghdqc2Hrg6CjtYkrsTF2q+Bd0LUmy9UYWMSiUrw/Ly0FcAP12ulBPXNwmeAj07VLuAEoakJuv79+2PLli3q/vDhwzFjxgzs378fb731Fjp27Fgfy9g8YYSONTR1SecIgcOIEQ/8VqJ3/QFsnes+NWfsciXeGiP0zk42Prii21jg0Cu11w6VCF2rroFZl9DMlbMzW3Wu/Tf984eKeHVe7nG3YJtFW+7IcBOiIsIQF61F6GwTJOrSvNDjaGD4+bUfZ3r/QAjbuYQieu2qtwgdv3urP22YZhx9mVRNpCA0saaIW265BQcOHLCNAZs0aRI+/fRTREVFYebMmfWxjM2TrocDD2Rp8y6FhmfJG0DWZuBqzWMxdtPXMMXGAQMm134u06sc1cbmCZugy/XuQUfczQDtPka7hQL6SUm3G/FX0DHlxCaPEx6r/Tfu36wfY/Qv1OBJ+PhHsXHNAipOxEZqAk6voStDDAqPvB9J7QcF/h7Dznb9OOf5cp9iFEnsi3yDaeuUXq7Nuo3okXZ2uiZbL1Lqcx9K7la3Wb+C0FiCjvVyOocccgj27NmDzZs3o1u3bmjbtg5+TS2NsAg5kDcmTJ0aivgtvOovdyPS6BfWpo+WPlQnE5N30UPBFxmvnbhdQZGYtlwbZO+ucaKhYBdq97HaZ+My++tFx7q7NV+4FnR6elkXuKFEVbkqsA+roF1NhLIsIexyVX9GBDKHXYektl5SfN68DilCnO1PaFtCaiqAMInu+ATHp928yvvzKLC4frO2An2OQ71SdFCzMsrfV//iURCCnXJdtGiRw+9xcXEYNWqUiLlA+HwasPzdgP5VqAMs4qc/mCHyYo5Kcu9Dx6vvm1YCvY/R0nQ8OdOJ3hNdjwCOe8j935mC/PAUzdW+sVFRhq5aBKR1D8DiZ+0RU1zuUsvk2n9UJCzkKEoHZp6EbpVaPaReOxdvbYogkVtmaSa1gfLx6dooKmfY+UrEXDj48Dvapm/DWJdMuEf7yQi1IDQ1QTdx4kT07NkT9913HzZuDKB4WrDDq7pACtCFupG3WzN1NniLaRE6D7YlRm7fDBx+tefnsKaMrvbu0KcPhEKzwNqvgNm3a/evXwyMpS9bEAVdZExodnlamx2KrT0PendrXHS4XZf/eSOwfV5gr8+OaAo2V+um62HAOR8DkU6j0gT3LH4VeNXF1A1XjLke6HtC/a/N1t1DfxqK0GLwW9Clp6fj9ttvx8KFCzFkyBCMGDECzz33HNLS0upnCZt7pyu7IYWGhZG41KEOEbrqlP5ao4IrdvwJPN3dP5uZXX87NF24nHnKbr1QaBbgrNVdfwX+/6yP81RDtPRN4NsrEXJYmx0qLBEOEbq4SHuEzhwWGXiXq566dpVST+qkpbrdpeSF2jBt7+u2GHE+MOCk+l+Lc+7T9v9QLCkQWhx+CzrWyd144434559/sGPHDpx99tnKWLhHjx4qeif4gfKia6B2d7bVP9zKs8hoKXQeBVy3yKGuibYlltNnuH4+6+UYvdMHz8++A/jpZu92Bkvf8PwcX73oKA4DHcnlC5XFdiuIX+8GvvbTlLX/ScDAU9z/nVHIvUtDVtCxVo7oNXScFqFjNkUG3uWqn+RjXYhdXhz88zJQIpEdv3zo2GnuC/zOrvumfm2h2NCy7C3tvkTohBCgTtPhmXq955578PTTTysfOkbtBD9I6txw5sJ6fdhOrauzRUBHeRYtu1oXznYh7DikZxgnG7j0vzIBUYn2/8/Z7v1k7upEboSjsvRaKk98fw0w/3HUr6CL1+4zRZi7y7//P+wqYNg57v+ue+6FgkWLAybUxKeiHFGOETpDDV0Nm5cCjdBRCCZ0cB295JSY3x+UCL3fgs7HBhWu32+vADI2oN6gaLTUaClXb523ghDKgo4Ruuuvv155z51//vkq/Tp79uzgLl1zZ/BU4NRXGua9UgdpjvUNYbYZKnx2HvDfR7XrFN87AfjtfoeHotKXI+yFAUDBvtqvw6t81tjp9hI8QXtLsTDd5qmujJz/hTYn1Bt6FLe+BBF96HRvL6YH/bUtYQTRUy0gI5EUirrfXajQeRS2X7QSuywdHZsiDBG63Pi+gVtS0LT6ji1ASs/af9NTraE4Ei1U4YWHNw86HXalc5QdrUvqC14AkmmfAyc9iyaPu6YwofkKunvvvVdF5phe3bt3L15++WUcPHgQH3/8sfKkE/ygXX+gvwvfs/qAszZ5dcsW+5YAI22MCrHT7c0xwObZ9hMoo2u0kjBg1k8Urg5quqmwjjfRQ+FFwRdsb6r66ogcPk2rOQpE0LHw/8MpnhsHQthcuKTSHpHVI3O6Hx35YejrwKFXBP+NdUFH2xLBNzhn+MSnfF+/7NimdUl9Czp9GkpThhdb/9cf+NPHOdVC8xB0f/31F+688041HWLWrFmYNm2asi4RAoDiYdGL/qe4AuHLCzT7hVinIeHNFWVkawGGnAkMmAJ8f522ninmmF5NHezwdEt0K/eCjhMdGE3TYSpVpVvcRMyYoqPHHf2wvJkbvzLS83OMEVV/Z6z6EynWa+D42ShgfU0z6oX/nqKR9No7493QM1/dMR+DvhyL1ih0iMzptXSk1CD4/Gbxa8BbR7n+m9iW+A8FWts+vj+/bf/6tS5hDfSYG7WLxWd7o0nDGmt2/rP2d9PPjb00QkMZCzPVKgQJ+n398bB2oHKVlgkWFCm06mDR//Dz0CLQr54TUoHTXgfengB8dTFw2NX2+jUDZmN9nDNsnjAaw/af5PnEwjmxV833zf+N24W+ePTO8iaYWM9VH2z7HUjuDrTrB/Q9HrjsVy1d5Qu2EWcpnmvo3E1MaEwqihBTkg6z9brWZltiTb2SaWsuB8xHA5Of9v/12cFeVer6b4wI9z+58U2lmxLzHtPS2L5mNWiWnbOt/pan/QAtarjqU+0Ckt6U/O43RfSyDl68djmssZdGaIymCKGOcCRSRGz9N0bohcGMlDDl2BLq6JhCYLEyBR2jkud8pF2t/3Sj1oziFKm06F2errzolr0NrPzQ/jsjb70nuvdW4/r1pd6NV/gU9R5TkSbgiBuAy3+rvwjXTzcBG76zLlMH7UToTmD6O7NW5993NHuUUMIahdS7XF01RVhY9M7IRSB4aozhtpz2GdDJS4TWeb/a/AtQ0EItolZ9rM1g9pVxNwOnvlp/y5O5CTiw1r7v+zthJZTguLRTXtZS2omp2ozx4tArkRA8I4KuMaEgSGoA6xIeBMOjNeH4eHvNIb+5Q1Fy1w77wPiOw4ALvwXuSweu+bv288MiYL57DzDCPtrOxqafHH3aeKBjrYm7E+uuBcCjbTTjaE/oUT9PXoTxbYDjH9E6JuvLgqHC0OVKGw12X+Zo0xN8EkUUzd4EHbt0Q63D2knQxbuI0FVawuvgQ5fnXoRT8Bdl+N4osmcJ8M4xwBfTXE+eaDFdrj42RRhraf9+Aciuh0jdopeAX++y7/tN2bqEx8lDLtUijNw3PzsX+Ooi7xNxhJBCBF1jw2hRfZsLUwgwVaEPX28pjRGuZkFSuFAkuYKND64GpTMNG2Noiqgs0mpN3IkeRmYY2fFWr8gIHXFlrWKsbdk6V2vs2LsEQYcHb6MPHYUj/dF8PQH2OxG4Y6t3OwllXRICUzFc+tBZI3MuaugqKfYC9aFjxMZdSpXr/fl+wHprZNTT9qcv4AeTtAvAy+f6P8mjOUBhxvS134KuUovs/XCda0uiupZ1JLS3i/bGNhfmhdk7E4G8PYGVXWyZo93nfjbpac326dc7Q9BuSHCHCLrGhifEjiPq9z3G3wlcOts+PLolCDpGmT71r27L9Mvt2nghX7pcibtuUB7YOWFAF0nuYHcca+0oNN2x/hvtSrm+miJUjZfFvqzePlugxIWgoOt7Ar4bQvNnk1PK1RChM4cHLuimvAQcZR2pZqDGbNEuHLiPeOtc5ntz3u/Ut4Ar5wPdjmiZJ1heRBFv3ylnaAbOdUdxsjjIFlFMS9IKqlVX4NpFmmF5Y8LAAD/nDh/qd12VRKycaf+92+HAyS9oj235NaiLKYSQoOvVqxdycmqHlvPz89Xf6hN22J5yyino1KkTTCYTfvjhB4e/WywWPPjgg8obLzY2Fscddxy2bavHothgwKtt1nrUF6y7YdE9r7oYneKJ1VsqsDnAZgN/T8SZm13X6KgIncE4lB2xbBpwVzOje9B5m1/Kk3rnQzxHHXjSaG1tmGEkLdhwHXEMGiMN+txVzhf1VdDN/R/wqQdTYWOETnUehxBJHbElzn4S1mvnoiPCEGbddK/FXw+c+GTg3o9t+zo8tGRHDkY++hsu/eBfWGit4c2Hjs1SN68CRkzT9heOUPvMh/UdZEzzHkXcmvcb/H3tCxCuHStp9eQvFCdjb9bKJIJpNMwpLyw3YJqS9cl62UJjoZdw+BvF1I2YOY7OyKiLtPGIm2cFZ/mE0BN0u3fvRk1NTa3HKyoqlJVJfVJSUoLhw4fj9ddfd/n3Z599Fq+88greeustLFu2DPHx8TjxxBNRXl5P/l3BoKpMK66tr6vuzI3AU13sQoUF/fkBhOSbGhRCdOn3B4q2MhdNEaMv14ap6/DEyoYWtxG6nFq1U2vT8vHP9mx10VHLumTF+57TOkzNMjLBlEqwYUSOY9B6jXd8zNcCb9YR+uKl1n0c0HE4Qopdf+PQvfZ1H28VdLxY1O/vrG5nL1XwB25njlFjdM3Ah4t3o7C8Ggu2ZMEcFuV53b00VNs3jA0q7HSuz3FW7tj9NyLyvExHqU9Y8nDC40D7gYH9/zH3aeJ69WfBu1Dm90TPevz+ELDxRzQqeoc+j33+wjpuZ0FHTnisfnwYhXrBZx+En376yXZ/7ty5aNXKHrGgwJs3b56a51qfTJ48Wd1cwRPlSy+9hPvvvx+nnXaaeuyjjz5CamqqiuSdd16I2nWwdoEptTt3uq/tqgsH12ppNVqjkPM+axljaiiEuhzq3/+w5s2VJ+CxD7o242VnmLuTh+Gkuz2zGFNf/wfMtL13yWgcOzDV/tzdi7QoGUWjK1hfp9fpmAMszveXERdoEQdfoHjVo3ueGHM9Qo69S3F41tcAJjjU0JHYqHAUVVTj2LI5wD9rgHG3+PfaTNNzzidTpIZU3MYD9v2iJiwK4e4idBSErppuKOzrGqnlRSQnqDBFd9zDmjhxVTtqwLR/BeKwAma8i0aBF0+MoLMrmFFkf2E0lGUnwbKJ4fq6cbn9961ztGjrIO3c0yjo0cdiDzW5rqgq1y7gWM/tTJ/jgrNsQmgJuqlTp9quXi+55BKHv0VGRiox9/zzz6Ox2LVrl5pYwTSrDkXn4YcfjiVLloSuoNOviniFVC+Cbp02FUEPw7OrtiWgInQG4eQLFLrOPnQ8+THKQoFjbIyY5CENx5OG4cSxZGeOEnNk4dYsR0HHNMm+f7043ncHzv4A9cKexcBHU4HrlwBtrOaoE//n+/+zXtCXqAlPdhTZrDfylopuKMxVtg5X59o51RhRVIGh1RuALQEIOr1A3rAfFFdUY2+u3ZduyclzMX6Am+gf9zta2uj+iDr8HgdaS8mOWkb8/nlFE+JM435/rXbsuSTEzWTTVgKfngncutHeue4vvChiFzftiyi0g7kfssSisbtcdaHfS7tA8Rle8Pc/SfOhdMXStzQrk8Gn130ZhdAQdGardxnHfi1fvhxt27ZFKEExRxiRM8Lf9b+5gqli3nQKCwttn1f/zMGEr8loou21EzqovLeZ3WypQ4L+fib6JKUOhUV/v11/w7T4ZVjO/8p389imBqMbF36viVcftqG+TWoGTkVY1yMc/yd3N8JmngTzpb8A3cbYH6c44UlXj3zaXqwapi8vhIUCwPr8HZn2E/DmA4WO+1VCB5iKDti3jzPnf117YkQwKS9CWE0FzLS10d+DJ3iKBifzZVeYSnNgodeat+Xb+CPCvrsK5rv3eq3xqfUdqSdM1RUOgi42Isz2nrq4KzOHw1JT6n77uKMkR/tex7S2rZtN6Y4XC/mVETDzO+jqtSuKtP+n+bTx71HxMFUU+b88/LyclrJlNjDsPFiOvFVd6Jlm36bKDDy+nsViq80xM5oTSISsrpQXaOuDdWp12S+2/YawH66F+d50zdg7ULbPg+mHa2C59h914Wjid4DfhQb0+Kz1Pako1tZR96P8W0csHzn3U/1Fa/3ZtP13Zd1jGdiI0ccmgDnIx61AXicikEhYc+Kpp57CI488UuvxrKyseqm940YqKChQGz6MYXszkGoKQ1H6VpQl+2Ey6gsWC9ofWIuSEVegJFOrq4jOTkfrHfOQtXsDzPF+RrCaElHdAG6+8kzft0mr7ghr2xOwrisSmbEHjJvmllSj2vB4q/n3ILxwH3KnOtbkhOfvQrttc5HXfxoqY7Tnb0m319ptPliIjIwMFekmsYhDq9JsZBxIs4+DckHSXw/CEhGHorH3IJjEZO0HzVWyCsthsa6rpIWPIjJ7I3LO/Nbr/0ec8BrMsSkwG9aNK6IqI8Cqwpx9W1GTZK078vU7Uk8kFhVoPnNWivNzUV6kbZdIk3YwrTCHo6q8BLlePp8zUQd2qs+bXVJjWzf/bnU0au297H4UZ/ZH6fDLav0/9y1OCM0vrUKl4b3Duk6Gqf041Pi5PCS23WiYu52Aip7HAXTwyMxEgjkKMSU5yPb0ehYzkruMQ0zaP8jet9X/2tQgEJudDhaKZOaXAqbAj8vR5RYwZpq9fyfMcYHPYI1N34ZWpTnIKKoCSjORZIpFZOEu5ASwXQLF+XsSl5sB5hByNy1EdTvfgwMmlgdYamDhxYcL4tofhsSlzyFr/25Y2DAlNMhxq6ioqH4EHRsNrr76asTExKj7nrj55nrs2PRAhw7aQYYnS3a56vD3ESPc24Lce++9uO222xwidF27dkW7du2QlGRIsQVxo/Nkzte3bfTEjkiMNCOxvQ+1SP5y83+IhwnxtiJ9rTaqbUQpUB/vFwpkb4Vp5YewHH2HTzUztm0SUYqwXX8CIy8CwiO1PxZqJ/yUTj2AVvb1ZWrdEcjbjPbO6zBXS58m9zvCVlu2v3Cj7c+F5fSna4X2SdYoh2U8LOa70L5N69pdcgX7YHpnIiznfAxTdSFQlYfYYG+z/dohoF2n7rbie/XZDiyr/dlc0f5Y397Hoo1KaxNj8brfufyO1Ad9xmLR1hKgFIgKN6FTR/sFTqt4Ng4VqwheuMni27owYhkAy+jL0bZbP61rmFlDdkUaaFu6A4llcUhw9dopSTBf/huSWafpUPNah+3f/sbaj7XpBNPGQq+fz3zs/cCHk9E2PgJhjXHc2MFpLolon1pHMVmmXUy0TYgC2tbhc2wrhyUmGe07WlPmw08H8g/xfz+pA7W+J6laN3ybrV/AMvht31/on89g+udlWO5yE6wZMRWmxU+iXckWzWZLaJDjFvVWvQi6F198ERdccIF6A953Bz9MYwk6poIp6ticoQs4ijN2u1533XVu/y86OlrdnOEGqa+TCdeTw+tPXw9TfZ24EpyuQlmPxc/HguvuhhRicyJrE7DsDZgm3OW12Nthm2RvQhi96DisnjYbBv+rMDZMGF+LArk0t/Z2y9qiUhhhrI0zmVBRXYO0PMfRUdsyS9Ax2Xql23Gourms5uFIsNJsmFi7xzRl4f7g7ydVJWr8XFiEVcAS1nKW5Xt/L9Yp0oX/iOts+5VbrOI2rCzHp21S6ztSHww7B2/N5fejFHHREQ7vpZsLLzIPwZmDjkArf5eDk0mmvOiwXTcfdLzirkCkSvu6XM/0T6PdhquaWHZGT37GsabTlzKEbb9pncbGucS84KkogolehO7GvbEeMGsj8ia/hVatu9fvNnEHly25W933f6vZdxjrzeryWvxuMtWqv8aAk9AYOHxPOC953dcwVZb4t57YHJPU2f3/sLauVTeEcdLLAB/n6LZQTEE8bgXyGhH+plkbM+VaXFyM7du3OyzL6tWrkZKSgm7dumH69Ol4/PHH0bdvXyXwHnjgAeVZpzd0hCz1dYCkWSQ7n055yf4YhQEP4s3ZuoRCgzVhrA3xB/rLETZG6IKOJzqahzoXpytrjzztRGksrs7arDUJWB/bm1Nqa4jQ2XKwCEf3swpt1kns/gtI6W23QLB9jgztJ5s7OImhPoyFh59Xu4haneQLNGf9cA+HCPoZLnsTGOliXJoztvFIITTvsmA/kio4paWVzabE2ZNuvnkUsoYcrdJ9fvsgssbS2jBiNlvUdjdSaYl0b1uSsVGbcDDhHscIHYXEms+sj/sh6Lgs9K87413txK/DrsyeR9vMlV2StVld6FSf+0vjea2xSzoYndJcl/TiDNQs2nlKhA6bLdL/075LenS/oVH1jbH+z5RVHnQemuV4LGOnv3NwQAg5mlRV/IoVKzBy5Eh1I0yV8j7NhMldd92Fm266SaWHDz30UCUA58yZE1DoskFZ8jrwyZnBf106hrsSbhzA3MfHVFlTRB1sU/3vYtNPkEYvuiFnAnduqy26KXp44HaexTnyAoeOyJ3ZtWd1bskwnNi5jJxo4cqNnZYlNFRlNDAYdhWu4Odw7lKl+KIYZm2NJ/SuPm9zXAnX1X0HtPUTKvzxEB6oerVWh6vx907IVvYmfsMo2jd2Kxp2t5ZWOvp3llsi3M/KZCfm0je0Tlcj+sQSf/cF3TPRuQSBESuaF3u6qLR2fieseqd+xs/5SWW1GRmFAdbR0YeTM57rmp047hHgZIOrw/4VwKdnNe40lF/uADb+4L9fpTsPOiO8CPC3e1ZocPxuijDWmzmHGimc+vTpo3zgGDULNhMmTKhtzOq0DI8++qi6NSlo6ZC2Ivivyw7XoS6EIu0KmjPOV8++os9eLXdhLuwMhd7Qs2o/3nuiw687s1wIuoNOgo4pMFfzfNVooXZaumnE+bVeOyis+lS7Qud4OJ0BJwP3+BDBtQk6H7/rTCOGEJaaKlRYmyJqCzrt0Hha+GJ0//03YPSuAOa42tfLJoP/nM7v8VMw9DCttrAW+oWC86gr/Xd/o7XuBB23Pcfkjb8HaOtmWawXODE7foGl2wigxzg0OD/foiJQ5ae8geNfXIj0/HI8dMogXDymfr1P3eIcTbdFoHMazxpK32e8iTNXxxlXHnTOrPlCO67Wx3FIaBxBt2rVKvz333/KTLh/f20My9atWxEeHo4BAwbgjTfewO23345FixZh0KBBwVnK5g6/gBQRlaXBO+kxtVWYBnQYVvtvTMOmrw6taEkw6XEU0CGAqQSMSvF/jWmleY8CB9YAFzp1fLqK/jGitulnTehZT5w7s+xXyxFhJlSbLdiWWaTmeYbr86U4CYL/68zoy+xFyIyiBeqS74ndf2vpQaOg8xWevJiKpleeL8y5V1u3E+9HKFBTXYFKRskMAk4n3irwKhEOk7kysO9fXGuPgm5p2Aigr5tIESNwLBtwTt8x9U78jcLYBJ1TGYK5WtVdaUbZbgRdeQEsETGoiWuH8MZKmdPWKSIaa/blY1+uVpP68E8b0KNNvL18wVfeHKdNP3Bn5u0Ls1lre7p9DrN+UdOYXnQUdPSTO89qQeIr09f7loJe+aF2XBNB13xSroy+0bw3PT0dK1euVLe0tDQcf/zxmDZtmhr/dfTRR+PWW2+tnyVujugGuP46fHtCH/XlStDt+huYdWv9eZv5AtOMNLWtD1gXdvjV/v8f6wsvnaWZjhojGK5OnkUZ2olhjyEFlbZcS3sY0mi7DCnXMb21q/jyKrODwazbCB0f16cM0CX/r+e0ubzBhMLBuS6K9UCvHgLs+NP7rFI36/mXdQdw7ccrsS7N4L2WvxdIX4VQwVxVafOhizdMidAnRZBqRMBUE8CEDgooQ4Ru44HaEbXOxeuBTbN83y5qwVKAI2/TUod+YdEMxp0jdDE+RKVZd9ZtLCysMfUlel0fMCIZnYSDhlQra1Nv/Ow/h++YT1B08ftbl4zK8ne1/dlVhK6xoEFwILYiTLf74i3YZyKw6y/lSSc0E0H33HPP4bHHHnOw9OBEhocffljNUo2Li1M1bRR6go/oRc/BLHqnKexpr9vd/43wZMBibBZYNwZM4bDjjmKpPqBQLM4K/GDNSKkOR3i5Kj5n8XHGekchRtHFE6Yh3avX0KUmRWNkt9au065t+7t+j4XPARt/ss/knf948OvoKFZrpfXigZzt9qYMd/BK3cVYtPzSSkz/cjXmbDiIx2bbLVtUo0lj7XMuYJSUnaYk1jlCZ+1ypeAzBTJyjZE1QzepHqFj5K91nPaeY0v/1AbGu4IRZkZonWEE/7iH3Lv6u4ONDzevqp0e57anubGrGcY6I6bBcuG3MFP8NVaEjvWc0Yk4WOBYO8e5uFd+uByF5X5sI9YheqsP9YQ+K5XNUsbXpGBuTHhs4DzZJzr5PhucF1jvHKtFQL3R+1it69/TZBuhaQk6GudlujBPpBGvPmUhOTkZlZV17CJqSbTpA3ByQ7IX6wd/4KgWdh+6siLQ6z+MV5gNCQu+ibtZqHWBEayZJwObAxxl9PphwF/POp1IklxH89iwYOwoo10KhbQ1HUthk1uifQ96to1H/9RE14KOo7bO+aj2e/z3oZbu1d9PLU9x8NM0zoKOV+u80vd28mbq3sXs21lrD6jCdbIxvdBe98ruQkb/QoTdkz7ELVU3OqRYdfSauiJLLEqj2/kfzb7iN22mL4+ZZVXYn6+lCQd0TEJSrCboSmrCgWo3xf19j3M9Q1i/YHE1czgQGJ1RI+88CDpGZCwWlPc8AZbGKoxXEbpEHDAIuqQYTXTvyCrB9C9WK4HuEzF1FHS6n6CxTpffeQrmIWeg0bjgW+DEJzQrInY1+wL3IzZ06Kl8T3QcoUUit/9R50UVQijlevnll+P7779XqVbeeP+KK66w2YP8+++/6NevHk7WzRUeYFgr5VzfUhcWveS+0YLzNEljWZfQ2oN8dGrwU4hMebAzMFA3e+d5rrwfk4SSimpcPnO5igaUsVuRB3DdusToQWcYl2XscO3VLgH9O9gF3VZjpyuh6DFeVfO+3q1LdNEV7AjdsHOA/pNqP+782dzVES14utbD3/2X5jC/NF0/CbPBgxE6X6MH9UxpJcclwGUNnf77z+ax+PCIX+pkLcRxbzoDOyYi0SpEims4VsyNbUnODiDbbtHkwNeXanVv/sDo7ozxrv929J0AR96546uLYfryApQNOrdudWd14awPVM2vMUL31oWHINka7Zy/ORPPzd3i22vxAo2R97pG6PydFV3f0D9SP7Y7d9+7g1kSHltcXbQ6w+/AUXcAnQ+p23IK9YbfR6kZM2bg2GOPVcPuu3fvrm68z8feeust9Rw2R7z77rv1sbxNGkZrXvxjG6Z/vw0v/m6NUhGKGtZH6dGYYPgRsZj/4FrXf6dw7Hui15ma9QaFD6FgyN0Z3Nc2ercFKuiM6adTXwWOuB6z1x5QJ40/NmXi5zXpBnNhg+jpMtrq6aWxy9Dh2qttPHq0iUNURJhtBJgN2mI8wckTu+2Pcd2wUFmPAgRaDO+Nw64CBp6C8qoax1okXwQdxbOTZQlf47+9jtGebbp47XMcMPWNkBF0XeffhGvDf3JZQ2eM2FHM+wWtK14YrNUbORkKD+iQhMRoTYSUmSOBKjeCjt9f1mO6gidgfyNMnlLdY27w3LnKi5roBJgYjd7/HxqFroeqlKZeQ8frqdE9UvD6+aNszUVvLdyB3zb4UIfMlPWEewNfFoqmMTfWtuv58kKA83Ibi2+v1JqcDIboPgk6NmX5avFEL8CBUwJfRiG0BF1CQgLeeecd5OTkqI5X3nj/7bffRny8VsTLSQ2exm21VMJNJrw6fzuW7inEou2G1BNrWBjpYFF9MKAwsNR4Hq5+wVeNN8aFYoEdYsSd6AwUvdg5ENsSVxG6TiOAtn2xO8cudrbrnasnPQeMusj+3FNedki57My2i69e7eIRER6GPu00YbY7p1SJKNv6qC5z7HTVowB6HRajW4Om+pYa8YedC1GTn4bTXvsHx/zfAny8xCoq6bF12NU+CDrHmqzvV9WuxdmWYV0P7KJkB3BjTBpwQWzeVrQ35Ts0QehwcgQ5Mmwdrl05xb/aMb3DPCyiVofrwI5JtgjdPks7VKYO95AKd2Piq0ymA+hydTcGjxeSnuqieIETk4zY7bNg+vBkNDisa/3tfpXi1yN0beKj1cXRuD5tcf/J9u7vD/X91xOclsGGnkDpMERLbdYy3TYFt7HNX1g/p1+M+rp/+OJB58zWuZpLghByBHxkpbCj1xxvvC94p1VcpKqlIhsOFNrqjNTVEaNlwWqK0OthDF12Lg+SrqwyGgLOWD17puZ9pHfjBgumsFis7KOgY0rw5s9X44nfd6O6xuwo6BhJmvs/FZVIt9ZAEVski/VEupUIBRhrEg3RJ2PEq2db7TsywJp2Zb3PDl0Y6qKtyNAkwpM5owCte9ifc86HQDvNKihofHoW8v773mZ2/Ms66z7BTl9PhfeMKlMkGKIUnIZgTLe6TC+v+RLY8D1CgppKVOpdrrVSrprAi0A1kqqytO+Lr+h1ldbvny7o+DXn9k+M0SJ0P5nHYd/kmb7XNuowPVYZREHH8oz5j3k+nsQkwxydDBNr/oxNQw0Bv4+LX0VN7m5kFmmCrmMre1fmpWN7oFuK1t25dGcu8qx1q25h9zZH1gUKj1mujlv8LjRWlyvrHBnRZ1f8tf+oi1Cf4DFm/F3+vdd3VwMcAyY0fUHHAbQ07mVnq55yZRMEO1/5N8Ezw7toHa0Ucw7eVMEUdPpVmqeavN8fAj48BQ0Ox0np0Y4OQ4Mv6GiKy8kOPnqjvf3XTsxadwA/b8jBgq1ZwAlPABd9b7cBWPKaqmeikanObl2obZ8HrJxp92iaYU+3Gk2F6T/XtXWsut+vg4vGCJ6g2YRgFNhsXGEUQBd7FIqcv2uMHtYV2qvUVCK/xr6ubJHIbX8AS7USCpdwX3USziv25Nnm1o7ubhcP2zIN4mPLbGDxawgF6C+n25a4mxSh/92vUVEGE18Kd10sd0+JU92zeoSOViJFJSWuU9BMmbmL0LF5yt9yCU+CjscJT12u9KGLaQWz3o3vLRUfbKzHxXxzrG2MXgeDoKOh/ImDtRILru95m2s37TnA2mJO4QgUlsf89oAbQdfA60ZHr5ljJzkjiOzC9zWVrXvp+QrLWfQMgtC0Bd3//vc/vPbaa3j66adtKdcnn3wSr776qpqdKnhmRFe7yFqTlu/USh8kQceZe8PO8zzLlNYlnMXZ0PVMmRuAZ3sCaSuBE5/UrFUaCXZf2urhrN1y6uSmW4johdMxSbYuRbInt1RFo5Sg49g2W4erfYYr/65H6Lq1iVPpVmJsjLCNAHM1LYJF8fsN1j98zssjgLVfBW8FWKM8eVVRtofYRahSwazF4ZxWd3A93bFFE9BWjNG5C4/oji5WEbs9s9je6craTX6uxhyRZIX+ctXWSRG6TYlOvDViV2U1HlYGvL6iX7DEtlb7AH0H9XSrsTvzjLC/MfLD/q7FIk2F3U3gmPo6MOVF+MVZM4Ejp7v+G48Tnrpcb1gGjLgAlmirIPR3VmhdsdYLZlXaTZY7JDn6pk0aYm+CmrPeS+ahrhfPFDOuanQbM0KnC7qwSG2qxj4fyncYZac1UpahntsXeBHnzdJIaBqC7sMPP1QND9dddx2GDRumbtdff72qq5s50036QLAxvKt90PZqY/E4Hb4ZsQoG7EI6Y4Zns0gKOtZtNfSJVT94sJ6KHnkGr66g8MP1WkrABzakFzqkRVV0iS35n56jCV3riaQmKtHB0JTR1QP8nZMA9JO36nC1p0PTC8pQYU2p97KmW4lb6xLa1tAwVoeRPxY516qdKgr6SSDbcKIke3JKfWuKMEAROHvdAVt064TBqehn/awOna59j9dMbrf9jsbmj9734Bfz4W5q6OoQoWPzx0U/ABFRternSIJV0FXAKqRdpXOvmhfciRr8vrkzI1YROg+R31Zd1HPMsa1h4X7hqyVGsLDu8xnl9gsPY4SOjOzaGu0StUjz39uyPDey8IKNqWN3c3QDHS3I+lDa1TRG0w+jucc/BqQO1o4dvMD0pVHmz8f9b0xTgk4idM1C0OXm5qouVmf4GP8meIY1NJHhWhRntTFCRy+yURcHZ/UxdZfnxZJEP7g3tBcdLUvYVcX0DQ/U314V2PBzjw0hvh1QfzJE50haXqnmk7ZtrnbAt0bocmtianlcqbSrLnpYv0JvPcNoLuMMVzZE6LD2R0+5bXUwF+7rGJFxFQXgmK1g2pbQMDe5GzIqHU+OKu3KZWF6lylyV2z9DXhxqE30/bEpA0Xl2nMnD+mobD/6pibUrqPjyaDTSG0dNzLrE8Zim6WLuh/vVEMXFR6muie3WTrj3qSn7HYQvnpA9j5G3XUl6PQaOr1+z6/6PMKoytt++MFx//zpJvcdqqxl5fHAVckMuyC/ukQZTdckdYXlzp1A18PQoNC/cNTF2Fdu30+NNXQkLMyEEwZp3xdeSC1k+YQ7dIuOQL3o3EXouG8z3elrx2gwoSgfdzOQ0hOIiPXNtoQNEcTfpghOHwqmZ6rQeIJu+PDhKuXqDB/j3wTPREeEo1+7ONtJv6DU6nBOIREscfXPK6rY3SO6ubCxEN8dvJKlmWkwameyt9gNhSPjgS2/AHsN47PqirurZyeYEp1VS9CV2esOKWYo2EZdgv1VteuVVGSPf2c3Macq8ArZ0FVsjPzRssRY76NH6Ri1oumsgiOgfjEUJ7NbzvlzBNLd6Ak6209fhzWW3q7FKnFXs8flK9iniUyVbrV3t545Shv03be9fb1t1ztdybjpwMBT0diM2P85Bpr2uKyh43biY8WIwzLLIP+6i9d/C6z4wIVlibY+dEGvT6lQjTxGKKxeHq51LbqCPov+TFlhfdx/H7keL6dHlq5b5Lr7mP+z8YeGj8oZYUfqqa9iX1mU25Src9p1rif7EmYGePFMdwF/ofjmBSmjls5wmzDdabQfaigoMjlVhg0rvh4n9H2Igt4fmLpn2l8IOfzeozne6/3338egQYOUmTBvvM90K8eCCd4Z1CGudh0du8y+NFhg1AVrV5pH+Pf70pUHmU/O6B9MBmYGwX+IBzs9NckTSOoQ4OB6BNW2xAcPupV78+xpQIOgs+hX7xQyTFOd+gr2lNeej6hEDyd8DDlLe7+7dgE9jrL9fafewWo1FTZirKOzebTl7QJWf+oUBXBKR7Pr0VfDUD/QOwd1aKmClN7apBF3sFaI4jc8AtnFFbaICCMnR/TSOl/7uYrQkcFTG9dR38qk9NcxMmy7S0FH4qMi0BqFuKTkQ/8mM1CIUQQZInQUcXpNoT1CF+k6QsdmHH5P3M3MVDVgxQE1afiNLuitDRGm907w3CxTH/A7nbnZYUqEc8qVcL/T6xPnb8pERbUb03JG0ukv6a5G0RNstrpphbYPO8O0PNOdjSHoaCPy1UXasV8dJ4p8E3SeajU9QYEvTZBNX9CNHz8eW7duxemnn478/Hx1O+OMM7BlyxYcdZT9hCa4Z3AHe8RmzT6roAuqbQmnG9hr9VzCtIC7Ljpn9KvzYJgAX70QOPYh++/B7HSl2OGBzAdBZ2yGiLKmwJmqyTXH2tchTyQZGx06XB3SkvSzOus97YDI9WmIcBinROhWNa4EnS2Cw6t+plP1fYDbprVTWuPS2cBpQewQpZ/Uc31RVpBdW6wyKsKGFbrPu4K1l1bLkp9Wp9tS0lNHdlbpL9KnvUHQGTtdCRtK3EWgGgJzDcJRgyq4borQRV68qRyXmL/zb6qK6ihNUaPfdBEysEOSivoZI3SrzH3w4pDvate26aLdrW1JgjbeydcpKzYbIzeCLmOjZmztKiXr3DFfnqdFZhuS1Z+oC8qDXgRdZHgYjhuoffeLKqqxeIebBgWWEfAzB6FjfNnOHDw+ayP28iJIt/BpjMYI7g/6cePQK4DuHoyidRiZG3G+/ylifnefcGriaopYLI0beQ4VH7pOnTrhiSeewLfffqtujz/+uLIsufpq34rRWzqDUu0n+NU2QZfUsIKOcDD4rFu9P48RA+PPusDZssb0FQVdzrbgeFvxavOq+UCfYz0+jX5zv1gL+KMjwnDysI62v+2tbg1Melo72K39Anh/koMHnUNKlVeorFV04RCv19Dx5N02wZ4qcm6MsEWuKOiIbl1y5R+ag7+RiKjg1udQeJRkYp/TbrcnxyoW2OjhLs3OZhDrCey7Vfbu1jNG2tM3rKOzdbpmFNk7XfW05J9PodGwRr8qrV2sriJ0bIzQu1wt/hTQ07oiLgUbnUZ+6SQZmiLSzG2BcMemFFudpLsLLn/HwHmL0PF9+N121enKYwlTk/p7xvjXLBMUGI2MtjcmtYqNrDWqTedEQ9rV7dQIfs43x9gmefgFSyOe6aHWQVWNGdd+shLvLtqFR2dt0NYRu0z9MaEOFvpFAMtYxt7km2k8Jz6c8pL/70VrFNLUO13fPxF4x/O5oqkRNMt2Tot47733gvVyzZquydFItg7oZspVneiCGaFj6N+XubC8+t7jQ/2aUWy5SwP5GhF69zjHdFGv8dpUAnY+1hU6t7PDVz/guGHJzhxkF2sn6IkD2ttqm8jekgjgiOuAVp21poiYJAdBlxKvibN9uWWo4ed4eRiw6WcH3zt2fLLLVX28tvG2yIznCJ1uLuzhqpcGsD9qw+SDQmUxLGERyHIKQDIVXV5eBrx+mLbNXHHU7cDkZ1QN4Pr9mnAZ3CkJfQ1ileidriWVXCeGN2K3KzvxGropR8fatVpt86FzFaGLsHW5Vvsj6KwRuk0Himo1RBhTru2Rh2l7HlDpRNcROjeCjobWV87TvAt9gfVeNJD15EOnltuFoGMDy3EP2+vN2NndCD50Fgo66/7jqn5O5+i+7RATqS3rbxsyajUzOTZFBHC8Ze0o/y+6lTIGz7PWQK9Jo/A1NZ51CfcZXtDyGMgLsQM+TOChNZIn/0F30H+SNHSnK7v+aUweLPYt02y0mhGhMYOnhcETvG5fQmGhivEp6Nh1WBfBpMPW+ZP+z/vzlBed43QD1xj+XpcTMA8yegOBDichcOC3r+lfb4ahv97jtWvQmG49dXgndE62m3CqbcFUYOYmrQsu2u5Bx87HUd20k2JljRnppYzqWMWaocOV6Vh9lTrXz5HkuCikJkXbInRK0LPTbPw9WmQwfRXwTE/biX5fbilW7M6FhV1pRm+6ulJRDAuv6PXPYGBvkUXrlnN38mZ9YaeR9mkXypKn9kWEy05X0nsiYAoHtv2GRsFkwpKIw3HAkqKitPo8UCOM2umCrrKidtrdLayv6noYNrvocDWmXKNNlRhdsrD2uCh+Jy78VqvRdAVT/Jwb7BzZcwetLGhS7c5sm40tFGyuInQcfTfuFvvvFIUNHYGqKEJ1ZIL6zrlLt+rQfmZCP01w5JRUYuUeF/svI90RMXafSX+giOEYvrAwhw7mrKIKFJZXAUdcC3QejQaHIpXimyx4ShuV5o2PTwcWBTAxQ10wm7Ta6oZk3dfAzgUN+55NDBF0jcSwLsmOadfh04AHsn0/SHvDKSr0/ao0jHt6PmYs3OHY6craC28H6O5jgXv2AZfN8b/FvVaHa//aaUPWZGyejTpDIbT8XSA8SqVDPlm6B7PWpmsmwFZYKP2r1Xg0IToCxwxojy6t4xytS2bdpi2PNUKnC7qOyTHobbAg2Z3LyKX1tY0drlnu6+ecI1f5pVXILKrQnN2PuVfrwGPtHs1bY5NVw8Kkl/7CWW8twTaeb4PZ5VpZguoI18tn6+J1J+gWPKMOrjsMtXG9XYjXfu46XVkS0G2MZn/SGEQn4q6oe/CfpZ/L+jkSHxWhOlG/rJ6Aklh7Wt4rFE99j3eoozTWE8ZGhisBWWHRfegqa/uk0ctON7h2JSp+vdv3mtbs7dp3wx2s/eT2cBWtoQG4wVbIMvZmFZltUCxmlEW0cmtZ4syJQ1K9mwyrEpcABZ21+9wYgbWVWRx5K9D3ODQ4I6YBV8z1vXmKF5HMBvjb4Up4jmIksiFTrvoV8prPgvN6NYbASTNq7hBB10iMMBgMq8YIHlSDUR/F2qc3xgBbfnUwwn30541KmPzfb1tQVmktptaLsQt8iLrx5NJ9jO8jZVzhZL5rY80XWjoxKJYlqWo9fr0iDff/sB43frYK57291NZ1+tfWbJtfGn2rYiLD0TXF/pmYSlUpKGu0oiomxfb8Tq1i0b2NQdAZTtgOHnRGyxKDADRiTPPaDIY5IJ0ROBWxYfqmLZbsyFHpSvW6hSbfutd85ZBLseLwl22/GrtSVR0dI0HuxD5Hoh1Yo03XsGIUu14jdITF217qHeuNqjLEVuTABLPL+jnCx9mJenf11ShoPczH1y3Xht1Xltr2D0ZjjaKREXpG6Ww+dM62JWwSWvC0e+NbFnIve8u716TOvzOAH5zqMZ259Bdl0VOLJa9qtbY67QdpUbuG5Mx3sPzwV22/pnpIuZKJA1LVuD3dvsShdtMYZQokG2LwoNuY7igI1cUN05gUwY0Jsz3e6iuZFmbZgfUCnfNv31+0C//t9TGdfs1fwJib0GAEu3mhzHDxUhHEcYqNjOtLUxewk9UT7HYVfGdYZ8PECAo6TlD4/mrg9Lc9D0X3BouYMzc6ONvP35xhq/WoqrGoL+24Pm01P7hzPgKSXHgqGeG4qfmPa7VlvLIdeYH/y8WroOxtwPDzav+NjRGMiFGMsmkiUNhQYL16Zp2czr+7czH55b9x6/H9sI61LlZOGa4dzFjPGBcZhtIqsxahS26lrcczZmAnxdZarXi6U3IserS1R/N2ZZdqJpscC8U0jAtTYW8ROrL5YCGO7tdOW8cUUTxp8vXCI9Q0C53syqjgRuhadcb2SO4XWh3JoT1SsNUaRVPWJVwWGiw7Q6HB6EZcG2zfXuwyCuXqsVqdro1pXbJzAX6ruRKH4g3ERbmOhOlCr4spE1U5u4FUHya5sKRgxtEovnCOSvmRHoaLAB0KuuxSN7YlFIRMmx11h+v30Oe4+loD5mmOqw67ml3+b75jgxXLJjZ8Cxz/aIMa6BotS7xF6Ng0MbZPW/y1NUtdxPI7NMRwvFVcH6D3JaOTNZVKJDrM4lYXcsXAgRlA2nLg2kVoUOh/x+PrZb9o5SvejhNOpsKPztqI71dpj50xqjPuO2kg2iZ4mIfNGuOGxLivs6Y7Kq6Or1eo/TzrA1UP2eIidK1atfJ46969Oy6+OEiTDloAbRKi0S1F2ynXpxegioKHaZG6FtQ6+UYRRquMLNXFDr/4g07TZr96gsvEMTE7/qyD1YQFmPY5MGCKa0HH1K8/Xl9erp43pDteddGS5OlfN9vGUyXHReLIvm1tEZOOrbSDF08AFn7BrVdwxoaIzskxDgJNRbGu+lM7ORhOburA7kXQDepkFxF6U4HqdKUo1SONTlGAv8xDgKlvBG+00H8fI3Wbvcj4sJ52PyoVXbr4J9c2Kfosz7g2tsgnC9EZwXTVWKBHQGt1uhLu85t/QUNjLtZ88/KQ4LZjMs4aVXsl8jW0Xenj7FRrito41cDVPpAQHam6XJ+vORcWCngjPBmzfpEF7lYYQflx9X7c9/06fLQy288u13zvTVL/vgMsdTG7VxlsG/43dzuw+JXgjqDzxseno/NmzajZWw2dzomD7WnXp37dhJxiP6dxuINlKm16q5o5XbDr7MgssTZFNHDTiL7P6I0rbFrwJuB5TGdHblJn9Z2k+NWhSfjE/1ugSlaM5SoOLH8PmHMfGgwKuMGna/eDYZfSpjfwUD4waKprQ+3mHqH74AP7F0oIDiO6JmNvbqka3r2jwARVhVXXA6Ve2Gw1FmYN1gKnMTjLdhrSaItfBXoerXmquYOWBky1crLAjnmBLRcjb9ZxSLXQZ9geXKsV2wdK/0mqbb+0sto2qYEWIRRu7/+zy0EHcTwVfat0OiZFYUd2mYpglqQMQgKDJzOnICGWfk7DbBG61MQYJV64zXZR0BlOujr6ezOS4E4sMEIXFRGm0uHr9hfYO133LQXOfE9tRx5ojcJ0VUlbYFgQ63M2/YzOKpo4xDbZgUKXdX1qnqu7CIz1oqMqOgV7cnNs82p1/zln+LpMZeudrsYmFKz6VBsD1n9yg0Z8qoqyUGaJV12u8da5rc7EWyN02ZZW6MYLGl+wit1dpVEeBR0jdGaE4dWq03BDSn84SBQKtah4FQFip+aCrZkqim/cfy+Mi0KYr9FaikxDSYBL9vyjbVdG4V2YlOeWVKoof/vYFPvndFfjF2wOrkdFXE+/BN0JgzrgidmbUFpZg3+25+CkV/7GS+eOxJjeVq+43x7Qygn8mXjADfDTjcDIi7ChrFetP6sGob6N1+WaUxWBa99ajPMPPwmnX+fFQoxNSfdnqu8cv5vO4rSwvFqVrHy9Mg3PnDkUAzo4bWvWb/J7O8mQjq9PGJU+5n6t5jZYXdYVRcCP12up427aTOemTvORpk0QY1fg6kxz3eYLuonQ/bBqf63WfZ4caK2hYJrPm3UJ6xdokUBBRxd0Xw1NjfCLSPHoCtazDDvXN+88T7Bbdvi5qlhZP/lRND8wZRC+vW4s+hrSf2cd4pgy6GTtOiUbB9+uFbZnrEdpsX17UNBRtOgpNHaf0tPOCE98FESe6ucIxeQga+cjBaAaAaZH6JgGSR2sfLf0VDkxFR9EDV36g3VAqyxGodn+udsnRds+G21XqpbMAD46rfb/MbI7+gqkWVJs+1ZvF+lWn+roWPzPzukGdtevKcpEjkVb/24jdNbHKegiyh3Nl92iag5N2FZgP7T2cCHodC+6CWGrUXJgi+MfK0tQaopRZQIv/rEVq/Y6ijmyu/eFQPvaM7XdngxdjKriBQP3Ye6z6gLQldFuq67IjuyIsc/8iVPeXYe0Muv+0pDWJZXFyGK5gZWOSd7reNslRuPti0ajjdVmKKOwAue/uxQv/L5V+85yO3GutD/w2LzqEzVhwTndSngRVEOfvuqy4Phq+kNlMTZk1WD57jw8/NNG13WDzljrtlfts2/Li8d0d/CSZH03a5DVPmKEpS0NaVtSkAak/QvcuV3r8K4rm3/Rph/RcoplEs0EEXSNCMWGzsp0pgRMdY/QtR8MnPOxivbwS/3NSnu69XBrSo3t/7biVwo13WXco6CzRuhYm6fXX/jDph+Bdd+4//sZb9etQJ4pa7a0l+Y6GLoO7qydtGk3MuvmI/HsWcPw/qWjcUh3x3E3jNDpqDo61omVFyKzMtpB0BFd9DCa5zxFwjjyy126VWdYF7uA3cAoHdcvu2Vn3w5s+wMb9FSs/v7IQficu4GCANa/KyqLkVelfW4WkafERaFHG60MgOeDgoJ8192RtNWY8gK2lCR6bIhw1elqG3Wmo0eOcg3d1w2AuTgHOdAFnfumCJKFVojyVdDx+5HYETtzyr1E6LT6uZciX9dOKkZSB2NZrOPUHTasHMm6VysLe9ysRdZ94eIfgAn3qPQZBfXHS3bjhs/+w2FPzsNRz/6Jo56ZjwIkuO5yveQn/BAxSUWSq80WLNxvvYBpKOsSTnWoKsXBikhbh3BSrG+JJUbmf73lKIzr08a2T78ybxvOf2cZysLj/b941gVMQnuHUgj9wo3H1Sy01gbX+5oODxZVpcip0tbRsIqVsPxfP20+uDuYLrV6WvKCQefYgal44dwR+OLqI2z1r7xAffmPrY7/z5IQrr+GmrTAKSY/XOe6pjcQCvZpNYesCW+MiGo9IYKuEaERq96NtZrF+hwj5etB2h2shxt0qhJga9MKbEXuh/ZojXMP7Wp72lI97cpoi7erSUa+OAaKflZsy2fthb+w6cNVh6sOl6Eu3WGMGDCatGcxNhrSlHoUjERHhOOc0V1VF5wzxghd+7VvAU93BSw1SC+3f1Y9Vdjd2BjBtKuBFQbfqz4ubDyMDDUUaq+loOt3AnD1Am2ma842B2FKimCNTASrfqmi2HYSYAG0ij4axMfB6gQtauN88uYMyIPrHTzoXFmWuGoA2Wa0LiG0TQiLaPAI3Z6jn8MllXer+95q6LIsydoFgy9Rj8OvAW7fZOtwZRZZr5U1onvR0RalytnjbuhZ+Cj+ctuvc6cfjd9uHY/rJvS2PVadsVnrqPQRRuImPr8AJ7z4Fx74cQNmrz2g6sAIU+E7iyNr+9Dx81qjeDqbCyO1Gb+GJqB6xdrVvb8swlbG4GzU7Yn2STH46PLDceeJ/W1eg2ySWpymXbAFJuhSbRE6lk0cP8h+PNkYMwKYvtbWnNVQVJ/xPp4t1+bLmmFCGD3iPHXEZ6yzHUdW6dOKGGSw2mlxLu6nVx6uBDT5dNleh++77fM1VJROP+b9eAMw77G6v16ZtVFIdfKLoBOCAC0zdMPR7VnFKOpzqlasWRf2LNYKnNkMsdI+c/GsQ7rgcOvQdH0GoT1C5+Uqi8vU9TAgqaPmGs+f/sATAz3oPAk61mO8OxGwFqv7jW7OmpBqu3rmcd9o6OoJY4QuoyLKdiW4z9qJyAkRNC0lPd1ZlwC2kWJkQv/2Pqfcbd23FFB874T2tRo7SizW2qEgXf2bB5+Jvyv72tKtzh2Zq2IOV55+Ks1khL9/fLrDAd5Vh6tO7/bx7jtdWYM4+Az7HMwGoqzKgjJr5ZpeK+eM/vgnNcfhvcPn+Fzjx8i4bl3DRhF+z90JukoLBZ3T9y9/H6oKtP2IIkQvFdDNqMmx254A/nrO+8Jwf3q2F1bN+1LrXDYQaZ1hTNaFDwIOd6qfYyT+sXZISP/H9tC2XLN2cdfRRxuXuhIZh9KzPsM/Fb19sixxBdfhDcf0UVEnnTR+r/2O0Gm+a2VRbWx1soycGkf5qcaIYEPR9OY47djuhtzIDjhg0b5DJRb9ws/DcaJQ86Bj6Y1+Acwoe6s4+wUs1/U147VaQUZn2VRmg408k56xd1zXBWYcvAUVuK3YKMRzFc8ldaXM2ijUWJM96gmJ0DUy+sQIap4Df38M7Jhftxfc/gfwzyvqi8qh6YRXWScP0yYi6B2Hq/Q6Oo5g8iS0yOrPNa84whScqyHe3g6EPLHQVNgdtP/QrxwDwXqwrY5rZxunReHlzjTWk6DbX67drznjPSws0bz6OiXbTyTGKBanQuhwQDejonr01VXtlBFGtfQr4LX787U07zPdtT8mdKgVoSsJcoQu69Db8XvNIep++0SroDMs86aiaK2zjBFDY3SKB8D4tjZTYeocT+llr52uZ75j72BrIHr8fgXOCPvLIRLnjC7gWQpRqteceuOnm1Hx420270J360VPuTJCV13pJOhmTcflBVqxPucA680mRjGjah992Q8YiSjNwf4ie63nTRP74Kcbx+HXW+zZgGVVvTVja4f/zVfTa/YW2YWfEoWMDLKmqSGIiMb+dkdrqUwfLEs8QVseNv2Q2VUjgbM/9O8FeGE7bjq25JuglyUP7JDkEJ3em5ENPNcXWP8dggZtbDLWa/6EbgifcyeODluj7hfrLTbuzIX5/WOUPamTsnRh6QgZaZ2CY+Tqo3vZLiR+35hhd0hgAxenYjDCVVdeHATM9dIxy32dTTisL6YYrSvlFHStgSOu1ya7NBNE0DUyI7rav0RJ6z7wXGfm85VHK/XlY6cSmTykg5qKQI7oqV3FsSZG+d+x+P8QF4aiRjhIXa/zmfco8Ddnr/oBDyC8+vfUSdu6pzZY+uB6BASnK7CGrSxeWZSQgQZrEG8wYqIXqu/hPFcKntYjkGXWrkCNlhzGk7QxQvfLevuB5qShHX2KHAyx1vix0yzP4KxQFJmimRxbn0dKEIPdrQ4PzkHUYkHR9qVIgibK2iVqJwG9hs5my3Lsg5rprDE6VZoDS1yKzVS4S2vXUShXdXRM7+mTN2zQh81Xk9xgYLGg9YF/0MpU4rGGLt6aik1FLi5fdbZvc4+zt6KkyJ6idi/otNfeYemE4jDH/dRSUYK8au2ior11uxB+h/VlzauJ9i1Sa21e2MGUqhWmbjmphqlgvTE5Kztbu5g0ikRrk8T2Irvg5VQT8+fTgMUu7Gzqg7w9iFz8AlpZ91NfOlw90dH6PV5VlIyaPsf7+c/DgeMfwSbdCNxqP2RsftqSXa2tN1+7on1BF2Yetnfyli/R06RlKUptkXw3gp/Lx7rppI5YZTASNtZ0Gy/Gbj/BfiH++OyNdiuTDT8AGUGYhcpoHzMBnmCKl5NlVONYEATdYVcDx/xPlTeoxqxmggi6Rsb4Jcquig5Ol2tMsmo31zlrtL3DzTHtmqsJQG9pToa59VmrLNz3deSQDlO0k5/WPJys3mqnvbYIj80ydGOx46p198BnxVpqgDZ9sSHLrooYJfOHrtZap10l2sElat4DiEO5Q0OEHs3ST6zGNJYx3XqyD4KODO1sSLuyjs56YNtUbD9JHGK9cqbNxetdntOGs9eVyhL0+ek0jA9b6xCh45xZPYqxm8bJ7I6Mb+M4taA0BxWRrVFcUe21fk6nj6HTdZtz2pX+Z28dGTx/PW9UliDcXGHrcnWbcrXamRQjFu0q9vrWDFSaixyz/bP28BKhu6bqNizrcpnD32rKC23pdXZr6rB2TB9Mn1vlR4SOtW8FVnGaxH03wlb/pQscU95ObbYnC8V1rDV1mVWOIqosIsnuRVjfZG9FjzUvIBYVQRF09JIkbWuyUfbHU/41d+xbDmRsdGiIYEkHt6X+/dmR7WXCSl0EHRtEXGGuQXhNBUqhLUMWkvFh/zeAzlr0vRZscOOs4O5HOtTPjezm2qvwzFFdbJNt6Jn54xrr9+DXu4BNs+o+2YglJt7SnodcCpzzoXYuoRNAIE4LRjqPAnoepc3Ldm5Kcobp4GAautcjIugamV5t420nlEw1CaCuPnQFKI9IxKJtWbboiR6VM3a6EhU+//lm4LsrPb+m7kNnE3S7/Dv5MkXLsV9Wnpu7GWvSCvDeol2qzd5G276aMAsEFmrftMKhM9TYEOELetPDhpquyB/3AFJ2/4JwaNE+o3caT6zdnaxL/E23uup0VYKux1HKfHldln09cN6sTl5+XnAOLtaTBKN+xho60t1gXaLS8hxH9nx/QxG+Cblh9v3IF0HnsdOVXbO8kGmozslSrWM119bl6i7lGmFbR5WmaN8KwMvykFkd5/D99hShI3p6Vqemoti+XQyCzridMmoSUBMW5dPyGC9Suqc4Lo/esLG/wiqUjI0R1q7XQjj+T5EpqeFsS6wXuBTVRBe0gaJfmLUz5SNh8TP+pY5/fwBY9KKDZYleo6tH6ejnVhMT5EJ7W4TOzbnB6lKgR+aqEIEVGOTeXDgiWotKJaZitbXDlaUfxlpAI8wQ3H+y3fz6uTlbtOMCDYzrOs+V3aYMEHib5c2RemxM6jdZdV7XmeXvaRH3zbO0KRueeHUU8Fwd/FEbEBF0jb0Bwky2gwwjdJa6Crouh2K5ZaCtxoNXV0bDV0ahdHFC65KaiDjvBalK0MXZBR19lniV5Cu/Pwj8+YS6W1hehUXb7RYQ3xoiiWoM2RQfHfnd4GBZ0sk/Xzu9zosO/jk1sbDAZKtH6dza0ftKT02yWDgtr8wh3XryMN+bRoYaBN3atHzN7f3gOocowNjebVQ0hTxy4Fpg4dOoM9b0jV5AbUzt9TRYl6gORzXJwAKseF97wkXfYV6P2/wTdIaThd557SDoSEN1upZo+5/dh067oHpzzZuY9O0k3LnwTpRUlRgidyYUhLUG2DnoCa6wslyklcd6j9BZSyDeinwRx2663/FlqqtcRuiMgubZ6vOw+1QfyjN6HYNdp35jE4jdDSl14+8FFutyGq1LBp6C+RN/VALBSK45ruHEt/XiRV9+PaIYKPqxtgjW9eBPRqQwHZbEjrYaXR5HOWbM+TtQEtEqyILOek4YcYFHwadH6MiE/TOAXX+7fr6aFfwMMnMLbOUPvLCMMBitu7KAOaa/1tlMc3BejKs0qLfvhDd0axVvIx+/vAD46iJt5FiPI+s2IpIsfAbY9ZfWFMGLE08RP6Z4ec5rAoigCwH0NMKqmp6oSB1VtxebcDeezjvGobvVmcN7adEV1pplV0Rogs0T/U9SQlHRpo/mdefKhNQd2VttDRF/bs60FeESjuLiZIc68+nZsPxyp232KU+EzidDb3SxHuw5sL330vtgggUW61fEmHJ1PlHTuiSQdKveuKHXN6pO1wn3AhMfsHW48uq4f4dEW2FyoTkmSBE6/UQZXSsSpEfobCllRmcZAV31sU386/Vz3jpcjZ2uehkeI8MV1YYDaIp1CkBeHUe/+Upyd8zp8yD2WdrZmiIW71+MN1a/gf3F+zFn9xx8teUrh8hdXliy99IECroLvsYvlcNt244Rck8p13DUILzK8SLup4m/4bWaqS4jdMbGiAzDfFO3xKVgUyQngZhcCky9zIAXLmpfN0boYpKwsbr2zM6D1YmAxdFQu96oKEKZKc72PUy1juirs6CzWAWdr9Yl3LZFB5EX0cZWamDsoDcKusX97gKOewhBg+MSz/vM/bSPiFj81uZC7LTYjzsTS34B9i1z/fy0FeqicFW6/TvsqiHCmXtPGmiruXxzwQ7UxLWru22JLnynuhg7V6spopVWb8s67vTVgb+nxWKwLWmj7cu+nM8aqiSkDoigCwH0zq3Pa47FztEP1um1LLm7cCBLH8cUbztgG6HHkE5aicl9N5QOD076IHWmRa9f7LtLPa98GJa3DnP+dZ1jZI8Hx7kbrI9tmQM82zswl/W8PSiptGgTFwKonyNdrOtKP3kYMXa5OluX/L0125ZuZZODURB5I8zQGMEr36xWQ1Ax6Axst9aZ0cuODQccOUYKzDGoKQ9Cl2tNFUrCEmyds8aUq8umj9FXaCc/iroXhyB+z3yfTIV1KI7G9daMcRnRfH+RIRrHAzVHStU1feMrCe2wPHkyiq1RGkbiZu+ajbiIOIztNBbXD78eg9sMVlFR3SdyRuyVwNibPL9uWBgsvY7B8tx4WzrTOF7OpW0Jo1/G+kTWQCl/OO1927kRdCeFLcXw78Z7P8ms/w7Jq96y/ersidfNsM/nxvcCTIbIx6pP0W/zG7Ve8vWIi4Cr/0SDkNITf0Zq3bjcFm3j6yjorMdavz0dGZGsqcCeylYu5zEbGyNWl3e0R52DAV0IwqPV7GWXxCbjo9iLscfSwfZQES/83DVRMEIdm4JV+wq91s85R9nplqAftw/G9QNSegdH0PU9wfPzeOyhRQo9UP95BUhbHvh7VpVpBuC6bYlaDg8R5+uXAbdubNDRhIEigi4E0NMIUahCbkYd0k48uL9+GKaY5zsIFGeMNXW7Ci1eI3TrVy3FRa//hvcZZtffhzUNvqCnNGKSUVZZo+ZSOntgfbvSWmQbnaDVNwUyiaIsDwer7NGQgASdUxROJyo8rNaJxBjp+PzfvX51tzrDjkOd9fsLlPkuU7nGk4Z+Ii+2xKCytI6NM6TLaJzf5ivssnRUxykaC+sY03I2WxZG0XjQ3TpH1b3sL9SK1NlAQY8+X7jPcIX/2vxtyCg07EO3bwHG3IAGYd+/6J8xy0FsbsrdhNLqUixOX4w31ryBDTkbHNKxq8x9vV/E5O9DyZzHEFuVV6tj2L2xcBTCagztzRXFOOPfaRgbtt6h+1hH3w84CTa+NM17dH37PHQ+8LvtV6PPoLPAe77PTMeO911/oVu+PcoTFxXm0nuxXhlwMu6vudL22d3NC/Y3Qke7mOXxEzT7DV+PY237Y2Op/bs6qGOiywhd1J6F2qzYYMELXU5JYPOQK0pz0SXvX0TDfmFQYol2f5zgMTa+HVYbRn6NdNHh6grj8/5qczZwupfImq+CbtELnp9H4c2pDmye4zaj7UqglFuj0IzQ8bXYPGL2kCXi994akAh1RNCFUIRuavgiHPnjUYF38FSXw1RTiUJrPUxXN+ke1orpV6qP5J2Iyps8eL9ZLBj04yR0SZ+Dx2ZvRGZROfD5ecD31/i2TBR+TNMmtMfCrZlqqL1e26cLh392ZGu1HJwaoBfK+gMFZnk+9ulzJtXB1v+5sMb0WFp4N3yEk9X9jsm1TyQ9DNMiygweZf6kW11NjFiTlu9QP6cLUz2CxohaVVkQBB0jQVZBxZFfxkiSQ4TOOAmDXWbHaqmk7cXRthOZr879FKfnH97NZl/yjNGoNMI3URgUNv6IYw7OtP1qNpVie57jPMeR7Ueqn7qPYe/y9aruyCPZW5Gw7HlbR6anxpj4qAglpCstEQgzGwRdZTE6lm1FNKpcplw7WFOONq8xbxGmsjzksObNSjcnkWkUdMaJEIryfGTXxNkEaP922v1+ZatQ8/KIBmmMqMjYiuqSvDp70OlwfWo2QCY8EnMH0Gu8b//IC5ob/8WfRV1cHmNYTxdtrXONzNsG/Pt28FJ0az7TatWcJ3nopP+Hp0vuR2vY9wXuHxXuBF1JNsxxbWxZBS47J2r4gjHNrJpDmE1hs0Kg8CJu8nNaTZtTpLq2oLMK6Dpbl5hgGXgavtwGfLA5DJYr57m/WGMjGKdTPNFJRaxDHRF0IVRDV6w7fAc6CcBaB1BoTSW5SrcSnoB1+5KiqjCsO+jhiru6AmGwoMwSrY5PTC+qqxpfrUvYZn7TSqD7WPy63p5unTy0I84apR0c+brf/5emmUYy1eTvrFJrCH2HwS8rkAgdT956tCmipgSda9JredDptEuIrmV34W+6VWe4IULHOjpjY4feqasXw/+v6nL8c5hmOlsXLOu+wcflnOVoqZXWo3WJXuytrEt0WEtn7RDNsyT6nG41cvvx/W2v/d2q/Vipj0r77yPgg5PQIJTmoIA1cVYqUYhRqaMQzbQWP2ZYJAa1GeRgLtyjeoc2mcHTSdoqcPI5F9VDhyvhBQJrJ1+tPh1PRE2vXeDupilCb17xaRqAdZkyq7TjQOu4SNu612GEVW/QuCT9MeA7+4WapSwfWZUxtovDrq3tJ/1w1js2QGNE2Ofn4pYIzaQ3NQiCjoX/+nepMu+A75NprNtd73DltjNeAHJ76hdCO0uiNSsOb9FTX9FLYtwI6JpybR8o1UU+S1tqDkdGaze2JV0PQ2bHiSit1C5ER/iQbnXlHGDavQh4smPdal95VUNrJOJOsPJa6bJFmL5tOJ76ZRMsPKfUJUKX1BGzBjyNu/+qxCM/b8RfW7NUCYpL6M7AyTiMojfwvOlAEEEXQilXvTU/YOsSa4ea3rHWtbX7lM8R1sYIpnY6fnu6VmzqCutBqQya0PmLdigBWJewCH7+Ji3dSgPfMb3a4IxDutjKEr5ZmQYLPdjYOeWvCz3/76o/8V3hANvB1tX8TF/Qo5o/VR+B1ebeLhsinK1L6pJuVe+ZYu+W40xX48ivWilXxCG9tI4dXipLk44OYLrD5HKckh5dslmX6OxcqH7og+196XA10jo+Cref0M/2+yM/b9CMSlmYvHeJ56v0YFGSjXxab1jpn9ILMyfNxJJpS/DZSZ/h8XGPI8rqBxhvbYzYX5WkpiZ4jEqV5aHGFGHryPRmXZMUE4n9aIeNlakum1UYFXM2bNYjtfZpAJ6PFeayXBywijJXFxvcj/ULv4rKKpgNkQ9GxvKtzQNdWseha3K0g5iv9wid2YzwwjRb80rHOlqWONfDPl/9BKrnPe7bPy14GjWvjLZ1hdKXzTlq39vaHJRjtq7nYHW66qKdx2IXx+nS4sJaXa7v10zGhtRTXb/e4ddgXuuz/E63Eo4G010SVudaL6DrYqJMB4Ql1gtUD40Jn6wvxw+bizHjr53YnnxU3cyAywuxexf9FrXz16HfHGZfBmf4fWBdKZsCG3jedCCIoAvFCF2ggs76f7pvlG7D4YrDrXV0SShFp8LVbhsjLFZBV249WPBqpoZTHXgisdo/eIT+Qk93x78bd6HI2h123MBUVXDOAwMtOfRuShWtmfY5MNrRaNUr4RHIbz0EGws0UTSwY+2Dra/wxEWerL4Ar9ac4dKyxN0UgEDSrfpJVfejY0G8muBhTYUwWmY8kZ8cthRj/rsDdaW0uMB2Re+c1jPWf1Gzp+UZIg3HPYL3Rn5j2x986XB15vzDutmMSpn2oZhXReQUdf6m2wOhNBs5VlHCCwp9/FpkeCSGthuKk3rZI4V6DV2mOcn7yas0F8VhfF1t3/M0Do1QsE0M+w9XVn7kwqYj1uV2iY4IV1Hk3ZYOuDXyQe3iygO5PU/FYvNgl5YlOvrFT74lHtXF9qhbWq+zscA8wvYcu6CzbvP6jtAVH0SYudIm6OpqKux8Ac1O1/JiH0VpUTrK6EXooiFCp7d1e9ssYPxxAvAEj82dRgJDztKK+V18lyssEahGhC1rwOkmVekbXb9exgZs2pXmV4erER5fyZ4K635Ql2YmmjXr68lomWOkJBvHrL4NA0xarfJvkccA424O/D03/YSbVp+KCGgXqiXMaLkzyqY9V0Kq1vwhgk7wBUaseOKweyMFKOi6HoqLO/+K7ZZOXiN0PLgz9VBmPTFb3Ai64pISVFgiUWbRhEVeaRW2V2kHWJ/SrryKL8/H7C32g9ukIR1c2qqoEzsLVH0tVNbJ2oqyH29DsrWGxF9DYSNdXIhg3V3eGWMdXaDpVld1dLqti/GkoUfROphy0CvfPiw9UMqLC1WDhXOHq6vi+V3GtGtYGFYU+Wcq7Crt9eApdqPSZ+duRlFclwazLrF0HIl/ynrYviOuxH9WaRZ+3/M7kpK0/TYbrbyfvNr1x9xwrSaLFyyuUvXOgm6IaTdONy1Uo/hISVIv3Fh5EzIsrd3a7nBfoOD7uWQgzFGe9/XVPa7E7+bR6r67/VOvqytAPMyGk+q/qedioVmzYGF6sav1e6CnlOs9QmcdB7fP0j6ogs7oRVdZ4j7N50DhAeSGaV3azrVkzhG6dEtbrOlygda9HQy6jwEOvRI46z17HZlx0arDbJYl+jHj2oifcfS6u2q/Fq/QZhyN9rt/sjWn+Vueon92tb+YIupmXcIoJu1Yhp3r8rMpSrIxvPhv29SeTXsOAHuXuc8qeaMsD6WIVQKYqIs7d9HUIs68tXYti6ATfEGN9GkVo+Y6jjZ/CEtn7QAcCHvyK5QFAdOO+ggnd+9Jb7BSi3bSKC1xLSKzo7qgf8WH+Ndi90D6IzMZuHWD3ZvOE2X5sEQlYu4m7QtD4Xp0P6sgBHDi4A62q8pZaw+gYtMc4Hc/PZxytqHjlo9tUx38NRR2FaEz4irl6myWe/JQTUQHinFihI5RmOqCjifyGHNZ3QqRec1QVmiI0LlKuTrNdDWwI6vY1v3rzmfNG2N7t7VFNLOLK/HayjIgLKJBDpr7j3wCH1Vq49P6OkUYX131qjIXnvj1RNy24DaY4rRu10xLa+zpOtW9+z5rmQaehvtLzlP3u3NOqpcoMb3oaFvCBoii8ipbJHCWeYzaNq62C6EnYRjMuCnsaxTs9GDfUFOFsh3/IAklHrtu9ZQrI0theh1TTTUidvyBNiiwRfs7WyN07Mx9LOkhbXRSfVKajWpTJNL0lGuQBJ1+gUZBZy7zMYpWdABp1cYO19oiqFfbBNvorU+TrwGStQagOnPSc8Dwadp3w0XN5PrUqZhc+YzDcjHtH26dIOEAt6+5GpsLtQv0QZ1aeZ3D7E7Q8TxTEplSt5QrhVS7fsAZb7tvTHCaFlK17z/g/RMCnv9sKctX0WidA1VxsLgTdEztjroEGDENuPhHhDqScg0ReDXPWZ3ZlZEorAisy9W89ms8VfQ/dZ8nWm/dh23io211FwUFrg9suSW1Q/x/bs/TClnZQu6N8gJURiSoyB45pn97hwMILSP0yQr0Ntq64T9g2Qz/OsT02kFrqtlVOsRXXHUGuxN0jDROHdEJpwzvhEvGdkddGGpojNAxXjlToFP42tLyrg7WfrC83Zl4sOpSDylX+wFP98QjHHOmN0pQ9Hlyl/fGvScNsHUGfv3fQZjP/QzoNwn1irkGe/bsVIa+pK/TuKO88jxlLqxTYtJmm1JgfdbxHqDDULcvnbF7E5Jq8nxKt+rblPYZUai2jf8q2bMKl4f/qk6XrrYLYWTdDBNuDP8epbv+df8GJVk4ZeVlGBW21aeU6/c1R+LTgdZaorI8nLn5VowK22aLZMYw6mgVQ98UDbE2MdUjA0/BTT1/tV14dKjjlIja5sKxsFT4+D0qTMf2cm1foU6n2bczRi+6sP0r/G/ucsM3C/7FqU98Abw8HGAjghPZxfZIlX7sY9NMJC/83ExmyLG08rt+zlV08tEOrwJH2qfG+AUvSsty1ag0S85OtyU8ZmtXv37s21SaaI+eBUB5YTbyDIIuuyYBVYVuyocGn65Z+fB85+G7HyqIoAsRtHSCBR9FPoWiNYFdCRQf2Ia+pn0eO1yNtEmIUle/d1VdhaxwezrBiHnPUsyOuhftYE9NcGRY+R9PA38+6X2hyguQZ7BNMKZbdc46pKvt/p8HIrUxK/6kc8rzUYYYFUKn+WhfwyD4oETo3JxIWM/00nkj8eq0kW7ngfoKbWTaOPm5OQtTdvnZ7SrqNi1iS01nrLT0d5tyZbRBs3cAvv0vTZv7azUFrqwxB5xudV7XerSWFw6bEo+wd7zVFwVpGPfDOIwN2+AyQqfblejsLd1kK57O3rfZY1QgYfa1uCPiS58FHVOulYhU/mG6oDPt+Rt3RHyl6vDcpVw1iwmTitaWFXuIMFm/QwXWmjd3KVdGE0kmWmNFhfW7aI3U6fVgeiRWF/onVP6O4lXfo75JL9IuKHlt6k7gBiroHq++EM/2eNen/6m6Yh7eLBxn27auolrsktejiA/k3QdsCML6MdfgrAXHY1j5credoCM2PocPIrUIXbcUbTY4jxMxZhddttYu9Rwk+mwo7Gp/0WtLl2RFAZEBRk4tZuwa9yxO+yUMRa+NR82KD1w+rahQq2/TS5JYjqAoDMy6pKwo117nCOCZ6mlYNc5NU8TW37T3YXr3lzu1VG8II4IuRNAOBCZ1RVye4eiJ5SvFBdkGDzrvgo5msgVIwFc1x2jjfFxQUZCJwWF7UIMwmxkqmxKz9m0Btv/h9T3MY2/B9OobbXVFxkHzOof2aG2LEsw/YBU1fhTHVxfn2LrxGHGh0AoU5xQiC9B164r6hNFU41xXdr3q3WQ6nBaxxdwN91ddhmJDR1sg9En7HieGaScJV6k9drNdOraHrabv2k9WqtSrnm4NhqAjR/axX0js+vcXNfy8Iee4LsqbgU83fYqNORtRba7GiHZaEwCJj4zHoJQBSI7XBOylB58A/nrW/WuX5dnqy9x1uG7J3YInlz2JmetnIi7ago3m7ni35iQUlWlRloqSIvs4NhdCm+i2G0U+CjouE0/yzhcMRoFD7d7VlIGT9j6v1URZo95ssKKw1AWMnrY9PWwRKtd+i3rlq4txWs57tmOVu6kb/qJfoNUgHAesBtne2FLRBgdqkryWdOhRugJLnBIOdaXGeuHGzmLWMZcV1I4khZdlI9akCV9uK5ZnsOGDVlO17DgqilGFSORa9/+RXf1riCAsJdCbmo4qnIXKnwKM0IVH4NWcQ7G+vL3yO8zKcl2fmh7dE09XnWfrHmfKvywiKeAI3cIhT+LSqrttv7M+dktRtGsP1c/Oxn8LvsO3qzNhoQ9dmoeIeAgggi5E0At+eRVSVuRjoa4TFUW5PnW46vAAz96o88LnozJDS604U16m1dYxNWvs4txQ3sanpohVBbFYWqzZMhzdt61tbqmzmJliTbvuN1tP8H6kK/bFD8Yn1cfVuSGC8MRljIw4i6r6ZJihMYKfwzllztqpA2iDT2qOR0ZF3Yx4D8v7CRPDVqn77iJB904eYIug5ZdW4fKZy7Fqr33fDKTD1dXQb53i3SuAv1+o35mJeoTCkgRTRD7+2P8Dnv73aZw761yM/XwswsPC8fCYh/HNKd/gn/P+wYwTZmBQB+0i5GB1EioL3DdFRFbQ5iPBbYSurLoM1/x+DT7f/DmeX/k8NpR/gVWWvniq+gIUVmiisbKsyO5Bl+C+ho6UWGJQ5WFqiG7Iy3ohRufclWDwQoudn+x4P7l8lprUUmHtdmUkw2gBpAtVisSqoiAOoHeBZf9/ykolmPVzJClWK1+YELYK96df793I/eB6xM2+HinQ1vUID2lK/SKH660wrw61ZVby87RtSJHPkhKXr1lZovYF3R+T3+cfzEdieMU7KKl22uZ9j8P4qC+Qi0TVjOfLecJT2rWLKQsWTrIIhII09N37JRJQqrxTy62ROGf2mLrgrZpTVUmSzl5TF1XnGQh7cstUZFxntGkzRi+9qfbrFWu+qS8sLcLt36xFcVznkG+MEEEXIugHLNYJVJYG1u5eXZrvkwedTpuEaGWw8HTku4g9uMLlcypKtStE1vowXarXPC3KSdQiAF5So4W/P4dTwxbbGiDccWhPrXMyG0lY3P48INmehvXG0ohD8bp1mHkghsKe6uicZ7j6y+rM1epE7gvDDScKfb6rEV55x6ACU8MWIe9A3bpBI6rL1BUvD+ruiqJZH/fa+SNtwm1HVgleX7DdpwhdZU0l9hTuQY2XkyXNd/V9f0lOglYA7c5CIAhYrAXceUhE+3aOKZuIsAh0iO+AM/udif4p/ZW4IwM6aNsiy5KEinw3aZ7KEsTWFCHLkuxW0MVGxGLG8TPQJkaz6tlSMg+JKMChps0oLtXSY5zTa2tWcROh0xtkfjUfhu3h7m1L8opKkGtJUBd5xiYXV1C06TWoxQU5qi5rj7m9esz4fehpjdBRuFrqcTupyFLhfuzVLUusn7mqpgpvr30bDy9+GH+l/RXQS1PYMioZjwoMMG+HxZurQNZm9Eqfpc3ddfqeOqObSXOdByNCl5ufa6uJ4zovLqx9vA2vLlUX3GxSolg1+kpmqrnAdth8k66ikibV1OXrlBd3go77OyOEgVyEWQ6swXUlbyAGVSqzpF+AOFO+bw2OClvr8NgZFQ+h5mgXXbw+cOyaW1UQQ6e1qRiDCv6qnc62pnQPWlO8u83tRdAJvtEhyW4uXONr55UTcxJOx4yaKX7V0FUhHNWWMFRYI3HOVJWXqFA/O5oYrdInTKwusVpX0GDYA30yfsFIa2H1hP610606o7q1VnUyfJ8nay70qwC1YNdKdDNpkZMBhvmKwaijc9cQ4Qu7CnapiMz5s8/HjnzvLuPj+7VTHn1MZ1w8pofL2ql4lOOlqDdQnaZF1wLBYrGo+hoKOm8jf2h++94lo9WUAe1/XReB61TUVOCD9R9gwlcTMOX7Kbjo14vUSdgdPKGMs6Zdt1dbu5/r8Sq4MDcDhZY4dYWe0MrRwJrp1jBT7WtcfZ9iasbkrqMvf5+q49xq6aLqi1zVe2WXZeOev+9Bv9aasXKluRT94pfg6+hHUVWkve52Sxf8bR5qi7Z4iua/WH02fg3XunVdsTHlWIyqeFulFllb5U3Q6eUaudkZ2JJ4OMZXvqTEpfFYotfQMUIXURFYJsEnCtJgsphrWZb8b9H/VCfyt9u+RXpx4NMC+L1myprk53mJNBamK5sLmnqzRtfTRaNuXbLf0hb5FXUf5p5foK1jbocplU9gTpfa/msRFHSWGHU85/eJ+15/017Mi7odhfu0WlGdorlP4PXIl1w2BAUi6LItrRDB0XX6zG4/v4skH/HqwqGywvWFb8fd3+GBiI/V/bYJUbbRgcbyD3/oUbwGraH9LyO1dl9Fp/3AarLNDneyriQZFonQCa74ZNMnWJmxslYk6PXq0/BbxDEBrbS/K/phidVE1Bc7Ca2mxqSu7ijcXLE0/BBMr7rB5vJP0UF2WjphUf/77PNX3RBdXaTC6fzi6F9GV7BmrF977QBTdWATynYuha8cv/MZ3Bj+g4N1QF0wrjuK2Cpzlaqv8ofy6nLcsfAONfB9e/52TJs9DT/v+Nnj/zAi9u4lozFn+tEuBTmjFHrrfmmAaXm9mzgW5SpN40uhOdN1b114iPKs0mFUTZ9zqovEn3b8pETcCytfQFFlERIjE3HDiBuUYa8vdXT6ybs+Bd2qzhdiTMWr6v6A5FE4tfep6J6kdSiPaG+vnzOip/F5YGeXr6toRFWbfhha+R7WW3oo0eMq8tE2tq0aK7bkwBLbY7kJmodXRZkWofsCJypTa65rd7ZD2uxdk2pUsnho0thrsJtxZ1li9KKjwDFbTCjMzcTeHHtBvTHaz/2S9XarzH2wOMIH26JAydc96LTjzZDOrdTx8tfd7ADWOCTVzWgrXwWdte42K9uzQXpl/n4cMCfbhIwnmw89an1b1fV4NeEW1JVd4X3Qv3wmNlm6qaYvZwuhGrMFd1ZeiTdrTlF1hnpk1wITeocdqJWircrYijamIpcNQf7Ai07u4npE2ucRagbysg6oWkN+rpuqbsIjia7tqmgtox/3jh+kZXkuCv8NqZ+4v5hxi9mMOEuJEpBMTbNbmdF6Ulbg6Kdntliww9LZNkrz5/LhyO57LkIZSbk2Aj/v+xnPrXgO1/5+LRbtX2QTNDGRYfjDfAgWVNhHI/nDqOwfMNS0Uwk148nWU8qV0FxYnwfozObKdphr1g7cybGRGN+vre2KcUbpBCDRMLbICY50ijOXqCt/ngi8hfcP6aFdCV0d/iOqfr0PvhJZVai+oKwF8kWgUHyYOZXADcYaGRr+frbpM5z989n494DvBbEUNVvzNLsIwrTrvwcd/5/LsLdQcz/3BdZOMfXNiGpZif9XxMY0zK81h2KTpbvL9cX14wwjs0+cbo+auopSfLftOxws0epOuiZ2xScnf4JxnbXOQE+M7dPGlqb6JebkerXD2J5VorpDyTFdJ+KJI5/ArNNnYcE5C3B2v7Nd/g9Tzuz4Ze3i+YkfaC2XTnCwfbXZpCLMnjpcj+5ytPqZGpeK8R1OQUyx5lVWUa4JqKrCDCbrVHTO3feFRelsZLk78gtML3zO7Xv1Xv0MZkS+oO57M73m95PL/nbNydiBLuiz7kV8F/VgLbNtfscYwaZZ8X1lF7ncV4JCx+F4MuUJFekiY3qlqO+UMX2dVpTm9v2/2vIVtuW5rgvWu8r1E3Wel1q3wsy9OGjRMhLDu3r2uORFF4/jZJeT+AqEzOIK1QTA+rFrw3/ClB2POPy9oLwam81dsdvS0XbBrIynrTV1xYWOF341RZm2KSlGH01/4fmF3a40sX+05lJUR/vv/Vmcn4VcfYwcTKqD3iUVRTbLEt0loQoRSCza6XcdXWlxnvIrVXWlKXFKgOvLkJnhWE6xM/VEHFvB75f2PWSwZFai62NEqCCCroH5ZdcveHWjFiEorynHTfNvwtzdc9XBm4XJo0xbMaRggd+vy1mp11d9hDFhG3w2e2XUjDVxS8yDsKfadbdTr6LlODPsLxUtYASJXwC9USB59y+o2OF+akFmQQkSTOXqwOnLMo3uri3DAUsbVT/jCzygx9cUqdpB1vp4MnPlCYBRpIt/vVj9dAfTnk+fMRQvnjscPVNr8OaaN1WU7YrfrsDtC263iRZPXDDwAvRvrdmCkD7JfXDf4fc51Jgx/XberPN8SseSVJtdRQwqSwIfK5RZWIH7qq9SUwCcU66ZpZm4+verMWvnrFr/d87ornjurGGqOeauSY4moNx/bz/kdtvvFHQL9y1UkTpvUJzoXXM3FlyAgnaBG2t7o//qJ3FzuDbsvZ/B3qZNbBu0jnH9HWBERq+NoidfldW2xUjiDxfjnojP1H0KOqaZi61zWY2c1e8s1XDx+1m/44Zh9yC8VBd0ZSr693TVM3g44kO0c9ouTGVzv9uZv1PdZxSGkz6izaXqu++KyOI0m7u+Ow86Hb3x4enq8/GveYCqNaR5sat6XDZG0GqlQ+VuZBfUzT7HHeURSZiZ0VtFb7hs7VtFYFjbYarOUb9AOlBywKXo3ZSzSXUSn/PzOXhu+XNq6oerCB2PM7dUXo+dNZ4n0/yXMAGf10xU94cb/CJn75ytvsdGePzh+ro4fC7eKLxJm1NcB5L2/4VPI58Ai2PamArRvXyzw99zS6txZ8QXOCJso4rQ8XjIyJMe0SordhR04eU5Ng86X+ydPAl22irloBXerzoBu8v8rzXeU52Cv83D1P3Twhbh5dK7XX63wiqL1TmEgvWQ7lppDuva1P7p59ixAwe1ND0DAIxK92qXoJwenqs6B9vNjvvB2n2ONX2RqEb+2l+C5i9YH4iga0D45Zi3dx4sVl8rwlSefsDh1d2U8KW43vKlzTneV9LzSpGAMhXl6OKlfo4nbUaGeDDkQWB61Y34rEobWeTM4WWLcGnEHJXmIfwfvfPxWtP3yF7yidv32Z9ThLeqT8FGcw+X/m7OjO6eYhudE1+R6b37jOmSwnLlhM8vqLcoBNPcrMFZnbUaL618yeUJl/CAcd5h3XD6yC54edXLKDGY+P625ze1/rzBNN6nJ3+qoj6MJjw/4Xn1kxRUFCjR9OuuX1FUVaTq7FxF6vYV7lOpWx29G5Un3PSqwMeMZRUUoZcpXTVYGCN03DfP+OkMLD2wFI8vfVwJYGfOHt0Vr18wyuXVPeegntjjRHWfaerDOhyGxCjfogB6HV1bSx42Lvf/gsZXOhSsQUdTjk+2KxRObGqhxUh0uz/QATmYEz4dB9bWXr74rNUqkqILHtZ5nTPrHKzPXu/4/vEdVMMFv0e0AWJ0PMvSCqUVVcgpqVTROQp25/o5puuP/+Z4nPbjaRj3+ThEJW5Wz0s0lan5v64ILy9QJytG1fSmAm+CrrdpP6ozNsNUnq8i66wZc+4wZWPEMNNO/BF9Fw7sok9f8En74w2cbFmo7nPec3R4NO4+7G78PPVnnNzrZPX9chVRZRPOI0seQY2lBtWWany08SMsO7jMpaBjluFH85HYWeb52PR9xWg1vcMYvWedKC/Ibpx3I0qt864d0tKwoDsOIstg+hsI4YVpGBO2UTVksCki3lyE8ir7cTGnpAqXhc9Ff9NubDW/gyM+OwIzttyHUuuZ3TmSH1eVb+tw9SWbceGvF6oL4C82f6FMt40MtDYLTQr7F2mb/LfzYHnBg9Xa3O44UwVGmrYhPc8xqsmLnL3VyWq0Geso6ZLAVHGGNWJakVf7GOWJ3aUxSsTT/ql7Sjx6t4tX0U821a0pdxR0I/+8GI9HaLY5hALy1oz7UL19HkIVEXQNCA/iTx/1NI7teKztseuHX48LB12o7vPAyTqWBFMZDhbYT+S+kJ6RiTCTRUWqungo5OdJ6tY/b8V5s8/D4v2LVSEtrzw4+ov1GEY4XzLCXK5OOvRj09Hr6Biqzs92H63aW1SDp6unYYOlh08ROrbQU7SkW1JUWLzGB+PIvVl5qlidNhRGewVX7C+yX1nllOfg7XVv1xLcnN/JejdGliimnNM2p/U+DcPaaVeV3uBJ6MExD+LH035Er1b2bsQZa2c41E9mlGbglj9vcUgDU+BdPOdi3DDvBtsJg5EiRkqvqroDX1RrqbtAKM1Ow/zoOzA6bKtNJG7I3oDpf05XYpNQxPKE5W/t4C2jbsHrx76O9054D4PbavWcvlz163V0l0bMxaB/6jB42wN837iqPHVC4/7oqSxh+cHlGPPZGNXUQYuR/ZY5yjqiV9hBZOx3suspyUZcZQ62mDVT5JKwDfhgwwfYV7QPF/1ykRKErlL8HP3FGp1DK97EBvRSwoxNL1qziuPJNiEqweE7vM3yDnJNUer5GYW1jxWMDEVVF6rjgS9jyNj0khAdjociPsLkrPcRUcna13glfJyngVCw5ln99rIyAzN39Ubchi9wZLgmhsf01lLypEtiF3UM/WrKVy5rM3/Y/gM25Dg2AvRIqt1gpGcZLgz/HeEH17hfELMZbXbPRnt+4ugIFdFhaYGe/mU95FW/XYV8Q4ek1mASp7zh0rLqNu+2qoy+hFpknhetrVCCtFy76MkrKVdiaEPrg9hZvlDV7P6XtQRhqXNwReXtWGIZYnsugwS3VF6LH2vGqYYIno+8ZRt2F+zGqsxVeGLZE5j41UR8svGTWo0R90d+gqgt7jMe7sjJPKDOPYT7abjJgvSM7FrlIXdWXYMXqs9RGSxdVOvmwml7fMtu6OwsDFcinueMbm1ibU0sjHBW7XN0eoguO2jzvuM5jxdsBywpyNq7BaGKCLoGhkXRdw29C+f2OxcXDrwQ1w6/1vY3XoGwVoCRtgN+CrqsLO2LmRsehoXFDyKjJMPlCY2Rl7XZa5VguW7edaiKn493I5/FcxFvIr/UMX2QV1qpojjllijVEGGseWJNEYtJK63dea7IzMxUlgxMzzBCR7HgqTONBximXTm9Yoe5I3aneT9Z7CkwY0zFa/jFfATi4jOxOdcxJWEkrdjxam7BvgW2Dsys8izcuuBWNb+TJ4SX/3sZMREx+Pzkz3H/4fejVXQrJEQmYPoh0x1eo7Cy0OEg54qOCXb/PnLTyJtU+kiHzQOPH/m4rcOSB1meJNgVybq7q36/yia0aC7MXuDcwpKA65cKC/LU4Kvs2BxbJIjiiw0CRpjSOlDs3wmbqVbWibmr/6KxLiNN3A9XHFxhEzqH9UxR0aC9lvZIqsoEnFJZweBgYTlao1CZqnorCKcAZ5RRp9pSjpKYfFRYIpGf5bgfWTI04bHF0hVhEUV4f8tThv+rxiurXlEWLs4YPRk5KSKzSDs5s/7JOULHfc9IlaUEmxPz1QizDBfmuPysrSzFqoMwus0SrM1ytH0wwguGp/59Ckmd/lAiLqKqAPGWImXW3SXFxVi4tvHIt9Yd5ef4l/LyldjS/baGCKOg04mLdH3xNqX3FFw97GpbapZ0S6o9UzW1FWsUoaZydMp134CVmbkfj1c/j5Fh21U9bXFVIf5v+f85PIfHU2MUkBcLugXMwQzv5RmeqKkoso9ntMQj0lSDNIPoKSwuAffSXa0d96+IyBLMM4/EphJ7hJzlAovNQ7DL0hF928fjjdVv4NQfTsW6rHUu35sZDB7fjPuy3qFNBnayd7pWFfr3OUsrq/Fa2d22ySq6f2pmtuP+pJ0HLba6R902hhdl51Y8gMXWJkCf3zd9vWqoCEc1ysK2I69mmzru3B7xFUYd+Nr2vKrqGiRX56hGKJZQ6B6sPD4VpAdm/N8QiKBrjJVuCsO9h92Luw69y+HEp0Xo4pCAchzMdzG2xQPpBZX4o2YkdqSuxv6yzSoK5AzrongFq8OTaU74AuSZohGHilpzW/k7H1cROmvKlZjCyjGqW7KK0MVU5TvMEnT4nAdWK0uGtqZ87K74Sx087lt0n0chwhoJWjccW/k8FhdZbSw8sDdXW0+mqCz8cPAhXD73cqzJcn3FrZ+gO8V3UpGkr0/52naV/+bmN/Fn2p+253655Ut1ZUovsnMHnItZU2fhxWNeVJ2KOoyyMf31zPJnVPrUV5h6feO4NzAwZaA6UdObbHAb7cDEdcM6vfQSu/DlyfjFlS/axn99FPk0njG9ioIy/9LyOsXFBfg9Pg77u83Bc+uvx597/1Tvyxo/CjJyfPfj8d2p36FrUm0/wO+3fe8QkfAVCmimbxiR5Pq9bO5ltggoo2W0ruEBk9HZ3APB73Tdvj9LCaZsS6JXU2TW1OndrzrhsXuQhVYoy3M8eaVvXaWE3m5LB4zsloSuSY7jy2475Db0bNWz1nvwoqhTVBn+jroF/YuX2SJ0pa4idAZB17d1X0zr/iD+yb0UwyrecxnN35NTqjoHP43qj734TK13nsCNIlWPRJ7505nK7Lgo5jdsjgpTUaBbqm7Eq3EDsC/62VpRatYTUijqhe3BpqioAK3Nedhnbq+Et6tJJp6i4rxg+nrK1ziu23FKmCdF1W7g4TQZlptkIg4WDzZRO7Zrn501WxQSvLB7+4S3kRxtr6W7Y/QdtlIDPeWq+4HmeOmg9YQ6TlYUI88Si7DodCy39MINlTdjd759G5YUF2FlTDQqIxyPwa0rTsaF4fPQu8ieCt27dzemR3yDdshFWthnqjaYtYjXz7te1WY6YzwG6Y08ozvY61spsJi6VV507ux83LAzqwQppiI1AYPNXvr6yst2LGc5UFCGtdFXKhHW0RpV1eoYTVhmGYjlB903t7kiKWMFHoz4EJGdvsFTq2/CZXMvRrvOK9QFSkRFnq3mcdve/epYwWaYYV1aYZzVAH2vuT3C8kPXXFgEXSNBIeccxWBImdGpxeZByMj3r4txQ2krXBszGRWJWgiaaQHnqMCx3Y5VJ2qjsDim9d2wWGK0E11xbUG32twHK819kWLtoFqSvgQnfnsiEtuuU6nUDZae6svpipLCHGSFh6Gw6/d4ff1jyC3PVSLIKCodnl9Vgt/zHkFYtBYVWrnbuxN9eNoyfB93MxK7vY2i6jwVeWR0y1VHKrsZV120CnPPmosrh16pDv46V/W7CjHhjicOo0hLjknGER2PsP3+1pq3lHjUmblhJvyBJ4Z3T3wX7534nqo90+E+cf8R96N1tL1An8LvttHaeJ3UxGh1wqfodxWZ8QVOIpmRrJ3kthdsws1/3ox7F92rRl0xnfXo2Efx/Pjn1TI68+3Wb/Hg4gfVZAWOy/IVFpDfPP9mlRLSocgxXvFzagQFHdm9y32Hoo30VcAnZ2qjqnxge2YJbq28Bn933I6/Sx7Es8ufVSl2RkJdMbzdcJhgUgKK9Vpx6KqiEShyjCJ8Wj4W51Q+oPzezh4xBB9M+gBXDb1K/S+jlWyQcUdMdDS6hmXBXJWrmlVGVLyJz2qOrRWhYwT1k5M+UduHTRUTuhxjO3xnFLkSdCVYbeqKgo6/q5pd1pTxBH7F3CtsZs/83Oy0t0euLfgpNQuxphLsNCWjOHUB8s07VC3gG2veQKW50p6uDIvEblMCiosCb85xx4aNWsSTETrWzwVCn9Z91AXYj1Ndz8Vm5Dk89RNM7xzL1mJVXuKK9DRN6LCBYoS1w3VI2yH4cNKHSuDwOHLJ4Esc/odNEZvjyzHVcjM2lwXeSVpYXo3vLYNxYbckxPd6BUW9Z+KX8D7YnW8vg8j9//bOAzyKcuvj/+3JZje9kxAggYReBUE6KKCiCPYGWD77tStYQK9XsVzrFbGiWBEbNqTYkCbSe+8lIb333fme887OlmRTSdndnJ9PJJudnZ2dd2fe857yP6US9haOQOjh6zAwcjj0ar2Qc4n174LrNb9icOVGlJTL451zch/u034LXcjf2Ja/xL6P3LJc3PbrbS65wuJzmOPxv9H/w4QOE8S98sKOF7roNNK9isKuJLgdaMlGVgPyBQ+fyUWwqkh42iicSYuh28rvFfOYM2nZhQhUlaAUensuJ0mNUDEfiQO3PyJ7+OpLRWE2Pg4MhS5oq3hM12l303jhnAhGAU7lypW2hw/L8+gZKRi94oLFd548dTTnnSgzCuknT8QnDbq5c+eiQ4cO8PPzw6BBg/DPP57df8055Lra2hPXVzyOUwUNC6elZZ2Ef7SjGTTdwGnCcq7CojAFTdS0gqULkyQbEoOSRM8/f5Qhq6ismkH3umUK3rVMFB46SoqlogIymjYUvYkfo87gIcu0GgUeywuzYaR7id71pk/5J1UTbImPd32M3TmbYUx4G08GzcZ5B1+u83MX56bh+WgNJJ2jmpJWne/veN+tJ9A5FONMjDEGd/a5U/we6heKF4a9ILyoNUG5Zc55UWTckAeqIZDnoFtYt2p/7xrWFR+N/wiR/pHCw/D2+W/bvQxU6XpYr8afIQUirNYYDln34KBeX83YJyg/8LLOl7kNmVJojnLDlNU75Yd9f9D9hFmVwbGD7ZIdCjRROL8PFUbQxEnekGNVcmncsW7DBtFPeO1P7pt6V2VvZgV+ClahInAfTpXswye7PxEhdiqQcce/+v4La65ZIzyVlAvZNaQPnqqYitdLJ9gnL8o7/WZ3AbZJSUIbbkKPGJFW8a9+/8L7F7yPZ857xu25pO8PhZxLQn/FZe2isTJ6OY7np0Lb+RUg/E8Y/V0XV7RPMjCpIICuXapO7qE6jCX6mSjLrF5Qk5qejvPD3oJa79qtgMZA6YBB3ubbet/m8ny+rgI7DQYMiXkVam2h/VjJ478xU84xqpDKENL+B1wUl4A3SwaedSVnVTacLMU7lrHYF7UdkZFNW1FIKRbzts4T0YIi3SacMAB7gjLc5iESeWnHhEwQ5Vw5d4joFNxJePjpO+IMGcu/nPoQqviF2N9uLQ7kNu4aJTIKSnEwYg/KdLKRoSoLx9Wq1ShOc4T8yLh4qnIajpX1xH+Hv4rlly/HrHNnCQ8v5X+Rx5dC+eKzZMoL5dKcczEg0rE4pYXHLT1uEQs6Z2jBPzJ+JF4c8aKQ9ZneQy5gqFrpulvqgENSLPak1l3RrnAqVT4W8tAN7RyG0pDtWNVhNfbmu3qQs2yiz6QZqIi8U0/fznF5KApfiyElfyKnSmSpJqjAIteShnkkoSUZMGfYHLHYig+X04dCUGCfy9blBArJkp1SR/S29dg+LykMH1nG4+byh/DPkeZte9dYfM6g+/LLL/HAAw9g9uzZ2Lx5M3r37o1x48aJfC5PR6xA1MUIiPgRW/IWNei1KbkrcWmJq+eAVlR6jevETZML5ZhQoj5568ICyOMjh1yzCqvn0JF4KUkfUNL07LWzkVHicK3rgrYgMPAfHD5TfZVOE521OBcWyQ/hZde4PNcvsp8wOJ3JKsmye7lUmjK8EVOGvYYjdRaHlBdk48X0LKjLHInPZCRRVWlD29pcl3Id7uh9hzg3F3a6sNbXU+7jOdEOYVUKwVCyelNBE8aCCQvw7vnvCgNT6TyxvugNzItPxTdhldiU1rhuERvKEmEtD3WRVFEMutqgBUHn4M72x+Sxqe9nps/w5pg38fNlP4tJkLxe4zo6wlQE3Tj9DQacWzYX72T1rtNQWLNdzpfctmd/vYyKrNTNCIhYVm3Suqmnw9PqTFRAlEuVLnkjtkpJopBhb5o8ef1zOBMzSl4RckPkaQhyEgMeGDPQPnZVOZx3WIScC42/C+O6RFeCDXlviBw8Q8RveGDtFaJStrbFH4Wmu6mPoSS/uvGbl3ES8wo2IDyDul/IBlzfyL6Y3t11Ur6px032RUWiuSfyDj+AJwtnosDP1RAcETcCgyMGC21Fktop9V8jjhURC5Ga37D0kLr4MTULr8fnwhqyBW/vfwh3/nqni6bj2UDG8O8nfheSUQorQguwN6N6rih9pw7lVoqq8nCzf7VKYZK5cb5HUJ7r7b/ejo92vy8eawzpyKl4r9HHuuPMIegCt4kUsu7GyfDPGYlHtF/DP9vhGS8tyhWGvVFdKbRMKVWA7h0UpqZczABVib39V0nuGSEcbfELx5tjXrPLwJDBRmkltUFFOe5kfeia+NQyGo8Gd8aWk/Xv3JGZLm+bprXi8+Mz4Re9GBpDBvIl1/OVnyN/D0mGRfHQkcxTqvEN/BlaiK8j87HlRP2Mq9S8UqyMOIYytQqSqgwzV80UHW38AlJxRIoWHmEl2rT5dLG4zivUfugeG+RSuKUz78Cf+86u9WJz4XMG3SuvvIJbb70V06dPR7du3fD222/DaDRi/vz58GRo5fjzsS8RlPQC1OFrkKb+uV7yGEQRKf+XluCRjBLEl92HdqZ2GNZumDDoaqJDkGwAUZXr65VTMKX8qWouczLwvtHPxl3axQg0qqpJUJgKY7Gt9G3kpVbPvyDvUYWkwlEpCp3NA3BBwgVCsuGNUW/g9dGvu+SiKSFi53CcVQWYVUXYdCyn1sRadVkewiq0SLI8KnJZKBftnbHv1Fsuwxm6uZGXjsKr9dl23th5mD14tiiaoHCucx5NU0AVfWRUEJT7ROGyvQV/KTqXWH56QYP3SZIHBTnJKDr0IGLKp4uwJ4Ud3LW8qgp5PDenb7Y/piKKmsR4a4IS1G/tdavwejlX/hJUTXmuaC1nRVnYfLz6z/wa8y2PZKfDUCbfyIMs2Thah4gr7Scycz++SD0JtSXI7pmgEGbV46gJkmmgIh/SsduTmi+8su+v/x8u06wR8iGX9Km9a4ozZBhTyM6ZLJWjeq7EUmwvhHEHFVRYbR6VksLq2+VlZ4jOo345Q7Fg3Mfiunhu6HN275zz95i8iJTP+/KwdyBVhCNIqsTXp1MRmdVdGIOUv/f4wMeF8fL29reFMarQXUrFn0fk8FVTkF1YhizN+8IYUlh1apXIAaztfNQX+vwzBs5w+RtVL24/U71y8aPt32KhJgLXVjwhvHN1LRBp8Vw18nDGtB2b0hp3fhYf+QRQSeJ631X8LcriPhGXfmVhtl2RoF3xXvxkeAKdA0pcKpkVD52cmlEqQoTqkixRnZwUFYQAfYCoRqcF4/gO49FYEiP18I/7BIbw37HoxFPVJFxqYk1uGHqUvo/DkXuwK9txfrLNu/HXcblIhc7lmcL9doOOohNUOf5/y/8P5ZK8oFoRqMFr256qlhvqDsorTc7oitAyh2F+VfJVGNtxGL60jMINFY8JD93OjH1IzFiGx7WfihxOf718zQzuFI4Q4w4Y232K5Zn3NyjlpKWou52AF1FeXo5NmzZh5kxHqEytVmPs2LFYt87RbseZsrIy8aOQny/nrlmtVvHT1NA+RaeCKvumfDeSR7Bq5L9bVZV4c8ubeGrwUy5eLNqOVv2U36DcnClfJkhVJCqFUoL6498XXyXCjvQ+dVVChhp1ohCDIM0k5bioXH1f3mb4qcpFSDbCZMQzKc9gSMwQUcJOE0HI0XFQ4W/kZqVV+zwnsorwgeUi8TM9xB93D3xM5GGQl8fdeaWKX1qFLz26VDzukR2DGwp3439HszChh/tuFMcy5c9NfSUTQk24oesNuD7lenHjbcjY1TQmdaFT6TA5abLLfpoLDTSY2m0q/rvJUWF3qmy7yBUcEFV/Id4zeSW4QvMnJqrXYaH/63j94nvqfexUzKDcsMlAp0m+Pt+xhnBeYiismY9jg/k0Pj6wB2p9Pu7re5/LZErCsdOX3YzhJGNQBESocrHzVF6t7a3I0xtQmYN20KI7ZgHhH2FU3CiMjBtZ73EjIWLyiE3XfY9rjwXg9cN/QCWpcFinxTFLAkYnhzfoO0CLrq8PfO32OY1KHu/a9udvCqaWLSgrykNxebEwKMgwp/EoJa+dCjCaw9Arogc+m/BZjddFUlCS+KE8Muq7/J1+tvAKheQMwxs3PCnC6xH+EcgozMBj5zwmtPlI9oe4ouQUFi3eieycSEwfkgCj/uymlL8PZ+GB9HK8Hg+U2GxPWpy9OOxFUQ3eFNdY34i+GJcwDsuPrUBlTn/4ZQyBPiHJvm+6d1Kh03cHv4N/nBYlx29C77gudb63QW3Ai8NfFDmH5LlWSxK6Z7cjZdBGHbdeikSABSiynYdo1WioLadEBfLp3GJEmfSwlhWS4i38AwJd3iPSpBeC8WaUQJV5GEa/ItGu8TvLUCSR9prVKtI4+kf2b/Q5pQrYV3c8BK1Z9pTnSgeE4U2Ldlq81wQZo0eyilEGIzpUXI1i7fPinCv8fnQNhsYNxNytc7E+8CsMtF4EXXEMqPMgXfvKd08huzgDlZZKcY+sjaOZhfi2+CrgSCXGDt6CNMtGcW8pr5C/sypYsS3jbyxd+g5SIv1wXmYuDsQF2c9PuZQLKf4nWFUqFGkLxWddctkSu3OisfNITTRmPz5l0GVmZsJisSAqytUAoMd797qXs5gzZw6eftq1nQqRkZGB0tLG5z/UNkh5eXli4MnYVDDBhPHtxmPJySUuVanXxF1jT5Cfu2cuFh+XCwoogb+DqQNuS74N2dlxQlyXhEBD9FYU5sh5AOkF9fDwlZZjuHobrtH8joVZT9tD02/vfBtrin7BtLgAhOSlY0D+SaSnl6N/QH/MGzxPyHwsOJUDFFCiaSZOnE4TiaoKu445LrpgrQUV+RWg/wptTZHdcX+X+xGsCsbxwtMw7gtAmH49thyk93X15insOJKLhZWj8LulL/obrI0Oq9c0Jp7GyJCR+ED/IXLKsyBZDIiVzkdYZViDPvf+04WIU2UgUX0aAVoLsjLrnwtCHq3FoxfDAotIvs7Pzgf915RodHuwMcwRuqEwfH5hPu5IuUM8Ti1Oxb3r70WJpQjLIosQmROB87Nz8M2BVAyMrvmGvuFYvuhhSZIlHUyhuL3v88Igb8i5C1ZbRS5VCCpQkmegixaSSsJzoeHoYuiAwtzsWr7d1ell6oWlWI+szCRUFqagF04gOfxLLAkwYXTsaOiKdbV66fV+JhSUqLArZhkGffE5ruxwJW5NvhXZxRUwVOSDdI6NgcEN+oyBRn/YpMHgZwpCmCUMYX7yd4yukSApCA93fxgzNsleLmtlIN7Gs7hihQYfrY7HLefGYmL3cGg1Kqw5swZ78/bixqQbRQ5gffh910lMt2TjojMx+Do2FYnmRMzuMxsx2pgmTZuZ2mEq+vhNQO5PCzBJOwcvp/W07z+7LFsYc4SKQpnx70OjSUR6et1efyOMuDHxRiw6uAyzjh3CsdJQ7DuWjWDlpDaA8LLReOfYAswLjMHuGAnd1eOQK30rigm2HTyFjqEG+EvyHKU1+LmcH52lBJ9o+sEQ9jt0J+7E5uwxWGsdi6XWgbjPKEtKnS1pJWk4UeCav1lYVoSS3BKkF9W8/9N5ZRhhXY9rdL/jo4DnMDjpJszdOxcRFcCI9Dh063wRNhzegK/2fwWorCgJ2YHSoIP4cZcBgyIGYVafWfjP1v+gQqpAVIkRpWcuR25mbp0e1D0nMxGGPCENNjHqWvSJm26fK4f6H8WLqlmYJPVHubUM24PK8Jbkh25mlf1c/Wfbf2BRO7zE/UwTYS2w2ufZpp5HCgrqn5PokwZdYyBvHuXcOXvo4uPjERERgcDA6uXuZwsNOn3xaP9VB/0B0wNYdvI3WFCGqIIozLrwdSTHO9pHTVFNsRt0lANCN8vo8Ggcy9QjVFUs2qOkxIcjMtLW5LweBIVaEKXKwQjtRrwi/Ym7/3lVFAT8eUZO8D+h1+JExBFsL1uN4ZGyZl4k5P0v2/KPMOiCpQIUqwMQH+m44eXvyMMbuv+JZNaI9nPrfUwzo2aKxOIr9yzHiMxknMqshCk41O3KP29/EU4hAqekCFwfH9Ggz13fMfE07ul7N95c8h3uyUvDF+EXoGO76nIYtWHJlESiNHld20cEN/qcNReVGWcgVbkvd43uKo6TdLGeXPckcsodYa1PQvzxa9k5iMyrdPksFK6hfE+lijbjQBFCSYMOZvTuGIm4aFdpkfqiDYzCq35ByDLJnmRivVGP27rkNPhcXhp5KfacSIZl/5s4IOVCCyNeyMhCuu5pPDBoFCJNte+vJHQLhps6oVIjG+WLji7C48Mex4ljOTgphePzytFIiIlq0HEFhkQCGcAea3sERrSzv9b5GpkQPQHpUjpe3fwqNnd4AgP3zMKn+jm4sHgOXvi9Eot352DeDZ3x+p7XkVOWg825m/Hsec+6VDTXxNbTexGvyoB/0WDMOe9BjG4/TOhBNjV0D4sKTcIHP34FM4qRXaayf9Ywa5jwdCpFTxUaK4r8DyAysn6hydvDb0eQdCEiDlyFHBSjwKq375u6ApHOJRUb1EWR5TTCUYpuOXG4//rXse5gDn7f2VeksWxL/xKWrDOQTDmoKNMgKizUZZxTT62EqdMrcsiWFthFfyJel4Ciiij0TYxGZGTdklD1OYfvBr6La366ERUohsaqwR3dnql2T6LqWfrMJFXULbQbOukmIkl1Gr3Uh9E1Pgy3DLgFB3LLccnaT5FtVSHHqseCowtcis7IsB7QYQAiAyIxKXISosOi8cCKl3Dw+LWA1YhynVnIxdRGZulJfG94Ungpeyee77J9WFgESvMAi8oRMl4ZXIZuoXsQGSmLM88cMhPHl2fgSOFudCnWwuQ30eWcN/U8QkWdbdqgCw8Ph0ajwZkzrsUB9Dg62r0L2GAwiJ+q0IA01+ROg+5u/1GmKAw034Kb9j6D5UUjoLZEuWzTO7I34kxxdpkBCst0Du2Mb3IP4rmK26nDJ+aHBjTouP31auyPPIKx5nYo0CxEWg5w0/KbXFzgFFa6KmVKtf22jwpD0T6D0BMiF3pSlBFvbn1TlPGfyi3FcFWmaNkS38Bjom1TOsbjC6q/kIDtp/IxJDHcrptF3ROWTVkmmjmT0ju1xOkQPvisxqumMfE0rki5HBu+2ovLVT/h3fy8Bh8vtZcy2rsR+Hnc56UChbX/7McGjRzauCnlXlydcrV4zmwwi7xQkt9Q0Be1w/7ci5FZXiDGkBY6JPRMYr5U7EESH5SacCijCMFSDNKtwTgvytzoz22NqsB8lUNaQW3VolNuV0ztN6ZR+wz012GC5nessPbHp5bz8VjFzQgzJyOuipadO7SGYlTahLEVD2pmaSZO5JRjo5SCjZUpeCK8YddeSHiEMOj+VzkJkeGRLq91vkam95yOiUkThbTNqSO9sfqXj5F/Up4g96Xl454VTwhjjqBUimuWXINXR75aqyGTnl+K9IwzCPQrhiY0ARcnNW1OalUizH5iISqE3HOL7Z+V/qWoiBLa61GQggfP+Ve9i6zIe90xPAg/WQaJNJiw3FKxT0rmv+PXO4RRN+/8eS4ySO7IKCjHdmsnHLbGol0w3ePKMa1yGqAugzlrtrxRO+CKsmh08TsNtbqP/bWjOpyHudvlXGTCJHXFc9oPcFTqguToy5rsuk8OS8ZNHV/B8n23ICYzGSfTg6Hu7rrvLRlbxD2b+PX4r+gSsBVj1HmiwjUp0gytRosrO1+Hv/7agTLoYMkuRkpcCv5OXY8Km1ROst/FaGd25KgOaTcEV7V7Hqv2LUG6KgQ7TucjIdyh1UjzF2krUrheyfM9kV0iNBYLYBJC987nwBCuwzUB0Sh38sAlFYRgWm/HuaL3//ziBbj/rQtxW/5p3J2TU01+rCnnkcbsw7Pu5meJXq9H//798dtvv7lYzfR48GC5F5+nc170eNyT/wI+s4wRoorO0JdlWNww+2NKaKe8NPqikgZWEfzrXKW4o1JnQYFTex/SiyPDUUFT2s2enO8MtU0ZWvY6PrKMw6H0QszfOV/8jPhyBLbnLRMrX9Fbth5tv6rSv30IXtS+I8LBm47KEwPlb5H2G12sw78cjkPZGbhc8xeGqnc06nN7K3qj7DkuLcqr1q6tJijviSRsSLyWNAdJLJqEVT2RwbGX4vmMLMRn9ESCbrzL95+KVp48dxYkSYWksgrclGbGPeofUF6UK6rYyLNCyv3EwdyD+OGQ3JJo/5lCzLNcgv9WXiXaHjWWnu1GY3JqJAyFCSjPGob8Q48gJWEmzIaGf8eV4gbq02lAhdCgJA26sKD6Hd+IqCugr3RUsZPe3O/Hf8dP21MRrzqD9qozQjurIUSSQUchqHpcU5Q7RKHUDonJuPruZ7HotiEYrd6MAP1xHC91tLYjUkJSRG9fZyiR3TmJft3hLISoCnHMGonIhK5obuj7pDMGiZZTObk5Lrmg8QHJ0Ba1x+tnMjAswFVepz5Q+y/KH/7KMhInsovFQpRkfqj7CnVcoPaL7ip3V55YKarZCZIbuaPifizXjhDfE+pVLRQHgla7vOaQQYOIANfvTFJoO3TMj7U/zlftwNdhFSjUBAsh36ZkYvc+eO6EGkOKgHWHqqdwVPXM7i/6C9/EH8AplUn0UiXou0b9VN+3XIQTOaVC9ufe5PfRIT8cYZVWjIyuXoVL+nDv6V/GJPVq7Dwlp37QGJLO6UXfXiTE2N/Y/Ibw7NPfT2fni+IltTGkWju7zjE9cEW+I2EiMTcGnSuvhbGKvJPJ4AedaSZuKvm3KPyjhaIn4VMGHUHh0/feew8LFizAnj17cMcdd6CoqEhUvXoDJEeQgRBReeVOsoPU/P+48g9RyUndDoiTOcV4VTcX12r/EJVADaWzpV+1v93X7z6UHb8dFbl9EWIhEdPqUHPzHAQKY/JgRq5ohK2QqvsEn4ZXokxnFj0rG8qADqEYpdmKPqpD2GirdH3+n+ddttkhPYlTfsUoUpurCbH6Mv4BsofIXyquph245PCSau3VqJiGKmTHfzMeazO/wnzrSLxeOdljDbqkjh3wZvbjOJo5BdtPVq9sHBA6AdKJa/D2mXQYrSo8qPsasaosURhBhUTOeoDkMSaj4cCZAoQjD+0DNS4ttxp8bPHRWJD7ADJP3IHg9MGYbN2KSd2qCzDXF7o2yqETBl031VFcol5j769bFx1CIvDy6UL0yAtFL/NEzB83H5EYid/3puN+7Td4ze89u2e7voxMicZ6awqmaFZhuE0dv74MjALeMMzDM9LPKDpyFzoEJoq/tze3x9yxc6u16yJxbmeB6rUHs3BMisaI8teQ2NuxcG1O/KiwhCZCanNV6shzGxs6E11OjMLo4hLExtevCtoZ0kyjYp0U1XGcyCkWlboFFY6cqMKKQjy95ll8vO6o8EwSn+35DPf8fo+QaaFFtWgFh1JEmeT7J4nbPq77FJ2CVri8V0VBN3QJcz1GMkB7l/RCSpns4bJWBOL6/FxozRENNk7rY7w+6vdvPFN5AzYczUZZpaskFVVzV1UdSCpVodQaKHrjim0C/RCkKUMsMsV8RhQXB2LcmXjMO1GG9iHVJVN6tAvCGSkU0apsce0T9NkmdpqIsQlj7aLJpDdJmqrqMtnoMwRWF6umyt9p2VZcdqQPCg8+jK2p98Kv41C4o09yJ3ursjUHG98JpDnwqZArcdVVV4mChlmzZiEtLQ19+vTB0qVLqxVKeCqktfOIdiEypCCk5lfvQaisjMPbyTdbWnlQ6PE89S7k6NuLdkINRR8wABHFy5Dhn4dzY4Zhes/r0T1kAMKL0vBy+QF8H+te/iQhzIj7tN+IfpKfFQJFOkeCPHlQLisowD/+dUuA1LTv3apwxKiy8OHxHGQUZWLlyZUu21hUJWhvKUa6f92SAr4EJbojDbJoaH6ZvTXS8qPLMWPVDFGRSK3FkkPl/MuF+xYKzTjKKcvAQkidjCg8OKPehkNL0zMuRHQgIbafrN5ijDTgwosjEWWwIMvUBShdhkhVLr49+DXW5Lp27CCv0MncfDFZ7zTcjx+MpOrfeJmGlOhAJKjSYIEa/VQH8LL+bVjiXWUwGoLZTyvCTAZVBcaoN+MG7a9YZ5bDU3VBk6Cuwh8T00Oxv8N16BPRFeM/+0s8F4xCxERF2yUX6gtNknm9B0N1Wt1wT6YxFHtiJuOck0thLYvFVTEvI6/jUiFvU1WPj0SVSfyb8qSuW3Id7u17L9YcpqiABL1Wg34J1Sfw5iArcgi6npovPNYUESEtN2LbiVxskjpjaNlrmNu5Yf1CCb1WjbuNv2FM5UpMzJ4nFsgnC06KziQEacBlHLoKs1bvwvLdqeje4098vvdz8Ryl1Dz79xxoSvtih98deEk/i6S/xT6thmA8ml6Km2NjAV02/Cq1eCg73e2C1mwIw6epaRhgeQ7qyiCco78L+6Nqrj5tLHTvTU5KxK7Np1BZacXW47kYJOSHHM/3DO8pjCvy1v203h+lmVb84O+PcbbzTfPWvcbluLj8F4zKfkfMazQeiVTGbSUNuuoe8EizAQc1oYiUcrHztFyMQO+1YPcCEW5VIEdDr8BLEKySPXDm4HC3zgmS7qKiqVBUoJ96I/pGOzQ3nRneToWOuv/iTctknM5tuLHfnPich464++67cezYMSFHsn79etEtwlsgD10v1SH0Ux+o5qEjWYG9afkuYbbc4gqhMURVrtqAxt0ENcFxOJo2FYWHHsa9PeZgSOwQsV+6AAZrdiPUz31Yz0+nQTdDJs5R70PBmaEuycvlWSPxTOEjOBJZt2CtO+jCLAuIRTtVpmhcnpGnxzeXfCMETgmdWo/S01ega2UBtKbGtQfyVvxD4/BixZWiz6CicL/29Fo8uupREXZLL0nHtKXThKQJhacX7l3o8vrYolAMVh0QGoSeSLBRjwcDlon8yF2n81FhsVY36CCvyDv0kK/tSOSgIrefkPqgdAESzV508SKhBp+eSyHNcphUpQgIiTrrBdebhrdwt2YxUtQnkK+LhKaR153ioaMq7Y3WLghQlaGIJILqaWhT6KxI8odJVSLuFZ/+fcweAoozFCM6Wm4o3lCCdnwIfSM1tiI79RDXLJ3vv/bn4K4+d9n7AyuQnhy1mlOS3qkTxWubX8PpwuN4SrsA3xnniHtLSxAVGogS0H1LhdO2tk95xRXYcCwbldAiXR2NlHbuxaHrgkJ7dF/OKa5AUZlFfBdJ3JlEvP9z7pvYf1q+r244mi5SIhSoh3Dv0CEIUMnXtiHAUZynCQhFiqUQBYfuQ3c8iQePRSOlshjhbr4zfgFB0FlVMJWbhbZgqaRDcHjjvhN1Mcm0B5/onhMGOYXOq/L22Lfx5cVf4sG+TyInvTdWWvsiK2qIyza6gGBxvorKLeKcnc4txV/WXvjUMhaxwX5u5wiLMUos5mi+OpVbgg92fGDve03QXEYdWzLyIPK5u5XOB+JcQ/9EuxB/pGpihYRXH/VBvKd/Bb0j3X8Hu8TH4HzNZnwxKQgzL2z+1IA27aHzdsIDDNiqMsIklYicIAVafdz52Sb8uiddNLD/YOoAMfGRO59unn6qCviZGzexRPtbMb7iODZak0XSPEH/UpEFEWCsZaVuDEVI3iEUFcTi45GfY136MlSWhuO1PSZshgr9I9x7GeuDISwe5gJZxHTVgQzcNiJRdBs4U3QGe06X45Zdm/GZdgwsEQ1fQXszwaEReN4ySfxO/Vzpu0GN12lidA7p7MjcIRJ5qSsD5fAoPJJ3HGf0m0WDck9lqH4/8sqL8WnF+dh/psCu1k7sTc0HqU6ts3TDOX36In+DERGqPPx9ugQfTH0ID53zkMu+6PVhNnmV4AhHXlGjFxqGMIRb8hBBRmVU9dZtDfXQvW25RPz+b+2Hok9vfZvR03Z7bO2d9qcXYO0hOfwTgBIkWY9AFXMLWpr4xB5Qr5YQr0rH6oP+IvxW9XtGVY8khZJWlGb/27jYafhydxTa69JhsKUUtAQdjaX4VPcsXqm8AhuPJmL1gSws3HAcxeUW0RC+T2ARDNoJjdq3wRyKwIISqGEVec7UJuutMW+J7iRLdjgK98oqtHik74uY+fetIseOtEZf2jILAZG9QV8xo8lh0BnM4QjMK4Za0mL/sQAES1RfakBnNx66rKjz0Onwp8JYpeKylLKP8HFKdWOmKegWoUO4ZifCKvJFHt19csTTjhJBURYc1Ic1yjzaZRuDOQx+uRViPqO8Q/LQ7bL2F967f9dwTajCOqEo/4T4feepPOGRHx0/WoT3J3eebO/m8+tW6g2tEtdXXHj17xe9xz2mP1FekIlsBIp2bwnxCW7fU633B8yxMBbK7+tJ+KSHzpshte9KnUmsup2LIpbtOiOMOYK6J1z1zt8i94JuFIHklibbKqhhOS8KkX6VeEv/BnqrD9m7RVB/PH+V/DuJVtaE3hwhqlyJspIQ3N33bsRqzxMtw0hpO9mv8eruwf0mi4ILYuV+R8sxKtBIzyO5LC1mV06Hpn3z3KQ8FQq1UfI5Jb5TYi7dLEn1nVqqKVydfLVo60RdQyi3auFFC4UivFTUDckVZRRnhyejMkcJOR2iah4deeiod+rNqtmIj22HNQFjcUyKwum8Urd9HQ+kFyJUJRt0kY2UK3Empl17RGsK0M/vNMzte53VvgL9dIgACR5niDZNpIhfXw8dheB2abtjmzVRtCxScsBu6mqFmlIdOl/QuIO6dxtw385GvVQdkYwdZjn3iIyif464thAjYk2x+GTCJ7i5x82iMndQ9CB01F4kniPJElWI+4m0OYgMNmGoZpfIwXzrz0OYv+aIOG5iiHo3hplkRYHGEGDL1aLiMFp4Ky20qOq6agecjFy9MPY6BModfCIMcYjMkr0/pkDHQt0U7Ngn6Q1Sjh0ZKcG20KXLZwuinEXnVBQVOkc3viCoNsLjutjHb8vxXNGRxh3UiUGPCjyvex/9NY6etIQpSP5sgbbzRV5naqk3MCCjxlSiooH34rqKx8XvO0/lo2dET9GJiLyhzq0Zj2UXYYR6m/AiJgS7z+nuqz+OMZrNiFTlIFcTCo22Fn9XSAcgx/Paf7FB54kYzKKUntzIJeUWscqd88sel032nSnAlLfXYs2hTLFCe7TiVujjHGXrDSGIcrLIcEOZSB51yFvIBp0psOabQEBIpGhqTEtApbExJbW2U2XhVu0SxAfU3ZKlJmJ7j8XKwIni941Hc0SrL4XjWcXCE0F9DDsEea6nqTmg8MBbutcxRr1FVBcTJB/x7gXvilAjhXSovZFzXmH38O54evAcFB6/HkZVKdQGR4m/J+IfEosoVXY1g47a3B3Plse+WyTJrqiwPmWGEEwlKERbld2psqgwERfnGv5rDO3aJaC7uQghSYOgaj/4rD10T+g+w3917+CYNRq7kIhAv/oHTpYHX4U3LI5uJUa9Btdfdgnw4H4gtGEahS6TVXAjz5MpAsfOfw8HJdlwpgINd+g0OtzX/z4RDnt26LM4lk2LV0mIXvtFNPK4G0FUeLjob2p20h8jgfTrz22PUbEWRMY2/lgCQyOQJxmFB5U8Ts5UNej2pRUiKSRJtMT7cdKPuDbudQRZ1S73Z0LqPB5dShcgj1St6f6rKoVFS9I01Q2eeF0+FuufFEbRRPVaLDE8jmhzM6VZBMuRGBq/cou1xpaNNEdQficREuEa/g0KlSusA1VFOJheKOagp3UfYZrWoflYlZ5xDm/bDlthhDtovqDc14HqfWgf7t5BoSPnBAoQjRyU+deh3XjBf4BhrpEAT4ANOg/kQNgovFcpr1jJA/Px2mOiDx1BPQUVGRDyzn2+/riQK6FedBGNvPkEBwWKmxrJWWQWygYdeTr2SAmYWXEzTIE155Dok0bg8cqbRaNwxZ1ORRp0URIREY3PWVKVF+HOyJ0iX4puEtQWSIEmdRKmpD6GHfTVE+d9GUrgpTGniWLrCcdnJwmbl4a/hBeGv1CtZyeRWUBjqxaGutbPsw26kMh2CFXJq3nnwghayBAztF/gjcIHxe99wiVxsyZ2nc6rdiOnBPeV1l6YFLgQ5vDGpwDYMUdDpdYCV34MpNTcL7k+mKgoQtJBr6oQhtn8gFsaVOATb7IiCg4v2F0jExGlLSZXP1qLYfE6JKhlj/ofNRh0CgNjBgqP+5HMIhHC9leVIzg2qYWOFEK7rEjlLzxeIUYd7h3TGWtnjMZ/JvWEX8kZILDxOWf+XUahd9n7IHlgZ4OOFqa0yKiaFqAYutRnO7vQir+t3XBO6VwEOKWtxEcEiapohWcrrsMyk5x+UZWwQMoHOyTSEchzFqfOgsrNfaFJ8A9GuS5QvA/hTr6EOJReZI/oREW7pj+YEocgsfQTHJLaiWpZglqXqf1qjhDFlhzAdr9bRTXxzlNyYYQ7aL4IJg06VQACalBdCI+MEbI5uQiAKra68oMLcf2ByBR4GmzQeSBFMYPxrXW4fYJ64/cDtjwE4LnLeuDr24eIpsEKFHq7UvMH4gMbN5xhJj+UQC8meiXkml1cLnSxvrCMQbC55vBcu869hTFJ0iWKh47c5STgSERFnEUngrICXHvsSfRWy675lfscYddj2Y73iG6Gyi1PhsIPldoAoalEicCK7IH8nEYYdu6gXr3kBaFE47JAz6rOqoo6pjcWGK4VFdT70grsIZy9qfJkEK7KgxQgr+iHpX6I+bqXxO87q3jovttyyvabCuP6dWkaQ2fgrcAtvwEl7r0QDUGnUcOq1gvZEj+UIdrUsLTmKeU/4CeDHHKihd4tyaXAS4nAiX/QWgT9+jDeMn0gfj+aVYzDtvtCbdCClbp4XKZ6DQHJ7mWSmgM6/0ZzKK7tHYy1M8bg/vO7IIzy0aiPZmEaENj4nEtnHb8TOY70GVqEVdWPVBYqCukFZaiAVkhYRQY79pPgX4qF+meE143YLHVBQbB7wyI8TA5hUliWUg5K9c1bOVwy9nn8YZWjRO4KIwj6LoTYDLrwKvms8eFmMY8Qm4/JizhKPdIaa86pVPnLhRQUJs0qKhc5xdWOq9wizif1/S7R1GwcdmrfXtxT20+Zg9jr5sIbYYPOA0nUZQs9Kpp8n/15j6jyJK7oHyeSw6kSdtFtg4W3juirOogXde8hzL9xqy+qdtxuTRStw5SiiOzCcrHquUy9CmEBNef0hGnLMNVvlfASUB6Pw0NXLKQdAsyNky0RmKIgqXWIU8urtb8OODR/aMVLFyhhMDWuCs2rMZhE2JHY4uSlqw0SFSbD5raKB5Bnqxb2VCpDO2N74m2iArHSKtk9GtSFQDHo9EGyIR8cGS8q3ap66Gi1/u0WOQdqqmYZpma82HQHuOq/wPym6WQgaQ3CE7lYPwt3ls9v0GsDg0JEegbx+IVdYTjyG0DV5tFnl9t3VoQloQNS7Q//cFqIuYMmXCoAE5N5RBegFo9Mc6C54Gl0OO8KV4kXKjAa9xyQ4F6LrD5E6Cvwu+Ehke/q7KHbbAtHkubaUv2j6K06KDyUzvptZICMVW/C/3RvuBTJmIxGnKvegziVfC+8XfMDBqgPuH//IDMqJI0Iy4ap8mHxb141gKBB16E8TM77I684pUdUVWmghTh5pLdpe0Ftcs35DjFI+NbwNEapt6DEtoAjz6mfqRZD1CRHgJTrf4ebsCt55whSbajQ12IctusPjJmNCV1bRjKnOWCDzgNJLt+FN/RzRZWpUulKuTEPXeDo6xoSoMfntwzCqOQIYTxVqrRQNTLRPcSox7WVT+BrywhHUURxOUapt2KW7hOEBNQsDKyqKMbTmIdu6mPCW0Ql/xQmphLxn/0mym7FxqJWQxUYi35B8oqObnoUQisorRC5fkEoRJEqAGiuMIIHYwlPQY4tj4aSkOtDZmGZ8HhRwUqE0cN1+yQrxvntQpxKDtntsOXR7UmTvwtUtWoOk1f4msAYsbImDxd9R5SJhPJ4lFSFUUFpMOa6JmE3mtwTwD/vAmV1e57qg1XrBzXkHrsav4YlrfftHC/ClC9N7oYJPWOAAyuAjiMAXdP3P603YUkIKM+AP0rrFXalhHWCFrH/qlyAFqfn5QBN5s5o9cCg284qrKbWB6CDKlUYGxS1UMKBilD6FM1fQvqGhHHJY6csiJXFV2fVKZyn3ilCwXb0JlHhraS03KP9Dl2l6h0niCCjXrT4I4OfrhcSFW5WUrfjX8GrxK+0CFM+p8Lx7CLxOcmr+H7iG9UMd5VGjx6qQ6JCmtCiUkSJ/EJqicBoDSjXByMS8j1QERh25liWfK4WVY7Ejo61NBigsT73TuD59sCOr+GNsEHngQTaK5kcbvo7RyaK3pvOBBi0+HD6QMwYGQUNVbU10niiEF6okZJlJXsOnZAtUZWJggsy+GrEX/aOUTIpsfpgJui+tc7aHcvi78NZE9oJPfwd7vuVBzLsKy4NJBTom/km5aFoL38fr1ReKX7fcjyn3gYd3Sw3+N2JpLLG6Yy1HCqM3XY/zlfLLaS2ncwVEyJJlhCR6nz4Bdtu9LYm9qTMT9890mokvtmshFuBZHMpYAvRnjUGm9FlbBrP8KKgmzC2/L+iWEXn3zCDzmBrA3dFzxCI7uIn/gY6n49WJVTuEDEwUJ5k1x/JElqZNXE0U76eSc+yR9kWtDhHVgH7l7v+7cwueVJ36pXbYNRqlKgDREiwtMIqUh6sZNDYDJ14vZI+oHifHWHXDOoSoSpFmcrfNadSpRJhQyosUMEKf5TDUMN3hl73kuEu/GrtJ2RZcF4T3I9r4/g6TDz9ujgud3l0B9OL7CHgLmFuUguoF7PGZFdtIBWDMeUvQ939strf1xwtrv2aDDolDWO91BWW2voDV5QCm0iYXAKM3qltygadBxIcIk8USuUVtXy5ZVgnoLwI+P3Zap4Bk1QkcgnOho8wC3O074tWUjRxCtkSlKNcZRB5JjWi1aNcE2DPi1i5X15dUZJ6T2MTtEVpfy6CIxxSE3/tz7CHL6g359eDvHMldbaQcU/fCyXMUFlFfNcdtOoPsFUum50q5zwSmsQCYxGrdkiXkOdXkeZ4KP4LYMBN8rbmaFhUOoTYqudIvoBy7n7aftru3Y7UFAIBTXST9gsCBv4fcHnDwqM1oVS1kofOUJvmY23GZXkhcGY3QMUarW3QhSUCGgPGxMnfyQqLhNVO6RJVOWrzoAglf/9WCHdtXgCsed3VmPvkMmDli+Q2Oqtdl2kD7akhVMR2MMPRYqyLv8Pb7JxHR9cyLajJs1amqd5Pt0Iv75MiOGqVBL9aZKWy248XFcdZIT0RntzM/cyDE6CxVti9ZetsuogKykLrYe2XmLpjmttdlDudL4UYN6LCzuiunI+PtXKlN3WMcIYM6G83y2kXkzSrMSygFhmaylJg6aPy7+bmEWBubtig80CCgmWDTsmNeXRCiqycvvdn4K8XgbVvVHlBPNCh8bkehFqjE1WutJIkHSa6oZCMSYW67tBNpV+IvXJJ0Yt7WLsIl578L86akTMQcsX/EBagt6/6nBsi19VA3Gf55VF8jCfErzRe1Hy+Ph46JQwWFNhy4q2NRWWORpK//Lmo4IakaxQSYqNEHqEgIgVbp+7HdinRnkf32550e+7p+B7R0JRkAcbwJjowFXDhS0CEIwXibBhVsVLIS5A4uL+tt2i96TgSeCIdCIoDEgYDjx6zS0i0GuS5fDwN8QMvtf+ptrDr0Uz5eqYiJ525FTwjZBTb+nzi2Drgwwlybta0n866iMZKxpeteIsWos5yHu1sucFhKtkI2W/z0FGUhDzNVKRm0Va/v+3r/gB+spxrX5w5Cw9X5fmuRzFvUBYWd/sLKjJUmxPb925wWKF9oZlfKns4f9x2GnP/kFMeaK6oKfxr9Quye+i6q45iq+FWhBfWniqhiuqO6FhZv4+KIqgHrsLfR7JETjfxlGEhIlJd20dWW6gpmL2z0I4NOg9EExCGQ9rOoqhgUMdQTOxlWy0c/FX+d91coNSpmo9yPSY6rTAbgVVntOvOUTspmgyPS5E4ZKi7tUll+6E4I8kra6XKSLQiMzaNF0hdWYzhSfK+KHSz2Fa5+IruLQw+8AraJCoNwtSORGtn+ZKaoImC2ksRQcEe7qEjzNFI0MmTHU1w39hW2l1UJ3DTkYfkXDZCpULX2EAoUlykRaesyokp/eJEsjO6u5d3aG3CNaSneAQDSudB1d1hBNULjVbkEYkTVFnWurlzzqjVGJwYBj+dPMX8sS+9RkkJynskyMvvH9QKKRSGQNmgO7oG+GQSENVTNuZsofyz4eCAWfjQMt5u0DkvSoIr0iEljsY36nEuHjrFIFliHYQNUVdV26euxyVCWNsCFb61DIV/ZM0V60G7PsaEou8RtvFVIHMfWsKgGxYu35eokHfDkWx8sPoI7vlii/DUEl3M5TAGuz+3p3rdiy8tI8XvlCcYrCqSOzPUxoEVuM/yof3hrlOOufHrjcp9QIKZ0oJqi2Q5h7adjTsvgg06TyQoDhEPrcM9112OD6ad48ihUKmBc+8Cpv3smlBKxp2l5hyV+qDSG+3eGxJ1JN6zXIxvou+v87X+l8/DZ5Jrw3Mq1NA3RfVpxj7guVhcGp7qovxPJKpOI1Dj2u+2zeAfjACr48ZVnzw6CrlS/gph8G/ZSsLGIEV0g84c7hJuJygPMCp9NaBxJIsbv7kBMwKX2zW9/rRtS71Xz6VG4T0mA7F94Yn06xQDrcqKc7p1Rkr7BoZ6CjOADy8CNn0IvNARSGtch4cm58/n4bfoagxJDLdXbboTfSaUwpXvNeNg6C63QWtR6F5K91Bq40aSNNd/02QTuqnzUKGrRlBhxGbbdarXqCFdPh+qi16GJlpuH0eeJFqwptsWxSutvXGm/cXV9plYsgMXqv9GDgLxQMWdtXcr0ZuAbFtHg6byUNcEecy7TUJCnPx5iVnf78IzPznyda8+Jx4pgRU15qgZu48XRRPOEao6xyLrIAZkfCfnvjnl0ZF3cMlOed6I9LOKcHCdIf3gBKDL+LMr5mtF2KDzUAINWozvFgmTwSl59LK3gfHPAbF9AKtFToImFlwMLDk71WqqkKWQq7PBRD31wo11f0V0KgkpVWw3KjcPsLVyOSsonEyTnrm6wRKiKoLB5J3Jq2eNOQa60iz4qy31li6hkOsy6zmYaPrCO1agwx9C4aWOPDVFuitSbUsed54UCtPRy0/uj0meAEXna1LfdnK4df27QHH1NlSeQFyE7DWYp3oOqqxDDXsxVXgfWw1sWiBPQuHyZNjq0OLz1CZRha/g3L7PWbKEciOJzRGXAp2rNAFtCcKSgHb95MmeOgA0oZezU+5a3KhZZveiK95I6nCg7zwKyE/FzZol9u0PnCkQxi8xTL0dyVL1cGPwkZ9wr26xWIB3VKUhxKCq3aDLOSr/HtDMBh1x5QIkDplst4dI+UCBRJvnTO4JVUl2jQZd+6IdmKz+yz6HuOSJ1oQpChprmT1Uq0iX/Lw9VaQQEVd0t+3Dr47IRPtzgZje8FbYoPNUqHR6w3uOx7nH5ZCKwhfXAD/eK/9eklu7K7ke7Ot2D+4u/5f9pkJ8pHsRN6Q9X/eLlz+Bd8tsyaQ2SJFcW0VnqFHojcJ4CSw+iW4xrl6lYHXxWReDeC02wdPzoirsXtW8kpor8kjKg3LtrKQNSL0hvWQF2jkiAH4612PtYiqRq6udPHQUno3VVvcATenXDji5AfjlYaDCtf2Sx0AhU+LQ7w0/RpqwidStQKeRstyGJxDaCSjOwtA4ba1eZKUgguR0LtZvBgocTetbjG6Xyl65ZsB4ag1u0imeY0ee67iIHLnA7dBvGJv5if3vVOmqhFypG0qPtB+q7VPlH4JwTTH6qw/gD8MD0BTJXVJq9JpJNn27pqryro3yIoSUpyIl2nGvplSIZy/rIUSbRbTpzr+B82xzVxX8j6zAg/pv7aLClSq94/qoCVu+Wwe9fP0rnuCvNtpSMgBM7Bkty/nUJRQ9+V1g1GPwVtig81ToS6wk6hJf3wwsvsPxuNslwK7vxCpYeOrqWnnUgTGsvWhR43zj8VeVCS2lul8ciiDJVen8wch3gaF1h2vrPTlkH8YIp9U+lcZTde/ZGrJeS/wg4M71iEuQCwEI5xZZ7rxzxKXq1Xgkfw68gpMboJ0TjQsiXb9bnfyLq+c3mSIRanX9/L3jgpAUaZbDkOSRDHSEgjyKdgOAATfLvze0x66zAdfa1a1VvV40yarSEGzTUdt8XJaecacRRjIcN518Aji1ET6FXzCCnfrEKgz2OyoXuJlj4FeRJzTXlDw6xUNH6RF6dxWs/iEIURfi2j62sEht92gSmI7uCfSYYpeYalZWvgAsmIiJvWPsfXHnXd8f1w1KcGxD3/GavG5+QQiyna+fLIOxbJAjN64ug25AWLndK0itw+j7RqREm5GcnAJM/QGI7gFfhg06T0VUXtkmMgoV0Y2OVhgKva8BIroCK2bLht9ZhtASc9fg31r54lFaeFFZvNZQjypSYxj8K/Ps+kNEXEgTVp9Sk/GcIxje2XWFuTD5NaDzBWiT0E0xMgW9Exxe0K3H6zboOqlTkVi2F14BhWUs5egf6ponmd3hIlH97IIpGsZy15DeZCqGIM7skBPdPdUrGdTOYYwpHrfGkORJBp280FBlH0ZfW0cbEgNX8uUUjtg06IRkCdEasiXNiV8QzBJ9NldDthP1nyaPmS2lJMSm40n5n0oOHRUw+Qe4ua/7BUNdWYrxnfR1G3T9pwK3r5YldqiAprmhHLS8k/i/89rjo+nn4LcHR2Bcd6eK0YI04ONJNed6+gfDSDJcsCIbgdC3P6fu9ySJkfPuQ5it0pV4+kdHRe8VA+KhouhWebFcPOTDsEHnDQbd4T+Ecj6SxrrmzoydDRxdJT93lp6q0NLjQrmcKKu02leIOv96TDDGMKhhtecwUCeCfx+9XpYAaAoor2X6L+ifEIIAW3seid6RJBtIrqGtsnQmzit3nOPa8ugyCsrtWmcWnZdIvdhW3j3MrkZAcMowoKrYaI8pUE9+F/GhckWcVq3CxN628ApNHp68MqdJjjwbRGO6vUz5ALjxe9kw9KT719QfxT2rX3uHkaYUBVSVLCEPnaAlvEgtiX+ICCcrCgJEhzAjAkqpT2w7exg00Sjnmu1LKxSiwgS19jOa3Rh0IR1k470kGxLlKtbQu1lQWQ6c3gLky5qMzU5IggjxaovSMDI5svrCnr7rNJ9RgYI7/IJE1xTKn5uiXY1z0r6o+z11/sD5TyMm0VEcQlqUyn1gUp9YYOfXwHMxcks3H4YNOk9FlNLbDLqDvwGR3arfsKkap9skYOIbQOez6ytpNJmFkLDzSpJaCunradARihZdkKoQIWUn5cTopoBW7dTvUktSCLJHKgZZGHbyXZEM32Y5+BsisjfZWwNRjlJN0hCkUq8Y6ZKucS3iWhwyPP2C0MHgmhvXJ2e5aDPkQngSkDgKD56fLIy6R8enIJS0C+l8kD6bs3fb08g7JU+6Pa+UP3NjWldR/pyn0XG4SMfol1CzQXfEWVTYFz10YUk42O5SaJyiF+J80JjTYpTu6b2uQkx4iN2TLutsSjiiiofGnZFO3+frvxaGnETfl9o8z9u/BN4dCSyaihaBPHREzjH3zxfbukfU1IkhOEF0c5h1YRfMSDqB4OM2qa66OL0F/Q3VRYPHdI1EmMkAlOTI3m/nvFsfhA06T2XSPGCcLdepKNN9fgxdyFcukN3qVDxwFvj5m4XquAGOldP4sudROeDWul8cdw7w8GEUGuXwgeKpa7JKStIb++hi4WmZOiRBJNkODMxGws435e4ZbZXAGKgKTqOPLaSVU1xhb4tWlcwCRxhHbfASg44wxyDUkmnvpkD5WMErnwAOrnDdrihLSGVM6mjBqkdG49bhnRzXyKVzgZQL4bEoeXCDbvfcsHBj2LdUJP73jg+2awRuOpbrNocuwM8AhHX2vZzY2D44Ofy/KIDj/jwgIVQ2wJMnyF7oye8iKM6h9ym3SVPh4ZDX3Gsn0iKFCuEG3IT0af/U/v5KTiZ5sVoCCiFTd43CGopblErzmgy6dv2guX4RLh/eFxG6ctmxUR+WP4n43e/YdQ8VrhwQ71Q46GOLBTewQeepBMcDJlvO2HWLgDFPNevbkWwJoeiUEVkIQkhoeP0KOALC0DFCvvjsrVuayqCj8A2FljP2YljnCGx58gL8d6JNDd/XJoCGQCGbfDLoHDeqLTXk0Sk5dCTamdfnNngN13wB1ajHRIVcuEmP+0Z1gIpW2wGR1dv2/DkHSK+SH0jV4YoAsaeitclk7PsZPkXGHmD92zDpNegSJSfB70vLt/d1LS6vtAuRn4oYBtyzse6KRm/DUin0Mk3KIpfS2shDd96/gL7Xy3/IOYZewU4KBgIJkaYaKpYpKvFCglwVTak3taFvYYOOJF8eT5O9xjV56Oj7XpMn2mqVjb6KEjk3vC7JEgVzNNSFZ1yUECLMBozoYptDS3PPunDQG2CDzlPZvkisOsSXm1ZkZ9mCpk4iUvCF39WogOwJ0aES7+pfgSmtjhUgQcf3xTW4PVbWTOoWYm1ag46MNlrR2QQyg4w66MopDKcCDF6gp9ZcUAl+fir6tg+us2MEiQoTa609YOjuwd4qdxXO/iGYfl5HbHzifEzrbbvBu6lyFRRWkXBY/RrwWQ2Ti6egsU3cq16GTxGaKE/KRZn2sCvJA263fUeP2goiiA6+2sKvJBvxn4/AYM0e8dDsp0XnUC1wcpOcpE98fAkGZyxyeRnpy80/McF9HrJtEav6dRaCfq9SHFSTQdeShnJt0jnUonLCizV7osnwerEjcGC5nHLkLKBfG9SqrSAVPdo55oPJ/dpBq/Qhp0VgG1j8s0HnqaTvBvb8ACy8Dlh8Z/O/X3gSfom8GYW20ACJVl6g3ghVUXUx0GrQxXl0DUaHZmHVI6Nw3623ANd+1bSrQpt0iR1yodPF3tyGridDRTLn3iFCWnV1jFA8dBeoNyA8dwe8hn2/AD/I+oiConT3mlqUG0NGf9VQzxkqiOgJj6a+XghvwyZdQkr+7gojFA06Ymru/4APvWihUV9sXqFzY2RP2uS+7aDOPgS8P1r+bhIBEQhTueaJUkEEFVO4vYeScaYzQsg6sTsAAC6iSURBVJV1EJr847W/v1Jko20hDx2x8iXgq2nun6PiJEoRqgnFCUD394bkhlKla8EZXNwrVkxHZoMW1ztLpVzyJnDlx/B1WqCOmWn0TZ4SZylcVJP7uikpL8IQaRu2IhD5MNkKJBpQdUcNuWk1KlbaRiC4DgHHhhLS0dWgI0X83teiTUOq5u3PBd0CEyMCRDL17tR8lFZY4KfTVOvjSjyi/xq63aVAwkB4BXknga2fAxe/JhvvVGhDyfZ0A6+KKdpVmJbCN9SQPNnDDQW6dnpdJX9WX4LkhojsQ+jX3mFUK/pgzgZdmLoQUPvgdETeKp0R0/sFY/gVw5EYYQIOykLDdl3EgAjoS7PQLtjf3lkhQKmKrcnYp3ywimK5KKI2qJhuxonGVU83Fkp/OFFDZIf6kVNVbsdh7p+nEDLlzZG2akN0TKm6Nrg9BsYZhVPBX6eRiyEUKMf8LPPMvYE27N7wcOhLTaXdpPLtLFfSXBSk4fYTD6Gb+rhdVFhQXy8beUeUCibyqmx4v2mP75xbgBGPOB5TkvuEenSx8GUoZLP3Z2HEKHl01PbKXc9MJeRqonFtyZv72UKGG10H1C6IIG8byWEEujHoKIE8xqmvZe5RoLzQsyVLlJQFqgpsqqpwT4HuHcMfEYuvjuEB1aqxFckSQgiT+2rSOklxlOWjc5QZaqoOIcOdCgdssjyiJVdhOrpEORQFAlS2llk1Xau2c1WnQUfacyKSUUeuXVNCxhXJpDh3NlJY87rcd7guryaFXo+tlYud6kPXicBdf4scPpJKcTHmygqBD8YBJzbA1/GxO4gPoazMqAycLpDmxnZj8LetDO0eugYZdLZJl4wM8qo0Je0HAUljHI+pPyFpGrVlKHF44bXAyX9c8uiqhl2p7VdJhdz+x6gq9T6DjlB0tOjmXOEqNGyHDP5+Nzoe56fKEx+JCnu6QXfibyCtihSLLzD6cSB+oGj51NcWdqVqbOpp6pxD52/Jlz2Vvghp65HXSiH/lJz/qhhZlD5QWYou0Q5vHOlF1io0Pe0nSLH9IGk90OsUTAVrknuPM80RNVW4Oodd6dr9cIJc+NEQJDeyTeSlp+vLU9riNSNs0HkqSoPglvDOEXpXgy5TCsKy2DuBIFs1aX08aP1suRGiFVkTFytQUuuaNxwVi9TH9hfX/rFtDpoAKXyRf7qKQZfrNn+O8JO8zaCzeTEU450qWecNcb8t5d2c2e143OE84JEjgDkKHg2FktU6YPST8DmoyviALDHTz+k7SmFXJeRK1csaumf4qofujjVC+NaOpdyRX0jQuN+5Dsm2SmDiN2s/nL5uZa0eOmn0kyjufjU8DkWLLreKFp2lQl6Y1WXQTV8iLwSI+hZFkHzVnHhgp5uevLRQouuLOiv5OD6YtOAjRHYFZue6d1s3BzaxWaMt1JqBYOzpNB3jFOmUuuji1IKLbs5Nvdq2WoAVT9pyJeLlyZsU09sylP1Locf8U2IyMOo1KC63VBNvVcKt1E7nVEAPdBQraC+BqldHPe7Ix6IinaoVrgrkFf7t38DjqfK5oRw6bymaoUR3muh9jT0/Ar89Azx22kVgeM3BTHvP0g5hAcCVi71rodEQqlZ0UucbN88r0i5EMfwQ2qGWdnX/vAdV+m5UDPDARS0JJl/9BRBtc0oobPpIXpinXFT768mIUxZw9S0YogiTtdK9/h0ZdJEp7KFjWhm6mEnXpyXQaFEW2BHlkmzjRyMLvQtXya1j6kPmQWDrF04euiYuEadVHeUV2qRL2oquUH216Kg8v3ecfD5S80px2pZc7eyho3ZpS/q/JwuaegtUvUqh1PDODg2uqhWuCuSJqyxxdFiZe47s1fUGKNePDB9fg6RLaEwKUsX3UxEYXrrTkS7RITxA7v2qeGN9jT+fB76pRaD99FZg7iB01mfbz89lfhvh91stHtvcY1BtnA9tRg09UVv7mqUc54Cw6qLDwx+qu+r877eBb25umEFHc6Upyn0aDnWVqWpc+ihesnxlWoKM6Wvxg/U88fs56n0YtfWB+nsNjq0BFt8he9IoTJxQQ1issdAFSx45pdK1jegK1UlsX3v/SyFYasPZS5dhq3AlD120UfK+BtVUMXf877o9dHRDJ2iVTouKrIOOv3k6o54Apv4AX5YuCTBokRIth9CUnE4iObAS+O4OIF3WavM5KG9MkSghr/HLXYEdX7saQBl7YSjNxLmdZCPo/MCTwP5fat6nbTGrP+2hif4U+tz8ievfkscDo5+o+7V5J2SvGi3cGpK6Y452b9Cd/29g4C1oC7BBx9gJC3BUBvmpGlEUQYmwFAod8yTQ68qmP7OkRZdzRL4p0g976IBxzwIXvljdoHNqsaSEXDuozmDKLwNk49ubWPUKsPpVh0FXtUuEs2wJQTd1SoQmPL3CVWHEw6J4wOegFAmq6CTtNdHHtPoirHNAMbDtc9kI90Vo4Un3ReX7W3Da9b6qeJyLMvDmtf3w5rV9MSYxoPYQtM3QkVqqA0RDOfArsHmBI49y0Y31L2Kjz0bn5OGDtgKLemKKqi4sTnQaIS982wBs0DF2/D+5EDMNX8m/owxWSrivb7m7kjNXnClPpkrYqykhzx/1jaW8qMdOAv1rEK9sa1CysSS5FEZscvLQKSFXe1s3b8tVEnmCtirXf20BBt/lfjvy3FHXBepOkLZT/p30CpnWg7xPpDlmk2RxFhhWaO9vyxP21aIIIcNhM1bzT7pq0BHCw64SBl1ogF6I4xosJYC+lnCjYsh5qredDHmS4iEor5U87DVV7NZ0vhr62c5/Wu6B7gxJn1C3GE89T00MG3SMg4piROtkKQEjVbvWpXHkjFK5RFWoVIW4f1nTn9l+NwBjZjke+1Ij87NpgP5MhAjrBBv1QmCY2HUqTwgME5k2D51dCsFWAOM1CBX4NIcxqjQcrwrl2zyRLiddn9kBRCTLBgXTutz4vX3x5c6gizHY8j1tqQM+6aGrKJIXXiQWr+STOWvF0f3TuSsP5VTWtvCyVWxKtRl9rQl51qiry9HVwI6v5MKmmq5bd+eLUn3e6NOw9wzpIBdkVC3KoWKMNjJXsEHHONCbEKKtEL8WqkxAtJNIa12Qizx+kHwjIpojv81SCWTsB46vB94aIhditHVEuEaSta2cJsxKq4Qdp2SvQIbioVPEor3NQ0e5MTTZZewDPp4EZMnhu2rQTVu5cY+b0yZa/XgNtmr9hDCj8EIphJsM8K+wCWH7ak5sh2HA1TZdTrpOqTl9VRWAye8A3S9zPO42Se4eUhMhCZA6j0NlsIdW+ivSJRRqjewO9L2+/q+NGwAExjnC1PXl9Ba5+IT0ORXSdriKjfs4bNAxDvRGdA3XICnShHZj74R6WgOStOkGdfNyR4irOfLbKNmdKhd3fQek75JXtm0dEiglbCFJ5zy6TcdyXEKuEYZK7zToKHeyXT85lH/4j9pX29/fDfwyQ/YG0OuY1mfdW8BLcpUyCQw769F1DDfKEk0jHvVdbyrJLJHXmD5fj8vlTidVv8OUTkKVvgq9rpB/aiIgHNI1C1EZ4aE5olSVnnCe3D3ogmca1qmCrlvq+lJTNXtNlOQCOxY5vPkUZqXiioY4JrwcNugYBzojIgxW/PrACNwx0unmUl+owlVJSm1qYWEl9Ear29Obbe/hoyv6hkB5Y5R0bvPQuRZGyC2WMgvkApddpiHA/btk+RdvosNQ4Nbf5e8XUduNnvLn9v0MfHal3E2EaX1ovMry7Hm1SscIuwYdGeujHoPPQlWuVNhD6Sik6+mu+GX/cjk0qEChypo80d5yXyKB4IcOuHb4qQ/0PVk31xHtqS9mW1GUokVHwsaUi8cGHdMmofy08bb+qN/fBXx8acNeP3cQsOxxWZW7OQw6KoYI6QicIoNO5X2GSXNAK1+6kdk8dNT8O9BPa5cuKSq32CUigsxmOcfEW8R2naGqZqoO1PrXnlxNla6iO8Ey/n54qBd5WOdw+1O94oNlnTDSYvNVKoqB356WUwZWviTndblrWr/+Hcfj724Httl0Pb2ZmiSGaoPa+1EaSUFqA98rSv5X8dBRIc45twKxDczF82I4ZsU4UMRb7RdVAxNJKexKulN3rG2+s0ru+Iw9snfOGw2T5uCmpfaiFGr+TR6QlfszkFlYLrx0CudjHfD9p8Clb8KroNDJ87YkcvJw1BZyVdp8mWN9tzeotxFkq+gkL3JEMnrFBeP1q/sgLa8UVw6IA76dJXtSblwMn0SJJJAY+sb5cj4ZNZOv6sVsSFGEL6PkUg6+u4GvC5FbISoGHRVmXPRftCXYoGMcUGUqrZRHPionljakypUgo4JyJpqzoohaQJEO2aVzm+89vI0qWk39bAYdsWyXQ5cpyXoUOPwnvA76PpFxRonWdUnVKKt0qnBlPAMyrgmlwhPApX2cZDtIJLyu/p7eDBlmai1QlCl7nRQD15mAcPneSWkF5HWnBXV9ZT58DZJkIcOsoS0KVSpg5AyH5hyFramamCRU2gjs4mAcnNrkyOOgMEFDRStp0t2/FPhkcvOdVeqD+PAB196xbR1SZf/hX/aHznl0K3Y7ehsGUgVzQ410T4HyJylc3PPy2rdLHC3/W1d7Iabl0OqBe7fVLDYuur74qAadYmhQCkrGXjmUSBWcVSEPnWSVzwW1W7RWtF2DjrCUyfe1hjLsAaD9IPn3b/8P2PQh2hJs0DEOaLInvSRCeOgaaNApOlLUuqU5b47UNme3D7ZJaiyUm7TzW/vD3vFB9p6QSgN0IlBd5r1hHDLoKKeIFh11bTflg+bpVMI0HtII0zo60bggPHQ+Hh4nGRISuiaqaqURlKrS+xrZqKPet9QDl7x2bZWB/wcMcSxS6036HuDIKqAoSw7xt6GCCIJDrowDmuzLZWFhXP6BXNzQ0NURVWaRoGRzQc3ZqXEzhWi6XdJ87+NNkBFTXgCU5gN+gTD76dAlyoy9aa7dOgJUpd5t0BH7fgHa9a/d4K/Li8e0PJTwTwsPUvOvChkuzp0TfJGLXwEyD8jSJe4MusgU4LK3HY//Zavkb6tc+FLjXrfhfeDYOmDcf+THbcygYw8d4+qhI1c/KZoL1e0G3mQpbKKEF5oLxQtI+SaMjDIZKu2xqoRdFUqTLwMGTPfOszb8Ifnfmvq4Mp4NLfRq6h7zf3967/eyIdIlFEKl3svuOiZQ4Y9YDPN97awwR8vSWSQorG97WpRs0DEOSOBz0O1yYu4vjwIHVjTs7JD8wN6f5LBBc6GICZ9zS/O9h7dBvU4JmxZdTS2W/HpcDPSYAq+EksqVKlfGO6VLnL6fbY6fHwTmDZbFsWvircHAzq+Bk5uAFzrIXXGYhmGKdhSXkP5dG1NCaFuflqkdEvic8AKg8wO2fm5L4m0ASsuVnrUonDcFs3OBi15u3vfwtirC8//tshp156ELO7NGblrvjZz4x9H+jfE+KMxIos+UFuBM+l5gTnubtqQPQ1ELyhX89Sn3z1NkQ5EuIRFm2ramnEOmbnHhXle2ydZ/bNAxDsqL5NUh5dGJKtdGyJY49/FrLtpIo+UGVRGed68s6WKDemaGOfXMDDHqoF3+GLD1M3glCUOA/tOB5AmtfSRME6UFCEqyZQPG1ys6FW212nIFKZeQDDq6DxMGc8scm68tHMK7OLpFtDHYoGMckEfu/dFypZC1shEGnS2/7dDvfFZbGqrsOvKX/SH1zHRusRRhNsiGurfKllDe0cTX3OcfMZ4PTbLjX6hezUqeKMKXZUsIJa+4trxk8tAVkofO1vLKWwuYWjtt6NK3gPdGe2804ixgg45xoLPdQIozbY/9G6eIruQ7MS3H328Ba107QDiHXcNNhratPs+0LgFhwLm3V28FRcUCzh4sX4V6UNfpoYuQIyN0ndI9VJE5YRpG2ja57Zdz56M2As+8jAO9zXtTWQYMexCISGnY2aEE1EePAoZmrHJlak46P77e5U/92ge7eujSinw/tMV4LlTlSp445+b05KGjnswk5+HLpFwE/POurB5QE9T9hu6h1FEith+nlpxNAQrRBnMQ2aBjqnvoiDGzGndmfD104kVVhOShG9ghFDtO5WFK31ggN9mRNMwwLc2fzwNR3V0Nuj7XAh2H+/5YUBurf22pfRulIpNy6dqyqHBTENq25EoU2KBjXD10lOtBjbIPr5QFXDlnyXsqXSnB3KnDh1ajxpe3nYuySiv8dBogeVVrHyWDtr7oqFIUwcaLg4O/AktnAkMfkLXUht7f0iPkGzxypM2GqzmHjnFAhsCM43Jj848vad4WXkzTQq2DEoY6EqqdiiOEMccwnlCBWFWLbv27wIYPWuuIPAsJQOZ+YMciYM+PrX003osxtM06ItigY6pDibmNKYpgWo/4c4DpP9csvJt5EHgmolqeHcO0GFQQkFfFoNvzA3B0NQ8CoYRZc45yrivTKDjkyrgyf4Ksa0Z4q8RFW6WyXJabUYpbnKHKOUt5m0wUZjxIUiJugFx0pXwPS3LliAAjV7kqBl1EVz4jTINhDx3jSsFpIPuw/Dt76LyLl7sA650afDujiJVylSvTWnQ+H7hxseuigqpcuZDK1UNHrRPbaMiQaSMG3bPPPoshQ4bAaDQiONi9ZtHx48dx0UUXiW0iIyPx8MMPo7KSWwU1uNLVUiGHR7QccvUqzDFAQWodBh17XZlWhMSt6UeBCnnYoJMhQ/fqz4Eh9wCdL2itEWK8GK8JuZaXl+OKK67A4MGD8cEH1ZNoLRaLMOaio6Oxdu1apKam4sYbb4ROp8Nzzz3XKsfsldCEH9sXmDS3tY+EaYoqQoUKxaBjYWGmlaCF4px2wMWvAf2nAlYrMOh2IO4cHhJnvTr6YRhf9tA9/fTTuP/++9GzZ0+3zy9fvhy7d+/Gp59+ij59+mDChAl45plnMHfuXGEMMvWE8uaUyZ/xei06O4ljgDvWAnruD8m0EiQeHBDpWHSQ7trY2a66dG2dnd8ASx8DCtpmL1KmjRh0dbFu3Tph7EVFRdn/Nm7cOOTn52PXrl2temxexZjZctuZd0e29pEwDYXC5DV56PwCZVFXRbyUYVoD6mWaf1L+nfQuqQexkg7AALu+A/6eC+z8ms8G47sh17pIS0tzMeYI5TE9VxNlZWXiR4EMQMJqtYqfpob2KUlSs+y7SYjtC5UpGji1GZKnHmNbG5P6QuGrc++UQ1lV2bcEqmNrIV3wH3g6PjMePkRTjYmKBLDzTsv3ltTtUC+4GNY7/2mTfTfdofIPhYrON0VK6jjXfJ14FtYmvm81Zj+tatDNmDEDL7zwQq3b7NmzBykpDewp2gDmzJkjwrlVycjIQGlpaZO/Hw1SXl6eGHi1B3pL9MdWIuTvuagM7YKs9HS0BTx9TBqOU9K5DdOBVfDf/yMy+vwLno7vjYf301RjYtYGw3BmNzLT02FIOwpqFJhRZIFkbRv3mrowV6pBWa75pVaU1nH/5evEs7A28X2roKDAuwy6Bx98ENOmTat1m06d6teTjYoh/vnnH5e/nTlzxv5cTcycORMPPPCAi4cuPj4eERERCAwMRHMMOqn30/49cbJSbdkMlWSF1j9QVAq3BTx9TOpN4RmoFt8JafSTQGwfl6dU1Pvc3+wVY+oz4+FDNNmYXPQf0ZYpUusHnJI9EBHxSXKaBwNVYKg4C4Fh0Qis41rl68SzsDbxfcvPz6/Br2nVq4g+OP00BVT9StIm6enp9klrxYoVwijr1q1bja8zGAzipyo0IM01mdCgN+f+zwpbFaRKb4TKE4+vmfDoMakvpF11+HeoqOF5XL/q3T/0AV4zpj4xHj5Gk4yJv5PkVGkuYAiCWhEyZ+wiy2pDQL3yXfk68SxUTXjfasw+vGZZRBpz2dnZ4l+SKNm6dav4e1JSEkwmEy644AJhuN1www148cUXRd7cE088gbvuusutwcbUgCImPI6lXrwOg1lMkPakc2co8ZwlS5jWJusQ8OO9snSJSgOEJ7X2EXkWva+SfximEXiNQTdr1iwsWLDA/rhv377i3z/++AMjR46ERqPBTz/9hDvuuEN46wICAjB16lT8+9//bsWj9lJhYRIUpopIxjurCKv2yySSLwQqmz4nlGEaLF1ydJXc3mrI3fIPwzBty6D76KOPxE9tJCQkYMmSJS12TD7bb7GyBNj9A9DtktY+GqaptOh6Xs7nkvGMbiZUx+nOi8wwzFnBCSqMK4mjgMjuwNHVfGa8EZItGXhr9b+f2CCHuximtT10pihZL/GLa4Alj/B4MEwTwQYd40pFCZC+S77xMt5H0hggcXT1v39/F7Dh/dY4IoZxnxaQe0JuRM8wTJPABh3jyon18r8lOXxmvBGaJNe/A1Q6xLKdq1wZptUZMQPoez1Qkg0YZZkOhmHOHjbomOpFEYQk8ZnxRrIPAb88Uj2PrrxQ7tPLMK1NlwuAhMHyotGfpIUZhmkKvKYogmkh9LZJn+UEvJPAOPlfCmmFOolyl5OHztRqh8UwdjL2A3t/kr3GbNAxTJPBBh3jXoeu3QA+M95a5Uo4e+isFsAczeEtxjM4swP47Wng+m+A6N6tfTQM4zOwQce4ooTlyhreR47xEA8reT3ynGQh1Brgvu2teVQMU92LbI4FTE3TKYhhGM6hY6pCkgLUJaLDUD433krKxTa9L4bxYC/yohuB4uzWPhqG8Rm4KIJxRaUCBt/l2nOR8S4ufRPoe53jMenPvZQEnNzUmkfFMDLKYiPrABdfMUwTwgYdw/galeWycKsChc+LMuTQK8O0NhqnTB+/oNY8EobxKdigYxhfY93/gHlDHI/Li+R/ucqV8RTGPi0bc87GHcMwZwUbdAzji0nnpPFFUiUuBh0LCzMeAnmQqSsNwzBNBht0DOPr0iUkKuysMcgwrY0xDOg4orWPgmF8CvZ3M4wv9spUDLrwzkCnkcD0pYDe3NpHxjAyIx/lM8EwTQwbdAzjawTaDLqCM/K/1C+TWi0xDMMwPgsbdAzja2gNwGOnHTlzB3+VJUvYK8IwDOOzcA4dw/gizgUQx9YBWz5tzaNhGIZhmhk26BjGF/nzeeDH+xxVrlzhyjAM49OwQccwvkhxFnD8b0eVKxt0DMMwPg0bdAzjq9IlimxJRTFLljAMw/g4XBTBML4qLlyWD5Tmy3pflWWtfUQMwzBMM8IGHcP4tBbdaaD/1NY+GoZhGKaZ4ZArw/gikV2By+cD5ijgzG4gzxZ+ZRiGYXwSNugYxhfxDwF6TJH//WoasG5uax8RwzAM04xwyJVhfJUtnwGBMSxbwjAM0wZgg45hfJWNH8ihV5YtYRiG8Xk45MowPi1dctomW+LUOYJhGIbxOdigYxhfli7JOSYbc4bA1j4ahmEYphnhkCvD+LJ0SeEZYOZJQKVq7aNhGIZhmhH20DGMrxLTB0i5CLBUtPaRMAzDMM0MG3QM46t0HAaMeBSYNxhI3dbaR8MwDMM0I2zQMYyvYrUCaduBrIOtfSQMwzBMM8MGHcP4KpJVFhUm9KbWPhqGYRimGWGDjmF8FY1TzZPO2JpHwjAMwzQzbNAxTFuAdegYhmF8GjboGMaXaT9E/pcNOoZhGJ+GdegYxpdp1w+oLAXUmtY+EoZhGKYZYYOOYXyZ858B1OyIZxiG8XX4Ts8wvgwbcwzDMG0CNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HG1rH4CnIUmS+Dc/P79Z9m+1WlFQUAA/Pz+o1WxPewI8Jp4Fj4fnwWPiefCY+PZ45NtsEMUmqQ9s0FWBBoSIj48/6wFhGIZhGIY5G5skKCioXtuqpIaYf23Eyj59+jTMZjNUKlWT75+sbjIWT5w4gcDAwCbfP9NweEw8Cx4Pz4PHxPPgMfHt8ZAkSRhzsbGx9fb4sYeuCnTi4uLi0NzQgLNB51nwmHgWPB6eB4+J58Fj4rvjEVRPz5wCJ3ExDMMwDMN4OWzQMQzDMAzDeDls0LUwBoMBs2fPFv8yngGPiWfB4+F58Jh4HjwmnoXBA+Z2LopgGIZhGIbxcthDxzAMwzAM4+WwQccwDMMwDOPlsEHHMAzDMAzj5bBBVw/++usvTJw4UQj8kdjw4sWLXZ4/c+YMpk2bJp43Go0YP348Dhw44LJNWloabrjhBkRHRyMgIAD9+vXDN99847JNdnY2rrvuOqFhExwcjJtvvhmFhYUu22zfvh3Dhg0T7UVIxPDFF19EW6MlxuPo0aPi/Hfs2BH+/v5ITEwUCa/l5eUu++HxaLkxcaasrAx9+vQR77V161Yek1a4RhR+/vlnDBo0SFwnISEhmDRpksvzx48fx0UXXSTeJzIyEg8//DAqKyvR1mipMdm/fz8uvfRShIeHi7lk6NCh+OOPP1y24TFpujE5dOgQLrvsMkRERIjzfeWVV4rXtcbczgZdPSgqKkLv3r0xd+5ct2rOdAM7fPgwvv/+e2zZsgUJCQkYO3aseJ3CjTfeiH379uGHH37Ajh07MHnyZDHwtL0CDfiuXbuwYsUK/PTTT+LL9n//938uStQXXHCB2P+mTZvw0ksv4amnnsK7776LtkRLjMfevXtF15B33nlHjMmrr76Kt99+G4899ph9HzweLTsmzjzyyCPiJlsVHpOWHQ8yJsjAmD59OrZt24Y1a9bg2muvtT9vsViEMUcLobVr12LBggX46KOPMGvWLLQ1WmpMLr74YmEw//7772KeoPekv5ExSPCYNN2Y0L80J5MxSOebvv/0XScjkeaPFp/bqfUXU3/olH333Xf2x/v27RN/27lzp/1vFotFioiIkN577z373wICAqSPP/7YZV+hoaH2bXbv3i32s2HDBvvzv/zyi6RSqaRTp06Jx2+99ZYUEhIilZWV2bd59NFHpeTk5DY7hM01Hu548cUXpY4dO9of83i0zpgsWbJESklJkXbt2iX2u2XLFh6TVhiPiooKqV27dtL7779f43vTWKnVaiktLc3+t3nz5kmBgYEu97G2RnONSUZGhtjPX3/9ZX8+Pz9f/G3FihXiMY9J043JsmXLxPc7Ly/Pvk1ubq6Yt5Xz3ZJzO3vozhIK/RDkJnVuH0ZaNKtXr7b/bciQIfjyyy+F65Us94ULF6K0tBQjR44Uz69bt064YgcMGGB/Da0EaF/r16+3bzN8+HDo9Xr7NuPGjRMrtpycnLP9KD5BU42HO/Ly8hAaGmp/zOPR8mNCoYxbb70Vn3zyiQiBVIXHpOXGY/PmzTh16pR4bd++fRETE4MJEyZg586dLuPRs2dPREVFudyzyCNBHgumacckLCwMycnJ+Pjjj4X3iDx1FGWgUHf//v15TJp4TGgb8s45a8/R9rSdsk1Lzu1s0J0lKSkpaN++PWbOnClOPLlbX3jhBZw8eRKpqan27RYtWoSKigpxwdHg33bbbfjuu++QlJQknid3OF10zmi1WmFAKK5y+tf5xkgoj5Vt2jpNNR5VOXjwIP73v/+J7RR4PFp2TGgRTfkst99+u8vN0Rkek5YbDwpFERQaeuKJJ0QoiXLoyLggg4PHo+XHhIyLX3/9VYQHzWazMC5eeeUVLF26VIwNj0nTjsm5554rchkfffRRFBcXCyP6oYceEmFtZZuWnNvZoDtLdDodvv32W5GISgNEXgNKQKWVKlngCk8++SRyc3PFxbZx40Y88MADIveB8iAYzx4P8kJQMuwVV1whvENM64wJGdQFBQXiBsu0/ngoOUKPP/44pkyZIjxAH374oTAqvvrqKx6iVhgTWvTcddddwoBYtWoV/vnnH5EHRjldzoYh0zRjQoUQ9F3/8ccfYTKZEBQUJMaHilWcx62l0Lb4O/ogdCOjSjsKyZEVT4NMVV+KF4GqYN58800Riujevbv4GyVi0gVHyZiUbE9VS+np6S77JXc5rXTpOYL+rVo9ozxWtmGaZjwUTp8+jVGjRolQR9UEVR6Plr1GKOmYQhNVW+vQPijpmBLueUxabjwoxEp069bNvl8am06dOokqSuUaIaPCGb5nNe81Qp5S8ihRRSXx1ltviWR8uj5mzJjBY9KEY0JQMQONTWZmpvC8UXiVvvd0HRAtObezh64JIeucBpzKmmn1RKXjBLlixcmuYrFrNBr7Knfw4MHCsqcKFwW6OOl5+gIp21B1DLncFehCpZwJxZ3ONM14KJ45Ch8pnoeq2/N4tOw18sYbb4hKSrrB0s+SJUvE3ymn6Nlnn+UxaeHxoOuCDDjK81GgexNJ/lC1nnKNkPfIeUKjexYZG86GINM0Y1LTNvTYea7hMWmaMXGGZGLImKN5m77vl1xyScvP7Q0qoWijFBQUiEo6+qFT9sorr4jfjx07Jp5ftGiR9Mcff0iHDh2SFi9eLCUkJEiTJ0+2v768vFxKSkqShg0bJq1fv146ePCg9N///ldUufz888/27caPHy/17dtXbLN69Wqpc+fO0jXXXONSPRMVFSXdcMMNovJm4cKFktFolN555x2pLdES43Hy5EmxzZgxY8Tvqamp9h8FHo+WHZOqHDlypFqVK49Jy47HvffeKypdqdpv79690s033yxFRkZK2dnZ4vnKykqpR48e0gUXXCBt3bpVWrp0qagSnDlzptTWaIkxoSrXsLAw8To631Sp+dBDD0k6nU48JnhMmm5MiPnz50vr1q0T4/HJJ5+IquMHHnjAZZuWmtvZoKsHNKA02FV/pk6dKp5//fXXpbi4OHHRtG/fXnriiSeqleTv379ffBHoZkcD1atXr2rl51lZWWKQTSaTKOufPn26+MI5s23bNmno0KGSwWAQN9Lnn39eamu0xHh8+OGHbt+j6hqIx6PlxqQ+Bh2PScuOBxkZDz74oNjGbDZLY8eOdZF5II4ePSpNmDBB8vf3l8LDw8X2JHnS1mipMSF5DDKgybCgMTn33HOFVIkzPCZNNyYkL0LGGG1DhtrLL78sWa3WVpnbVfS/BnofGYZhGIZhGA+Cc+gYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGYRiGYbwcNugYhmEYhmG8HDboGIZhGIZhvBw26BiGaTNMmzYNkyZNOuv9qFQqLF68GC0N9Uml96ZetmfDU089hT59+jTZcTEM0/qwQccwTLMzceJEjB8/3u1zq1atEkbK9u3beSTqID4+HqmpqejRowefK4ZhXGCDjmGYZufmm2/GihUrcPLkyWrPffjhhxgwYAB69erVZkaivLy8Ua/TaDSIjo6GVqtt8mNiGMa7YYOOYZhm5+KLL0ZERAQ++ugjl78XFhbiq6++EgYf8c0336B79+4wGAzo0KEDXn75ZZfty8rK8OijjwpPFW2TlJSEDz74QDxnsVjEfjp27Ah/f38kJyfj9ddfd3s8Tz/9tDiewMBA3H777S4GFr3va6+95rI9hScpTFkTdExdunSB0WhEp06d8OSTT6KioqJaiPP9998Xx+fn54ePP/4YYWFh4jM5QyHhG264oV4h1z///FM8/u2334RRTO8/ZMgQ7Nu3z+V1zz//PKKiomA2m8U5Ki0trbZvOrauXbuKY0tJScFbb71lf+6mm24SBrdyrHS++vbtixtvvLHGc8IwTMvCBh3DMM0OeZRo8ieDTpIk+9/JmCND7JprrsGmTZtw5ZVX4uqrr8aOHTuEEUSGkbMRSPv44osv8MYbb2DPnj145513YDKZxHNWqxVxcXFin7t378asWbPw2GOPYdGiRS7HQsYPvZaMIdrXt99+Kwy8s4EMJTpOel8yIt977z28+uqrLtscPHhQGKz0fmSQXXHFFeKz//DDD/Zt0tPT8fPPPwsDqiE8/vjjwvjduHGjONfOr6fPT+fyueeeE8/HxMS4GGvEZ599Js7Xs88+K84NbUvnfsGCBeJ5Ot9FRUWYMWOG/f1yc3Px5ptvNup8MQzTDEgMwzAtwJ49e8iSk/744w/734YNGyZdf/314vdrr71WOv/8811e8/DDD0vdunUTv+/bt0+8fsWKFfV+z7vuukuaMmWK/fHUqVOl0NBQqaioyP63efPmSSaTSbJYLOJxQkKC9Oqrr7rsp3fv3tLs2bPtj+k4vvvuuxrf96WXXpL69+9vf0yv1el0Unp6ust2d9xxhzRhwgT745dfflnq1KmTZLVa3e73yJEj4r23bNkiHtO5pMe//vqrfZuff/5Z/K2kpEQ8Hjx4sHTnnXe67GfQoEHiMykkJiZKn3/+ucs2zzzzjHitwtq1a8VnePLJJyWtViutWrWqxs/PMEzLwx46hmFaBArjUThw/vz5do8VFUQo4VbyDJ133nkur6HHBw4cEJ4s8mpRDtmIESNqfI+5c+eif//+IpxKnrt3330Xx48fd9mmd+/eIjSpMHjwYBH6PXHiRKM/25dffimOlfLb6H2feOKJau+bkJAgjsuZW2+9FcuXL8epU6fEY/LyUSUuhVEbgnP+IXngFG+fcl4HDRrksj19ZgXyvB06dEiMAx278vOf//xH/N35NQ899BCeeeYZPPjggxg6dGiDjpFhmOaFDTqGYVoMMhoo7FhQUCCKIRITE2s10JyhvLjaWLhwoTA46D3ISCIDcPr06Q0uQFCr1S5hYcI5H64q69atw3XXXYcLL7wQP/30E7Zs2SJCklXfNyAgoNprKQ+NDEzKp6OQ865du4RB11B0Op39d8UYpBB0fSBjlqAwMZ0z5Wfnzp34+++/7dvR/tasWSOMajLGGYbxLNigYximxaAcOTKYPv/8c2HEUK6XYoBQQj4ZDM7QYyo2ICOiZ8+ewqhYuXKl233TtuQBvPPOO4WhRAUTzh4mhW3btqGkpMT+mIwW8khRoQVBXjSSBlHIz8/HkSNHavxMa9euFd43MuKoMKFz5844duxYvc/JLbfcIjxzZOCOHTvWfhxNBZ3X9evXu/zN2VCjYonY2FgcPnxYnDPnHyrgUHjppZewd+9ecf6XLl0qjpdhGM+BDTqGYVoMMpyuuuoqzJw5UxhNzt4oCuNRwQKF9Pbv3y8S8inpnrxuSvXp1KlThRFIor5kZFFhg1L0QIYUJf0vW7ZMvJ6S+jds2FDtGMhzRl48KmBYsmQJZs+ejbvvvlsYmsTo0aPxySefiHAwFWfQe5JBWRP0vhReJQ8hGZBUQPDdd9/V+5xce+21Qs6FPGQNLYaoD/fee68Ic5MBRueFPi95Ap2hopA5c+aIY6dt6HPT9q+88op4nryOVDRBlbAUWqa/037JCGQYxkNohbw9hmHaMJRcT7eeCy+8sNpzX3/9tSiCoOT79u3bi+ICZyjR//7775diYmIkvV4vJSUlSfPnzxfPlZaWStOmTZOCgoKk4OBgUXAwY8YMl+R/Koq49NJLpVmzZklhYWGiGOLWW28Vr1XIy8uTrrrqKikwMFCKj4+XPvroozqLIqh4Q9kfvZaKKug4FOi1zsdRlRtuuEEUazgfR0OKInJycuzb0HP0N9pW4dlnn5XCw8PF8dE5eOSRR6odz2effSb16dNHnNeQkBBp+PDh0rfffivOOY3J//3f/7lsf8kll0hDhgyRKisraz1mhmFaBhX9r7WNSoZhmLbMmDFjhP4eecgYhmEaAxt0DMMwrUROTo4IG19++eUiBExiyAzDMI2B+8cwDMO0ElS8QUbdCy+8wMYcwzBnBXvoGIZhGIZhvByucmUYhmEYhvFy2KBjGIZhGIbxctigYxiGYRiG8XLYoGMYhmEYhvFy2KBjGIZhGIbxctigYxiGYRiG8XLYoGMYhmEYhvFy2KBjGIZhGIbxctigYxiGYRiGgXfz/3bb0nXpnIQ3AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_logits_with_temperature(\n", " next_token_logits, start=19_800, end=19_900,\n", " temps=(0.5, 5.0),\n", "):\n", " x = torch.arange(start, end)\n", " logits_orig = next_token_logits[0, start:end].float().cpu()\n", "\n", " # Apply temperature scaling\n", " logits_scaled = [\n", " scale_logits_by_temperature(logits_orig, T) for T in temps\n", " ]\n", " # Plot logits\n", " plt.plot(x, logits_orig, label=\"Original logits\", lw=2)\n", " plt.plot(\n", " x, logits_scaled[0],\n", " label=f\"T={temps[0]} (sharper)\", ls=\"--\", lw=1\n", " )\n", " plt.plot(\n", " x, logits_scaled[1],\n", " label=f\"T={temps[1]} (flatter)\", ls=\":\", lw=3\n", " )\n", "\n", " # Highlight max logit\n", " max_idx = torch.argmax(logits_orig)\n", " plt.annotate(\n", " \"Berlin\",\n", " xy=(x[max_idx], logits_orig[max_idx]),\n", " xytext=(x[max_idx] - 25, logits_orig[max_idx] + 2),\n", " arrowprops={\"facecolor\": \"black\", \"arrowstyle\": \"->\", \"lw\": 1.5},\n", " fontsize=12,\n", " )\n", "\n", " plt.xlabel(\"Vocabulary index\")\n", " plt.ylabel(\"Logit value\")\n", " plt.legend()\n", " plt.grid(alpha=0.3)\n", " plt.tight_layout()\n", " plt.show()\n", "\n", "plot_logits_with_temperature(\n", " next_token_logits,\n", " temps=(0.5, 5.0)\n", ")" ] }, { "cell_type": "markdown", "id": "519ee3b9-b4e3-4b14-82d7-1d483af31ef3", "metadata": {}, "source": [ " \n", "### 4.4.3 Sampling the next token from a probability distribution" ] }, { "cell_type": "markdown", "id": "dfa52395-8506-427d-9f7e-7f5e5e46eb2b", "metadata": {}, "source": [ "- Convert logits scores to probability scores, which sum up to 1" ] }, { "cell_type": "markdown", "id": "6fdb8cd9-bd3e-476a-a90a-e8f2361f6d8b", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 14, "id": "9c84d516-ac92-4ebd-9dda-cddb35aca7fd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Probability sum: tensor(1., dtype=torch.bfloat16)\n" ] } ], "source": [ "# Step 3.2: Rescale next-token scores\n", "rescaled_logits = scale_logits_by_temperature(next_token_logits, 5.0)\n", "\n", "# Step 3.3 Convert rescaled logits into probability scores\n", "next_token_probas = torch.softmax(\n", " rescaled_logits, dim=-1\n", ")\n", "\n", "print(\"Probability sum:\", torch.sum(next_token_probas))" ] }, { "cell_type": "code", "execution_count": 15, "id": "7be14ac5-3b98-41e3-8c81-2e19d2fae10e", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnQAAAHWCAYAAAD+VRS3AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQ2NJREFUeJzt3QmcjeUf///PjGHsO2PNKLLvskXqSxHK9suSbAkVRZIta/jakqUI9bX1sJdUSMmSiuxlF6VIY4sZS9aZ+/f4XL//ff7nzAzNMGe557yej8cxc+5znfu+XW5n3nNtd4hlWZYAAADAsUL9fQIAAAC4NwQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcLszfJxDM4uLi5K+//pIsWbJISEiIv08HAAAEEL33w6VLl6RAgQISGnrnNjgCnR9pmCtcuLA/TwEAAAS4EydOSKFChe5YhkDnR9oyZ/9DZc2a1astgWfPnpU8efL8a8IHdZ9acN1T98GI6z511f3FixdNw4+dF+6EQOdHdjerhjlvB7pr166ZYxDofIu69x/qnroPRlz3qbPukzIsi+YaAAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcGH+PgEAcLrIAas8nv8+trHfzgVAcKKFDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcLiAC3bRp0yQyMlLSp08v1atXl23btt2x/LJly6RkyZKmfLly5WT16tUer1uWJUOHDpX8+fNLhgwZpH79+nLkyBGPMufPn5d27dpJ1qxZJXv27NKlSxe5fPmy6/WNGzdK06ZNzT4yZcokFStWlAULFnjsY+7cuRISEuLx0HMCAAAIqkC3ZMkS6dOnjwwbNkx27dolFSpUkAYNGsiZM2cSLb9582Zp27atCWC7d++WZs2amce+fftcZcaPHy9Tp06VGTNmyNatW00g031eu3bNVUbD3P79+2Xt2rWycuVK2bRpk3Tr1s3jOOXLl5dPPvlE9uzZI507d5YOHTqYsu40EEZFRbkef/zxh1fqCQAA4LYsP6tWrZrVo0cP1/PY2FirQIEC1pgxYxIt36pVK6tx48Ye26pXr251797dfB8XF2fly5fPmjBhguv16OhoKzw83Fq0aJF5fuDAAUv/6tu3b3eV+fLLL62QkBDr5MmTtz3XRo0aWZ07d3Y9nzNnjpUtWzbrbsXExJjz0K/epHUaFRVlvsK3qPvgqPsi/Vd6PIId1z11H4xivfCZk5ycECZ+dOPGDdm5c6cMHDjQtS00NNR0kW7ZsiXR9+h2bdFzp61vK1asMN8fO3ZMTp06ZfZhy5Ytm+nK1fe2adPGfNVu1qpVq7rKaHk9trboNW/ePNFjx8TESKlSpTy2aTdtkSJFJC4uTipXriz//e9/pUyZMom+//r16+Zhu3jxovmq79WHt+i+tRvam8cAdR9ofHndh4qV4NjBjM8c6j4YxXnhMyc5+/JroDt37pzExsZKRESEx3Z9fujQoUTfo2EtsfK63X7d3nanMnnz5vV4PSwsTHLmzOkqE9/SpUtl+/btMnPmTNe2EiVKyOzZs03XrIa9t99+W2rVqmW6cgsVKpRgH2PGjJERI0Yk2H727FmP7uCUpheEnp9eaBpa4TvUfXDUfakcnoHudkNGggXXPXUfjOK88Jlz6dIlZwQ6p9iwYYMZQ/fBBx94tL7VrFnTPGwa5rQFT0PfyJEjE+xHWyLdWxe1ha5w4cKSJ08eMxbPmxeZTtjQ4xDofIu6D466P3ghxON5/F8Ygw3XPXUfjOK88JmTnImWfg10uXPnljRp0sjp06c9tuvzfPnyJfoe3X6n8vZX3aYzVN3L6ExVu0z836Bv3bplZr7GP+63334rTz31lEyaNMlMiriTtGnTSqVKleTo0aOJvh4eHm4e8ek/vLd/4OhF5ovjgLoPJL667uPEM9Dx/4zPHH/i8z711H1y9uPXn+7p0qWTKlWqyLp16zwSrj53b/lyp9vdyyudqWqXL1q0qAll7mW0JUzHxtll9Gt0dLQZv2dbv369ObaOtXNfuqRx48Yybtw4jxmwt6Pdx3v37vUIkgAAAN7m9y5X7YLs2LGjmaBQrVo1mTx5sly5csV0cSptFStYsKAZf6Z69eoldevWlYkTJ5qwtXjxYtmxY4fMmjXLlY579+4to0aNkuLFi5uAN2TIEClQoIBZ3kRpt2jDhg2la9euZmmTmzdvSs+ePc2ECS1nd7M2adLEHK9ly5ausXUaQnWsnXrrrbekRo0aUqxYMRMQJ0yYYJYteeGFF/xSlwAAIDj5PdC1bt3aTArQhYA1NGm36Jo1a1yTGo4fP+7R5Kjj1BYuXCiDBw+WQYMGmdCmM1zLli3rKtOvXz8TCrVVTYNW7dq1zT7d+6J1kWANcfXq1TP719Cma9fZ5s2bJ//8848JknaYVBomteVOXbhwwYRCPe8cOXKY1kZdv6506dJerzcAAABbiK5d4noGn9KuYF1SRWfFeHtShI4Z1IHajO3xLeo+OOo+csAqj+e/j20swYzrnroPRnFe+MxJTk5ghDwAAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAhyPQAQAAOByBDgAAwOEIdAAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcLiAC3bRp0yQyMlLSp08v1atXl23btt2x/LJly6RkyZKmfLly5WT16tUer1uWJUOHDpX8+fNLhgwZpH79+nLkyBGPMufPn5d27dpJ1qxZJXv27NKlSxe5fPmy6/WNGzdK06ZNzT4yZcokFStWlAULFiT7XAAAAFJ9oFuyZIn06dNHhg0bJrt27ZIKFSpIgwYN5MyZM4mW37x5s7Rt29YEsN27d0uzZs3MY9++fa4y48ePl6lTp8qMGTNk69atJpDpPq9du+Yqo2Fu//79snbtWlm5cqVs2rRJunXr5nGc8uXLyyeffCJ79uyRzp07S4cOHUzZ5JwLAACAt4VY2pzlR9oi99BDD8l7771nnsfFxUnhwoXllVdekQEDBiQo37p1a7ly5YpHsKpRo4ZpQdMAp3+dAgUKyOuvvy59+/Y1r8fExEhERITMnTtX2rRpIwcPHpTSpUvL9u3bpWrVqqbMmjVrpFGjRvLnn3+a9yemcePGZj+zZ89O0rn8m4sXL0q2bNnM+WlLobdonWpAzps3r4SG+j3DBxXqPjjqPnLAKo/nv49tLMGM6566D0ZxXvjMSU5OCBM/unHjhuzcuVMGDhzo2qaVoF2kW7ZsSfQ9ul1b9Nxp69uKFSvM98eOHZNTp06Zfdi0MjQ46ns10OlX7Wa1w5zS8npsbdFr3rx5osfWCi1VqlSSzyW+69evm4f7P5R9EejDW3TfGnS9eQxQ94HGl9d9qHj+Xhzs/9f4zKHug1GcFz5zkrMvvwa6c+fOSWxsrGn1cqfPDx06lOh7NKwlVl6326/b2+5URhO0u7CwMMmZM6erTHxLly41LXozZ85M8rnEN2bMGBkxYkSC7WfPnvXoDk5pekFoGNULjRY636Lug6PuS+XwDHS3GzISLLjuqftgFOeFz5xLly45I9A5xYYNG8wYug8++EDKlClz1/vRlkj3Fj1todPu5Tx58ni9yzUkJMQch0DnW9R9cNT9wQshHs/j/8IYbLjuqftgFOeFzxydcOmIQJc7d25JkyaNnD592mO7Ps+XL1+i79Htdypvf9VtOkPVvYyObbPLxP8N+tatW2bma/zjfvvtt/LUU0/JpEmTzKSI5JxLfOHh4eYRn/7De/sHjl5kvjgOqPtA4qvrPk48Ax3/z/jM8Sc+71NP3SdnP3796Z4uXTqpUqWKrFu3ziPh6vOaNWsm+h7d7l5e6UxVu3zRokVNoHIvoy1hOjbOLqNfo6Ojzfg92/r1682xdayd+9IlOhFi3LhxHjNgk3ouAAAAvuD3LlftguzYsaOZoFCtWjWZPHmymTmqXZxKW8UKFixoxp+pXr16Sd26dWXixIkmbC1evFh27Nghs2bNcqXj3r17y6hRo6R48eIm4A0ZMsTMXNUlRZRObGjYsKF07drVzEa9efOm9OzZ00yYsGe4ajdrkyZNzPFatmzpGhenIVTH2iXlXAAAAIIi0OnSHzopQBcC1tCk3aK6hIg92eD48eMeTY61atWShQsXyuDBg2XQoEEmtOms0rJly7rK9OvXz4RCbVXTlrjatWubfbr3ResiwRri6tWrZ/avoU3XrrPNmzdP/vnnHxMk7TCpNMBpy11SzwUAACDVr0MXzFiHLvVjPa7gqHvWofNf3YO6DxRxfl6Hjv9pAAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAAwRjo9HZaH374oQwcOFDOnz9vtu3atUtOnjyZ0ucHAACAlL6X6549e6R+/frmVhS///67ucG93qx++fLl5r6r8+fPT+4uAQAA4MsWuj59+kinTp3kyJEjHje7b9SokWzatOlezgUAAAC+CHTbt2+X7t27J9hesGBBOXXq1N2cAwAAAHwZ6MLDw+XixYsJtv/yyy+SJ0+eezkXAAAA+CLQPf300/LWW2/JzZs3zfOQkBAzdq5///7SsmXLuzkHAAAA+DLQTZw4US5fvix58+aVq1evSt26daVYsWKSJUsWGT169L2cCwAAAHwxy1Vnt65du1a+//57M+NVw13lypXNzFcAAAA4INDZateubR4AAABwWKDT8XN3MnTo0Hs5HwAAAHg70H366acez3VyxLFjxyQsLEweeOABAh0AAECgB7rdu3cn2KbLmOhiw82bN0+p8wIAAIA37+UaX9asWWXEiBEyZMiQlNgdAAAAfB3oVExMjHkAAAAgwLtcp06d6vHcsiyJioqSjz76SJ588smUPDcAAAB4I9BNmjTJ43loaKi55VfHjh1l4MCByd0dAAAAfB3odEYrAAAAUuEYOgAAAARwC12LFi2SvMPly5ffy/kAAADAG4FO798KAAAABwe6OXPmeP9MAAAAcFcYQwcAABBss1zVxx9/LEuXLpXjx4/LjRs3PF7btWtXSp0bAAAAvNFCpwsLd+7cWSIiIsx9XatVqya5cuWS3377jYWFAQAAnBDopk+fLrNmzZJ3331X0qVLJ/369ZO1a9fKq6++yq2/AAAAnBDotJu1Vq1a5vsMGTLIpUuXzPft27eXRYsWpfwZAgAAIGUDXb58+eT8+fPm+/vuu09+/PFH1x0k9L6uAAAACPBA95///Ec+//xz872OpXvttdfk8ccfl9atW0vz5s29cY4AAABIyVmuOn4uLi7OfN+jRw8zIWLz5s3y9NNPS/fu3ZO7OwAAAPg60IWGhpqHrU2bNuYBAAAAh3S5FitWTIYPHy6//PKLd84IAAAA3g102s26atUqKVWqlDz00EMyZcoUOXXqVHJ3AwAAAH8FOp0EsX37djl48KA0atRIpk2bJoULF5YnnnhC5s+fn1LnBQAAAG/fy/XBBx+UESNGmK7X7777Ts6ePWtmvQIAAMAB93K1bdu2TRYuXChLliyRixcvyjPPPJNyZwYAAADvBDptkVuwYIG5K4QuJqzr0o0bN05atGghmTNnTu7uAAAA4OtAV7JkSTMZQidH6HIlERER93oOAAAA8GWgO3z4sBQvXvxejgkAAAB/ToogzAEAAKSSWa4AAAAIDAQ6AAAAhyPQAQAABFug27Bhg3fOBAAAAL4JdA0bNpQHHnhARo0aJSdOnLi7owIAAMB/ge7kyZPSs2dP+fjjj+X++++XBg0ayNKlS+XGjRspd1YAAADwXqDLnTu3vPbaa/LTTz/J1q1bzT1dX375ZSlQoIC8+uqr8vPPPyd3lwAAAPDXpIjKlSvLwIEDTYvd5cuXZfbs2VKlShWpU6eO7N+//152DQAAAG8Gups3b5ou10aNGkmRIkXkq6++kvfee09Onz4tR48eNdueeeaZu9k1AAAAvH3rr1deeUUWLVoklmVJ+/btZfz48VK2bFnX65kyZZK3337bdMECAAAgAFvoDhw4IO+++6789ddfMnnyZI8w5z7OLqnLm0ybNk0iIyMlffr0Ur16ddm2bdsdyy9btkxKlixpypcrV05Wr17t8boGzaFDh0r+/PklQ4YMUr9+fTly5IhHmfPnz0u7du0ka9askj17dunSpYvpMrZdu3ZNOnXqZPYfFhYmzZo1S3AeGzdulJCQkASPU6dOJenvDQAA4LdAN2zYMNOdGh4e7rH91q1bsmnTJvO9hqC6dev+676WLFkiffr0MfvctWuXVKhQwcyaPXPmTKLlN2/eLG3btjUBbPfu3SZo6WPfvn2uMtpiOHXqVJkxY4aZtKEthrpPDWk2DXM6xm/t2rWycuVKc97dunVzvR4bG2vCoE7y0EB4J4cPH5aoqCjXI2/evP/69wYAAEhRVjKFhoZap0+fTrD93Llz5rXkqFatmtWjRw/X89jYWKtAgQLWmDFjEi3fqlUrq3Hjxh7bqlevbnXv3t18HxcXZ+XLl8+aMGGC6/Xo6GgrPDzcWrRokXl+4MABS//a27dvd5X58ssvrZCQEOvkyZMJjtmxY0eradOmCbZv2LDB7OfChQvW3YqJiTH70K/epPUaFRVlvsK3qPvgqPsi/Vd6PIId1z11H4xivfCZk5yckOwxdNqlqV2L8f3999+mNSypdN26nTt3mlmyttDQUNMitmXLlkTfo9u1Rc+dtr6tWLHCfH/s2DHT5eneqpYtWzbTlavvbdOmjfmq3axVq1Z1ldHyemxt0WvevLkkR8WKFeX69eum63n48OHy8MMP37asltOH7eLFi+ZrXFyceXiL7lv/3bx5DFD3gcaX132oWAmOHcz4zKHug1GcFz5zkrOvJAe6Fi1amK8a5nR8mXuXq3ZR7tmzR2rVqpXkA587d868LyIiwmO7Pj906FCi79Gwllh5e9ya/fXfysTvFtUu4pw5cyZr/JuO0dNuXQ2GGtI+/PBDefTRR00o1OVcEjNmzBgZMWJEgu1nz5716BJOaXpBxMTEmAtNgyt8h7oPjrovlcMz0N1u2Eiw4Lqn7oNRnBc+cy5dupTygU5bupSeaJYsWcwYM1u6dOmkRo0a0rVrVwkWJUqUMA+bhtlff/1VJk2aJB999FGi79HWSPcWRm2hK1y4sOTJk8dM0PDmRaZBXI9DoPMt6j446v7gBc9ei2AfS8t1T90HozgvfOboBNAUD3Rz5swxX3VGat++fZPVvZoYnQmbJk0as3adO32eL1++RN+j2+9U3v6q27QFzb2Mdo3aZeL/9qwTOnTm6+2Om1TVqlWT77///rava6tm/MkkSv/hvf0DRy8yXxwH1H0g8dV1HyeegY7/Z3zm+BOf96mn7pOzn7ua5XqvYc5u1dO7Sqxbt84j3erzmjVrJvoe3e5eXulMVbt80aJFTShzL6OtYNoNapfRr9HR0Wb8nm39+vXm2DrW7l7o7dDcgyQAAIAvJKmFTseEaUjKkSOHVKpUKdFJETZdfiSptPuxY8eOZhyatm7punZXrlyRzp07m9c7dOggBQsWNGPPVK9evcxyKBMnTpTGjRvL4sWLZceOHTJr1izzup5X7969ZdSoUVK8eHET8IYMGWIWObbXkitVqpQ0bNjQdA/rGDi964XeukwnTLgvhqzr7enEDW250z5sDWvKbunTc9X9lylTxox/0zF0Ggy//vrrJP/9AQAAfBbomjZt6uoqTGyR3bvVunVrMyFAFwLWCQkaltasWeOa1HD8+HGP5kYdp7Zw4UIZPHiwDBo0yIQ2neHqvrhxv379TCjUdeW0Ja527dpmn+790AsWLDAhrl69emb/LVu2NGvXudPbmv3xxx+u5xpk7TGESsPe66+/LidPnpSMGTNK+fLl5ZtvvpHHHnssxeoHAAAgKUJ07ZIklUSK0+5gnWyis2K8PSlCxw3qQG3G9vgWdR8cdR85YJXH89/HNpZgxnVP3QejOC985iQnJzBCHgAAIBi6XHXs3J3GzbnTMWcAAAAIsECnEwAAAADg4ECnM1EBAADg4ECng/LswXj2/Udvx5uD+wEAAHAPY+iioqLMzA29sX1i4+l0sqxu1/uzAgAAIMACnS6YqzevVxs2bPD2OQEAACClA53enSGx7wEAAOCQQBffhQsX5H//+58cPHjQPC9durS5XZfdigcAAADfSfbCwps2bZLIyEhzqywNdvrQ7/W+pvoaAAAAAryFrkePHuYerO+//76kSZPGbNOJEC+//LJ5be/evd44TwAAAKRUC93Ro0fNTentMKf0+z59+pjXAAAAEOCBrnLlyq6xc+50W4UKFVLqvAAAAJCSXa579uxxff/qq69Kr169TGtcjRo1zLYff/xRpk2bJmPHjk3qcQEAAODLQFexYkWzaLAuHmzr169fgnLPPvusGV8HAACAAAt0x44d8/6ZAAAAwHuBrkiRIne3dwAAAATmwsLqwIEDcvz4cblx44bH9qeffjolzgsAAADeCnS//fabNG/e3Kw35z6uTr+316QDAABAAC9bojNc9a4QZ86ckYwZM8r+/fvNHSKqVq0qGzdu9M5ZAgAAIOVa6LZs2SLr16+X3LlzS2hoqHnUrl1bxowZY5Y02b17d3J3CQAAAF+20GmXapYsWcz3Gur++usv18SJw4cP38u5AAAAwBctdGXLlpWff/7ZdLtWr15dxo8fL+nSpZNZs2bJ/ffffzfnAAAAAF8GusGDB8uVK1fM92+99ZY0adJE6tSpI7ly5ZIlS5bcy7kAAADAF4GuQYMGru+LFSsmhw4dkvPnz0uOHDlcM10BAADggHXo1IkTJ8zXwoULp9T5AAAAwNuTIm7duiVDhgyRbNmySWRkpHno99oVe/PmzeTuDgAAAL5uoXvllVdk+fLlZjJEzZo1XUuZDB8+XP7++295//337/WcAAAA4M1At3DhQlm8eLE8+eSTrm3ly5c33a5t27Yl0AEAAAR6l2t4eLjpZo1PlzHR5UsAAAAQ4IGuZ8+eMnLkSLl+/bprm34/evRo8xoAAAACsMu1RYsWHs+/+eYbKVSokFSoUME814WGb9y4IfXq1fPOWQIAAODeAp3OYnXXsmVLj+csWwIAABDggW7OnDnePxMAAAD4dmHhs2fPyuHDh833JUqUkDx58tztrgAAAODLSRF6H9fnn39e8ufPL4888oh5FChQQLp06SL//PPPvZwLAAAAfBHo+vTpI99++6188cUXEh0dbR6fffaZ2fb666/fzTkAAADAl12un3zyiXz88cfy6KOPurY1atRIMmTIIK1atWJhYQAAgEBvodNu1YiIiATb8+bNS5crAACAEwKd3r912LBhcu3aNde2q1evyogRI1z3dgUAAEAAd7lOnjxZGjZsmGBh4fTp08tXX33ljXMEAABASga6cuXKyZEjR2TBggVy6NAhs61t27bSrl07M44OAAAAARzobt68KSVLlpSVK1dK165dvXdWAAAA8M4YurRp03qMnQMAAIADJ0X06NFDxo0bJ7du3fLOGQEAAMC7Y+i2b98u69atk6+//tqMp8uUKZPH68uXL0/uLgEAAODLQJc9e3Zp2bLlvRwTAAAA/gx0c+bMScnjAwAAwFdj6OLi4szYuYcfflgeeughGTBggFlQGAAAAA4JdKNHj5ZBgwZJ5syZpWDBgjJlyhQzQQIAAAAOCXTz58+X6dOnm7tBrFixQr744guzuLC23AEAAMABge748ePSqFEj1/P69etLSEiI/PXXX946NwAAAKRkoNN15/R+rfEXGta7RwAAAMABs1wty5JOnTpJeHi4a5veNeLFF1/0WIuOdegAAAACNNB17Ngxwbbnnnsupc8HAAAA3gp0rD8HAACQSu7lCgAAgMBCoAMAAHA4vwe6adOmSWRkpJlBW716ddm2bdsdyy9btkxKlixpypcrV05Wr16dYPLG0KFDJX/+/JIhQwazvMqRI0c8ypw/f17atWsnWbNmNfem7dKli1y+fNljsodOANH9h4WFSbNmzRI9l40bN0rlypXNRJFixYrJ3Llz76kuAAAAHBfolixZIn369JFhw4bJrl27pEKFCtKgQQM5c+ZMouU3b94sbdu2NQFs9+7dJmjpY9++fa4y48ePl6lTp8qMGTNk69atZgau7lNDmk3D3P79+2Xt2rWycuVK2bRpk3Tr1s31emxsrAmDr776qgmEiTl27Jg0btxYHnvsMfnpp5+kd+/e8sILL5iFlwEAAHwpxNImLT/RFjm9L+x7771nnutdJwoXLiyvvPKKuVdsfK1bt5YrV66YEGarUaOGVKxY0QQ4/asUKFBAXn/9denbt695PSYmRiIiIkzrWZs2beTgwYNSunRp2b59u1StWtWUWbNmjVk0+c8//zTvd6ctddHR0ebuGO769+8vq1at8giTun8tq/tLiosXL0q2bNnMOWprobdovWpIzps3r4SG+r1RNqhQ98FR95EDVnk8/31sYwlmXPfUfTCK88JnTnJygt9+ut+4cUN27tzp0QKmFaDPt2zZkuh7dHv8FjNtfbPLa6vZqVOnPMpoRWhwtMvoV+1mtcOc0vJ6bG3RS6p/OxcAAICAW7YkpZ07d850bWrrmTt9fujQoUTfo2EtsfK63X7d3nanMpqe3ek4uZw5c7rKJMXtzkXT9NWrV02XbXzXr183D5uWtVO9N++Jq/vW1kvuu+t71H1w1H2oeHZ0BPv/Na576j4YxXnhMyc5+/JboAtGY8aMkREjRiTYfvbsWY8xfilNLwhtrtULjS5X36Lug6PuS+XwDHS3GwccLLjuqftgFOeFz5xLly4FfqDLnTu3pEmTRk6fPu2xXZ/ny5cv0ffo9juVt7/qNp3l6l5Gx9nZZeJ/2Op9anXm6+2Om5xz0T7uxFrn1MCBA80kEPcWOh0zmCdPHq+PoQsJCTHHIdD5FnUfHHV/8EKIx/P4vQDBhuueug9GcV74zNEVPQI+0KVLl06qVKki69atcy0LopWhz3v27Jnoe2rWrGle1xmlNp2pqttV0aJFTdDSMnaA09CkY+Neeukl1z504oKO39Pjq/Xr15tj61i7pNL9xF8yxf1cEqPLm7jfC9em//De/oGjF5kvjgPqPpD46rqPE89Ax/8zPnP8ic/71FP3ydmPX7tctbVK7xGrExSqVasmkydPNrNYO3fubF7v0KGDFCxY0HRVql69ekndunVl4sSJZsmQxYsXy44dO2TWrFmuitSwN2rUKClevLgJeEOGDDEzV+3QWKpUKWnYsKF07drVzIy9efOmCZA6Q9V9huuBAwfMxA1tudMmT12aRNlB8cUXXzSzc/v16yfPP/+8CYVLly41M18BAAB8ya+BTpch0fFjuhCwTjLQsKRLftiTDY4fP+6RTmvVqiULFy6UwYMHy6BBg0xo0+VEypYt6yqjAUtDoa4rpy1xtWvXNvt0b7ZcsGCBCXH16tUz+2/ZsqVZu86dLmPyxx9/uJ5XqlTJfLVXedGwqOHttddekylTpkihQoXkww8/NDNdAQAAgmYdumDHOnSpH+txBUfdsw6d/+oe1H2giAvWdegAAACQMgh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADhcQgW7atGkSGRkp6dOnl+rVq8u2bdvuWH7ZsmVSsmRJU75cuXKyevVqj9cty5KhQ4dK/vz5JUOGDFK/fn05cuSIR5nz589Lu3btJGvWrJI9e3bp0qWLXL582aPMnj17pE6dOuY4hQsXlvHjx3u8PnfuXAkJCfF4aFkAAICgCnRLliyRPn36yLBhw2TXrl1SoUIFadCggZw5cybR8ps3b5a2bduaALZ7925p1qyZeezbt89VRoPX1KlTZcaMGbJ161bJlCmT2ee1a9dcZTTM7d+/X9auXSsrV66UTZs2Sbdu3VyvX7x4UZ544gkpUqSI7Ny5UyZMmCDDhw+XWbNmeZyPBsKoqCjX448//vBKPQEAANyW5WfVqlWzevTo4XoeGxtrFShQwBozZkyi5Vu1amU1btzYY1v16tWt7t27m+/j4uKsfPnyWRMmTHC9Hh0dbYWHh1uLFi0yzw8cOGDpX3379u2uMl9++aUVEhJinTx50jyfPn26lSNHDuv69euuMv3797dKlCjhej5nzhwrW7Zsd/13j4mJMeehX71J6zQqKsp8hW9R98FR90X6r/R4BDuue+o+GMV64TMnOTnBry10N27cMK1f2iVqCw0NNc+3bNmS6Ht0u3t5pa1vdvljx47JqVOnPMpky5bNdOXaZfSrdrNWrVrVVUbL67G1Rc8u88gjj0i6dOk8jnP48GG5cOGCa5t202ornnbJNm3a1LT6AQAA+FKY+NG5c+ckNjZWIiIiPLbr80OHDiX6Hg1riZXX7fbr9rY7lcmbN6/H62FhYZIzZ06PMkWLFk2wD/u1HDlySIkSJWT27NlSvnx5iYmJkbfffltq1aplQl2hQoUSnPv169fNw71bV8XFxZmHt+i+dVyhN48B6j7Q+PK6DxUrwbGDGZ851H0wivPCZ05y9uXXQOd0NWvWNA+bhrlSpUrJzJkzZeTIkQnKjxkzRkaMGJFg+9mzZz3G96U0vSA0cOqFpq2Q8B3qPjjqvlQOz0B3uzHAwYLrnroPRnFe+My5dOmSMwJd7ty5JU2aNHL69GmP7fo8X758ib5Ht9+pvP1Vt+ksV/cyFStWdJWJ/4F769YtM/PVfT+JHcf9GPGlTZtWKlWqJEePHk309YEDB5oJIO4tdNpVmydPHjO5wpsXmc7A1eMQ6HyLug+Ouj94IcTjefwegGDDdU/dB6M4L3zmJGflDL8GOh2fVqVKFVm3bp2ZqWpXiD7v2bNnou/RFjF9vXfv3q5tOlPVbinTblINXFrGDnAanHRs3EsvveTaR3R0tBm/p8dX69evN8fWsXZ2mTfffFNu3rxpgpp9HO1m1e7WxGj38d69e6VRo0aJvh4eHm4e8ek/vLd/4OhF5ovjgLoPJL667uPEM9Dx/4zPHH/i8z711H1y9uP3n+7aYvXBBx/IvHnz5ODBgyZ0XblyRTp37mxe79Chg2nZsvXq1UvWrFkjEydONOPsdCmRHTt2uAKgVqaGvVGjRsnnn39uApbuo0CBAq7QqN2iDRs2lK5du5o173744Qfz/jZt2phy6tlnnzWBU5dH0TFxurzKlClTPFrY3nrrLfn666/lt99+M0uuPPfcc2bZkhdeeMHHtQgAAIKZ38fQtW7d2owh04WAdbKBtqppYLMnIBw/ftwjoeo4tYULF8rgwYNl0KBBUrx4cVmxYoWULVvWVaZfv34mFOq6ctoSV7t2bbNP96bLBQsWmBBXr149s/+WLVuatevcZ8ZqWOvRo4dpxdPuYT1H97XqdLarhkJ7koSW03XySpcu7YOaAwAA+H9CdO2S/+97+Jh2BWtw1EGU3h5Dp2MGdVwPXUG+Rd0HR91HDljl8fz3sY0lmHHdU/fBKM4LnznJyQl+73IFAADAvSHQAQAAOByBDgAAwOH8PikCAHBnjNED8G9ooQMAAHA4Ah0AAIDD0eUKOBTdcAAAGy10AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAABwuzN8nAABASoocsMrj+e9jG1PBSPVooQMAAHA4Ah0AAIDD0eUKIGDRdQYASUMLHQAAgMMR6AAAAByOLlfAj+hSBACkBAJdEASGULGkVA5LDl4Ikd/GNvH3KQGAz/HLE1I7Ah2AFMUPTgDwPQId4CW0jgIAfIVJEQAAAA5HoAMAAHA4Ah0AAIDDEegAAAAcjkAHAADgcAQ6AAAAh2PZEgAAvLwm4+9jG1PH8CoCHYCAwILEAHD3CHQAEEAItgDuBoEOKSoYfhg5/e/o9PMHACREoANSCYIaAD5PgheBDgBSMQbmJ61uFK3VcDICHZAC+MGAQBAI16GeQ6hYUiqHJQcvhMhvY5v4/Bzg+e+RUtdEIFxfuD0CHeAAfJAiENDaBwQuAh0AIIFg/SWC0Aqn4k4RAAAADkcLXRAKht+8g+HvCACAjUCHFA1OgDcR1IHAQhd14KDLFQAAwOFooUPAoPUFAMDPnLtDoIPXEdQQjEMNuO4B7//fY3z0/49ABwQYPrCoLwBILgIdggIhiboHUtNnTGq8A0SgTrCIDNDzio9AB8fz9oeRk2fyBsoHNeBNTvmBC3gTgQ7wISeHw5RE0EQw4rqHNxHoELD48EMwIOQDSAkEOgBBGbqdfO5OQ10D3kegA5AqA0SoWFIqhyUHL4TIb2Ob+PuUUmXgupegxri31Mkf4d3JE1JSEoEOjrpgU1Iw/B2TirrwTd1Qz6njh7evJ2IF82eTr0U6uO4D4tZf06ZNk8jISEmfPr1Ur15dtm3bdsfyy5Ytk5IlS5ry5cqVk9WrV3u8blmWDB06VPLnzy8ZMmSQ+vXry5EjRzzKnD9/Xtq1aydZs2aV7NmzS5cuXeTy5cseZfbs2SN16tQxxylcuLCMHz8+2efilAvY/XGv5QB4393+f+T/MQLhc9zX12FkEPz88nsL3ZIlS6RPnz4yY8YME+YmT54sDRo0kMOHD0vevHkTlN+8ebO0bdtWxowZI02aNJGFCxdKs2bNZNeuXVK2bFlTRoPX1KlTZd68eVK0aFEZMmSI2eeBAwdM8FIa5qKiomTt2rVy8+ZN6dy5s3Tr1s3sT128eFGeeOIJEwb13Pbu3SvPP/+8CX9aLqnngpTFb8ZIqWvHH9cXqPtgHGrgjy79YOT3QPfOO+9I165dTaBSGp5WrVols2fPlgEDBiQoP2XKFGnYsKG88cYb5vnIkSNNKHvvvffMe7V1TkPh4MGDpWnTpqbM/PnzJSIiQlasWCFt2rSRgwcPypo1a2T79u1StWpVU+bdd9+VRo0aydtvvy0FChSQBQsWyI0bN8x5pEuXTsqUKSM//fSTOV870P3buQBOxQcpEFj/1xhzePf1+nuQBEG/drlqYNq5c6dpBXOdUGioeb5ly5ZE36Pb3csrbX2zyx87dkxOnTrlUSZbtmym9c8uo1+1pc0Oc0rL67G3bt3qKvPII4+YMOd+HG05vHDhQpLOBcHd/J3a8G8GrjkgcPm1he7cuXMSGxtrWs/c6fNDhw4l+h4Na4mV1+326/a2O5WJ350bFhYmOXPm9Cij3bXx92G/liNHjn89l/iuX79uHraYmBjzNTo6WuLi4sQrrl/RUYVy65olcj3EHOv/bbu9xMokdRv7il831L3/rgnqPn79+G5fd657/51XMOyLuo/2R92LmJ/jOlxLG4K0gSgl6P6U9j7+K8uPTp48qWdobd682WP7G2+8YVWrVi3R96RNm9ZauHChx7Zp06ZZefPmNd//8MMPZp9//fWXR5lnnnnGatWqlfl+9OjR1oMPPphg33ny5LGmT59uvn/88cetbt26eby+f/9+s+8DBw4k6VziGzZsmHk/D+qAa4BrgGuAa4BrgGtAklgHJ06csP6NX1vocufOLWnSpJHTp097bNfn+fLlS/Q9uv1O5e2vuk1nubqXqVixoqvMmTNnPPZx69YtM/PVfT+JHcf9GP92LvENHDjQTACxaZrXY+bKlUtCQkLEWzTh6yzdEydOmFm98B3q3n+oe+o+GHHdp66615a5S5cumbH9/8avgU6bJatUqSLr1q0zs0PtkKPPe/bsmeh7atasaV7v3bu3a5tORNDtSrtJNVBpGTvAaSXr2LiXXnrJtQ9tItXxe3p8tX79enNsHWtnl3nzzTfNDNi0adO6jlOiRAnT3ZqUc4kvPDzcPNzpWD5f0QuMQOcf1L3/UPfUfTDiuk89da/zAJLE8rPFixdb4eHh1ty5c01XpnZzZs+e3Tp16pR5vX379taAAQNc5bVLNSwszHr77betgwcPmm5M7frcu3evq8zYsWPNPj777DNrz549VtOmTa2iRYtaV69edZVp2LChValSJWvr1q3W999/bxUvXtxq27at6/Xo6GgrIiLCHH/fvn3mPDNmzGjNnDkzWecSCGJiYkyTrX4FdR8suO6p+2DEdR+8de/3QKfeffdd67777rPSpUtnxs79+OOPrtfq1q1rdezY0aP80qVLzRg4LV+mTBlr1apVHq/HxcVZQ4YMMYFMw2K9evWsw4cPe5T5+++/TYDLnDmzlTVrVqtz587WpUuXPMr8/PPPVu3atc0+ChYsaIJifP92LoHA3xdZMKPuqftgxHVP3QejGD//rA3RP1KsXRABSWfW6uLHOoYvfpcvqPvUiuueug9GXPfBW/cEOgAAAIcLiHu5AgAA4O4R6AAAAByOQAcAAOBwBLoAtGnTJnnqqafMQoK64PCKFSsSLF7cqVMn83rGjBmlYcOGcuTIEY8yevux9u3bmzX5MmXKJJUrV5ZPPvnEo4wuatyuXTuzXo6uh9elSxe5fPmyR5k9e/ZInTp1JH369GbBxPHjx0tq5ou6//33301d65qJGTJkkAceeECGDRtm7m3sjrr3znXvPoBZ16rUf+effvqJuvfBZ45atWqVWe9Tr31d09Neg9R2/Phxady4sTmO3qLxjTfeMAu/p1a++rz/5ZdfpGnTpmZBf/3Mr127tmzYsMGjDHW/Itl1/+uvv0rz5s0lT548pl5btWqV4IYDvvpZS6ALQFeuXJEKFSrItGnTErymk5L1A/C3336Tzz77THbv3i1FihSR+vXrm/fZOnToIIcPH5bPP/9c9u7dKy1atDAXmpa36QW2f/9+sxjyypUrzQdLt27dXK/rgsxPPPGE2b8uwjxhwgQZPny4zJo1S1IrX9S93qdYF7GeOXOmqf9JkybJjBkzZNCgQa59UPfeqXt3/fr1S3T1deree3WvIUODR+fOneXnn3+WH374QZ599lnX63pvbw1z+svN5s2bZd68eTJ37lwZOnSopFa++rxv0qSJCca6iL5+nusxdZt973HqPvl1r1/1Z6QGca1XvZ712tWA7n5/dp/9rPXLYilIMv0n+vTTT13PdT093aaLHdtiY2PNfWg/+OAD17ZMmTJZ8+fP99hXzpw5XWV0EWfdz/bt212vf/nll1ZISIi5x67S+9rmyJHDun79uqtM//79rRIlSgTFv6C36j4x48ePN4tf26h779b96tWrrZIlS7ruz7x7927q3svX/c2bN816nh9++OFt/x/ov0toaKhrYXn1/vvvm7VC3T+HUitv1f3Zs2fNfjZt2uR6/eLFi2bb2rVrzXPqXpJd91999ZW5Xt3XndObEujPUbteffmzlhY6h9FuIqXNsrbQ0FCz5s3333/v2larVi1ZsmSJaerV3xQWL14s165dk0cffdS8vmXLFtP0W7VqVdd79DcP3ZfeJs0u88gjj5hbtNkaNGhgfhO8cOGCBJuUqvvExMTESM6cOV3PqXvv1b12h3Tt2lU++ugj040SH3XvnbrftWuXnDx50ry3UqVK5l7bTz75pOzbt8+j7suVKycREREenznagqEtHMEmpepe7xeut62cP3++aVXSljrtIdAubfv2l9R98utey2jrnPuac1pey9llfPmzlkDnMCVLlpT77rvPLFyo/9DavDtu3Dj5888/JSoqylVu6dKl5j60+h9ZL7bu3bvLp59+KsWKFTOvazO7/md2FxYWZkKF3QSvX90/WJX93C4TTFKq7uM7evSovPvuu6acjbr3Tt1rI4iOiXnxxRc9PmDdUffeqXvtulLalTR48GDT9aRj6DR0aBCh7r33maOh45tvvjHdhlmyZDGh45133pE1a9a47k3OdZ/8uq9Ro4YZs9i/f3/5559/TFju27ev6b62y/jyZy2BzmHSpk0ry5cvNwNc9YLQFgYd2Kq/6Writw0ZMkSio6PNf+IdO3ZInz59zJgKHV+BwKl7bbHQgbbPPPOMaTWCd+teg/OlS5fMhzR8e93bY4refPNNadmypWkZmjNnjgkby5Yt45/Di3Wvv8j06NHDBIvvvvtOtm3bZsaH6Vgv92CI5NW9ToTQa/eLL76QzJkzS7Zs2cy/g05Kcf/38ZUwnx8R90w/CHVWnnbT6W8NelHprDG7xUFn3bz33numK6NMmTJmmw6A1f/IOvBWB+DrbKgzZ8547Feb4fU3ZX1N6df4s3Xs53aZYJMSdW/766+/5LHHHjPdJfEHv1L33ql7Hbis3Rvxb8uj+9CByzoIn7r3Tt1rF6sqXbq0a7/673D//feb2ZX2da9hwx2fOSlz3WuLqLY06UxLNX36dDNIX6/5AQMGUPd3cd0rncyg/wbnzp0zLW/avarXsV7Xypc/a2mhczD9bUAvMJ1Grb+V6ZR0pU2/Kv5vCGnSpHH9llyzZk3zm4TOqLHpf3p9XS9Yu4zOxtGmfJt+AOhYDLuZPljdS93bLXPa1WS3UsQvT917p+6nTp1qZlfqh7Q+Vq9ebbbr+KPRo0dT91687vVa1wCn44Js+tmiy/jo7D77utdWJfcfgPqZoyHEPQgGo3up+9uV0efuPxOo++TVvTtdDkbDnP4c1ev36aef9v3P2mRNoYBPXLp0ycy604f+E73zzjvm+z/++MO8vnTpUmvDhg3Wr7/+aq1YscIqUqSI1aJFC9f7b9y4YRUrVsyqU6eOtXXrVuvo0aPW22+/bWbVrFq1ylWuYcOGVqVKlUyZ77//3ipevLjVtm1bj9k6ERERVvv27c1Mn8WLF1sZM2a0Zs6cmWqvBF/U/Z9//mnK1KtXz3wfFRXletioe+9d9+6OHTuWYJYrde+9uu/Vq5eZ6aqzAw8dOmR16dLFyps3r3X+/Hnz+q1bt6yyZctaTzzxhPXTTz9Za9asMbMKBw4caKVWvvjM0VmuuXLlMu/TetUZnH379rXSpk1rnivqXpJd92r27NnWli1bTL1/9NFHZnZxnz59PMr46mctgS4A6QWk/7HjPzp27GhenzJlilWoUCHzn/G+++6zBg8enGBK/y+//GIuPP2w1AujfPnyCaa1//333+aiypw5s1kWoHPnzubDxd3PP/9s1a5d2woPDzcfxGPHjrVSM1/U/Zw5cxI9Rvzfr6h771z3/xboqHvvfeZo+Hj99ddNmSxZslj169f3WBZC/f7779aTTz5pZciQwcqdO7cpr0uepFa++rzXZTM0KGvg0LqvUaOGWarEHXUvya57XV5Ew5iW0aA2ceJEKy4uzi8/a0P0j6S35wEAACDQMIYOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAAOBwBDoAAACHI9ABAAA4HIEOAJKgU6dO0qxZs3uuq5CQEFmxYoXP61zvmarH1nvY3ovhw4dLxYoVU+y8AKQMAh2AgPbUU09Jw4YNE33tu+++MyFlz549Pj8vpylcuLBERUVJ2bJl/X0qALyAQAcgoHXp0kXWrl0rf/75Z4LX5syZI1WrVpXy5ctLsLhx48ZdvS9NmjSSL18+CQsLS/FzAuB/BDoAAa1JkyaSJ08emTt3rsf2y5cvy7Jly0zgU5988omUKVNGwsPDJTIyUiZOnOhR/vr169K/f3/TUqVlihUrJv/73//Ma7GxsWY/RYsWlQwZMkiJEiVkypQpiZ7PiBEjzPlkzZpVXnzxRY+ApcedPHmyR3ntntRuytvRc3rwwQclY8aMcv/998uQIUPk5s2bCbo4P/zwQ3N+6dOnl/nz50uuXLnM38mddgm3b98+SV2uGzduNM/XrVtnQrEev1atWnL48GGP940dO1YiIiIkS5Yspo6uXbuWYN96bqVKlTLnVrJkSZk+fbrrteeff94Ebvtctb4qVaokHTp0uG2dAEg+Ah2AgKYtSvrDXwOdZVmu7RrmNIi1bdtWdu7cKa1atZI2bdrI3r17TQjSYOQeAnUfixYtkqlTp8rBgwdl5syZkjlzZvNaXFycFCpUyOzzwIEDMnToUBk0aJAsXbrU41w0/Oh7NQzpvpYvX24C3r3QoKTnqcfVEPnBBx/IpEmTPMocPXrUBFY9ngayZ555xvzdP//8c1eZM2fOyKpVq0yASo4333zThN8dO3aYunZ/v/79tS7/+9//mtfz58/vEdbUggULTH2NHj3a1I2W1bqfN2+eeV3r+8qVKzJgwADX8aKjo+W99967q/oCcBsWAAS4gwcPapKzNmzY4NpWp04d67nnnjPfP/vss9bjjz/u8Z433njDKl26tPn+8OHD5v1r165N8jF79OhhtWzZ0vW8Y8eOVs6cOa0rV664tr3//vtW5syZrdjYWPO8SJEi1qRJkzz2U6FCBWvYsGGu53oen3766W2PO2HCBKtKlSqu5/retGnTWmfOnPEo99JLL1lPPvmk6/nEiROt+++/34qLi0t0v8eOHTPH3r17t3mudanPv/nmG1eZVatWmW1Xr141z2vWrGm9/PLLHvupXr26+TvZHnjgAWvhwoUeZUaOHGnea9u8ebP5OwwZMsQKCwuzvvvuu9v+/QHcHVroAAQ87cbT7sDZs2e7Wqx0QoTd3aotQw8//LDHe/T5kSNHTEuWtmrpGLK6deve9hjTpk2TKlWqmO5UbbmbNWuWHD9+3KNMhQoVTNekrWbNmqbr98SJE3f9d1uyZIk5Vx3fpscdPHhwguMWKVLEnJe7rl27ytdffy0nT540z7WVT2fiajdqcriPP9QWOLu1z67X6tWre5TXv7NNW95+/fVX8++g524/Ro0aZba7v6dv374ycuRIef3116V27drJOkcA/45AB8ARNDRot+OlS5fMZIgHHnjgjgHNnY6Lu5PFixebwKHH0JCkAbBz587JnoAQGhrq0S2s3MfDxbdlyxZp166dNGrUSFauXCm7d+82XZLxj5spU6YE79VxaBowdTyddjnv37/fBLrkSps2ret7OwxqF3RSaJhV2k2sdWY/9u3bJz/++KOrnO7vhx9+MKFawziAlEegA+AIOkZOA9PChQtNiNGxXnYA0QH5Ghjc6XOdbKAholy5ciZUfPvtt4nuW8tqC+DLL79sgpJOmHBvYbL9/PPPcvXqVddzDS3aIqUTLZS2ounSILaLFy/KsWPHbvt32rx5s2l90xCnExOKFy8uf/zxR5Lr5IUXXjAtcxpw69ev7zqPlKL1unXrVo9t7kFNJ0sUKFBAfvvtN1Nn7g+dwGGbMGGCHDp0yNT/mjVrzPkCSFkEOgCOoMGpdevWMnDgQBOa3FujtBtPJyxol94vv/xiBuTroHttdbNnn3bs2NGEQF3UV0OWTmywJz1okNJB/1999ZV5vw7q3759e4Jz0JYzbcXTCQyrV6+WYcOGSc+ePU3QVP/5z3/ko48+Mt3BOjlDj6mB8nb0uNq9qi2EGiB1AsGnn36a5Dp59tlnzXIu2kKW3MkQSdGrVy/Tza0BTOtF/77aEuhOJ4WMGTPGnLuW0b+3ln/nnXfM69rqqJMmdCasdi3rdt2vhkAAKegux94BgM/p4Hr92GrUqFGC1z7++GMzCUIH3993331mcoE7Hej/2muvWfnz57fSpUtnFStWzJo9e7Z57dq1a1anTp2sbNmyWdmzZzcTDgYMGOAx+F8nRTRt2tQaOnSolStXLjMZomvXrua9tpiYGKt169ZW1qxZrcKFC1tz587910kROnnD3p++VydV6HnY9L3u5xFf+/btzWQN9/NIzqSICxcuuMroa7pNy9pGjx5t5c6d25yf1kG/fv0SnM+CBQusihUrmnrNkSOH9cgjj1jLly83da7/Jt26dfMo//TTT1u1atWybt26dcdzBpB0IfpHSgZEAIDv1KtXz6y/py1kAIIXgQ4AHOjChQum2/j//J//Y7qAdTFkAMGLe8AAgAPp5A0NdePGjSPMAaCFDgAAwOmY5QoAAOBwBDoAAACHI9ABAAA4HIEOAADA4Qh0AAAADkegAwAAcDgCHQAAgMMR6AAAAByOQAcAACDO9n8BLG2d7P9PezAAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_scores_bar(\n", " next_token_probas, arrow=False, ylabel=\"Probability value\"\n", ")" ] }, { "cell_type": "code", "execution_count": 16, "id": "802fce1a-50ab-4836-a131-eac1683552ee", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Token ID 19,846 probability: tensor([0.0003], dtype=torch.bfloat16)\n" ] } ], "source": [ "print(\"Token ID 19,846 probability:\", next_token_probas[:, 19846])" ] }, { "cell_type": "code", "execution_count": 17, "id": "8421674d-b69b-4749-a2f3-464dd3d2b7a3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Highest probability: tensor(0.0003, dtype=torch.bfloat16)\n" ] } ], "source": [ "print(\"Highest probability:\", max(next_token_probas.squeeze(0)))" ] }, { "cell_type": "code", "execution_count": 18, "id": "7f9f5b82-0d48-4f2d-a3c7-b9dc7c9d541e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sampled token: tensor([[65094]])\n" ] } ], "source": [ "# Step 3.4: Sample token according to probabilities\n", "torch.manual_seed(123)\n", "print(\n", " \"Sampled token:\",\n", " torch.multinomial(next_token_probas.cpu(), num_samples=1)\n", ")" ] }, { "cell_type": "code", "execution_count": 19, "id": "243eeba0-82fc-44b2-a09c-41174e0169f8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " mistress\n" ] } ], "source": [ "print(tokenizer.decode([65094]))" ] }, { "cell_type": "code", "execution_count": 20, "id": "b3e0bf3e-4647-4bff-b9a9-44e40209c6c9", "metadata": {}, "outputs": [], "source": [ "def count_samples(probas, num_samples=1000, threshold=1, tokenizer=None):\n", " # Draw samples according to probabilities\n", " samples = torch.multinomial(\n", " probas.cpu(), num_samples=num_samples, replacement=True\n", " )\n", " \n", " # Count how often each index was selected\n", " counts = torch.bincount(samples.squeeze(0), minlength=1)\n", " \n", " # Print results\n", " for i, c in enumerate(counts):\n", " if c > threshold:\n", " if tokenizer is None:\n", " print(f\"Vocab index {i}: {c.item()}x\")\n", " else:\n", " print(f\"'{tokenizer.decode([i])}': {c.item()}x\")" ] }, { "cell_type": "code", "execution_count": 21, "id": "eb8cadb2-d1f9-4a47-bfaa-7a91a3306c50", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "'}': 2x\n", "' " ] }, { "cell_type": "markdown", "id": "607316ee-5e8b-4f11-addb-ae747e1e2970", "metadata": {}, "source": [ " \n", "### 4.5.1 Selecting a subset of top-p tokens" ] }, { "cell_type": "code", "execution_count": 26, "id": "049a584b-0dbc-4fa6-b424-1c8e4a13c4d3", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAG2CAYAAACDLKdOAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAJ2JJREFUeJzt3Ql8TPf+//GPWGKPJSRoKqULLoIoDdVVS5XW7UJpLSnpQ9EqpRVLUNVQpaqCa217yxWtpb24VFNKK712tavSyk+tRRAkyPwfn+/jP3MzkiiRZCbfvJ6Px7nmnDln5jvH3M7bdy3gcDgcAgAAYAkfTxcAAAAgOxFuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVPBpu1qxZI23atJHKlStLgQIFZPHixX95zerVq6VBgwbi6+srd955p3zyySe5UlYAAJA3eDTcJCUlSUhIiMTExNzQ+QcPHpQnn3xSHn74Ydm6dau88cYb0r17d1mxYkWOlxUAAOQNBbxl4UytuVm0aJG0bds203PefvttWbp0qezYscN17IUXXpAzZ87I8uXLc6mkAADAmxWSPCQ+Pl6aN2/udqxFixamBiczycnJZnNKTU2VU6dOSfny5U2gAgAA3k/rYs6dO2e6svj4+NgTbo4ePSoBAQFux3T/7NmzcvHiRSlWrFi6a6Kjo2XEiBG5WEoAAJBTEhIS5LbbbrMn3GRFZGSk9OvXz7WfmJgot99+u7k5pUuX9mjZAADAjdGKjKCgIClVqtRfnpunwk1gYKAcO3bM7Zjua0jJqNZG6agq3a6l1xBuAADIW26kS0memucmLCxM4uLi3I6tXLnSHAcAAPB4uDl//rwZ0q2bc6i3Pj506JCrSalz586u83v06CEHDhyQt956S/bs2SOTJ0+W+fPnS9++fT32GQAAgHfxaLjZuHGj1K9f32xK+8bo46ioKLN/5MgRV9BRd9xxhxkKrrU1Oj/OuHHjZMaMGWbEFAAAgFfNc5ObHZL8/PxMx2L63AAAYN/vd57qcwMAAPBXCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAVvF4uImJiZHg4GApWrSoNG7cWNavX3/d8ydMmCD33HOPFCtWTIKCgqRv375y6dKlXCsvAADwbh4NN7GxsdKvXz8ZNmyYbN68WUJCQqRFixZy/PjxDM+fO3euDBw40Jy/e/dumTlzpnmNQYMG5XrZAQCAd/JouBk/frxERERIeHi41KpVS6ZOnSrFixeXWbNmZXj+unXrpGnTptKxY0dT2/P4449Lhw4d/rK2BwAA5B8eCzcpKSmyadMmad68+f8K4+Nj9uPj4zO8pkmTJuYaZ5g5cOCALFu2TFq1apXp+yQnJ8vZs2fdNgAAYK9CnnrjkydPytWrVyUgIMDtuO7v2bMnw2u0xkavu//++8XhcMiVK1ekR48e122Wio6OlhEjRmR7+QEAgHfyeIfim7F69Wp57733ZPLkyaaPzsKFC2Xp0qUycuTITK+JjIyUxMRE15aQkJCrZQYAAPmk5sbf318KFiwox44dczuu+4GBgRleM3ToUOnUqZN0797d7NepU0eSkpLklVdekcGDB5tmrWv5+vqaDQAA5A8eq7kpUqSIhIaGSlxcnOtYamqq2Q8LC8vwmgsXLqQLMBqQlDZTAQAAeKzmRukw8C5dukjDhg2lUaNGZg4brYnR0VOqc+fOUqVKFdNvRrVp08aMsKpfv76ZE2f//v2mNkePO0MOAADI3zwabtq3by8nTpyQqKgoOXr0qNSrV0+WL1/u6mR86NAht5qaIUOGSIECBcyfhw8flgoVKphgM2rUKA9+CgAA4E0KOPJZe44OBffz8zOdi0uXLu3p4gAAgGz+/c5To6UAAAD+CuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCoeDzcxMTESHBwsRYsWlcaNG8v69euve/6ZM2ekV69eUqlSJfH19ZW7775bli1blmvlBQAA3q2QJ988NjZW+vXrJ1OnTjXBZsKECdKiRQvZu3evVKxYMd35KSkp8thjj5nnvvzyS6lSpYr8/vvvUqZMGY+UHwAAeJ8CDofD4ak310Bz7733yqRJk8x+amqqBAUFyWuvvSYDBw5Md76GoLFjx8qePXukcOHCWXrPs2fPip+fnyQmJkrp0qVv+TMAAICcdzO/3x5rltJamE2bNknz5s3/VxgfH7MfHx+f4TVff/21hIWFmWapgIAAqV27trz33nty9erVTN8nOTnZ3JC0GwAAsJfHws3JkydNKNGQkpbuHz16NMNrDhw4YJqj9DrtZzN06FAZN26cvPvuu5m+T3R0tEl6zk1rhgAAgL083qH4Zmizlfa3mTZtmoSGhkr79u1l8ODBprkqM5GRkaYKy7klJCTkapkBAEA+6VDs7+8vBQsWlGPHjrkd1/3AwMAMr9ERUtrXRq9zqlmzpqnp0WauIkWKpLtGR1TpBgAA8ocs1dysWrXqlt9Yg4jWvsTFxbnVzOi+9qvJSNOmTWX//v3mPKd9+/aZ0JNRsAEAAPlPlsJNy5YtpXr16qavy6008+gw8OnTp8unn34qu3fvlldffVWSkpIkPDzcPN+5c2fTrOSkz586dUr69OljQs3SpUtNh2LtYAwAAJDlcHP48GHp3bu36dxbrVo1MzfN/PnzTdPQzdA+Mx988IFERUVJvXr1ZOvWrbJ8+XJXJ+NDhw7JkSNHXOdrZ+AVK1bIhg0bpG7duvL666+boJPRsHEAAJA/3fI8N5s3b5bZs2fLv/71L7PfsWNH6datm4SEhIg3Yp4bAADynlyd56ZBgwam6Uhrcs6fPy+zZs0yfWmaNWsmO3fuvNWXBwAAuClZDjeXL182zVKtWrWSqlWrmuYinWlYRztpp1899vzzz2f15QEAAHKvWUqXR9BmKL20U6dO0r17dzNbcFo6PLty5cpuI5u8Ac1SAADkPTfz+52leW527dolH3/8sTzzzDOZziGj89hkx5BxAACAHG+WGjZsmGlyujbYXLlyRdasWWMeFypUSB588MGsvDwAAEDuhpuHH37YzDdzLa0q0ucAAADyVLjRvjYFChRId/zPP/+UEiVKZEe5AAAAsuSm+txoHxulwaZr165uzVK6UvfPP/8sTZo0yVpJAAAAcjvcaC9lZ81NqVKlpFixYq7ndG2n++67TyIiIrKjXAAAADkfbnQmYhUcHCz9+/enCQoAANi3/EJewzw3AADkPTkyz40usxAXFydly5aV+vXrZ9ihOO16UwAAAJ5ww+Hm6aefdnUgbtu2bU6WCQAAIMtolgIAAF4vV1cFBwAAyJPNUtrX5nr9bNLKaPZiAAAArwo3EyZMyNmSAAAA5Ga46dKlS3a8HwAAgHeEG+3I4+zAo4+v5686+gAAAHhFn5sjR45IxYoVpUyZMhn2v3EuqKnrTAEAAHh1uPnuu++kXLly5vGqVatyskwAAABZxjw3AAAgfy6/cK3Tp0/LzJkzZffu3Wa/Vq1aEh4e7qrdAQAA8IQsTeK3Zs0aszL4xIkTTcjRTR/fcccd5jkAAIA81SxVp04dCQsLkylTpkjBggXNMe1E3LNnT1m3bp1s375dvBWrggMAkPfk+PIL+/fvlzfffNMVbJQ+7tevn3kOAADAU7IUbho0aODqa5OWHgsJCcmOcgEAAGTJDXco/vnnn12PX3/9denTp4+ppbnvvvvMsZ9++kliYmJk9OjRWSsJAABAbva58fHxMRP0/dXp3j6JH31uAADIe3JkKPjBgwezo2wAAAA56obDTdWqVXO2JAAAANkgy5P4qV27dsmhQ4ckJSXF7fhTTz11q+UCAADIvXBz4MAB+fvf/27ms0nbD8e5mKY397kBAAB2y9JQcB0ppbMRHz9+XIoXLy47d+40MxM3bNhQVq9enf2lBAAAyMmam/j4eLNKuL+/vxlFpdv9998v0dHRZpj4li1bsvKyAAAAnqm50WanUqVKmccacP744w9Xp+O9e/feeqkAAABys+amdu3asm3bNtM01bhxY3n//felSJEiMm3aNKlWrVpWywIAAOCZcDNkyBBJSkoyj9955x1p3bq1NGvWTMqXLy+xsbG3XioAAIDcXBU8I6dOnZKyZcu6Rkx5K2YoBgAg78mRGYozk5CQYP4MCgq61ZcCAADwTIfiK1euyNChQ02CCg4ONps+1uaqy5cv33qpAAAAsihLNTevvfaaLFy40HQkDgsLcw0PHz58uPz5558yZcqUrJYHAAAg9/vcaC3NvHnz5IknnnA7vmzZMunQoYNpD/NW9LkBACDvuZnf7yw1S/n6+pqmqGvp0HAdEg4AAOApWQo3vXv3lpEjR0pycrLrmD4eNWqUeQ4AAMDr+9w888wzbvvffvut3HbbbRISEmL2dVI/XR380Ucfzf5SAgAAZHe40XautJ599lm3fYaCAwCAPBVuZs+enbMlAeD1Ply5T7xN38fu9nQRAHiZW5rE78SJE66FMu+55x6pUKFCdpULAAAg9zoU67pSL7/8slSqVEkeeOABs1WuXFm6desmFy5cyFpJAAAAPBVu+vXrJ99//738+9//ljNnzpjtq6++MsfefPPN7CgXAABA7jVLLViwQL788kt56KGHXMdatWolxYoVk3bt2jFDMQAAyFs1N9r0FBAQkO54xYoVaZYCAAB5L9zoelLDhg2TS5cuuY5dvHhRRowY4VprCgAAIM80S02YMEFatmyZbhK/okWLyooVK7K7jAAAADkbburUqSO//PKLzJkzR/bs2WOO6YKZL774oul3AwAAkGfCzeXLl6VGjRqyZMkSiYiIyJlSAQAA5Fafm8KFC7v1tQEAAMjzHYp79eolY8aMkStXrmR/iQAAAHK7z82GDRskLi5OvvnmG9P/pkSJEm7PL1y48FbKBAAAkLvhpkyZMulWBQcAAPAGNxVuUlNTZezYsbJv3z5JSUmRRx55RIYPH84IKQAAkDf73IwaNUoGDRokJUuWlCpVqsjEiRNN/xsAAIA8GW4+++wzmTx5spmob/HixWbhTJ3rRmt0bkVMTIwEBwebSQAbN24s69evv6Hr5s2bJwUKFJC2bdve0vsDAIB8Gm4OHTpkFsh0at68uQkXf/zxR5YLEBsba1YZ1+UcNm/ebGY8btGihRw/fvy61/3222/Sv39/adasWZbfGwAA5PNwo0O/tXbl2nlvdGK/rBo/fryZDDA8PFxq1aolU6dOleLFi8usWbMyvebq1atmNmRdy6patWpZfm8AAJDPOxQ7HA7p2rWr+Pr6uo7phH49evRwGw5+o0PBtVPypk2bJDIy0nXMx8fH1AjFx8dnet0777xjViDv1q2brF279rrvkZycbDans2fP3lDZAABAPgg3Xbp0SXfspZdeyvKbnzx50tTCBAQEuB3XfeeaVdf64YcfZObMmbJ169Ybeo/o6GhTwwMAAPKHmwo3s2fPFk86d+6cdOrUSaZPny7+/v43dI3WCmmfnrQ1N0FBQTlYSgAAkOcm8csuGlAKFiwox44dczuu+4GBgenO//XXX01H4jZt2riOOUdqFSpUSPbu3SvVq1d3u0ab0NI2owEAALtlaW2p7FKkSBEJDQ01SzmkDSu6HxYWlu58XY18+/btpknKuT311FPy8MMPm8fUyAAAAI/W3ChtMtK+PA0bNpRGjRrJhAkTJCkpyYyeUp07dzYTBmrfGR2pVbt27XRLQahrjwMAgPzJ4+Gmffv2cuLECYmKipKjR49KvXr1ZPny5a5Oxjq3jo6gAgAAuBEFHDq+Ox/RDsV+fn6SmJgopUuX9nRxgDzlw5X7xNv0fexuTxcBgJf9flMlAgAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALBKIU8XAAAAZN2HK/d53e3r+9jdHn1/am4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABW8YpwExMTI8HBwVK0aFFp3LixrF+/PtNzp0+fLs2aNZOyZcuarXnz5tc9HwAA5C8eDzexsbHSr18/GTZsmGzevFlCQkKkRYsWcvz48QzPX716tXTo0EFWrVol8fHxEhQUJI8//rgcPnw418sOAAC8j8fDzfjx4yUiIkLCw8OlVq1aMnXqVClevLjMmjUrw/PnzJkjPXv2lHr16kmNGjVkxowZkpqaKnFxcbledgAA4H08Gm5SUlJk06ZNpmnJVSAfH7OvtTI34sKFC3L58mUpV65chs8nJyfL2bNn3TYAAGAvj4abkydPytWrVyUgIMDtuO4fPXr0hl7j7bfflsqVK7sFpLSio6PFz8/PtWkzFgAAsJfHm6VuxejRo2XevHmyaNEi0xk5I5GRkZKYmOjaEhIScr2cAAAg9xQSD/L395eCBQvKsWPH3I7rfmBg4HWv/eCDD0y4+fbbb6Vu3bqZnufr62s2AACQP3i05qZIkSISGhrq1hnY2Tk4LCws0+vef/99GTlypCxfvlwaNmyYS6UFAAB5gUdrbpQOA+/SpYsJKY0aNZIJEyZIUlKSGT2lOnfuLFWqVDF9Z9SYMWMkKipK5s6da+bGcfbNKVmypNkAAED+5vFw0759ezlx4oQJLBpUdIi31sg4OxkfOnTIjKBymjJlihll9dxzz7m9js6TM3z48FwvPwAA8C4eDzeqd+/eZsts0r60fvvtt1wqFQAAyIvy9GgpAACAaxFuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGCVQp4uAAAA3uDDlfvE2/R97G5PFyFPouYGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVGC2FPI3RDQCAa1FzAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWYfkFAPBSLC8CZA01NwAAwCrU3GQz/qUFAIBnUXMDAACsQrgBAABWoVkKgPVoLgbyF2puAACAVai5ATyAmgQAyDnU3AAAAKsQbgAAgFW8ItzExMRIcHCwFC1aVBo3bizr16+/7vlffPGF1KhRw5xfp04dWbZsWa6VFQAAeDeP97mJjY2Vfv36ydSpU02wmTBhgrRo0UL27t0rFStWTHf+unXrpEOHDhIdHS2tW7eWuXPnStu2bWXz5s1Su3Ztj3wGAMD/0KcMkt9rbsaPHy8RERESHh4utWrVMiGnePHiMmvWrAzP/+ijj6Rly5YyYMAAqVmzpowcOVIaNGggkyZNyvWyAwAA7+PRmpuUlBTZtGmTREZGuo75+PhI8+bNJT4+PsNr9LjW9KSlNT2LFy/O8Pzk5GSzOSUmJpo/z549KznhUtJ58TY38lljvtsv3qbXI3dae78pN/eb70nu4n7nrpz4jXW+psPh+OuTHR50+PBhLaFj3bp1bscHDBjgaNSoUYbXFC5c2DF37ly3YzExMY6KFStmeP6wYcPMe7BxD/gO8B3gO8B3gO+A5Pl7kJCQ8Jf5wuN9bnKa1gqlrelJTU2VU6dOSfny5aVAgQLijTSdBgUFSUJCgpQuXdrTxbEe95v7bTO+39xvW2iNzblz56Ry5cp/ea5Hw42/v78ULFhQjh075nZc9wMDAzO8Ro/fzPm+vr5mS6tMmTKSF2iwIdxwv23F95v7bTO+3znDz8/P+zsUFylSREJDQyUuLs6tZkX3w8LCMrxGj6c9X61cuTLT8wEAQP7i8WYpbTLq0qWLNGzYUBo1amSGgiclJZnRU6pz585SpUoVM/Rb9enTRx588EEZN26cPPnkkzJv3jzZuHGjTJs2zcOfBAAAeAOPh5v27dvLiRMnJCoqSo4ePSr16tWT5cuXS0BAgHn+0KFDZgSVU5MmTczcNkOGDJFBgwbJXXfdZUZK2TTHjTajDRs2LF1zGrjfNuD7zf22Gd9v71BAexV7uhAAAADWTOIHAACQnQg3AADAKoQbAABgFcINAACwCuHGC8XExEhwcLAULVrUrJS+fv16TxfJSjq9wL333iulSpUyK9Dr6vK6Gj1yx+jRo80s4W+88Qa3PIccPnxYXnrpJTMje7FixaROnTpm6gxkv6tXr8rQoUPljjvuMPe6evXqZmFnxux4BuHGy8TGxpq5f3Qo+ObNmyUkJMQsDHr8+HFPF80633//vfTq1Ut++uknMxHk5cuX5fHHHzfzLCFnbdiwQf7xj39I3bp1udU55PTp09K0aVMpXLiw/Oc//5Fdu3aZ+cHKli3LPc8BY8aMkSlTpsikSZNk9+7dZv/999+Xjz/+mPvtAQwF9zJaU6O1Cfp/EOeMzbrO1GuvvSYDBw70dPGspvMtaQ2Ohp4HHnjA08Wx1vnz56VBgwYyefJkeffdd83cVjp5J7KX/vfixx9/lLVr13Jrc0Hr1q3N/GwzZ850HXv22WdNLc7nn3/O30Euo+bGi6SkpMimTZukefPmrmM6gaHux8fHe7Rs+UFiYqL5s1y5cp4uitW0tkxnF0/7PUf2+/rrr83M788//7wJ7fXr15fp06dzq3OITjCrSwPt27fP7G/btk1++OEHeeKJJ7jn+XGGYvzPyZMnTbutc3ZmJ93fs2cPtyoHaQ2Z9v3QanybZrv2Nrpcija3arMUctaBAwdMM4k2c+ts7nrPX3/9dbOmny55g+yvKdMV2GvUqGEWhNb/lo8aNUpefPFFbrUHEG6A/1+bsGPHDvMvLeSMhIQEszac9m/SzvLI+cCuNTfvvfee2deaG/2OT506lXCTA+bPny9z5swxywP97W9/k61bt5p/MFWuXJn77QGEGy/i7+9vEv+xY8fcjut+YGCgx8plu969e8uSJUtkzZo1ctttt3m6ONbSJlftGK/9bZz0X7d637WPWXJysvn+I3tUqlRJatWq5XasZs2asmDBAm5xDhgwYICpvXnhhRfMvo5M+/33382oTGrKch99bryIVheHhoaadtu0//rS/bCwMI+WzUY6RFODzaJFi+S7774zQziRcx599FHZvn27+Retc9OaBa2218cEm+ylTazXTm2g/UGqVq2aze8EdeHCBbdFnpV+p/W/4ch91Nx4GW0f15Sv/9Fv1KiRGUWiQ5PDw8M9XTTraFOUViF/9dVXZq4bXZVe+fn5mREOyF56j6/tz1SiRAkzBwv9nLJf3759TSdXbZZq166dmS9r2rRpZkP2a9Omjeljc/vtt5tmqS1btsj48ePl5Zdf5nZ7AEPBvZBW0Y8dO9b82Oow2YkTJ5oh4sheOoFcRmbPni1du3bldueChx56iKHgOUibWyMjI+WXX34xNZP6j6eIiIicfMt869y5c2YSP60J1uZX7WvToUMHiYqKMrXyyF2EGwAAYBX63AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AZArdGLEtm3bZsvki4sXL5bc9ttvv5n31qUibsXw4cPNxIUAcg7hBsgH08K3bNkyw+fWrl1rfrB//vnnXC9XXhMUFCRHjhxhqQggDyDcAJbr1q2brFy5Uv7v//4vw6UmdB2zunXrSn6RkpKSpet0EcTAwEApVIgl+QBvR7gBLNe6dWupUKGCfPLJJ27Hz58/L1988YUJP2rBggVmwT9fX18JDg6WcePGuZ2fnJwsb7/9tqnB0HPuvPNOmTlzpnnu6tWr5nV0/SJddPSee+6Rjz76KMPyjBgxwpSndOnS0qNHD7ewoe+ri8WmpU042pSTGS3T3XffLcWLF5dq1aqZ9X0uX76crhloxowZpnxFixaVzz77zCzYqZ8pLW0269Sp0w01S61evdrsx8XFmYCo768LVV67Evfo0aMlICDALByq9+jSpUvpXlvLVrNmTVO2GjVqyOTJk13P6cKLGj6dZdX7Vb9+fencuXOm9wTI9xwArDdgwABH9erVHampqa5js2bNchQrVsxx5swZx8aNGx0+Pj6Od955x7F3717H7NmzzXP6p1O7du0cQUFBjoULFzp+/fVXx7fffuuYN2+eeS4lJcURFRXl2LBhg+PAgQOOzz//3FG8eHFHbGys6/ouXbo4SpYs6Wjfvr1jx44djiVLljgqVKjgGDRokOucqlWrOj788EO3soeEhDiGDRvm2hcRx6JFi1z7I0eOdPz444+OgwcPOr7++mtHQECAY8yYMa7n9doSJUo4WrZs6di8ebNj27ZtjgsXLjj8/Pwc8+fPd5137NgxR6FChRzfffddhvdQX1/fe8uWLWZ/1apVZr9x48aO1atXO3bu3Olo1qyZo0mTJq5r9PP7+vo6ZsyY4dizZ49j8ODBjlKlSpnP5KT3qlKlSo4FCxaYe6d/litXzvHJJ5+Y58+dO+eoVq2a44033jD7/fv3dwQHBzsSExNv4G8eyJ8IN0A+sHv3bvNDrD/ITvpD/NJLL5nHHTt2dDz22GPpAlGtWrXMYw08ev3KlStv+D179erlePbZZ93Cjf5oJyUluY5NmTLFBJ6rV69mOdxca+zYsY7Q0FDXvl5buHBhx/Hjx93Oe/XVVx1PPPGEa3/cuHEmRKQNgDcSbjTkOS1dutQcu3jxotkPCwtz9OzZ0+11NAylDTcaOufOnet2jgY2vdZp3bp15jMMHTrUBLC1a9dm+vkBOBw0SwH5gDZ1aJPJrFmzzP7+/ftNZ2Jnk9Tu3buladOmbtfo/i+//GKanLQpRvucPPjgg5m+R0xMjISGhpomp5IlS8q0adPk0KFDbueEhISY5hunsLAw0zyWkJCQ5c8WGxtryqr9YfR9hwwZku59q1atasqVVkREhHzzzTdy+PBhs6/NdjqiS5uabkba/kqVKlUyfx4/ftx1Xxs3bux2vn5mp6SkJPn111/N34OW3bm9++675njaa/r37y8jR46UN998U+6///6bKiOQ3xBugHxCf0C1X825c+dMR+Lq1atfN6ykpf1ormfevHnmx1ffQwODhqHw8PCb7rzr4+Ojtclux9L2n7lWfHy8vPjii9KqVStZsmSJbNmyRQYPHpzufUuUKJHuWu23omFL+99s2rRJdu7cacLNzSpcuLDrsTMYpaam3tC1GuzU9OnTzT1zbjt27JCffvrJdZ6+3o8//mgCpgZTANdHuAHyiXbt2pnwMHfuXPODrh1VnT/G2plVfzzT0n3tqKs/qHXq1DE/sN9//32Gr63nas1Qz549TWjQzsZpax6ctm3bJhcvXnTt6w+41lRoJ2WltSs63Nrp7NmzcvDgwUw/07p160ytjAYa7dR71113ye+//37D96R79+6mxkbDXvPmzV3lyC56X//73/+6HUsbWrSjceXKleXAgQPmnqXdtPOz09ixY2XPnj3m/i9fvtyUF0DmCDdAPqEhon379hIZGWkCRNpaCm3q0FE/2uyxb98++fTTT2XSpEmmNsY5iqlLly4mEOkEeho4dLTQ/PnzzfMaKjZu3CgrVqww1+uIpQ0bNqQrg9aoaO3Orl27ZNmyZTJs2DDp3bu3CV3qkUcekX/+85+myWz79u3mPTVcZUbfV5ugtOZIw9TEiRNl0aJFN3xPOnbsaIbIa82Jfrbs1qdPH9MUqGFE74t+Xq0hunb0WHR0tCm7nqOfW88fP368eV5ro6KiosyIKm1+0+P6uhqIAGSCjkdA/qEdU/X/9q1atUr33Jdffmk6EGvH1dtvv910zE1LO8n27dvXjOwpUqSI48477zQjrtSlS5ccXbt2NSOQypQpYzrrDhw40K3jrHYofvrpp82oqvLly5uOxBEREeZaJx0BpKOpSpcubUZm6Yihv+pQrB2fna+n12qHZC2Hk16bthzX6tSpk+nonLYcN9Oh+PTp065z9Dk9puc6jRo1yuHv72/Kp/fgrbfeSleeOXPmOOrVq2fua9myZR0PPPCAGZWm91z/Tl555RW385966ikzKuvKlSvXLTOQXxXQ/8ks+ACA7R599FEzv4/WnACwA+EGQL50+vRp07T23HPPmWYynXgQgB2YRxxAvqQdnzXgjBkzhmADWIaaGwAAYBVGSwEAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAscn/AxcuOX9lVwaHAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Step 3.1: Get logits (here: use toy logits for 10 tokens)\n", "toy_logits = torch.tensor(\n", " [-0.7, -3.0, 0.1, -1.2, 2.0, -1.0, -0.5, -2.0, 0.3, 1.5]\n", ")\n", "\n", "# Step 3.2: Apply temperature scaling\n", "toy_logits_scaled = scale_logits_by_temperature(toy_logits, 1.0)\n", "\n", "# Step 3.3: Convert to probabilities\n", "toy_probas = torch.softmax(toy_logits_scaled, dim=-1)\n", "\n", "plt.bar(\n", " torch.arange(len(toy_logits_scaled)), toy_probas,\n", " alpha=0.5\n", ")\n", "\n", "plt.ylim([0, 1])\n", "plt.xlabel(\"Vocabulary index\")\n", "plt.ylabel(\"Probability\")\n", "# plt.savefig(\"12.pdf\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 27, "id": "1fcbc146-0feb-42cf-a87e-8991e1a0b3f3", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAG2CAYAAACDLKdOAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALWZJREFUeJzt3Ql4VNX5x/E3CSQQZDUmMZAal8pS2RGISqmVGkVQ1FrEKkg1/VMRLFGqQQQUWUTBKEYoKNhWkFgV64IoRRCVKAguKJuImpQdEQJBkwD3/7ynznQmG1nnzpx8P88zOvfm3pkzd4bML2cNcxzHEQAAAEuEu10AAACAmkS4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWcTXcrFq1Svr37y8JCQkSFhYmL7/88knPWblypXTp0kWioqLknHPOkWeeeSYgZQUAAKHB1XCTn58vHTt2lMzMzAod//XXX8sVV1whF198sXzyySfy5z//WW699VZ58803a72sAAAgNIQFy8KZWnOzePFiGTBgQJnH3H333fL666/L559/7t13/fXXy8GDB2Xp0qUBKikAAAhm9SSEZGdnS58+ffz2paSkmBqcshQUFJibx4kTJ+TAgQNy6qmnmkAFAACCn9bFHD582HRlCQ8Ptyfc7N69W+Li4vz26XZeXp788MMP0rBhwxLnTJkyRe6///4AlhIAANSW3NxcadWqlT3hpirS09MlLS3Nu33o0CH52c9+Zi5OkyZNXC0bACDACvNFprcO3cse207kxpe0L0ftPH796Np77GrSiozExERp3LjxSY8NqXATHx8ve/bs8dun2xpSSqu1UTqqSm/F6TmEGwCoBu2yWXQ0tC5heD2RqJ++vO/aJhIZLSEliMNHoFSkS0lIhZvk5GRZsmSJ375ly5aZ/QCAAAebeSkiuR+G7mXXYBPZyO1SwLah4EeOHDFDuvXmGeqt93NycrxNSoMHD/YeP2zYMNm+fbv85S9/kc2bN8uTTz4pzz//vIwaNcq11wAAdZLW2IRysEns+d9aEFjJ1Zqbjz76yMxZ4+HpGzNkyBAzOd+uXbu8QUedeeaZZii4hpnHHnvMdCh66qmnzIgpAIBLaN5BkAmaeW4C2SGpadOmpmMxfW4AoBodcycn/Pf+mJ007yCovr9ZWwoAAFiFcAMAAKwSUqOlAMBKoTikujDEyos6hXADAG6yYUg1EGRolgIANzGkGqhx1NwAQLBgSDVQIwg3ABAsmDEXqBE0SwEAAKsQbgAAgFUINwAAwCqEGwAAYBU6FAOwB5PhASDcALAGk+EB+AnNUgDswGR4AH5CsxQA+zAZHlCnEW4A2IfJ8IA6jWYpAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrMBQcQEksYwAghBFuAPhjGQMAIY5mKQD+WMYAQIij5gZA2VjGAEAIItwAKBvLGAAIQTRLAQAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrMEMxUJtYXRsAAo5wA9QWVtcGAFfQLAXUFlbXBgBXUHMDBAKrawNAwBBugEBgdW0ACBiapQAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALBKPbcLAFSI44gUHQ2ti1UYYuUFAEsQbhAawWZeikjuh26XBAAQAlxvlsrMzJSkpCRp0KCB9OjRQ9asWVPu8RkZGdK6dWtp2LChJCYmyqhRo+THH38MWHnhAq2xCeVgk9hTpH6026UAgDrD1ZqbrKwsSUtLk9mzZ5tgo8ElJSVFtmzZIrGxsSWOX7hwodxzzz0yb948ueCCC2Tr1q1y8803S1hYmMyYMcOV14AAu2ubSGSIBQUNNmFhbpcCAOoMV8ONBpLU1FQZOnSo2daQ8/rrr5vwoiGmuNWrV8uFF14oN9xwg9nWGp9BgwbJhx+G8F/1qBwNNpGNuGoAgOBrliosLJR169ZJnz59/leY8HCznZ2dXeo5Wluj53iarrZv3y5LliyRvn37lvk8BQUFkpeX53cDAAD2cq3mZv/+/XL8+HGJi4vz26/bmzdvLvUcrbHR8y666CJxHEeOHTsmw4YNkzFjxpT5PFOmTJH777+/xssPAACCk+sdiitj5cqVMnnyZHnyySdl/fr18tJLL5lmrIkTJ5Z5Tnp6uhw6dMh7y83NDWiZAQBAHam5iYmJkYiICNmzZ4/fft2Oj48v9Zz77rtPbrrpJrn11lvNdvv27SU/P1/++Mc/yr333muatYqLiooyNwAAUDe4VnMTGRkpXbt2leXLl3v3nThxwmwnJyeXes7Ro0dLBBgNSEqbqQAAAFwdLaXDwIcMGSLdunWT7t27m6HgWhPjGT01ePBgadmypek3o/r3729GWHXu3NkMHd+2bZupzdH9npADAADqNlfDzcCBA2Xfvn0ybtw42b17t3Tq1EmWLl3q7WSck5PjV1MzduxYM6eN/n/Hjh1y2mmnmWAzadIkF18FAAAIJmFOHWvP0aHgTZs2NZ2LmzRp4nZxUBGF+SKTE/57f8xO5rkBgDoorxLf3yE1WgoAAOBkCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVeq5XQAEmOOIFB0NrcteGGLlBQC4inBT14LNvBSR3A/dLgkAALWGZqm6RGtsQjnYJPYUqR/tdikAAEGOmpu66q5tIpEhFhQ02ISFuV0KAECQI9zUVRpsIhu5XQoAAGoczVIAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKq6Hm8zMTElKSpIGDRpIjx49ZM2aNeUef/DgQRk+fLicfvrpEhUVJeeee64sWbIkYOUFAADBrZ6bT56VlSVpaWkye/ZsE2wyMjIkJSVFtmzZIrGxsSWOLywslN/85jfmZy+88IK0bNlSvv32W2nWrJkr5QcAAMHH1XAzY8YMSU1NlaFDh5ptDTmvv/66zJs3T+65554Sx+v+AwcOyOrVq6V+/fpmn9b6AAAAuN4spbUw69atkz59+vyvMOHhZjs7O7vUc1555RVJTk42zVJxcXFy3nnnyeTJk+X48eNlPk9BQYHk5eX53QAAgL1cCzf79+83oURDii/d3r17d6nnbN++3TRH6Xnaz+a+++6T6dOny4MPPljm80yZMkWaNm3qvSUmJtb4awEAAMHD9Q7FlXHixAnT32bOnDnStWtXGThwoNx7772mOass6enpcujQIe8tNzc3oGUGAAB1pM9NTEyMREREyJ49e/z263Z8fHyp5+gIKe1ro+d5tG3b1tT0aDNXZGRkiXN0RJXeAABA3VClmpsVK1ZU+4k1iGjty/Lly/1qZnRb+9WU5sILL5Rt27aZ4zy2bt1qQk9pwQYAANQ9VQo3l112mZx99tmmr0t1mnl0GPjcuXPlb3/7m2zatEn+9Kc/SX5+vnf01ODBg02zkof+XEdL3XHHHSbU6Mgq7VCsHYwBAACqHG527Nght99+u+nce9ZZZ5m5aZ5//nnTNFQZ2mfmkUcekXHjxkmnTp3kk08+kaVLl3o7Gefk5MiuXbu8x2tn4DfffFPWrl0rHTp0kJEjR5qgU9qwcQAAUDeFOY7jVOcB1q9fL/Pnz5fnnnvObN9www1yyy23SMeOHSUY6VBwHTWlnYubNGkidUphvsjkhP/eH7NTJLKR2yUCAKDGv7+rPVqqS5cupulIa3KOHDliJtrTvjS9evWSL774oroPDwAAUClVDjdFRUWmWapv375yxhlnmOaiJ554wox20k6/uu+6666r6sMDAAAEbij4iBEjTDOUtmjddNNNMm3aNDNbsEejRo1MX5qEhJ+aQAAAAII53GzcuFFmzpwp11xzTZlzyOg8NjUxZBwAAKDWm6XGjx9vmpyKB5tjx47JqlWrzP169epJ7969q/LwAAAAgQ03F198sZlvpjjtwaw/AwAACKlwo31twsLCSuz/7rvvTH8bAACAkOhzo31slAabm2++2a9ZSlfq/uyzz+SCCy6o+VICAADURrjRyXM8NTeNGzeWhg0ben+mazv17NlTUlNTK/OQAAAA7oUbnYlYJSUlyV133UUTFAAAsGMouI6WAgAACOlwo8ssLF++XJo3by6dO3cutUOx73pTAAAAQR1urrrqKm8H4gEDBtRmmQAAANxbFTzUsCo4q4IDAEJPQFcFBwAACMlmKe1rU14/G1+lzV4MAAAQVOEmIyOjdksCAAAQyHAzZMiQmng+AACA4Ag32pHH04FH75fnZB19AAAAgqLPza5duyQ2NlaaNWtWav8bz4Kaus4UAABAUIebt99+W1q0aGHur1ixojbLBAAAUPvhpnfv3qXeBwAACPm1pdT3338vTz/9tGzatMlst2vXToYOHeqt3QEAAHBDlSbxW7VqlVkZ/PHHHzchR296/8wzzzQ/AwAACKmam+HDh8vAgQNl1qxZEhERYfZpJ+LbbrvN/GzDhg01XU4AAIDaq7nZtm2b3Hnnnd5go/R+Wlqa+RkAAEBIhZsuXbp4+9r40n0dO3asiXIBAADUbrPUZ5995r0/cuRIueOOO0wtTc+ePc2+Dz74QDIzM2Xq1KlVKwkAAEANCHN05r0KCA8PNxP0nezwYJ/ErzJLplunMF9kcsJ/74/ZKRLZyO0SAQBQ49/fFa65+frrryt6KAAAgGsqHG7OOOOM2i0JAACAm5P4qY0bN0pOTo4UFhb67b/yyiurWy4AAIDAhZvt27fL1Vdfbeaz8e2H41lMM5j73AAAALtVaSi4jpTS2Yj37t0r0dHR8sUXX5iZibt16yYrV66s+VICAADUZs1Ndna2WSU8JibGjKLS20UXXSRTpkwxw8Q//vjjqjwsAACAOzU32uzUuHFjc18Dzs6dO72djrds2VL9UgEAAASy5ua8886TTz/91DRN9ejRQ6ZNmyaRkZEyZ84cOeuss6paFgAAAHfCzdixYyU/P9/cf+CBB6Rfv37Sq1cvOfXUUyUrK6v6pQIAAAhkuElJSfHeP+ecc2Tz5s1y4MABad68uXfEFAAAQMjNc6Nyc3PN/xMTE2uiPAAAAIHvUHzs2DG57777zBoPSUlJ5qb3tbmqqKioeiUCAAAIdM3NiBEj5KWXXjIdiZOTk73DwydMmCDfffedzJo1qzplAgAACGy4WbhwoSxatEguv/xy774OHTqYpqlBgwYRbgAAQGg1S0VFRZmmqOJ0aLgOCQcAAAipcHP77bfLxIkTpaCgwLtP70+aNMn8DAAAIOibpa655hq/7X//+9/SqlUr6dixo9nWSf10dfBLLrmk5ktZl+gipEVHa+exC2vpcQEACMVwo6OhfF177bV+2wwFryEabCYn1NSjAQBQ51Q43MyfP792S2KJR5dtrdb59Y7/ICOkliX2FKkfXdvPAgBA6E3it2/fPu9Cma1bt5bTTjutpspVZx0LbyAze66q0ccc8etz/HdosGEmaQCApaoUbnRdKZ3r5u9//7ucOHHC7IuIiJDBgwfLzJkzJTqaWoEqCwuTYxENpUZFNqrZxwMAwLbRUmlpafLOO+/Iq6++KgcPHjS3f/3rX2bfnXfeWfOlBAAAqM2amxdffFFeeOEF+dWvfuXd17dvX2nYsKH87ne/YxI/AAAQWjU3R48elbi4uBL7Y2Njzc8AAABCKtzoelLjx4+XH3/80bvvhx9+kPvvv9+71hQAAEDINEtlZGTIZZddVmISvwYNGsibb75Z02UEAACo3XDTvn17+fLLL2XBggWyefNms08XzPz9739v+t0AAACETLgpKiqSNm3ayGuvvSapqam1UyoAAIBA9bmpX7++X18bAACAkO9QPHz4cHnooYfk2LFjNV8iAACAQPe5Wbt2rSxfvlzeeust0/+mUSP/GXBfeuml6pQJAAAgsOGmWbNmJVYFBwAACAaVCje6jtTDDz8sW7dulcLCQvn1r38tEyZMYIQUAAAIzT43kyZNkjFjxsgpp5wiLVu2lMcff9z0vwEAAAjJcKOrgD/55JNmor6XX37ZLJypc914VgavqszMTElKSjKTAPbo0UPWrFlTofMWLVokYWFhMmDAgGo9PwAAqKPhJicnxyyQ6dGnTx8TLnbu3FnlAmRlZZlVxnU5h/Xr15sZj1NSUmTv3r3lnvfNN9/IXXfdJb169arycwMAgDoebnTot9auFJ/3Rif2q6oZM2aYyQCHDh0q7dq1k9mzZ0t0dLTMmzevzHOOHz9uZkPWtazOOuusKj83AACo4x2KHceRm2++WaKiorz7dEK/YcOG+Q0Hr+hQcO2UvG7dOklPT/fuCw8PNzVC2dnZZZ73wAMPmBXIb7nlFnn33XfLfY6CggJz88jLy6tQ2QAAQB0IN0OGDCmx78Ybb6zyk+/fv9/UwsTFxfnt123PmlXFvffee/L000/LJ598UqHnmDJliqnhAQAAdUOlws38+fPFTYcPH5abbrpJ5s6dKzExMRU6R2uFtE+Pb81NYmJiLZYSAACE3CR+NUUDSkREhOzZs8dvv27Hx8eXOP6rr74yHYn79+/v3ecZqVWvXj3ZsmWLnH322X7naBOabzMaAACwW5XWlqopkZGR0rVrV7OUg29Y0e3k5OQSx+tq5Bs2bDBNUp7blVdeKRdffLG5T40MAABwteZGaZOR9uXp1q2bdO/eXTIyMiQ/P9+MnlKDBw82EwZq3xkdqXXeeeeVWApCFd8PAADqJtfDzcCBA2Xfvn0ybtw42b17t3Tq1EmWLl3q7WSsc+voCCoAAICKCHN0fHcdoh2KmzZtKocOHZImTZrU+OM/umyrBJtRvznX7SIAABCw72+qRAAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWqed2ARAcHl22VYLNqN+c63YRAAAhiJobAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVYIi3GRmZkpSUpI0aNBAevToIWvWrCnz2Llz50qvXr2kefPm5tanT59yjwcAAHWL6+EmKytL0tLSZPz48bJ+/Xrp2LGjpKSkyN69e0s9fuXKlTJo0CBZsWKFZGdnS2Jiolx66aWyY8eOgJcdAAAEH9fDzYwZMyQ1NVWGDh0q7dq1k9mzZ0t0dLTMmzev1OMXLFggt912m3Tq1EnatGkjTz31lJw4cUKWL18e8LIDAIDg42q4KSwslHXr1pmmJW+BwsPNttbKVMTRo0elqKhIWrRoUerPCwoKJC8vz+8GAADs5Wq42b9/vxw/flzi4uL89uv27t27K/QYd999tyQkJPgFJF9TpkyRpk2bem/ajAUAAOzlerNUdUydOlUWLVokixcvNp2RS5Oeni6HDh3y3nJzcwNeTgAAEDj1xEUxMTESEREhe/bs8duv2/Hx8eWe+8gjj5hw8+9//1s6dOhQ5nFRUVHmBgAA6gZXa24iIyOla9eufp2BPZ2Dk5OTyzxv2rRpMnHiRFm6dKl069YtQKUFAAChwNWaG6XDwIcMGWJCSvfu3SUjI0Py8/PN6Ck1ePBgadmypek7ox566CEZN26cLFy40MyN4+mbc8opp5gbAACo21wPNwMHDpR9+/aZwKJBRYd4a42Mp5NxTk6OGUHlMWvWLDPK6re//a3f4+g8ORMmTAh4+QEAQHBxPdyo22+/3dzKmrTP1zfffBOgUgEAgFAU0qOlAAAAiiPcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAFiFcAMAAKxCuAEAAFYh3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAq9dwuAFAdjy7bGnQXcNRvznW7CABQp1FzAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoyWAlzAKC8AqD3U3AAAAKsQbgAAgFUINwAAwCqEGwAAYBXCDQAAsArhBgAAWIVwAwAArEK4AQAAViHcAAAAqxBuAACAVVh+AUCFsWwEgFBAzQ0AALAKNTcArEeNE1C3UHMDAACsQrgBAABWoVkKAIIUzWlA1VBzAwAArELNDQCgRlHjBLdRcwMAAKxCuAEAAFYJinCTmZkpSUlJ0qBBA+nRo4esWbOm3OP/+c9/Sps2bczx7du3lyVLlgSsrAAAILi53ucmKytL0tLSZPbs2SbYZGRkSEpKimzZskViY2NLHL969WoZNGiQTJkyRfr16ycLFy6UAQMGyPr16+W8885z5TUAAEIffYXs4XrNzYwZMyQ1NVWGDh0q7dq1MyEnOjpa5s2bV+rxjz32mFx22WUyevRoadu2rUycOFG6dOkiTzzxRMDLDgAAgo+rNTeFhYWybt06SU9P9+4LDw+XPn36SHZ2dqnn6H6t6fGlNT0vv/xyqccXFBSYm8ehQ4fM//Py8qQ2/Jh/RIJNRV4r5eZ68zkJLK4317umPieZb2+TYDP81+fU2rVwHOfkBzsu2rFjh5bQWb16td/+0aNHO927dy/1nPr16zsLFy7025eZmenExsaWevz48ePNc3DjGvAZ4DPAZ4DPAJ8BCflrkJube9J84Xqfm9qmtUK+NT0nTpyQAwcOyKmnniphYWESjDSdJiYmSm5urjRp0sTt4liP6831thmfb663LbTG5vDhw5KQkHDSY10NNzExMRIRESF79uzx26/b8fHxpZ6j+ytzfFRUlLn5atasmYQCDTaEG663rfh8c71txue7djRt2jT4OxRHRkZK165dZfny5X41K7qdnJxc6jm63/d4tWzZsjKPBwAAdYvrzVLaZDRkyBDp1q2bdO/e3QwFz8/PN6On1ODBg6Vly5Zm6Le64447pHfv3jJ9+nS54oorZNGiRfLRRx/JnDlzXH4lAAAgGLgebgYOHCj79u2TcePGye7du6VTp06ydOlSiYuLMz/PyckxI6g8LrjgAjO3zdixY2XMmDHy85//3IyUsmmOG21GGz9+fInmNHC9bcDnm+ttMz7fwSFMexW7XQgAAABrJvEDAACoSYQbAABgFcINAACwCuEGAABYhXAThDIzMyUpKUkaNGhgVkpfs2aN20Wykk4vcP7550vjxo3NCvS6uryuRo/AmDp1qpkl/M9//jOXvJbs2LFDbrzxRjMje8OGDaV9+/Zm6gzUvOPHj8t9990nZ555prnWZ599tlnYmTE77iDcBJmsrCwz948OBV+/fr107NjRLAy6d+9et4tmnXfeeUeGDx8uH3zwgZkIsqioSC699FIzzxJq19q1a+Wvf/2rdOjQgUtdS77//nu58MILpX79+vLGG2/Ixo0bzfxgzZs355rXgoceekhmzZolTzzxhGzatMlsT5s2TWbOnMn1dgFDwYOM1tRobYL+A/HM2KzrTI0YMULuuecet4tnNZ1vSWtwNPT88pe/dLs41jpy5Ih06dJFnnzySXnwwQfN3FY6eSdqlv6+eP/99+Xdd9/l0gZAv379zPxsTz/9tHfftddea2pxnn32Wd6DAKPmJogUFhbKunXrpE+fPt59OoGhbmdnZ7tatrrg0KFD5v8tWrRwuyhW09oynV3c93OOmvfKK6+Ymd+vu+46E9o7d+4sc+fO5VLXEp1gVpcG2rp1q9n+9NNP5b333pPLL7+ca14XZyjG/+zfv9+023pmZ/bQ7c2bN3OpapHWkGnfD63Gt2m262Cjy6Voc6s2S6F2bd++3TSTaDO3zuau13zkyJFmTT9d8gY1X1OmK7C3adPGLAitv8snTZokv//977nULiDcAD/VJnz++efmLy3UjtzcXLM2nPZv0s7yqP3ArjU3kydPNttac6Of8dmzZxNuasHzzz8vCxYsMMsD/eIXv5BPPvnE/MGUkJDA9XYB4SaIxMTEmMS/Z88ev/26HR8f71q5bHf77bfLa6+9JqtWrZJWrVq5XRxraZOrdozX/jYe+tetXnftY1ZQUGA+/6gZp59+urRr185vX9u2beXFF1/kEteC0aNHm9qb66+/3mzryLRvv/3WjMqkpizw6HMTRLS6uGvXrqbd1vevL91OTk52tWw20iGaGmwWL14sb7/9thnCidpzySWXyIYNG8xftJ6b1ixotb3eJ9jULG1iLT61gfYHOeOMM2r4maCOHj3qt8iz0s+0/g5H4FFzE2S0fVxTvv7S7969uxlFokOThw4d6nbRrKNNUVqF/K9//cvMdaOr0qumTZuaEQ6oWXqNi/dnatSokZmDhX5ONW/UqFGmk6s2S/3ud78z82XNmTPH3FDz+vfvb/rY/OxnPzPNUh9//LHMmDFD/vCHP3C5XcBQ8CCkVfQPP/yw+bLVYbKPP/64GSKOmqUTyJVm/vz5cvPNN3O5A+BXv/oVQ8FrkTa3pqeny5dffmlqJvWPp9TU1Np8yjrr8OHDZhI/rQnW5lftazNo0CAZN26cqZVHYBFuAACAVehzAwAArEK4AQAAViHcAAAAqxBuAACAVQg3AADAKoQbAABgFcINAACwCuEGqKJvvvnGTASoSwfYNKmeLvZXWTp52R//+EcJFc8884w0a9bMqve2qu+dr5UrV5rXffDgwQpfuwkTJpiJGD10AswBAwZUqxyFhYWSlJQkH330UbUeB3UX4QZ1mv4iL++mv7hRPp1J+7HHHpN7773X1UCCwBg4cKBZo6os+lnQ96o6oUtn9L3rrrvk7rvvrlZZUXexthTqtF27dnnvZ2VlmanSfRcbPOWUUyTY6V+5bk7v/tRTT5k1jGpzQcaioiKpq9x+f4vTddfKW3tN12arCbqg6p133ilffPGFWasJqAxqblCnxcfHe2/6S1lrazzbsbGxZuG7Vq1aSVRUlKl6X7p0aZmPdfz4cbNIXps2bSQnJ8fs00U5u3TpIg0aNJCzzjpL7r//fjl27Jj3HH0+DQdXX321REdHy89//nN55ZVXyi2zVtdPnDhRBg8eLE2aNPE2B+lfueeee655HH0ubSryDQWe5oN//OMf5jH09V5//fVmTZyyvP766+a4BQsWlHnMokWLzKKBvl544QVp3769+RLUhTH79OljFoBVukryAw88UOZ19TQJadjs3bu3uXb6/Lp47KFDh0rUqhUUFJi/8lu2bGkW4tR12LR5xZfWJOiChnpt9Fp/9913UhGbN282wU3LoIt7vvPOO94V5c855xx55JFH/I7XZiwt27Zt20p9PE+TjX4OTjvtNPP+DRs2zAQY35oOXa1eaztiYmIkJSXF7Nfn1sV09Zqdfvrpcs899/h9lpRu67n6num5+hnQsnroe6+L8uoipvoZv+GGG8w6SMW9//770qFDB/O6e/bsKZ9//nmFa9B8m6X0vpZba3M879vXX39doWvXvHlzs7K5fr6ASnMAGPPnz3eaNm3qvRozZsxwmjRp4jz33HPO5s2bnb/85S9O/fr1na1bt5qff/311/qt4Xz88cfOjz/+6Fx99dVO586dnb1795qfr1q1ypz/zDPPOF999ZXz1ltvOUlJSc6ECRO8z6Hnt2rVylm4cKHz5ZdfOiNHjnROOeUU57vvvivzXTnjjDPM4z7yyCPOtm3bzE1NnDjRef/99025XnnlFScuLs556KGHvOeNHz/ePPY111zjbNiwwZQvPj7eGTNmjPeY3r17O3fccYe5v2DBAqdx48bOq6++WmZZtJxhYWHOBx984N23c+dOp169eub6aVk+++wzJzMz0zl8+HClrqteqxdffNHZvn2788033zgZGRnmvF27dpmb5/FuvfVW54ILLjCvR6/Fww8/7ERFRXkfT8sWHh5ursWWLVucxx57zGnWrJnfe12cpwz63rzwwgvOxo0bzfPo9di/f785ZtKkSU67du38ztP375e//GWZjztkyBDzHgwcOND5/PPPnddee8057bTTSrwHeszo0aPN9dHbf/7zHyc6Otq57bbbnE2bNjmLFy92YmJizHta/Dx9//ScZ5991pwzZ84c7zFPP/20s2TJEvN5zM7OdpKTk53LL7/c+/MVK1aY1922bVvzedX3rl+/fua9KCwsLPXfiZahY8eOfq/xqquuMvcPHjxoniM1NdX7vh07dqzC1+7uu+82rwuoLMIN8JPiv7QTEhLML2Ff559/vvmC8f0CfPfdd51LLrnEueiii8wvcw/dN3nyZL/z//GPfzinn376//4Bijhjx471bh85csTse+ONN8oNNwMGDDjp+6Zf8l27dvX7EtIvu7y8PO8+/QLt0aNHiXDzxBNPmGuxcuXKcp9Dg52WNycnx7tv3bp1Zp8GktJU9LpqmCnv/VHffvutExER4ezYscNvv1779PR0c3/QoEFO3759/X6u4aIi4Wbq1KnefUVFRSbseAKjPqc+94cffmi29ctfA4eG2bLoF3+LFi2c/Px8775Zs2aZUHL8+HHve6Ah2ZeGn9atWzsnTpzw7tPAWPw8DSW+x2g40H1lWbt2rXmdnqDoCTeLFi3yC7ANGzZ0srKyKh1uigdmj4peOw2iGqyAyqJZCihFXl6e7Ny501SL+9LtTZs2+e0bNGiQaXJ56623/PobfPrpp6b5RfvteG6pqammn8/Ro0e9x2n1v4c2q2hTRWlNBb60aaE4bcbR8mlzgz7X2LFjvc1jHtocpU0SHtq8Ufy5tElp1KhRsmzZMtMsVJ4ffvjB/F+bLzw6duwol1xyiWmWuu6662Tu3Lny/fffV/q6lvYai9uwYYNpDtTmON/rrE0hX331lTlGH1ebqnwlJyef9LGLH1evXj1TJk85ExIS5IorrpB58+aZ7VdffdU0kelrLo9eH20e832OI0eOSG5urndf165d/c7R59TjtNnG95rpef/5z3+8+7QJyfcYPefLL78010itW7fONCFqE51+Djzvb/HPie/rbtGihbRu3brE+1MdFb122qzp+28FqCjCDVBNffv2lc8++0yys7P99usXj/at0L4Enpt+GeuXjW8YqF+/vt95+uWk/VLKoyHIlz63dsDUsrz22mvy8ccfm9FLvn05KvpcnTt3Nv1B9IvHt79GabRfh/KEFxUREWGC0RtvvCHt2rWTmTNnmi9H7WtRGcVfY2n0Guvz6Ze273XWL2Lt51Hbbr31VtMnREPe/PnzzUgi3+BSVRV57ZWlAVz772h41j5Ma9eulcWLF5ufFf+cBEJFrt2BAwfMZxGoLMINUAr9AtC/LrVjpS/d1i9sX3/6059k6tSpcuWVV3o7nCrtSKwjr7TzZPFbeHjN/tNbvXq1Ga2kgUZrF7Rj8rffflulxzr77LNlxYoVpjP0iBEjTnqsXquNGzeWCE1as6DhToOWjvbRL9LKXNfi9DE8NRC+QUz3ae1T8WusNViqbdu28uGHH/qd98EHH1ToWvgep511NUTp43lomNQgMmvWLNMpWjuUn4zW6HlqvDzPobVNiYmJZZ6jz6kB1jds6jXT2hftmO1R2uvUz4IGQO0crR2p9bPaq1cv0/G9rBpC39etwVWHfvu+7soo7X2r6LXTjsz6HgOVxVBwoAyjR4+W8ePHmy9wHdGjf11qrUBpI4c0BOgv8H79+pkai4suusgMK9dtbQL47W9/awKNfrHpL+wHH3ywRq+7foFp04L+JXz++eebUU6ev8qrQpt5NODoyB1tjsnIyCj1OH1NOhLqvffe846Q0S/Y5cuXy6WXXmpGnOn2vn37vF+OlbmuxZvUtKZGH9vTtKPl1BorHTk2ffp080Woz6XHaHOfNn2MHDnSBC0dnXPVVVfJm2++We6oN1+ZmZnm2mrZH330UfNF7/slrKFBRwSlp6eb4yrS3KW1JLfccotpNtSRYXotdIRTeYH3tttuM++Bfs70WA3Nel5aWprfefoZ0H3/93//J+vXrze1ZnpdlH4ONWjoPh2hpZ9DHXVXGm1O1VFucXFxJjBrDV1VJ+bT900/A/paNcRpM5eWuSLX7t133y2zjEC5Kt1LB7BU8Y6S2lFTRza1bNnSjObRTpO+HX19R0t5TJ8+3Yyo0VFLaunSpWYkj3bI1JE+3bt39xu9oufryBdfWgYtS3kdih999NES+7Vz8KmnnuodjaPHlNfxU+kx+nhldf7UUUKxsbFOWlpameXR0Td6jTwdW/WclJQUMwpIRy2de+65zsyZM6t1XT2GDRtmXqP+3DNSSDujjhs3znQ81cfTDts6ck1H+viOEtLOwPo+9O/f34w0q0iHYh3Fpu9ZZGSkGd3z9ttvlzhWRx7psdOmTXNOxtPZVsvrea90JJGOtiuvA67Szt3a8VrLoqPctLOwdnL2PU87Zes10s9a8+bNTUdk3w7G+nr0Oun7oqOYdFSd77X2dCjWEXK/+MUvzHPp6//000+9j1HZDsU6Qq1nz57m2utj67WtyLVbvXq1GdV29OjRk15XoLgw/U/58QcAyv0DyXTY1U7I2rm6rtHaBe1ArR2CtaajPFpToUsbvPzyywErX6heO+2DozV0Y8aMca18CF30uQFQLdq/Zs6cOSUmlLOdju7RkUo6maCO8jlZsEHFr5023eloOw3MQFUQbgBUm/aduemmm+rUlXzuuedMJ26tiZk2bZrbxbHq2mnfIO2TVN4yD0B5aJYCAABWoeYGAABYhXADAACsQrgBAABWIdwAAACrEG4AAIBVCDcAAMAqhBsAAGAVwg0AALAK4QYAAIhN/h9yk6C1RM90yQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Step 4.1: Sort by descending probability\n", "sorted_probas, sorted_idx = torch.sort(toy_probas, descending=True)\n", "\n", "# Step 4.2: Compute cumulative sum\n", "cumsum = torch.cumsum(sorted_probas, dim=-1)\n", "\n", "plt.bar(\n", " torch.arange(len(sorted_probas)), sorted_probas, \n", " alpha=0.5\n", ")\n", "plt.step(\n", " torch.arange(len(cumsum)), cumsum, \n", " where=\"mid\", color=\"C1\", label=\"Cumulative sum\"\n", ")\n", "\n", "plt.ylim([0, 1])\n", "plt.xlabel(\"Token rank (sorted by probability)\")\n", "plt.ylabel(\"Probability\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 44, "id": "951989fd-197c-420e-b50a-ff9e6c6cd801", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([0.4538, 0.7290, 0.8119, 0.8798, 0.9170, 0.9475, 0.9701, 0.9886, 0.9969,\n", " 1.0000])" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [] }, { "cell_type": "code", "execution_count": 45, "id": "ead2d1d9-f2de-40f5-8e2d-e69444a0511e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cumulative sum: tensor([0.4538, 0.7290, 0.8119, 0.8798, 0.9170, 0.9475, 0.9701, 0.9886, 0.9969,\n", " 1.0000])\n", "Tokens kept: 2\n" ] } ], "source": [ "# Step 4.3.1: Apply top-p threshold (e.g., keep tokens until cumulative mass > 0.8)\n", "top_p = 0.8\n", "keep_mask = cumsum <= top_p\n", "n_kept = torch.sum(keep_mask).item()\n", "print(\"Cumulative sum:\", cumsum)\n", "print(\"Tokens kept:\", n_kept)" ] }, { "cell_type": "code", "execution_count": 47, "id": "53f6df8c-1d2c-40df-b8e5-b757530605a2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tokens kept: 3\n" ] } ], "source": [ "# A more common variant is to include the token that crosses the threshold\n", "keep_mask = (cumsum - sorted_probas) < top_p\n", "n_kept = keep_mask.sum().item()\n", "print(\"Tokens kept:\", n_kept)" ] }, { "cell_type": "code", "execution_count": 48, "id": "1c45fb7c-c00d-4970-be0a-faf1adca3db6", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAWhtJREFUeJzt3Qd4FNX6x/ETSCgBQZBexC7gBUSagIoFRazYLuoVEBVFxAKKClfBThFBBISLInqV5sW/XbHQRGlSVJSi2EBApEMIkJDM//kdnXU3JJu2M7vZfD/Pszq7zE5mz87OvHPOe85JcBzHMQAAAHGiRLR3AAAAIJIIbgAAQFwhuAEAAHGF4AYAAMQVghsAABBXCG4AAEBcIbgBAABxJdEUM5mZmWbTpk3miCOOMAkJCdHeHQAAkAcalm/v3r2mVq1apkSJ8HUzxS64UWBTt27daO8GAAAogA0bNpg6deqEXafYBTeqsXELp0KFCiZWa5e2bt1qqlatmmt0Csq7qOH4przjGce3d/bs2WMrJ9zreDjFLrhxm6IU2MRycHPgwAG7fwQ3lHe84fimvOMZx7f38pJSUuyCG8AvGRkZZtGiRXb59NNPNyVLlqTwAcAHBDeAh8HNp59+apdbtGhBcAMAPiG4ATyiJsUmTZoElgEA/iC4Abz6cSUmmk6dOlG+AOAzbicBAEBcIbgBAABxhWYpwCNpaWlmxIgRdrlv376mVKlSlDUA+IDgBvDQwYMHKV8A8BnBDeCRpKQk07t378AyAMAfBDeAh6NoHnXUUZQvAPiMhGIAABBXohrcfPbZZ+bSSy+105frLvett97K9T1z5841p512mildurQ54YQTzMsvv+zLvgIFGaF4yZIl9qFlAEAxCG727dtnR3AdO3Zsntb/+eefzcUXX2zOOecc89VXX5l77rnH3HLLLeajjz7yfF+B/FJA8+GHH9oHwQ0AFJOcm44dO9pHXo0fP94ce+yx5plnnrHPGzRoYD7//HMzcuRI06FDBw/3FMg/TbnQsGHDwDIAFIrjGHMo1dtCTExWwqAp6opUQvHChQtN+/btQ15TUKManHBdcYO74+7ZsycwLb0esUj75ThOzO5fvPGqvBXQXHXVVSF/BxzffuN84m+QYH/n6ftM5sG9OglEbsOOYxJeb2cStn5lvJTZe48xSeVMLMrPObRIBTe///67qV69eshreq6AZf/+/aZs2bKHvWfw4MHm0UcfPez1rVu3mgMHDphY/QJ3795tL7jc8VPe8Ybjm/KOKscxlT++3JTa9qUnm1c4U8MUXVu3bjVO4j4Ti/bu3RufwU1B9O/f344O61IgVLduXVO1alVToUIFE6snfyVYax8JbijveMPxTXlHVfo+U8KjwMYPTtVTjfPPeZ41HVWN4WapMmXKxGdwU6NGDbNly5aQ1/RcQUp2tTaiXlV6ZKWgIZYDBwU3sb6P8cSL8k5PTzejR4+2y3feeScD+Xlc3ohCefuRAxJpGfv/Xr59S8SbYBS8q/bDq5vThMRk+30WRyXyUZ5FKrhp3bq1+eCDD0Je++STT+zrQKxRs6JbjaplIK7omJ52hjGbFpgiS4FNpPNLlMOnZh1tl+A9aqIa3KSkpJh169aFdPVWF+/KlSubo48+2jYpbdy40fz3v/+1/96zZ08zZswYc//995ubbrrJzJ4927z++uvm/fffj+KnALKXmJhobrvttsAyEFdUY1OUA5tabf/sGYS4FNUz7tKlS+2YNS43N6Zbt252cL7Nmzeb9evXB/5d3cAVyPTp08eMGjXK1KlTx7z44ot0A0fMVqGqKRWIex4073guhnNLUMSDm7PPPjtsdX12ow/rPStWrPB4zwAAUW3eAQqBunLAIxqVeOXKlXa5UaNGpmTJkpQ1APiA4AbwMLh5++237bJGKia4QVR6HWloCW07fV9kE1y1PSBGEdwAHubcnHjiiYFlIBq9jnTkhQ59CsQ/ghvAqx9XYqK5/vrrKV+ER68jIOIIbgAgVhTBQeXodYRYRHADALGCQeWAiCC4ATyi6RfGjx8fGIAyKSmJsgYAHxDcAB7RGE47duwILAMA/EFwA3j140pMNN27dw8sAwD8wRkX8IiSNzVHGnxUFGepZrwYIOIIbgDEh3iYpRpARBDcAB5RF9zVq1fb5QYNGjCQn9cYLwbAXwhuAI8cOnTIzJgxwy7379/flCpVirL2C7NUA8UawQ3gkYSEBFOvXr3AMnzELNVAsUZwA3hE49rceOONlC8A+IzZ/AAAQFwhuAEAAHGFZinAw+kXJk6caJdvvvlmpl8AAJ8Q3AAe0ZQLW7ZsCSwDAPxBcAN49eNKTDQ33HBDYBkA4A/OuICH0y8cf/zxRbN8vZzGIDPTJGjbmnagRATT/pjGAMBfCG4A+DqNgcKZ6pQ5AA8R3AAeTr+wbt06u3zCCScUnekXmMYAQBFHcAN4OP3C1KlTi/b0Cx5MY6Cgb+vWraZq1areBHyJyRoSOvLbBVBkENwAHtGUC7Vq1QosF0leTGOQmWmcxH1/breo1GYBKFIIbgAPp1/o0aMH5QsAPuO2CQAAxBWCGwAAEFdolgI8nH7h1VdftctdunRh+gUA8AnBDeARTbmwYcOGwDIAwB8EN4BXP67ERNO5c+fAMgDAH5xxAY9oDJf69etTvgDgMxKKAQBAXKHmBvCIRuJd/+uvxmQcMEfXrVN0pl9gAkoARRzBDeCRQ+np5pX//tcu9y/3pCmVkE5ZA4APCG4AjyRk7DdVS/zx53JRLOVabf+cpwkAihiCG8DD6Rd6JT/v2QSUnmMCSgBFFMENUFQnoAQAZKuIZDgCAADkDTU3gIfTL0zb38UuX5uebpKSKGoA8APBDeARTbnwU8bxgWUAgD8IbgCvflyJieaK0m/8tdyXcgYAnxDcAB7RoH2Nk1a6TyhnAPAJZ1wAABBXqLkBPJx+YXNGLbtcMzOTOwkA8AnBDeCRQ4cOmRf332qX+x86ZEqVpqgBwA8EN4BHEhISTMWEXYFlAIA/CG4AD6dfuKfcs389eYJyBgCfkFAMAADiCsENAACIKzRLAR4mFM/Yf61dvvrQIZPI9AsA4AuCG8DDruBrM+oHlgEA/iC4ATxSsmRJc0npd/5a7kM5A4BPCG4AD4ObZknL3SeUMwD4hIRiAAAQV6i5ATziOI7ZmlHVLld1HMMwfgDgD4IbwCPp6elm3P477HL/9HRTqhRFDQB+ILhB0eA4xhxK9WbbmZkmQdtO32dMiQi21KanmmSzL3LbAwDkCcENikZgM+0MYzYt8GTzCmeqe7BdVdT0K+8+GeTBXwAAZIeEYsQ+1ap4FNj4olZbYxKTo70XAFBsRL3mZuzYsebpp582v//+u2nSpIkZPXq0admyZY7rP/vss2bcuHFm/fr1pkqVKubqq682gwcPNmXKlPF1vxElt28xJqlcRDepAfa2bt1qqlatakpEslnKpcCGWcEBoHgEN9OnTzd9+/Y148ePN61atbKBS4cOHczatWtNtWrVDlt/ypQp5sEHHzQvvfSSadOmjfn+++/NjTfeaBISEsyIESPy98f37ct+7BG9Fhwoab2c6EJYtmzB1k1N/bO5JTtZXw+3ri6ayUG1Avv32xySHJUrV7B1DxwwJiMjMusmB13sDx7UPAXh13VptTSVTw7rqnzd4CQtTRm9OW83eF3tr7arR3axjY4H91jJbbtB6x7av9+88+67dvmyDh1MYmKWn1vp0sa4r2mb2nZOgtdVeanccqLM5aSk/K+r70xlkROt52ZF52ddHWM61oJznHRM6/ei7yDculmpDFQWot+EthOJdfPzu4+Fc0TW331+zifxeI7I67oFPUfk43evdUOO73Dr5ud3X5zOEeHWzSsnilq2bOnccccdgecZGRlOrVq1nMGDB2e7vtY999xzQ17r27ev07Zt2xz/xoEDB5zdu3cHHhs2bNAv3dn950/+sEdmx452P9xHZnJytuvZddu1C123SpWc123ePHTdevVyXrdhQ2fTpk1Oenr6n+s2bJjzuvXqhW63efOc161SJXTddu1yXjc5OXTdjh1zXFePkHWvuir8unv2/L1u167h1/39dyfjwB7HGW4cp40Jv+6PP/693XvvDb/uN9/8vc8PPxx+3UWL/l536NDw686aFVh3/3PPOY888oh9HExKOnzdd975e7sTJ4bf7rRpf687bVr4dSdO/Hvdd94Jv+7o0X+vO2tW+HWHDv173UWLwq6bOXDg3+t+8034de+99+91f/wx/Lq33/73ur//Hn7drl3/XnfPnvDrXnVVyDEcdt0YOUeErJvLOSLkfBKP5wh33dtv9+QcoeM5r+eIQ0OG5Pkcod9f2HU5RzjB54idO3f+ef3evTvX+CJqNTdpaWlm2bJlpn///oHX1CTQvn17s3Dhwmzfo9qa1157zSxZssQ2Xf3000/mgw8+MF26dMnx76jJ6tFHH83zfh1MSzO7/vgj8LxamPFJ0tPSzI7gdXVHmsO6h9LTzfagdatmZJiSYSZc3LVrlx0nRWVy1KFDJqc5FzMzMszWoO0elZ6e47pOZqb5I2jdymlpNuk123UdJ2TdI9PSTLiGv5B1Dx4Mu66agJy/7mArHjhgyuayrklPzlPC7/bt203GX3ezR6SmmnCNVzt27DCH/trncvv2mSPCrbtzZ2Dd5JQUUyHMuvre0v5at/TevabDzJl2uWQ2d6m7d+82B/9at+zevaZimO3u3rMnsG7pPXtMpTDr7t271+x31929O+y6KXv3mtS/1i21a5epHG7dlJTAuok7d5oqYdbdt2+fSXHX3bEj7Lqpqalm71/rlty+3VQNt+7+/YF1E7ZtC3tcHDhwwOx2101NDbvuwYMHQ373NYrAOSJ43XDniIyMjNDzSRyeI7QvcsT+/WF/9wU9R5Tft8+Uz+M5omxKStjfcvA5Innv3rDnE84RoecIndvyKkERjomCTZs2mdq1a5sFCxaY1q1bB16///77zbx588zixYuzfd9zzz1n7rvvPnsw6wfes2dPm4MT7qSlh2vPnj2mbt26Zudvv5kKFSrEZJVzpgZ/27fv7xyQ4t4sdSjVlBhTwTZLZfbYnHPOTQGrnDMPHDDbNm+2OVzZ5twUsFmKKufsq5yV47Rt27a/y5tmKU+bpQ47n8TjOSKGmqXydT6hWcrkp1lK1+9KlSrZoC/b63csJRTnx9y5c81TTz1lnn/+eZujs27dOnP33Xebxx9/3Dz88MPZvqd06dL2kVWJI46wj1zlZZ2CrFs+zH3AXzkJ+mHYH0e4dcOdQCK5bvCJNJLrBp/Mc5L51wki8c/vLU8JxTqB5DXJXOuVK/fnMZFbQnF+tqvjLptjr9DrKmjIa/tzftZ1Aw0v1g3+bfx1Isu2vLOum5t4Xjc/v/v8nE/i8RxRkHXze47w4nzCOSJfv/v8dPiIWnCjqFYTC27ZsiXkdT2vUSP7SmEFMGqCuuWWW+zzRo0a2ervW2+91fz73//2pqcLUECqXdQdhlSsWNEmvgMAvBe1aKBUqVKmWbNmZtasWSHV1Xoe3EyVtd0tawCjAEmi1LoGhJ1+YdSoUfahZQCAP6LaLKVu4N26dTPNmze3CcLqCq6amO7du9t/79q1q83LUVKwXHrppbbLd9OmTQPNUqrN0etukAPEkqS8Nt0AAOIjuOncubPNdB84cKAdxO/UU081M2fONNWr/9mvQQP1BdfUPPTQQ7ZqX//fuHGjTZBTYPPkk09G8VMAOddODhgwgOIBAJ9FrbdUtCjbWvkPecm2jhY1z6nbpAYyJI9I7Tv7jHnur4TJu1I8GaGY8vYP5e0vypvyLo7XbzJwAQBAXClSXcGBokTjMGmQSbnooosOn34BAOAJam4AD5sDVqxYYR9aBgD4g1tJwCPqwXfOOecElgEA/iC4ATyigOass86ifAHAZzRLAQCAuELNDeARjbKgUbUlOTmZ6RcAwCfU3AAe0ZQLw4cPtw+mXwAA/xDcAACAuEKzFODh9AuDBg2ifAHAZ9TcAACAuEJwAwAA4grNUoCH0y98+umndrl9+/ZMvwAAPqHmBvCIplxYvHixfTD9AgD4h5obwMMRis8444zAMgDAHwQ3xY3jGHPoz4Hlioz0faYoUkBz3nnnRXs3AKDYIbgpboHNtDOM2bQg2nsCAIBnCG6KE9XYFOXAplZbYxKTTVGafsEdmTgpKYnpFwDAJwQ3xdXtW4xJKmeKFAU2CQmmqFBgM3jwYLvcv39/O6gfAMB7BDfFlQKbohbcAACQBwQ3gEfUFKUaG3cZAOAPghvAIwkJCTRFAUAUMIgfAACIK9TcAB7JyMgwc+fOtctnn302A/kBgE+ouQE8DG4+//xz+9AyAMAf1NwAHilRooRp1apVYBkA4A+CG8CrH1diornwwgspXwDwGbeTAAAgrhDcAACAuEKzFOCRtLQ0pl8AgCig5gYAAMQVam4Aj2jKhfvuuy+wDADwB8EN4OH0C+XKMTkpAPiNZikAABBXqLkBPKJRib/44gu73LZtW6ZfAACfENwAHgY3c+bMscunn346wQ0A+ITgBvCIplxo2rRpYBkA4A+CG8CrH1diornssssoXwDwGbeTAAAgrhDcAACAuEKzFODh9AvDhw+3yxrMr1SpUpQ1APiA4AbwUHp6OuULAD4juAE8oikX7r777sAyAMAfBDeAh9MvHHnkkZQvAPiM4AYAIjBgY6w2QWZmZtp9O3DgAOMtUd4xT7mJkRgXjOAG8PCC9+WXX9rlFi1aMEJxHHIcx/z+++9m165dJpb3UQHO3r17bW0iKO9YpsDm2GOPLXQHDIIbwMPg5qOPPrLLp512GsFNHHIDm2rVqpnk5OSYDB4U3Bw6dMgOKhmL+xdvKO+CUxC+adMms3nzZnP00UcX6ngluAE8vANp1KhRYBnxF7y6gc1RRx1lYhUXW8q7KKlataoNcBSQF6YjBsEN4BHdKV955ZWUb5xyc2xUYwMgMtzmKN08FCa44XYSAAqBph4g9n5PBDcAACCuENwAHk6/8PTTT9uHlgHk7w7+rbfeipntoGghuAE8lJqaah9ALPb0uvPOO81xxx1nSpcuberWrWsuvfRSM2vWLFMUPfLII+bUU0897HX1vOnYsWNU9gnRQ0Ix4BElw91+++2BZSBW/PLLL6Zt27Z2BG3VLKpXnxKkNXTBHXfcYdasWRPtXYyYGjVqRHsXEAXU3AAeUXW4ugnrQdIpYkmvXr3sMblkyRJz1VVXmZNOOsmccsoppm/fvmbRokWBAEjrfPXVV4H3qeu7Xps7d659rv/ruYKipk2bmrJly5pzzz3X/PHHH+bDDz80DRo0MBUqVDDXX399SA3mMcccY5599tmQfVKti2pfcvLAAw/Y/VTvNNU2Pfzww4Eeay+//LJ59NFHzddff233Rw+9lrVZqk2bNnY7wbZu3WpvPj777DP7/ODBg+a+++4ztWvXNuXKlTOtWrUKfN6cutprvzUui2rA9L4+ffqEbRZTUOnun1vOr7/+ujnzzDNtGWrQz++//94OAtq8eXNTvnx5W/ukfUXeUHMDAJHiOMYcilIzZGKyrqS5rrZjxw4zc+ZM8+STT9qLd1YFmQ9NF/cxY8bYwOOf//ynfehCP2XKFJOSkmKuuOIKM3r06MMCi/w44ogjbEBQq1Yts3LlStOjRw/72v333286d+5svv32W/u5Pv30U7t+xYoVD9vGv/71LzNs2DAzZMiQwA3H9OnT7TYVWEjv3r3NqlWrzLRp0+zrb775prnwwgvt3zzxxBMP2+Ybb7xhRo4caddXgKhmsBUrVuT78w0aNMgGfAqSbrrpJhsQ6vONGjUqUK4DBw4048aNK0DpFT8FCm7mzJljzjnnnMjvDRBHNE6De9eru9KSJUtGe5fgNQU2z5WPTjnflWJM0uHBSlbr1q2ztQ3169eP2J9+4oknbDOX3HzzzaZ///7mxx9/tDUscvXVV9vrRmGCm4ceeiik5ke1KwooFNyotkO1GxpbKlwzlAKEe+65x3z++eeBYEYB2HXXXWeDnfXr15tJkybZ/yuwEf0dBU16/amnnjpsm1pXf7N9+/a2Bki5SxqRPL/0dzp06GCX7777brtPyn8KLle3tgceNUspij3++OPtAb1hw4aCbAIoFsHNe++9Zx9aBmKBAptIa9y4cWC5evXqgaaj4NfUVFUYqmHRhV6BhAIZBTsKLPI7+u0FF1xgJk+ebJ///PPPZuHChbZGR1Q7o9+qmr/0N9zHvHnzbLCWnWuuucbs37/ffl7VJqmmR6PrFrYMxR3hPFJlWJwUqOZm48aN5tVXXzWvvPKKbedUG6uiyk6dOhV6sisgXmjKhZNPPjmwjGJATUOqQYnW384DNa2oliK3pGH3mA0OhnKa+Tw4YV7bzppAr9c0b1DwtrMGWeFmVXcDEF1vVLuhJifV2jzzzDMmv7Sdu+66yzaTqdZGAYQbRKgJTTWsy5YtO6ymVUFOdlRTs3btWtsc9sknn9iEbNUsKSDS9VCfPS+fNWsZZvdacBkivAKdcatUqWITplTlvnjxYhvlKkFN1Xg6aJTUlVdjx461B0KZMmVs4pYS3MJRQpsOnpo1a9o2Xf3tDz74oCAfA/CUqsivvfZa+9AyigFdlNQ0FI1HHkd2rVy5sg0QdO7dt2/fYf/uznCuWg5RDokrOLm4MLTt4O3u2bPH1qLkZMGCBaZevXrm3//+t02wVYD266+/hqyjQCIvNaSXX365OXDggG1qUnDj1tqIkqK1DdWQnHDCCSGPcM1dahZTN/rnnnvONr8pKVu1QNl91h9++IHhIXxQ6NtJtS2qfVVJWIp6X3rpJdOsWTPbnvndd9/lWs2o7HwlUi1fvtw0adLE/uhyqnrTQGjnn3++zS6fMWOGjZZfeOEFm50OAMgbBTa6iLds2dImxOqCu3r1antxbt26deCCffrpp9vkW/2baiKC814KQ7X9qv2fP3++DQK6desWNidNwYyaoFRbo+Yh7aeaf4LpJlkBkgKwbdu22V5P2VEStVoZ1NtKn0u5LS7dLCvY6dq1q/m///s/uz3dcA8ePNi8//772W5PeTATJ060Cc0//fSTee2112zZKRhzP6uSrZVkvHTpUtOzZ0+Ghojl4EbVagowLrroIvslqiugvsAtW7bYhDW9prbIcEaMGGHbKLt3724aNmxoxo8fb9tqFSBlR68r01/d6tT2qoO5Xbt2NigCAOSN8kN0Q6mOIffee6/5xz/+YW8clcAa3BtH51zlj+iGVYm4yrOMBN0Q69x9ySWXmIsvvtgGG8rjzMlll11mWwt0E63kfNXkKDgJpi7tygfVZ1JtydSpU3PcngIYtTDoJly9k4IpcVjBjcpFzcraN3XJzrpecO8y3WTrmqS8GZWhAi93png1nanpSn9LPaCUOMxkq95LcAqQXaZRLXXg6K1dunQxt9xyi/1xZB39Us1UObURqhZGX7ACJB08LkXwqhZ9++23D3uPAilVqep9+ncdwDpYlIGfU9Sv6D04glf1pw60nTt32vEXYpHKTOMZ6PNFNFcjfZ8pMebPz5zZe0+eelYUB16Vt24A3AuFBvNjID9vy9tvatpQLfKxxx5rm9VjmY5Fjj/Ku6j8rlRj5qarBNP1u1KlSmb37t25Xr8LlAigMQCUjHXllVfavJec8nLU9pgTVRuqWtTNCnfpeU6Jbqrymz17to26lWejGiLl+uiHq6at7Kg6UUloWenkqkKM1ZO/vjwFj5E8+SccSjXVgz6/k3h4e3tx5FV567jUdkVNrVxcvC1vv+n71WdRzUZBesf4ReXs5qIwmCTlHev0W9Lvavv27YedM/fu3Zvn7RQouFEgoZEesyZJaqdUXXjWWWfZf1O1YyTpA2u01wkTJtiaGlWVqueWhg/PKbhR9afyerLW3OiuMZZrbnQS8qLmxmWTBam58bS8tV31IhQlIxblC3mROL59ppsjnWx1risKCeME15R3UaDfks4LatbLWnOTnxrSAv0i1aap7G8FGsF0N6Z/y0vGump2FKAoRyeYnueUla4eUvqBBjdBaXhvNYGpmSu7buiqWcqudkmFF8snVp38I76PQduy243hzx8P5a1t1alTJ2LbiyeeHN8+0767Q/3Hco2Iam7c/Yvl/YwXlHfhuL+n7M4P+TlflCjslxdM1UjZDeedHQUiqnkJnoFWd3R67mbrZ6WELTVFBefxaP4NBT2MrwMAAPJdc6McG1Fgc+ONN4bUiKi25ptvvrHNVXml5iIlEGvcAnVJ1LwaGndBvadEGevq5q28GTcpUz2yNDS1kprVfVHDYWtsHSDWKAhX91BRwn1RrqUAgLgNbtyJyFRzowm91JffpZoTjYmgrt15pcnOlNiqycDUtKQufhpYyU0y1rgGwRcE5cqoy7m6BKrLnQIfBTqFma8E8Ipy0NyxODSPD7WLABCDwY36/wdPWpbXJqhwNG6BHtnJbpp5NVlp9Ecg1qmG051fh1wHAPBPgXtLAQhPye8aBwoAEKPBjaZZULKvBtDR/Bvh7kQ18iUAAEBMBzeabMxNIA4eURgAAKBIBjfBTVE0SwF5G8FWc86IEu0ZRA2x4uyzz7YdONRDtTj43//+Z+ei0nQZmoRz6NChdjqfcCZPnmyGDRtme+WqM03Hjh3tgLHunFGIbfRNBTyiXoXqDWinusj/FG4AIkCj5mvmb40Wrpm51fKghztMQ3a++OILOxSJ3vPdd9/Z4Eizg+enNzCKSHCjXBtNWpmXB4A/hxHXOE56FIXh+VE8aIyyefPmmVGjRgVGg1WNhuh1jTmmFAQNjvrggw+GzJulGh+3h6tqMzTSvGpE8hq8q6ft448/boMN9bbVcB5jx441XtLn1Gzh/fr1syPa6+8rh1RjpuVk4cKFdl81hpomRj3jjDPMbbfdZgMcFA15PuMWl+pLIFI0RpNOkCiG9oWZlFbTxwTPkRNuXY3zFTSeWI7r5mNYDl3sNbK7BpZ87LHH7Gua50vz9KmpRsHPf//7XzuBsWoqNJ/PI488Enj/K6+8Yms0dKFfunSpufXWW83RRx+d51oNNe0MGDDATmisccs0VtlJJ51kzj///BybhxRYhPPhhx+aM888M8dAJXh+QenQoYN56623ctyehhzRPmqCZjVHaeLbGTNm5NqUhSIY3OjuEwCQB+XL5/xvukC+//7fzzVHX2pq9utq8uHg8b4ULG/bdvh6+Wj2VI2LBpRMTk4Omcfv+eeftwOlqkZDtTkaeHLTpk12kFQNtOoOqKp1Ro4cadc5+eSTzcqVK+3zvAY3mkZHNUKioEZNQHp/TsHNZZddZlq1ahV2m6oByokGiHUHhnXpuV4Pt48KqjTQrCZIVe3VpZde6nktE6IQ3Gg2bXcWbS2HE6uzbQN+T7+gO2T3JM70C4hlq1evtjUWwcN86CKfkpJifvvtN1s7IxqJPngdveeZZ56xU/AET2qck6xzB+p5uJYBjYavh59WrVpla5QU1KmWRxNFq1mrZ8+eZuLEib7uCzwObpRz484EfuSRR2Y7zo07oWZeZgUH4p3u9qZPn26X+/fvz/QLxUlKSs7/ljUA+OOPnNfNOh/ZX7kxxUlhm6VUO7Vly5aQ1/Q8uNYqK81nqMBOAY1ouh/lCOlvPPHEEzYfCXES3MyePTuQLDxnzhwv9wmICwr0VYXvLqMYyc/UNF6tG4aapbLehCrZ9o033gjcpIqajFRrUqdOncB6ixcvDnmfpsNR9+q81Nq462d9rr+dk8I2S6lmSAPQ3nPPPYHXPvnkk8NqkIKlpqYe1gnA/Xz0fCwinGJm9+7dapy2/49VGRkZzubNm+3/IyotxXGGmz8fWoa35Y24Lu/9+/c7q1atsv+PZZmZmU5aWpr9v6tHjx5OixYtnJ9//tnZunWr/S5+++03Jzk52bnjjjuc1atXO2+99ZZTpUoVZ9CgQYH3tWvXzilfvrzTp08fZ82aNc6UKVOccuXKOePHj8/TvtSrV8+pUKGCM3ToUGft2rXOmDFjnJIlSzozZ850vPLFF184iYmJzvDhw+3n0udJSkpyVq5cGVjnwQcfdLp06RJ4PmnSJPue559/3vnxxx+dzz//3GnevLnTsmXLApU3IvO7ys/1u8D9U3fu3GnbHtVOKw0bNjTdu3enKzgAxDhNfKxOIjpv79+/3/z888+2Z596B6kppkmTJvZcrl5RDz30UMh7Nf6L3qMu46rNUG6Kekzl1b333mt7Wam3lPIzR4wYYfNavNKmTRszZcoU+znUA0q1TOoppd5iLqVcrF+/PvBcPcb27t1rk6u1v0rFOPfcc+3gfygaEhTh5PdNn332mc0cV9Z98+bN7WvLli0zu3btMu+++64566yzTKxSMrT2e/fu3TGb+KxEVHU9VH5TRJNQ0/cZ89xfvTjuSjEmKTJV3EWdZ+WNuC5v9aJRUKBxUNRdOlbpFK/8LzWzFLZ5tLAjGyuAUvNQcBNRvIlkeRdHB8L8rvJz/S5Qzc0dd9xhu8iNGzcu0A6p9ttevXrZf1PXQKC40/QLL7/8cuBOkOkXAMAfBbptWrduna2qC04g07IGStK/AfjzDk7jhOhBEiLi3fz580358uVzfAB+KlDNjYauVq6NBnAKptfUVgvgz+kXNMy8/aEx/QLiwNzgAQWzUIrCV199Ffb97jQPQMwEN998801gWfNtKIlMtTQa0MntzqfRG4cMGeLNngJFjPJJNHgfUByULVvWnHDCCdHeDSB/wY2SyJQcFVy9fv/99x+23vXXX2/zcQAAAGI6uFH2MoD89QpyfzfK/C/KPYMAIC6Dm3r16nm7J0CcUXfQ1157zS4z/QIA+KfAg/i5k4tp4KO0tLTDhssGijs147qzETPeBQDEeHDz008/mSuuuMKOZxOch+OewJk4EzB2XBvNIgwA8FeBkgDUU0o5BBplNDk52Xz33Xd21GJ1BQzXVRAAgFinG3ZNKaEpKHTTri7u2b0WL84+++y4GzW6QMHNwoULzWOPPWaqVKlikyT1OOOMM+w08eomDgCITbowh3s88sgjprhftGfOnGlHF3/vvffsvFOahyq717zcJ30P9evXN+XKlTOVKlUy7du3P2xG9oJuuzgoULOUmp2OOOIIu6wARyOwakA/JR2vXbs20vsIFNnpFyZPnmyX//WvfzH9AmKCLsyu6dOnm4EDB4actxlN2Jgff/zR1KxZ0066Ge41L2mMLE3cedxxx9mJSkeOHGkuuOACO75c1apVfdmHYldzo4j166+/tsutWrUyw4YNM1988YWtzdEXAeDPqu1ff/3VPph+AbGiRo0agYcmIVRtjftck5lqlu46deqY0qVL2/HNVGMRPMKw1p82bZq9yGtiQ10P5s2bF/ZvqsZDM2trNm7Nyq33aSbwDRs2hH3fb7/9Zkf5VlOQajCU+uDWXmi+tk6dOoWsr1oL1V64/679GjVqVKBWyh0hWa9rVnN9RgUsDz74oO3d6L7vzjvvtJ1l9B5N9pnda9nZvn27HetN/679bdSokZk6dWrg38PtU1bajmprdE095ZRT7PeiiSODB9QNVtDPm53333/fHhvuzZm+p3/+85/2O9R3cfnll4fst/tdDB8+3G7/qKOOsvNM6gbP9fzzzwe+e3W0uPrqq42nnAKYOXOm88Ybb9jlH374wTn55JOdhIQEp0qVKs6sWbOcWLZ7925lP9v/x6qMjAxn8+bN9v8RlZbiOMPNnw8tw9Py1va+/fZb+4j4d1mEeXZ8+2z//v3OqlWr7P+zOnjwoH1kZmYGXjt06JB9LT09PeLrhqN109LSQt7jmjRpklOxYsXA8xEjRjgVKlRwpk6d6qxZs8a5//77naSkJOf777+3//7zzz/b82edOnWcGTNm2M9/yy23OEcccYSzbdu2HPdBf0fbad68ubNgwQJn6dKlTsuWLZ02bdrk+J69e/c6xx13nHPmmWc68+fPt9ea6dOn2/dLt27dnMsvvzzkPXfffbfTrl07u7xr1y6ndevWTo8ePezxpofK6rfffnOSk5OdXr16OatXr3befPNNe+0aNGhQ4H2PPfaY/Yx6zx9//JHta9nRtocNG+YsWbLEWbdunfPcc885JUuWdBYvXhx2n3Kj7/zpp5+239XWrVuzXaegn1dUZio7mTx5sv0+3333Xftcx06DBg2cm266yfnmm2/sd3799dfb6772y/0udNz07NnT/g29V39zwoQJ9t+//PJLWw5TpkxxfvnlF2f58uXOqFGjnPz+rvJz/S5QcJOd7du3Z/vjiTUENwQ38XqxLSripbzDnYQfeeQR+0hJ+fsmYt68efa1t99+O2TdJ5980r6+c+fOwGsLFy60r7k3kS5dOPX6li1bAq8pUIhUcFOrVi27P8FatGhhL4zBwc2QIUMC/64ATBf9oUOH5rgP+jt636JFiwKv6SKo19wLf1b/+c9/7EVW15bs5BbcZL1ouwYMGGAvzMHlMXbsWKd8+fKBY3LkyJFOvXr1Qt6X3Wt5Ke+LL77Yuffee8PuU04UJJQrV85WHui7UdAUTkE/b7u/3jdmzBh7PMydOzew7quvvnrY+xXUlC1b1vnoo48C34XKJjhQu+aaa5zOnTvbZR3HCn727NmT62eOVHBT6CFTVV2lh5tBDgAoetTkofzJtm3bhryu55oUOVjr1q0Dy5oUVs1F7jpqQnFnAu/YsWPIei1atAg8V7Ksmjmybtul3khNmza115ZI0t/T/gdfr/QZU1JSbDNYYSgf9fHHH7f7raYZlcFHH31km7MK4pxzzrHlsGDBAnPhhRfapiH1Uvbi886YMcP06dPHfPLJJ6Zdu3aB15WCojwf5dm636u+kwMHDtg8JJe+95IlSwaeq3nK3dfzzz/f5uSqia1Lly62uSs1NdXEXEKx2uoeffRR89xzz9kCEn1gtUkOGjSIxEngr+kX3JOHchiYfqH40IjU7lhHwRcUTTSc9Ti47777DltXQcBpp5122LoahiPrusqLiSUffPBBINdCk2kWVG7vVdlkzWULzvGIhqefftpeF5V7ou9F10XlAWUd6DavlLejyUj10LGjnJWJEycGjq9Iatq0qVm+fLl56aWXbLDqBkO6xjdr1iyQfxMsOLE5+JgUvV/nQFFgpG1rqJiPP/7YJrGrN9iXX35pA1wvFKjmRkHMhAkTbCLxihUr7EPLKnS6ggN/3wRMmjTJPsIl7yH+lCpVyj6C75Z1V6vXVIMR6XUjoUKFCqZWrVq2c0gwPW/YsGHIa4sWLQos69hetmyZadCggX2uO3T3gly7du2Q9ZYuXRp4rh5au3btCrwvq8aNG9taix07dmT777qwBvf8kqxjz6issg4qq7+n4UyCAyN9Rl2AdRNSGNqORuhX78gmTZrYmorvv/8+133KKwULBw8ezPHfC/N5jz/+eDNnzhzz9ttv22u8S0H2Dz/8YJPN3e/VfSjpOK90LCtBWrGCkqKVkDx79mzjlQIFN1OmTLHZ77fddps9APXQsoIb/RuAP+9cVH1Lky2Kin79+pmhQ4faLuIKPtSrRgGDW2PkGjt2rHnzzTfNmjVrbK+YnTt3mptuuinstnVnr4umejspGFIPG9VGqBdPdtRLSj241AtHF2ONjP/GG2/YC7Wce+65Nlj673//ay++ajX49ttvQ7ahXkv6e7qQbtu2zQYHvXr1sqkU2hftvy7mem/fvn0LXbuqmpVPP/3U7qOag3Rd3LJlS677lNW+ffvMgAEDbBCp3pYqL5Xvxo0bzTXXXJPj3y/s5z3ppJNsgKNydsfLUaCmIV/UQ2r+/Pl2MmDVwKgiI6/NeBobSDVaOpb0efSdad80hIxnnAKoWrWqTfjJSq8pCzuWkVBMQnG8JrgWFfFS3uESH2NJfhKK9Z0oYbl27dq2d1OTJk2cDz/8MPDvbkKxer2ot1OpUqWchg0bOrNnzw67D+7fUWKpekCVLl3aad++vfPrr7+GfZ961lx11VU2GVW9b9TbKjgBeeDAgU716tXttvv06eP07t07JKF47dq1zumnn26TX7Xf2n9RwqwSpbX/NWrUcB544IGQnmkFTShW8rOSnJWsW61aNeehhx5yunbtGpL4nNM+BdMxdcUVV9gkYu1jzZo1ncsuuyzXhOKCft52WRKRdVxr//v27Wuf6/eqz6Hru747fYfqleUm9uaW3K3eblquVKmS3bfGjRvbnm9eJhQn6D/5DYg0no0iQFW3q9+8qKrs5ptvtpGrosJYTppTVdru3bttNWwsUkSrRCxVA0Y0TyN9nzHPlf9z+a4UY5LKRW7bRZhn5Y24Lm8lVOouVlPRaOyOWKVTvJqE1CxQ2E4fqhHQ51UqQn5yfVTTr5oANUPFu0iWd3F0IMzvKj/X7zwnFF955ZUhz1X1pvY6tSu6GdVKmjrvvPPy90kAAAAiKM/BTdbEoauuuirked26dSO3V0Ac0N3b66+/bpfVhTNrcigAwBt5PtuqCQpA/ppflOjoLgNFnRJWCzKViJKH9QD8Uqhbya1btwYmXFPWM5N5ASaki656GLjLAIAYDm7UTU3dytzuXO7Ju2vXrmb06NEmOTk50vsJFDn6TcTaAGsAUBwUqKuC+sdrltF3333XZr/rob7zeu3ee++N/F4CAAB4WXOjAX40D4U7tbxcdNFFdrhsJU6OGzeuIJsF4rLLsxT1bs8AEPfBjSa8ql69+mGv6wTu9WRYcU/Jeun7TMKh1D/HpYn0ODfwtbfUf/7zH7usuWA0NDoAIEaDG80wqoH6lHPjDrKzf/9+O5lm8GyxKIBDqabEmArm8NARRY0G8NL8Le4yACCGg5tnn33WTr+edRA/BTqa3r04G/lJ6CRp+ZWYmWr+nrLMI7XaGpNI0rfXNJeO8tMAFIxSH5SUr2uO39R1Xfmkb731lqcjOmuepnPOOcfOz6UZsrOO5qzZs7UPWScFze/fKW4KFNw0atTIjt+hKdA1DYM7yZkm2CrMFPcw5lBCWTP6lOUmOWOfSS1ZTrf8hS6WO889MfQFBTbUJAAxe5OTX33OPynfw3gMHDjQvP/++3Zix0qVKtkbVb3Wtm3bIhuQFEVt2rSxs5vnNMP2fffdFzJLd3ZBlwbR1TY0wSUKGNykp6eb+vXr21k+e/Tokd+3IzcJCeZQiWRzyMm0/49IEMIcUgCyjDCv6XJeeeUVc9xxx9kAZ9asWWb79u0FLidtryjklcXafmpfNPt5TsqXL28fuQ07EW4bxVGJglS1a2IrALknFP/vf/+zDy0DsUB3/fPnzzdDhw61zSH16tUzLVu2tEnvl112WWC99evX20EodWHVJIXqCasgyKXmEtXQvPjii4FJDlWroCFBRo0aZfPM9FCTiXz77bemY8eOdnvqkNKlSxezbdu2kPHTNFaa/r1mzZrmmWeeyfWzuPugxH3VXmiMNe2nJlZ0aZ86depknnzySVOrVi074KysXLnSnHvuuba14aijjjK33nqrSUlJOexvKJdUA9SqDHr27GmDI9fMmTPNGWecYZuTtI1LLrnE/Pjjj4dtQy0cqqFRGf3jH/+wZRTcLKVyymlSUfczussKSDX0ilu+er/KWMvBTVff5lLe6vGsVhj387dv395+B/GiQF1x7rjjDvvD4IQNhO8KvmrVKvtg+gXECrcmQM0aBw8ezHYdHa8KbHbs2GEvxJ988on56aefTOfOnUPWW7dunR0a5P/+7//shVVBjTqVqFZfzSR6KOjQhVuBRNOmTc3SpUttUKBASYGIq1+/fvZv6cL98ccf24v28uXLc/082gfN4aZx17Rd5Z306tUrZB3VSmk0fX0OtTroIt6hQwfbHPfll1/aGxBNBt27d+/D3rd69Wq7L1OnTrWfU8GOS9tRXp0+k9bVcA+aZDrr712fTWPAad9UPpdeemmBasnURKUyU86rW74KmrLalUt5631KJbnpppsCn0/7XZCpNeIq50YHg75IHYCK/MqVKxfy7zoAgOJOVcW6c3KXgVigCVyVtKoAZPz48ea0004z7dq1M9dee61p3LixXUfnd9Vs/Pzzz4FJkdU79pRTTrHn/xYtWtjXVIuh14On3lEzi2pQgptJxowZYy+0Tz31VOC1l156yW77+++/tzUqEydONK+99po577zz7L+rhkKdVnKjlgTtQ+3ate1zjZJ/8cUX25ofdx90jVINk9sc9cILLwTe516/tI8KOnTj7g51ovW1n/o8+uyPPfaYDVQef/xxG8hknUBa66osdEMTnNiroMldV+PAKdjQ573//vvz9d0pKFVNi4LScM1QY3Ipb9VQqXJCAY1q7kTX8nhSoJobVcHpi1Lkq4NSiVDBDwB/BjSq7teD4AaxROfvTZs2mXfeecfWAujOXUGOgh7R3bwuhG5gIw0bNrTnfv2bSxfGvMwpqN60c+bMCdQa6aHcTVEzjh4KlFq1ahV4T+XKlQNNSOEcffTRgcBGVDOimhN33kP3wh2cZ6PPoATq4BtzJVJnfZ/WCZ5OSNtWYLBhwwb7XB1rVAOivCU1W2liUXH/Pfh9wcFl8+bNQ8ox0r7Opbz1uRREqlyuueYaG+ypt1Y8yVfNjb74p59+2kZ+OhBV7aU2QHpIAUDRovyP888/3z4efvhhc8stt9jxy/Ize3fWWvucKCBwa0WyUn6Nmpa8lNf9zC99JgV4Cg50o69rpHJqgvNyoiEll/LWzZaa6BYsWGBbYFTb9e9//9ssXrzY5k8Vu5obJWQNGDDARoGKlJ977jmbfwPgcGq/Vru6HvHUlo34pJoZN6G0QYMGtvYhuAZCTS3K5dB64aiGJCMjI+Q11Qp99913tmbjhBNOCHko8Dj++ONtZxVdXF2qSdCNdG6U+KxaKNeiRYtsk1G4Wh99PtVuBCfQfvHFF4e9T+togNrgbev6pxot/a5Vy/PQQw/ZWhBtM6faD73PpeagZcuW2fULIrvyzeq0XMpblICs2irlECkXSNt98803TbzIV3Cj9snnn3/eDtSnZDQlcGmsG5IlgeyHTVDbtx5aBmKBLsqqdVd+yzfffGPzapRQO2zYMJtELOo5oyYLjV2mpN4lS5bYnkzKzVGTSji6oCpIUQ8e9c7R9UE3wUpOVhOOcnbUNKLrSPfu3e2FWgHDzTffbPNZZs+ebXv6qAYpL/OxqQaqW7duNhBRL7C77rrLJs6Gy0nR53Lfp7+lJhyNJaMeRcFTC6kGRvulwO6DDz6wNVvKn9F+KRlZvYwmTJhga5603zkN2jl27FgbOKjXlMpCQZCSeQtC5avvTYGVyje7c8sduZS3vh/l4yjZWMGh8mQ19lFBA64i3yylQtAEmS79ABT9KWrOS+IXUNyULl062ruAKMjvoHp+UiCh3JaRI0fai54ujqqJUIKxauZF53X1WtIF/6yzzrIXc+XmqPkiLz16FDSohke1HgqedEFWzcgDDzxgLrjgApsQq+YcbdMNYJTy4DanaNoS9S4K7tKdE9VGKDFW1yZd0NUdWzfh4SiPRhf7u+++2yZH67nykEaMGBGynmpkTjzxRFsG2mcFC0rFEO33tGnTbDClpijV+Kg1I3hCadeQIUPsQz3KtL/KdSrogHv6npQjpSBT5aXAzM31cdWqVStseSs/6LPPPrMDLe7Zs8f+mxKw3Q4Q8SDByUd9udrpfv/995AEMh2EiiKLSjudvkglPetHoy84JkcmdRyTnJFiUkuWj8ggfrF8oo2l2buZuZvyzg/1ttGF2x3jJVbpFK+mECWyxtscZ3mZmsBv8Vzefgj3u8rP9Tsxv1+aqgqD70a1IxrYKDhhi67gAAAgWvIV3KiqMasbbrghkvsDAADgX3AzadKkwv01oBhR1bRGQxXlAaiaGkBkm6XcHBig0IP4RZoyyZUQpfY1JbopMz8vlMylNk3NGwLEYi6PenDoQY9CAChGwc306dNt9zl1sVOXQ42cqJGPleAZjroZKiv/zDPP9G1fgfxQAr56FOrBCMXxizGMgNj7PUU9uFHXO3VtU/97dR3UXCfqlqd5MHKifvoap0CDD2nYayAWKaDRIFl6ENzEHw06J6mpqdHeFSBupP01unNhz5mJ0f4QGqmxf//+gdfUB193ugsXLszxfZq8TN12NbiSBm0KR/37g2e+VVcyUTOBJ00FkYg6tQ33EQE0ieRePrpboJz8ES/lrSZxdUtVLbM+j27KYrXrr8aycYMxUN6xPiyHpnRSLJD1HJGfc0ZUgxuNrqhamOARIUXPNZJjdj7//HM7m2pexzUYPHhwyBT1Lo3GqG7skabxaQrPMaWdA8bY77HwJ8vcmviKO/1gNG6CLlB5GRE1r7Q9964+li988VLe0aDvVInimzdvjunvV2Ve1Mu6KKG8C0fnS12js9q7d2+et1Gkum/og2l4bE1SltfRHVUrFDwktmpuNBqnBiL0YhC/1JK5j6iZK1trY0xqicgM4qdaLoQ/EenCpGMikhcA1UxqaHbRSKHBsxIXZ16Vd7ToZkw3abE6xYbKWyP3apbteCjvWEd5F5zOC6phzOk4zc9gmVENbhSgqF1ty5YtIa/reXbzgmiocCUSa3jurNVUunvSXBuagC2YBhzMbgh8FZ4nP/RI3b1pO+6jkDih5aW4EyJ+TARvz7PjrYjyoryjSZ8jVpt9dI7UMP26G46X8o5llLd38nP8RjW40Z1ss2bNzKxZswLduXVg6LkmJ8uqfv36ZuXKlSGvaUZW1eiMGjXK1sgAsULH98MPPxzt3QCAYifqzVJqMtLIx5oErGXLlnYiL01Dr95Toploa9eubXNnVCWlCcqCHXnkkfb/WV8HAADFU9SDm86dO9vEoYEDB9pJOU899VQzc+bMQJKxZiKnKhUAABSZ4EbUBJVdM5RoavdwXn75ZY/2Cij89AsfffSRXdbAlEy/AAD+ILsM8Ijyx5YuXWofRX1MFwAoSmKi5gaIR+oJ2K5du8AyAMAfBDeARxTQnH322ZQvAPiMZikAABBXqLkBPKLpBdx5zTSQZCwPzw8A8YSaG8AjGo5/6NCh9hGrQ/MDQDwiuAEAAHGFZinAI5prSNODCANRAoB/CG4AjyjHhi7gAOA/mqUAAEBcoeYG8EhGRoad4V7OO+88anEAwCfU3AAeBjcLFy60Dy0DAPxBzQ3gEeXbtG7dOrAMAPAHwQ3gEQU0F1xwAeULAD6jWQoAAMQVam4AD6dfyMzMDIxzw/QLAOAPghvAI5pyYfDgwXa5f//+plSpUpQ1APiAZikAABBXqLkBPJx+4YEHHggsAwD8QXADeEQ5NmXKlKF8AcBnNEsBAIC4Qs0N4BGNSjx//ny7fOaZZzKQHwD4hOAG8DC4mTdvnl1u06YNwQ0A+ITgBvCIxrZp3rx5YBkA4A+CG8CrH1diorn44ospXwDwGbeTAAAgrhDcAACAuEKzFOCRtLQ0M3ToULuswfyYfgEA/EFwA3jInTgTAOAfghvAI5pyoU+fPoFlAIA/CG4AD6dfqFChAuULAD4joRgAAMQVam4AD0coXrRokV0+/fTTGaEYAHxCcAN4GNx8+umndrlFixYENwDgE4IbwCOacqFJkyaBZQCAPwhuAK9+XImJplOnTpQvAPiM20kAABBXqLmBNfKT72OuJPqcf1K0dwEAUAQR3AAeTr8wYsQIu9y3b1+mXwAAnxDcAB46ePAg5QsAPiO4ATyiKRd69+4dWAYA+IPgBvBw+oWjjjqK8gUAn9FbCgAAxBVqbgAPRyhetmyZXW7WrBkjFAOATwhuAA+Dmw8//NAun3rqqQQ3AOATghvAI5pyoWHDhoFlAIA/CG4Ar35ciYnmmmuuoXwBwGfcTgIAgLhCcAMAAOIKzVKAR9LT083o0aPt8p133slAfgDgE4IbwCOO45i9e/cGlgEA/iC4Abz6cSUmmttuuy2wDADwB2dcwCPq/l2jRg3KFwB8RkIxAACIK9TcAB6OULxy5Uq73KhRI0YoBgCfENwAHgY3b7/9tl3WSMUlS5akrAHABwQ3gIc5NyeeeGJgGQDgD4IbwKsfV2Kiuf766ylfAPAZt5MAACCuENwAAIC4EhPBzdixY80xxxxjypQpY1q1amWWLFmS47ovvPCCOfPMM02lSpXso3379mHXB6I9/YIeWgYAFJPgZvr06aZv375m0KBBZvny5aZJkyamQ4cO5o8//sh2/blz55rrrrvOzJkzxyxcuNDUrVvXXHDBBWbjxo2+7zsQjqZc2LFjh30w/QIAFKOE4hEjRpgePXqY7t272+fjx48377//vnnppZfMgw8+eNj6kydPDnn+4osvmjfeeMPMmjXLdO3a9bD1Dx48aB+uPXv22P9nZmbaR8RFYg4hbcN9RECePmcMzn3kyfeTw99R8BHpv6ceUjfeeGNg2a/PE+u8Km9Q3rGA49s7+TlnRDW4SUtLM8uWLTP9+/cPvKaLgJqaVCuTF6mpqbbKv3Llytn+++DBg82jjz562Otbt241Bw4cMJGWnJESga04prRzwBj7PSYUems51YJFfr8jKy/7HakfzO7du+0FN9JdtkuXLm3/v23btohutyjzsrxBeUcbx7d33ImIYz640QlfA51Vr1495HU9X7NmTZ628cADD5hatWrZgCg7CpzU7BVcc6OmrKpVq5oKFSqYSEstubvwG7G1NsaklihvTELhg5tq1ar5s98Rlpf9jtTJKCEhwR4TXGwp73jD8U15xwvl5RaZZqnCGDJkiJk2bZrNw8npQ+vO2b17DqaLmCcXsggEI4HtuI9CytPnjNR+R5CfgYaCm0gfE7qorF692i43aNCAwMnj8kbOKG9/Ud7eyM/5IqpnlipVqtgh6bds2RLyup7nNpvy8OHDbXDz8ccfm8aNG3u8p0D+HTp0yMyYMcM+tAwA8EdUg5tSpUqZZs2a2WTg4LtdPW/dunWO7xs2bJh5/PHHzcyZM03z5s192lsg/3dv9erVsw8tAwD8EfVmKeXDdOvWzQYpLVu2NM8++6zZt29foPeUekDVrl3bJgbL0KFDzcCBA82UKVPs2Di///67fb18+fL2AcSKpKSkQG8pAEAxCm46d+5sey4pYFGgcuqpp9oaGTfJeP369SHtbOPGjbO9rK6++uqQ7WicnEceecT3/QcAALEl6sGN9O7d2z6yo2ThYL/88otPewUAAIqimAhugHik8ZcmTpxol2+++WbbTAUA8B7BDeARDVLn9gRk+gUA8A/BDeDVjysx0dxwww2BZQCAPzjjAh5RIvzxxx9P+QKAzxgeFAAAxBVqbgCPaEDKdevW2eUTTjiBqQYAwCfU3AAe0ZQLU6dOtQ+mXwAA/1BzA3hEUy5oxnp3GQDgD4IbwCMa16ZHjx6ULwD4jGYpAAAQVwhuAABAXKFZCvBw+oVXX33VLnfp0oXpFwDAJwQ3gEc05cKGDRsCywAAfxDcAF79uBITTefOnQPLAAB/cMYFPJx+oX79+pQvAPiMhGIAABBXqLkBPJx+Yf369Xb56KOPZvoFAPAJNTeARzTlwiuvvGIfTL8AAP6h5gbwiKZcqFq1amAZAOAPghvAw+kXevXqRfkCgM9olgIAAHGF4AYAAMQVmqUAD6dfmDZtml2+9tprmX4BAHxCcAN4RFMu/PTTT4FlAIA/CG4Ar35ciYnmiiuuCCwDAPzBGRfwcPqFxo0bU74A4DMSigEAQFyh5gbwcPqFzZs32+WaNWsy/QIA+ISaG8AjmnLhxRdftA+mXwAA/1BzA3hEUy5UrFgxsAwA8AfBDeDh9Av33HMP5QsAPqNZCgAAxBWCGwAAEFdolgI8oiTiGTNm2OWrr76agfwAwCcENyjSRn7yfeE34jgmOSPFpJbcrczfQm+uz/knBbqCr127NrAMAPAHwQ3gkZIlS5pLLrkksAwA8AfBDeARBTTNmjWjfAHAZyQUAwCAuELNDeARx3HM1q1b7XLVqlUZyA8AfEJwA3gkPT3djBs3zi7379/flCpVKrKJ0BHmJkIDQFFHcAN4KDk5mfIFAJ8R3AAeUU1Nv379KF8A8BkJxQAAIK4Q3AAAgLhCsxTg4fQL77zzjl2+7LLLmH4BAHxCzQ3gEU25sHLlSvtg+gUA8A81N4CHIxR36NAhsAwA8AfBDeARBTSnn3465QsAPqNZCgAAxBVqbgAPp1/YvXu3Xa5YsSLTLwCATwhuAA+nXxg1alS20y8UVRGZNsJxTHJGikktuduYhIRCb45pIwBkRXADeCgpKYnyBQCfEdwAHlFNzYABAyjfGMBEpUDxQkIxAACIKwQ3AAAgrtAsBXg4/cIHH3xgly+66CKmX0C+kcANFAzBDeARTbmwYsUKu3zhhRdSzig2yHFCtBHcAB6OUHzOOecElgEA/iC4ATyigOass86ifAGgOCYUjx071hxzzDGmTJkyplWrVmbJkiVh1//f//5n6tevb9dv1KhRIK8BAAAg6jU306dPN3379jXjx4+3gc2zzz5rZ1Jeu3atqVat2mHrL1iwwFx33XVm8ODB5pJLLjFTpkwxnTp1MsuXLzf/+Mc/ovIZgJymX0hNTbXLycnJTL8AxDgSuONH1GtuRowYYXr06GG6d+9uGjZsaIMcXQheeumlbNfXcPZKzuzXr59p0KCBefzxx81pp51mxowZ4/u+A7lNvzB8+HD70DIAoBjU3KSlpZlly5bZeXdcJUqUMO3btzcLFy7M9j16XTU9wVTT89Zbb2W7/sGDB+3D5U5kuGvXLtubJdIOpOwt/EYcxyRkppgDJZyIzL2jz+rLfkeYb/vtUXnr+D5w4EDgteC5pShvju+ifnyHw/Htb3mPm/OjiTW3n3N8xLe5Z8+eQK14rpwo2rhxo/bQWbBgQcjr/fr1c1q2bJnte5KSkpwpU6aEvDZ27FinWrVq2a4/aNAg+zd4UAYcAxwDHAMcAxwDpsiXwYYNG3KNL6Kec+M11QoF1/SotmbHjh3mqKOOitkcCEWndevWNRs2bDAVKlSI9u7EPcqb8o5nHN+Ud7xQjc3evXtNrVq1cl03qsFNlSpVbHfZLVu2hLyu5zVq1Mj2PXo9P+uXLl3aPoIdeeSRpihQYENwQ3nHK45vyjuecXx7o2LFirGfUKwchGbNmplZs2aF1KzoeevWrbN9j14PXl8++eSTHNcHAADFS9SbpdRk1K1bN9O8eXPTsmVL2xV83759tveUdO3a1dSuXdt2/Za7777btGvXzjzzzDPm4osvNtOmTTNLly41EyZMiPInAQAAsSDqwU3nzp3N1q1bzcCBA83vv/9uTj31VDNz5kxTvXp1++/r16+3Pahcbdq0sWPbPPTQQ2bAgAHmxBNPtD2l4mmMGzWjDRo06LDmNFDe8YDjm/KOZxzfsSFBWcXR3gkAAIC4GcQPAAAgkghuAABAXCG4AQAAcYXgBgAAxBWCmxg0duxYc8wxx5gyZcrYmdKXLFkS7V2KSxpeoEWLFuaII46wM9BrdnnNRg9/DBkyxI4Sfs8991DkHtm4caO54YYb7IjsZcuWNY0aNbJDZyDyMjIyzMMPP2yOPfZYW9bHH3+8ndiZPjvRQXATY6ZPn27H/lFX8OXLl5smTZrYiUH/+OOPaO9a3Jk3b5654447zKJFi+xAkJq5+4ILLrDjLMFbX375pfnPf/5jGjduTFF7ZOfOnaZt27YmKSnJfPjhh2bVqlV2fLBKlSpR5h4YOnSoGTdunBkzZoxZvXq1fT5s2DAzevRoyjsK6AoeY1RTo9oE/UDcEZs1z9Sdd95pHnzwwWjvXlzTeEuqwVHQc9ZZZ0V7d+JWSkqKOe2008zzzz9vnnjiCTu2lQbvRGTpfPHFF1+Y+fPnU7Q+uOSSS+z4bBMnTgy8dtVVV9lanNdee43vwGfU3MSQtLQ0s2zZMtO+ffvAaxrAUM8XLlwY1X0rDnbv3m3/X7ly5WjvSlxTbZlGFw8+zhF577zzjh35/ZprrrFBe9OmTc0LL7xAUXtEA8xqaqDvv//ePv/666/N559/bjp27EiZF8cRivG3bdu22XZbd3Rml56vWbOGovKQasiU+6Fq/Hga7TrWaLoUNbeqWQre+umnn2wziZq5NZq7yvyuu+6yc/ppyhtEvqZMM7DXr1/fTgitc/mTTz5p/vWvf1HUUUBwA/xVm/Dtt9/aOy14Y8OGDXZuOOU3KVke3gfsqrl56qmn7HPV3OgYHz9+PMGNB15//XUzefJkOz3QKaecYr766it7w1SrVi3KOwoIbmJIlSpVbMS/ZcuWkNf1vEaNGlHbr3jXu3dv895775nPPvvM1KlTJ9q7E7fU5KrEeOXbuHR3q3JXjtnBgwft8Y/IqFmzpmnYsGHIaw0aNDBvvPEGReyBfv362dqba6+91j5Xz7Rff/3V9sqkpsx/5NzEEFUXN2vWzLbbBt996Xnr1q2jum/xSF00Fdi8+eabZvbs2bYLJ7xz3nnnmZUrV9o7WvehmgVV22uZwCay1MSadWgD5YPUq1cvwn8JkpqaGjLJs+iY1jkc/qPmJsaofVxRvk76LVu2tL1I1DW5e/fu0d61uGyKUhXy22+/bce60az0UrFiRdvDAZGlMs6az1SuXDk7Bgt5TpHXp08fm+SqZql//vOfdrysCRMm2Aci79JLL7U5NkcffbRtllqxYoUZMWKEuemmmyjuKKAreAxSFf3TTz9tL7bqJvvcc8/ZLuKILA0gl51JkyaZG2+8keL2wdlnn01XcA+pubV///7mhx9+sDWTunnq0aOHl3+y2Nq7d68dxE81wWp+Va7NddddZwYOHGhr5eEvghsAABBXyLkBAABxheAGAADEFYIbAAAQVwhuAABAXCG4AQAAcYXgBgAAxBWCGwAAEFcIbgAAQFwhuAEK6JdffrGjHGtepHgaMVgzGeeXRma99dZbTVHx8ssvmyOPPDKuvtuCfnfB5s6daz/3rl278lx2jzzyiB1l2qXRvTt16lSo/UhLSzPHHHOMWbp0aaG2g+KL4AbFmk7k4R46cSM8TRMyatQo8+9//zuqAQn80blzZzsBZ050LOi7KkzQpekK7rvvPvPAAw8Ual9RfDFxJoq1zZs3B5anT59u54EJnkm5fPnyJtbpLjeac9e8+OKLdoJGL2ebTk9PN8VVtL/frDSpbLiJZTXxbCRotvh7773XfPfdd3YiSiA/qLlBsVajRo3AQydl1da4z6tVq2Zn9a1Tp44pXbq0rXqfOXNmjtvKyMiwMwDXr1/frF+/3r6mGcdPO+00U6ZMGXPccceZRx991Bw6dCjwHv09BQdXXHGFSU5ONieeeKJ55513wu6zqusff/xx07VrV1OhQoVAc5Duck866SS7Hf0tNRUFBwVu88Grr75qt6HPe+2119oJ/3Ly/vvv2/UmT56c4zrTpk2zMyIHmzFjhmnUqJG9CGrW7/bt29vZ7SUzM9M89thjOZar2ySkYLNdu3a27PT3u3fvbnbv3n1YrdrBgwftXX7t2rXtLOOaZFbNK8FUk6DZmlU2Kuvt27ebvFizZo0N3LQPmrl83rx59nXHccwJJ5xghg8fHrK+mrG0b+vWrct2e26TjY6DqlWr2u+vZ8+eNoAJruno3bu3re2oUqWK6dChg31df7tly5a2zGrWrGkefPDBkGNJ9Fzv1Xem9+oY0L669N03b97cztCuY/z666+3kzxm9cUXX5jGjRvbz3366aebb7/9Ns81aMHNUlrWfqs2x/3efv755zyVXaVKlUzbtm3t8QXkmwPAmjRpklOxYsVAaYwYMcKpUKGCM3XqVGfNmjXO/fff7yQlJTnff/+9/feff/5ZVw1nxYoVzoEDB5wrrrjCadq0qfPHH3/Yf//ss8/s+19++WXnxx9/dD7++GPnmGOOcR555JHA39D769Sp40yZMsX54YcfnLvuusspX768s3379hy/lXr16tntDh8+3Fm3bp19yOOPP+588cUXdr/eeecdp3r16s7QoUMD7xs0aJDd9pVXXumsXLnS7l+NGjWcAQMGBNZp166dc/fdd9vlyZMnO0cccYTz7rvv5rgv2s+EhARn0aJFgdc2bdrkJCYm2vLTvnzzzTfO2LFjnb179+arXFVWb7zxhvPTTz85v/zyi/Pss8/a923evNk+3O3dcsstTps2beznUVk8/fTTTunSpQPb076VKFHClsXatWudUaNGOUceeWTId52Vuw/6bmbMmOGsWrXK/h2Vx7Zt2+w6Tz75pNOwYcOQ9+n7O+uss3Lcbrdu3ex30LlzZ+fbb7913nvvPadq1aqHfQdap1+/frZ89Pjtt9+c5ORkp1evXs7q1audN99806lSpYr9TrO+T9+f3vPaa6/Z90yYMCGwzsSJE50PPvjAHo8LFy50Wrdu7XTs2DHw73PmzLGfu0GDBvZ41Xd3ySWX2O8iLS0t29+J9qFJkyYhn/Hyyy+3y7t27bJ/o0ePHoHv7dChQ3kuuwceeMB+LiC/CG6Av2Q9adeqVcuehIO1aNHCXmCCL4Dz5893zjvvPOeMM86wJ3OXXnvqqadC3v/qq686NWvW/PsHaIzz0EMPBZ6npKTY1z788MOwwU2nTp1y/d50kW/WrFnIRUgXuz179gRe0wW0VatWhwU3Y8aMsWUxd+7csH9DgZ32d/369YHXli1bZl9TQJKdvJargplw34/8+uuvTsmSJZ2NGzeGvK6y79+/v12+7rrrnIsuuijk3xVc5CW4GTJkSOC19PR0G+y4AaP+pv724sWL7XNd/BVwKJjNiS78lStXdvbt2xd4bdy4cTYoycjICHwHCpKDKfg5+eSTnczMzMBrChizvk9BSfA6Cg70Wk6+/PJL+zndQNENbqZNmxYSwJYtW9aZPn16voObrAGzK69lp0BUgRWQXzRLAdnYs2eP2bRpk60WD6bnq1evDnntuuuus00uH3/8cUi+wddff22bX5S34z569Ohh83xSU1MD66n636VmFTVVZNdUEExNC1mpGUf7p+YG/a2HHnoo0DzmUnOUmiRcat7I+rfUpNSnTx/zySef2GahcPbv32//r+YLV5MmTcx5551nm6WuueYa88ILL5idO3fmu1yz+4xZrVy50jYHqjkuuJzVFPLjjz/adbRdNVUFa926da7bzrpeYmKi3Sd3P2vVqmUuvvhi89JLL9nn7777rm0i02cOR+Wj5rHgv5GSkmI2bNgQeK1Zs2Yh79Hf1HpqtgkuM73vt99+C7ymJqTgdfSeH374wZaRLFu2zDYhqolOx4H7/WY9ToI/d+XKlc3JJ5982PdTGHktOzVrBv9WgLwiuAEK6aKLLjLffPONWbhwYcjruvAot0K5BO5DF2NdbIKDgaSkpJD36eKkvJRwFAQF099WAqb25b333jMrVqywvZeCczny+reaNm1q80F04QnO18iO8jrEDV6kZMmSNjD68MMPTcOGDc3o0aPtxVG5FvmR9TNmR2Wsv6eLdnA560KsPA+v3XLLLTYnREHepEmTbE+i4MCloPLy2fNLAbjydxQ8K4fpyy+/NG+++ab9t6zHiR/yUnY7duywxyKQXwQ3QDZ0AdDdpRIrg+m5LtjBbr/9djNkyBBz2WWXBRJORYnE6nml5MmsjxIlIvvTW7Bgge2tpIBGtQtKTP71118LtK3jjz/ezJkzxyZD33nnnbmuq7JatWrVYUGTahYU3CnQUm8fXUjzU65ZaRtuDURwIKbXVPuUtYxVgyUNGjQwixcvDnnfokWL8lQWwespWVdBlLbnUjCpQGTcuHE2KVoJ5blRjZ5b4+X+DdU21a1bN8f36G8qgA0ONlVmqn1RYrYru8+pY0EBoJKjlUitY/XMM8+0ie851RAGf24Frur6Hfy58yO77y2vZadEZn3HQH7RFRzIQb9+/cygQYPsBVw9enR3qVqB7HoOKQjQCfySSy6xNRZnnHGG7Vau52oCuPrqq21AowubTthPPPFERMtdFzA1LehOuEWLFraXk3tXXhBq5lGAo547ao559tlns11Pn0k9oT7//PNADxldYGfNmmUuuOAC2+NMz7du3Rq4OOanXLM2qammRtt2m3a0n6qxUs+xZ555xl4I9be0jpr71PRx11132UBLvXMuv/xy89FHH4Xt9RZs7Nixtmy17yNHjrQX+uCLsIIG9Qjq37+/XS8vzV2qJbn55ptts6F6hqks1MMpXMDbq1cv+x3oONO6Cpr1vr59+4a8T8eAXrvtttvM8uXLba2ZykV0HCrQ0GvqoaXjUL3usqPmVPVyq169ug2YVUNX0IH59L3pGNBnVRCnZi7tc17Kbv78+TnuIxBWvrN0gDiVNVFSiZrq2VS7dm3bm0dJk8GJvsG9pVzPPPOM7VGjXksyc+ZM25NHCZnq6dOyZcuQ3it6v3q+BNM+aF/CJRSPHDnysNeVHHzUUUcFeuNonXCJn6J1tL2ckj/VS6hatWpO3759c9wf9b5RGbmJrXpPhw4dbC8g9Vo66aSTnNGjRxeqXF09e/a0n1H/7vYUUjLqwIEDbeKptqeEbfVcU0+f4F5CSgbW93DppZfanmZ5SShWLzZ9Z6VKlbK9e2bPnn3Yuup5pHWHDRvm5MZNttX+ut+VehKpt124BFxRcrcSr7Uv6uWmZGElOQe/T0nZKiMda5UqVbKJyMEJxvo8Kid9L+rFpF51wWXtJhSrh9wpp5xi/5Y+/9dffx3YRn4TitVD7fTTT7dlr22rbPNSdgsWLLC92lJTU3MtVyCrBP0nfPgDAGFvkGzCrpKQlVxd3Kh2QQnUSghWTUc4qqnQ1AZvvfWWb/tXVMtOOTiqoRswYEDU9g9FFzk3AApF+TUTJkw4bEC5eKfePeqppMEE1csnt8AGeS87Nd2pt50CZqAgCG4AFJpyZ7p06VKsSnLq1Kk2iVs1McOGDYv27sRV2Sk3SDlJ4aZ5AMKhWQoAAMQVam4AAEBcIbgBAABxheAGAADEFYIbAAAQVwhuAABAXCG4AQAAcYXgBgAAxBWCGwAAYOLJ/wPqJpZr0IE7ugAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.bar(\n", " torch.arange(len(sorted_probas)), sorted_probas, \n", " alpha=0.5, label=\"Sorted probabilities\"\n", ")\n", "plt.step(\n", " torch.arange(len(cumsum)), cumsum, where=\"mid\",\n", " color=\"darkorange\", label=\"Cumulative sum\"\n", ")\n", "\n", "# Highlight cutoff\n", "plt.axhline(\n", " top_p, color=\"red\", linestyle=\"--\",\n", " label=f\"top_p = {top_p}\"\n", ")\n", "plt.axvline(\n", " n_kept - 0.5, color=\"gray\", linestyle=\":\",\n", " label=f\"Top-p cutoff at {n_kept} tokens\"\n", ")\n", "\n", "plt.xlabel(\"Token rank (sorted by probability)\")\n", "plt.ylabel(\"Probability\")\n", "plt.legend()\n", "plt.grid(alpha=0.3)\n", "plt.ylim(0, 1.05)\n", "# plt.savefig(\"14.pdf\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 31, "id": "42f1d126-bfc2-44dc-953a-c8d0a2720adc", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.4538, 0.0000, 0.0000, 0.0000, 0.0829,\n", " 0.2752])\n" ] } ], "source": [ "# Step 4.3.2: Zero out beyond cutoff\n", "kept_sorted = torch.where(\n", " keep_mask, sorted_probas,\n", " torch.zeros_like(sorted_probas)\n", ")\n", "\n", "# Step 4.3.3: Map back to original order\n", "filtered = torch.zeros_like(toy_probas).scatter(0, sorted_idx, kept_sorted)\n", "\n", "print(filtered)" ] }, { "cell_type": "code", "execution_count": 32, "id": "3ebc6a43-be43-4328-a567-41237403cac3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.5589, 0.0000, 0.0000, 0.0000, 0.1021,\n", " 0.3390])\n" ] } ], "source": [ "# Step 4.4: Renormalize to sum to 1\n", "denom = torch.sum(filtered).clamp_min(1e-12)\n", "renormalized = filtered / denom\n", "print(renormalized)" ] }, { "cell_type": "markdown", "id": "5d1f1ab9-8af6-47b0-9fec-30d3fcbf57cb", "metadata": {}, "source": [ " \n", "### 4.5.2 Adding a top-p filter to the text generation function" ] }, { "cell_type": "markdown", "id": "c4013204-fcb0-4bf0-8037-2071da740eb6", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 33, "id": "c06c9c20-da8e-4f87-a232-8d4c1ab501a7", "metadata": {}, "outputs": [], "source": [ "def top_p_filter(probas, top_p):\n", " if top_p is None or top_p >= 1.0:\n", " return probas\n", "\n", " # Step 4.1: Sort by descending probability\n", " sorted_probas, sorted_idx = torch.sort(probas, dim=1, descending=True)\n", "\n", " # Step 4.2: Cumulative sum\n", " cumprobas = torch.cumsum(sorted_probas, dim=1)\n", "\n", " # Step 4.3.1: Keep tokens where prefix cumulative mass (before token) is < top_p\n", " # Example: [0.5, 0.41, 0.09] with top_p=0.9 should keep the first two tokens\n", " prefix = cumprobas - sorted_probas # cumulative mass before each token\n", " keep = prefix < top_p\n", " # Always keep at least one token (fallback for very small/non-positive top_p)\n", " keep[:, 0] = True \n", "\n", " # Step 4.3.2: Zero out beyond cutoff\n", " kept_sorted = torch.where(\n", " keep, sorted_probas,\n", " torch.zeros_like(sorted_probas)\n", " )\n", " # Step 4.3.3: Map back to original order\n", " filtered = torch.zeros_like(probas).scatter(1, sorted_idx, kept_sorted)\n", "\n", " # Step 4.4: Renormalize to sum to 1\n", " denom = torch.sum(filtered, dim=1, keepdim=True).clamp_min(1e-12)\n", " return filtered / denom" ] }, { "cell_type": "code", "execution_count": 34, "id": "71fa717b-adee-4083-a5b0-2436094f8666", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "torch.Size([1, 151936])\n" ] } ], "source": [ "with torch.inference_mode():\n", " next_token_logits = model(input_token_ids)[:, -1]\n", "print(next_token_logits.shape)" ] }, { "cell_type": "code", "execution_count": 35, "id": "344a3b86-c8e2-40e3-af1e-afb9b45f4ae5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "' __': 158x\n", "' Berlin': 435x\n", "' ____': 169x\n", "' ______': 209x\n", "' Munich': 3x\n", "' Hamburg': 3x\n", "' _____': 18x\n" ] } ], "source": [ "torch.manual_seed(123)\n", "probas_lowT = torch.softmax(\n", " scale_logits_by_temperature(next_token_logits, 0.35), dim=-1\n", ")\n", "count_samples(probas_lowT, threshold=1, tokenizer=tokenizer)" ] }, { "cell_type": "code", "execution_count": 36, "id": "9258c857-674f-48e9-b595-0fe78ff2be8b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "' Berlin': 534x\n", "' ____': 217x\n", "' ______': 249x\n" ] } ], "source": [ "torch.manual_seed(123)\n", "probas_lowT = torch.softmax(\n", " scale_logits_by_temperature(next_token_logits, 0.35), dim=-1\n", ")\n", "probas_lowT_filtered = top_p_filter(probas_lowT, top_p=0.8)\n", "\n", "count_samples(probas_lowT_filtered, threshold=1, tokenizer=tokenizer)" ] }, { "cell_type": "code", "execution_count": 37, "id": "161f5841-8fa2-43a5-8db6-25639c6e33f3", "metadata": {}, "outputs": [], "source": [ "@torch.inference_mode()\n", "def generate_text_top_p_stream_cache(\n", " model,\n", " token_ids,\n", " max_new_tokens,\n", " eos_token_id=None,\n", " temperature=0.,\n", " top_p=None\n", "):\n", " model.eval()\n", " cache = KVCache(n_layers=model.cfg[\"n_layers\"])\n", " model.reset_kv_cache()\n", "\n", " # Step 3.1: Get logits\n", " out = model(token_ids, cache=cache)[:, -1]\n", " for _ in range(max_new_tokens):\n", "\n", " orig_device = token_ids.device\n", "\n", " if temperature is None or temperature == 0.0:\n", " next_token = torch.argmax(out, dim=-1, keepdim=True)\n", "\n", " else:\n", " # Step 3.2: Apply temperature scaling on logits\n", " logits = scale_logits_by_temperature(out, temperature)\n", "\n", " # Step 3.3: Convert to probabilities\n", " probas = torch.softmax(logits, dim=-1)\n", "\n", " # (New) Step 4: Apply top-p filter to probabilities\n", " probas = top_p_filter(probas, top_p) \n", "\n", " # Step 3.4: Sample token according to probabilities\n", " next_token = torch.multinomial(probas.cpu(), num_samples=1)\n", " next_token = next_token.to(orig_device)\n", "\n", " if (eos_token_id is not None\n", " and torch.all(next_token == eos_token_id)):\n", " break\n", "\n", " yield next_token\n", " out = model(next_token, cache=cache)[:, -1]" ] }, { "cell_type": "code", "execution_count": 38, "id": "820dc305-8377-47cb-b633-8de906b82a68", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \\boxed{18}" ] } ], "source": [ "torch.manual_seed(123)\n", "response = generate_text_stream_concat_flex(\n", " model, tokenizer, prompt, device,\n", " max_new_tokens=2048, verbose=True,\n", " generate_func=generate_text_top_p_stream_cache,\n", " temperature=0.5,\n", " top_p=0.8, \n", ")" ] }, { "cell_type": "markdown", "id": "0e3aea84-8eda-4ca1-8b48-498f66b6bc23", "metadata": {}, "source": [ " \n", "## 4.6 Improving response accuracy with self-consistency" ] }, { "cell_type": "markdown", "id": "59027b99-9f3d-49e9-9425-7a69242aab96", "metadata": {}, "source": [ "- Generate multiple answers (/reasoning chains) via sampling and vote" ] }, { "cell_type": "markdown", "id": "e6e7e5c3-544b-4529-ae2d-98ab63b6a93d", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "5216f529-52c4-4958-8a43-734307a30ab3", "metadata": {}, "source": [ "- Similar to classic majority voting, but instead of different models, use different temperature settings\n", "- This method is also described in the [Self-Consistency Improves Chain-of-Thought Reasoning in Language Models](https://arxiv.org/abs/2203.11171) paper\n", "- In short, this method\n", " - Samples multiple reasoning chains with high temperature and top-p\n", " - Extracts the final answer from each chain\n", " - Chooses the most frequent final answer\n", "- Nit: this is called [plurality vote](https://en.wikipedia.org/wiki/Plurality_(voting)) (in American English), not majority vote" ] }, { "cell_type": "markdown", "id": "1cc8df4b-a966-4fc1-ac17-0831c3175a38", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 39, "id": "b64e487d-653d-4097-bd22-8e2c68773796", "metadata": {}, "outputs": [], "source": [ "from reasoning_from_scratch.ch03 import extract_final_candidate\n", "from collections import Counter\n", "\n", "def self_consistency_vote(\n", " model, tokenizer, prompt, device,\n", " num_samples=10, temperature=0.8, top_p=0.9, max_new_tokens=2048,\n", " show_progress=True, show_long_answer=False, seed=None,\n", "):\n", " full_answers, short_answers = [], []\n", "\n", " # 1) Sample multiple answers\n", " for i in range(num_samples):\n", " if seed is not None:\n", " torch.manual_seed(seed + i + 1)\n", "\n", " answer = generate_text_stream_concat_flex(\n", " model=model, tokenizer=tokenizer, prompt=prompt, device=device,\n", " max_new_tokens=max_new_tokens, verbose=show_long_answer,\n", " generate_func=generate_text_top_p_stream_cache,\n", " temperature=temperature, top_p=top_p,\n", " )\n", "\n", " # 2) Extract the final (short) answer from each answer\n", " short = extract_final_candidate(\n", " answer, fallback=\"number_then_full\"\n", " )\n", " full_answers.append(answer)\n", " short_answers.append(short)\n", " if show_progress:\n", " print(f\"[Sample {i+1}/{num_samples}] → {short!r}\")\n", "\n", " # 3) Choose the most frequent final answer (self-consistency vote)\n", " counts = Counter(short_answers)\n", " groups = {s: [] for s in counts}\n", " for idx, s in enumerate(short_answers):\n", " groups[s].append(idx)\n", "\n", " mc = counts.most_common()\n", " if not mc:\n", " majority_winners, final_answer = [], None\n", " else:\n", " top_freq = mc[0][1]\n", " majority_winners = [s for s, f in mc if f == top_freq]\n", " final_answer = mc[0][0] if len(majority_winners) == 1 else None\n", "\n", " return {\n", " \"full_answers\": full_answers,\n", " \"short_answers\": short_answers,\n", " \"counts\": dict(counts),\n", " \"groups\": groups,\n", " \"majority_winners\": majority_winners,\n", " \"final_answer\": final_answer,\n", " }" ] }, { "cell_type": "markdown", "id": "5cad1e56-9ef5-458b-b6f6-5d501cac7966", "metadata": {}, "source": [ "- Additional tips\n", " - If all (long) answers look nearly identical, increase temperature gently to increase diversity \n", " - If (long) answers look off, decrease temperature" ] }, { "cell_type": "code", "execution_count": 40, "id": "aacc62be-e98b-4476-8ef1-c8c9dfccbb95", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Sample 1/5] → '83'\n", "[Sample 2/5] → '22'\n", "[Sample 3/5] → '54'\n", "[Sample 4/5] → '83'\n", "[Sample 5/5] → '61'\n" ] } ], "source": [ "results = self_consistency_vote(\n", " model,\n", " tokenizer,\n", " prompt,\n", " device=device,\n", " num_samples=5,\n", " temperature=0.8,\n", " top_p=0.9,\n", " max_new_tokens=2048,\n", " seed=123,\n", " show_progress=True,\n", ")" ] }, { "cell_type": "code", "execution_count": 41, "id": "08616229-7233-435c-9ee8-c7e962245883", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "83\n" ] } ], "source": [ "print(results[\"final_answer\"])" ] }, { "cell_type": "code", "execution_count": 42, "id": "3f629c53-8e8f-4b16-a376-d361ba9e1f4b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " To find the value of \\( x \\), let's solve the equation step by step.\n", "\n", "1. **Given Equation:**\n", " \\[\n", " \\frac{1}{2} \\times (3x - 9) = x + 37\n", " \\]\n", "\n", "2. **Multiply Both Sides by 2 to Eliminate the Fraction:**\n", " \\[\n", " 2 \\times \\frac{1}{2} \\times (3x - 9) = 2 \\times (x + 37)\n", " \\]\n", " \\[\n", " 3x - 9 = 2x + 74\n", " \\]\n", "\n", "3. **Subtract \\( 2x \\) from Both Sides to Get:**\n", " \\[\n", " 3x - 2x - 9 = 74\n", " \\]\n", " \\[\n", " x - 9 = 74\n", " \\]\n", "\n", "4. **Add 9 to Both Sides to Solve for \\( x \\):**\n", " \\[\n", " x = 74 + 9\n", " \\]\n", " \\[\n", " x = 83\n", " \\]\n", "\n", "**Final Answer:**\n", "\\[\n", "\\boxed{83}\n", "\\]\n" ] } ], "source": [ "print(results[\"full_answers\"][0])" ] }, { "cell_type": "code", "execution_count": 43, "id": "b359357b-c7c5-45e6-ab13-d79f17b07b33", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Sample 1/5] → 'x = 83'\n", "[Sample 2/5] → '83'\n", "[Sample 3/5] → '83'\n", "[Sample 4/5] → '83'\n", "[Sample 5/5] → '83'\n" ] } ], "source": [ "results = self_consistency_vote(\n", " model,\n", " tokenizer,\n", " prompt + \"\\n\\nExplain step by step.\",\n", " device=device,\n", " num_samples=5,\n", " temperature=0.8,\n", " top_p=0.9,\n", " max_new_tokens=2048,\n", " seed=123,\n", " show_progress=True,\n", ")" ] }, { "cell_type": "markdown", "id": "0a6e8834-71a6-4875-a0a3-3e26c7288cc5", "metadata": {}, "source": [ "- Parallel sampling like this is also done by Claude 4 (https://www.anthropic.com/news/claude-4); in addition, Claude 4 uses an internal model for scoring responses\n", "- A scorer can be similar to the LLM judge in Appendix F (F.5 Judging responses with other LLMs)\n", "- In the next chapter, we will also introduce a scoring method based on confidence scores (we can also use it as a tie-breaker)" ] }, { "cell_type": "markdown", "id": "495b12ba-ff54-4d5e-99ee-087be561af8a", "metadata": {}, "source": [ "Below are the results of the different methods on a cuda device (DGX Spark) evaluated on the full MATH-500 test set with all 500 problems" ] }, { "cell_type": "markdown", "id": "7ef20dfd-d621-46d5-b2ab-ab289bf0041e", "metadata": {}, "source": [ "| | Method | Model | Accuracy | Time |\n", "|----|----------------------------------------------|-----------|----------|------------|\n", "| 1 | Baseline (chapter 3), greedy decoding | Base | 15.2% | 10.1 min |\n", "| 2 | Baseline (chapter 3), greedy decoding | Reasoning | 48.2% | 182.1 min |\n", "| 3 | Chain-of-thought prompting (\"CoT\") | Base | 40.6% | 84.5 min |\n", "| 4 | Temperature and top-p (\"Top-p\") | Base | 17.8% | 30.7 min |\n", "| 5 | \"Top-p\" + Self-consistency (n=3) | Base | 29.6% | 97.6 min |\n", "| 6 | \"Top-p\" + Self-consistency (n=5) | Base | 27.8% | 116.8 min |\n", "| 7 | \"Top-p\" + Self-consistency (n=10) | Base | 31.6% | 300.4 min |\n", "| 8 | \"Top-p\" + \"CoT\" | Base | 33.4% | 129.2 min |\n", "| 9 | Self-consistency (n=3) + \"Top-p\" + \"CoT\" | Base | 42.2% | 211.6 min |\n", "| 10 | Self-consistency (n=5) + \"Top-p\" + \"CoT\" | Base | 48.0% | 452.9 min |\n", "| 11 | Self-consistency (n=10) + \"Top-p\" + \"CoT\" | Base | 52.0% | 862.6 min |\n", "| 12 | Self-consistency (n=3) + \"Top-p\" + \"CoT\" | Reasoning | 55.2% | 544.4 min |" ] }, { "cell_type": "markdown", "id": "2a7fbd6e-723a-4fd1-8e66-bd84eb455eb8", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "bedb5807-5446-4b13-9f78-2490c41d7632", "metadata": {}, "source": [ " \n", "## 4.7 Summary" ] }, { "cell_type": "markdown", "id": "87368c74-a360-40ce-9164-37b21777340a", "metadata": {}, "source": [ "- No code in this section" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" } }, "nbformat": 4, "nbformat_minor": 5 }