{ "cells": [ { "cell_type": "markdown", "id": "168fe6a6-c485-4e61-a868-f96963ff834e", "metadata": {}, "source": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "Supplementary code for the Build a Large Language Model From Scratch book by Sebastian Raschka
\n", "
Code repository: https://github.com/rasbt/LLMs-from-scratch\n", "\n", "		\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "eff5f507-d51b-4d13-827c-600d1159bc2d", "metadata": {}, "source": [ "# Chapter 8: Distilling Reasoning Models for Efficient Reasoning" ] }, { "cell_type": "markdown", "id": "e4a1fcf6-d439-47a2-ae63-bdf68377ec1c", "metadata": {}, "source": [ "Packages that are being used in this notebook:" ] }, { "cell_type": "code", "execution_count": 1, "id": "689035a8-ab9c-449b-8812-52c7d564e509", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "reasoning_from_scratch version: 0.1.17\n", "torch version: 2.10.0\n", "tokenizers version: 0.21.4\n" ] } ], "source": [ "from importlib.metadata import version\n", "\n", "used_libraries = [\n", " \"reasoning_from_scratch\",\n", " \"torch\",\n", " \"tokenizers\" # Used by reasoning_from_scratch\n", "]\n", "\n", "for lib in used_libraries:\n", " print(f\"{lib} version: {version(lib)}\")" ] }, { "cell_type": "markdown", "id": "d05af359-a684-4671-a310-d6739a9b214f", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "62a1e8d8-90e8-47a0-93ab-090c0da018f2", "metadata": {}, "source": [ " \n", "## 8.1 Introduction to model distillation for reasoning tasks" ] }, { "cell_type": "markdown", "id": "f43f6135-3804-4094-9b6b-d51aa790e8fc", "metadata": {}, "source": [ "- Distillation means training a smaller LLM (the \"student\") on the outputs of a larger LLM (the \"teacher\")\n", "- Distillation can be more effective than training with RLVR from scratch\n", " - For example, the [DeepSeek-R1 paper](https://arxiv.org/abs/2501.12948) reported that for the smaller DeepSeek-R1 model variants, they were able to achieve a better performance through distillation than RLVR\n", " - In particular, they trained the largest DeepSeek R1 model (671B parameters), and then they used it to train smaller models via distillation" ] }, { "cell_type": "markdown", "id": "7adf74ae-74d6-40dd-a914-fc95dc5a209f", "metadata": {}, "source": [ "- There are two main types of distillation: hard distillation and soft distillation\n", " 1. Hard distillation: The student is trained on text generated by the teacher (the teacher's outputs are treated as ground-truth targets)\n", " 2. Soft distillation: The student is trained to match the teacher's probability distribution over the vocabulary (typically by minimizing KL divergence between the teacher and student distributions derived from their logits)" ] }, { "cell_type": "markdown", "id": "41eb0635-2dc4-44db-9772-a1ef1bf774e1", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "cf61235a-369b-4c37-89c5-411d0cb822ee", "metadata": {}, "source": [ "- 1. Pure hard distillation:\n", " - We only use teacher-generated text as targets (here, the teacher outputs are considered the ground truth targets)\n", " - Loss: `L = CrossEntropy(y_teacher_tokens, y_student)`\n", " - Technically, this is standard supervised fine-tuning on synthetic data\n", " - We don't require access to teacher logits\n", " - This is what the smaller models in DeepSeek-R1 distillation do: generate reasoning traces via the large 671B DeepSeek-R1 (teacher) and fine-tune smaller LLMs (students) on them" ] }, { "cell_type": "markdown", "id": "ced3b579-fee0-4906-a30e-0d66a561dcc6", "metadata": {}, "source": [ "- 2. Pure soft distillation\n", " - We match the teacher's output probability distribution directly\n", " - Loss: `L = KL(p_teacher || p_student)`\n", " - This requires teacher logits (or log-probabilities) at training time" ] }, { "cell_type": "markdown", "id": "9c8080eb-c634-436e-b0ad-3834e60927a1", "metadata": {}, "source": [ "- 3. Combined hard and soft distillation\n", " - This is classic knowledge distillation (originated in computer vision) described in [Distilling the Knowledge in a Neural Network](https://arxiv.org/abs/1503.02531)\n", " - We train on the teacher's actual output tokens, but we also force the student to match the teacher's full probability distribution\n", " - Loss: `L = CE(y_teacher_tokens, y_student) + lambda * KL(p_teacher || p_student)`, where `lambda` is a hyperparameter to tune the individual contribution of the KL term" ] }, { "cell_type": "markdown", "id": "d0a14c35-9e02-43b4-b6c2-0a1b241713a3", "metadata": {}, "source": [ "- Hard distillation is more common in practice\n", " - Logits are usually inaccessible; i.e., proprietary systems such as OpenAI ChatGPT or Anthropic Claude do not expose full vocabulary logits, which makes classical soft distillation impossible\n", " - Even if logits are accessible, soft-distillation requires that student and teacher LLMs use the same tokenization; otherwise, the vocabularies won't match (hence, soft-distillation is more useful when staying within this same model family)\n", " - Note that even generating and reusing the text outputs for distillation may violate certain providers' policies\n", " - Even when logits are available, storing full vocabulary distributions for long sequences is expensive in terms of bandwidth and disk space; generating and storing plain text outputs is significantly cheaper" ] }, { "cell_type": "markdown", "id": "1e40d79f-fab2-42e2-9b15-e25f0bac5d0c", "metadata": {}, "source": [ "- If both soft and hard distillation are feasible, which one performs better?\n", " - Appropriately designed objectives for soft distillation can outperform hard distillation (https://arxiv.org/abs/2306.08543)\n", " - But there is no recent large-scale head-to-head comparison on modern LLM benchmarks where soft vs hard is the only varying factor for a fair comparison\n", " - Data generation strategy can matter more than whether supervision is soft or hard (https://arxiv.org/abs/2306.13649)" ] }, { "cell_type": "markdown", "id": "7b020c1c-3efa-4c5f-a18e-f87c1c84dc14", "metadata": {}, "source": [ "- In this chapter, we implement hard distillation similar to DeepSeek-R1, which is the more common and practical approach" ] }, { "cell_type": "markdown", "id": "bce96969-bc27-4f8b-aae1-917ad5698312", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "940dc532-4fe7-4610-9068-587d6aa9820e", "metadata": {}, "source": [ " \n", "## 8.2 Generating a dataset for reasoning distillation" ] }, { "cell_type": "markdown", "id": "cc92fb71-5189-4e44-a3b0-c518711a9574", "metadata": {}, "source": [ "- The first step in distillation is to create the dataset for training the student model (here, the Qwen3 0.6B base model we worked with throughout this book)\n", "- We will use the 12,000 math problems from the MATH dataset that are non-overlapping with the MATH-500 test set (https://github.com/rasbt/math_full_minus_math500)\n", "- These are the same 12,000 math problems we used in chapters 6 and 7 for RLVR\n", "- We feed these 12,000 math problems to an existing reasoning model (here, DeepSeek-R1) and collect the responses for training" ] }, { "cell_type": "markdown", "id": "d6bc6482-22a7-476a-b649-ef523371ec64", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "71379a68-aa8d-43be-a175-220d98b76971", "metadata": {}, "source": [ "- In chapters 6 and 7, we trained the Qwen3 base model with RLVR to generate the correct solution\n", "- The correct solution, using a verifier, was then compared to the reference solution to compute the training signal\n", "- In distillation, we compare the teacher's solution to the student's solution to compute the training signal" ] }, { "cell_type": "markdown", "id": "caa5714b-2a64-4c4c-8a83-5407403e3937", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "62a88573-8a56-4b0d-853d-185ba12df674", "metadata": {}, "source": [ "- For our distillation approach, we can use the teacher model to generate the distillation dataset ahead of time, before we start training the student model\n", "- Since generating the teacher answers for all 12,000 math problems can be time- and resource-intensive, I generated the dataset ahead of time using a 671-billion-parameter DeepSeek-R1 model hosted in the cloud via OpenRouter (the dataset cost approximately $50 in API costs to generate)\n", "- In the next section, we will load this pre-generated dataset, so it's not necessary for you to generate this dataset yourself\n", "- However, if you are curious, you can find the code I used, alongside the usage instructions, in the supplementary materials at [https://github.com/rasbt/reasoning-from-scratch/tree/main/ch08/02_generate_distillation_data](https://github.com/rasbt/reasoning-from-scratch/tree/main/ch08/02_generate_distillation_data) " ] }, { "cell_type": "markdown", "id": "ae220ae9-fadd-4ffe-bf63-9bc532a29157", "metadata": {}, "source": [ " \n", "## 8.3 Loading the MATH training dataset for distillation" ] }, { "cell_type": "markdown", "id": "a568d31b-7b17-44ca-b0d3-50070c7077d3", "metadata": {}, "source": [ "- In this section, we load a dataset of math problems and answers generated by DeepSeek-R1 as described in the previous section" ] }, { "cell_type": "markdown", "id": "7b0c3388-38b8-431c-8f74-3ebe6ea9c585", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "80c379e5-e299-4a72-b1cf-d6eae190c2b1", "metadata": {}, "source": [ "- The dataset is available from the Hugging Face model hub at https://huggingface.co/datasets/rasbt/math_distill\n", "- The following code loads it into our Python session:" ] }, { "cell_type": "code", "execution_count": 2, "id": "ebdc80fd-53db-4b22-b167-14e16763f0ab", "metadata": {}, "outputs": [], "source": [ "import json\n", "import requests\n", "from pathlib import Path\n", "\n", "\n", "def load_distill_data(\n", " local_path=None,\n", " partition=\"deepseek-r1-math-train\",\n", " save_copy=True,\n", "):\n", "\n", " if local_path is None:\n", " local_path = f\"{partition}.json\"\n", " local_path = Path(local_path)\n", "\n", " url = (\n", " \"https://huggingface.co/datasets/rasbt/math_distill\"\n", " \"/resolve/main/data/\"\n", " f\"{partition}.json\"\n", " )\n", " backup_url = (\n", " \"https://f001.backblazeb2.com/file/reasoning-from-scratch/\"\n", " f\"MATH/{partition}.json\"\n", " )\n", "\n", " if local_path.exists():\n", " with local_path.open(\"r\", encoding=\"utf-8\") as f:\n", " data = json.load(f)\n", "\n", " size_kb = local_path.stat().st_size / 1e3\n", " print(f\"{local_path}: {size_kb:.1f} KB (cached)\")\n", " return data\n", "\n", " assert partition in (\n", " \"deepseek-r1-math-train\",\n", " \"deepseek-r1-math500\",\n", " \"qwen3-235b-a22b-math-train\",\n", " \"qwen3-235b-a22b-math500\",\n", " )\n", "\n", " try:\n", " r = requests.get(url, timeout=30)\n", " r.raise_for_status()\n", " except requests.RequestException:\n", " print(\"Using backup URL.\")\n", " r = requests.get(backup_url, timeout=30)\n", " r.raise_for_status()\n", "\n", " data = r.json()\n", "\n", " if save_copy:\n", " with local_path.open(\"w\", encoding=\"utf-8\") as f:\n", " json.dump(data, f, indent=2)\n", "\n", " size_kb = local_path.stat().st_size / 1e3\n", " print(f\"{local_path}: {size_kb:.1f} KB\")\n", "\n", " return data" ] }, { "cell_type": "code", "execution_count": 3, "id": "dcc2920f-902e-479a-972c-c62377095c0b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "deepseek-r1-math-train.json: 107538.0 KB (cached)\n", "Dataset size: 12000\n" ] } ], "source": [ "math_train = load_distill_data(partition=\"deepseek-r1-math-train\")\n", "\n", "print(\"Dataset size:\", len(math_train))" ] }, { "cell_type": "markdown", "id": "588e79fb-3a47-4f23-8522-2d89264343f3", "metadata": {}, "source": [ "- Let's look at one of the training examples to check the structure:" ] }, { "cell_type": "code", "execution_count": 4, "id": "994212e1-d115-4574-9689-a112879a430f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'gtruth_answer': '6',\n", " 'message_content': 'Sam worked \\\\( x \\\\) days and did not work \\\\( y \\\\) '\n", " 'days. We know:\\n'\n", " '\\\\[\\n'\n", " 'x + y = 20\\n'\n", " '\\\\]\\n'\n", " 'His total earnings are calculated as:\\n'\n", " '\\\\[\\n'\n", " '60x - 30y = 660\\n'\n", " '\\\\]\\n'\n", " 'Substituting \\\\( x = 20 - y \\\\) into the earnings '\n", " 'equation:\\n'\n", " '\\\\[\\n'\n", " '60(20 - y) - 30y = 660\\n'\n", " '\\\\]\\n'\n", " 'Simplifying:\\n'\n", " '\\\\[\\n'\n", " '1200 - 60y - 30y = 660 \\\\\\\\\\n'\n", " '1200 - 90y = 660 \\\\\\\\\\n'\n", " '-90y = 660 - 1200 \\\\\\\\\\n'\n", " '-90y = -540 \\\\\\\\\\n'\n", " 'y = \\\\frac{-540}{-90} = 6\\n'\n", " '\\\\]\\n'\n", " 'Sam did not work \\\\(\\\\boxed{6}\\\\) days.',\n", " 'message_thinking': \"Okay, let's see. Sam was hired for 20 days. Each day he \"\n", " \"works, he earns $60, but if he doesn't work a day, they \"\n", " 'deduct $30 from his earnings. At the end of the 20 days, '\n", " \"he got $660. I need to find out how many days he didn't \"\n", " 'work.\\n'\n", " '\\n'\n", " \"Hmm, so let me think. Let's break this down. Let's say \"\n", " \"he worked x days and didn't work y days. So total days x \"\n", " \"+ y = 20. That's straightforward. Now, his earnings. For \"\n", " 'each day worked, he gets $60, so total earnings from '\n", " \"working would be 60x. But for each day he didn't work, \"\n", " 'they subtract $30. Wait, how exactly does that '\n", " 'subtraction work? Is it that he loses $30 per day not '\n", " 'worked from his total earnings?\\n'\n", " '\\n'\n", " \"Wait, maybe. So if he didn't work y days, then he loses \"\n", " '30*y dollars. But wait, does he get a base salary and '\n", " 'then deductions? Or is his total earnings 60x minus '\n", " '30y?\\n'\n", " '\\n'\n", " 'The problem says, \"On days that he works, he earns $60. '\n", " 'For each day that he does not work, $30 is subtracted '\n", " 'from his earnings.\" So probably, his total earnings are '\n", " 'calculated by 60 times the number of days worked minus '\n", " '30 times the number of days not worked. So total '\n", " 'earnings = 60x - 30y. And x + y = 20. And we know total '\n", " 'earnings is $660.\\n'\n", " '\\n'\n", " \"So substituting x as 20 - y into the equation. Let's set \"\n", " 'that up.\\n'\n", " '\\n'\n", " 'Total earnings: 60x - 30y = 660\\n'\n", " '\\n'\n", " 'But x = 20 - y. Substitute:\\n'\n", " '\\n'\n", " '60*(20 - y) - 30y = 660\\n'\n", " '\\n'\n", " 'Let me compute this. First, multiply 60 by 20 and '\n", " 'subtract 60y.\\n'\n", " '\\n'\n", " '60*20 = 1200. So 1200 -60y -30y = 660. Combine like '\n", " 'terms:\\n'\n", " '\\n'\n", " '1200 -90y = 660\\n'\n", " '\\n'\n", " 'Now, subtract 1200 from both sides:\\n'\n", " '\\n'\n", " '-90y = 660 -1200\\n'\n", " '\\n'\n", " '660 -1200 is -540. So:\\n'\n", " '\\n'\n", " '-90y = -540\\n'\n", " '\\n'\n", " 'Divide both sides by -90:\\n'\n", " '\\n'\n", " 'y = (-540)/(-90) = 6.\\n'\n", " '\\n'\n", " \"Wait, so y is 6. So he didn't work 6 days.\\n\"\n", " '\\n'\n", " \"Let me check this. Let's verify. If he didn't work 6 \"\n", " 'days, then he worked 14 days.\\n'\n", " '\\n'\n", " 'Earnings from working: 14 * 60 = 840\\n'\n", " '\\n'\n", " 'Deductions for not working: 6 * 30 = 180\\n'\n", " '\\n'\n", " 'Total earnings: 840 - 180 = 660. Which matches the given '\n", " 'total. So yes, that works.\\n'\n", " '\\n'\n", " 'Alternatively, another way to think. Maybe he earns a '\n", " 'base of something and then deductions, but the wording '\n", " 'seems to be that he only gets paid for days worked minus '\n", " 'deductions for days not worked. Which is how I solved '\n", " 'it. Seems correct. So answer is 6 days not worked.\\n',\n", " 'problem': 'Sam is hired for a 20-day period. On days that he works, he earns '\n", " '$\\\\$$60. For each day that he does not work, $\\\\$$30 is '\n", " 'subtracted from his earnings. At the end of the 20-day period, he '\n", " 'received $\\\\$$660. How many days did he not work?'}\n" ] } ], "source": [ "from pprint import pprint\n", "\n", "pprint(math_train[4])" ] }, { "cell_type": "code", "execution_count": 5, "id": "1bac26e4-a90d-4f00-a900-de0f0aa4b9c3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict_keys(['problem', 'gtruth_answer', 'message_thinking', 'message_content'])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "math_train[4].keys()" ] }, { "cell_type": "markdown", "id": "542fb68a-28a9-4124-9c78-6685d172424a", "metadata": {}, "source": [ "- The dataset contains four fields:\n", " - `gtruth_answer`: The correct reference answer (ground truth answer)\n", " - `message_thinking`: The reasoning trace written by the model\n", " - `message_content`: The final answer, following the reasoning trace\n", "- We can use the following code to format the answer:" ] }, { "cell_type": "code", "execution_count": 6, "id": "cd131bb4-ea29-47d7-99ee-e1eea8755b0a", "metadata": {}, "outputs": [], "source": [ "def format_distilled_answer(entry):\n", " content = str(entry[\"message_content\"]).strip()\n", " if not content:\n", " raise ValueError(\"Missing non-empty 'message_content' field.\")\n", "\n", " thinking = str(entry[\"message_thinking\"]).strip()\n", " return f\"{thinking}\\n\\n{content}\"" ] }, { "cell_type": "code", "execution_count": 7, "id": "80f04e8d-a595-4010-af9d-396fe2880079", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Okay, let's see. Sam was hired for 20 days. Each day he works, he earns $60, but if he doesn't work a day, they deduct $30 from his earnings. At the end of the 20 days, he got $660. I need to find out how many days he didn't work.\n", "\n", "Hmm, so let me think. Let's break this down. Let's say he worked x days and didn't work y days. So total days x + y = 20. That's straightforward. Now, his earnings. For each day worked, he gets $60, so total earnings from working would be 60x. But for each day he didn't work, they subtract $30. Wait, how exactly does that subtraction work? Is it that he loses $30 per day not worked from his total earnings?\n", "\n", "Wait, maybe. So if he didn't work y days, then he loses 30*y dollars. But wait, does he get a base salary and then deductions? Or is his total earnings 60x minus 30y?\n", "\n", "The problem says, \"On days that he works, he earns $60. For each day that he does not work, $30 is subtracted from his earnings.\" So probably, his total earnings are calculated by 60 times the number of days worked minus 30 times the number of days not worked. So total earnings = 60x - 30y. And x + y = 20. And we know total earnings is $660.\n", "\n", "So substituting x as 20 - y into the equation. Let's set that up.\n", "\n", "Total earnings: 60x - 30y = 660\n", "\n", "But x = 20 - y. Substitute:\n", "\n", "60*(20 - y) - 30y = 660\n", "\n", "Let me compute this. First, multiply 60 by 20 and subtract 60y.\n", "\n", "60*20 = 1200. So 1200 -60y -30y = 660. Combine like terms:\n", "\n", "1200 -90y = 660\n", "\n", "Now, subtract 1200 from both sides:\n", "\n", "-90y = 660 -1200\n", "\n", "660 -1200 is -540. So:\n", "\n", "-90y = -540\n", "\n", "Divide both sides by -90:\n", "\n", "y = (-540)/(-90) = 6.\n", "\n", "Wait, so y is 6. So he didn't work 6 days.\n", "\n", "Let me check this. Let's verify. If he didn't work 6 days, then he worked 14 days.\n", "\n", "Earnings from working: 14 * 60 = 840\n", "\n", "Deductions for not working: 6 * 30 = 180\n", "\n", "Total earnings: 840 - 180 = 660. Which matches the given total. So yes, that works.\n", "\n", "Alternatively, another way to think. Maybe he earns a base of something and then deductions, but the wording seems to be that he only gets paid for days worked minus deductions for days not worked. Which is how I solved it. Seems correct. So answer is 6 days not worked.\n", "\n", "Sam worked \\( x \\) days and did not work \\( y \\) days. We know:\n", "\\[\n", "x + y = 20\n", "\\]\n", "His total earnings are calculated as:\n", "\\[\n", "60x - 30y = 660\n", "\\]\n", "Substituting \\( x = 20 - y \\) into the earnings equation:\n", "\\[\n", "60(20 - y) - 30y = 660\n", "\\]\n", "Simplifying:\n", "\\[\n", "1200 - 60y - 30y = 660 \\\\\n", "1200 - 90y = 660 \\\\\n", "-90y = 660 - 1200 \\\\\n", "-90y = -540 \\\\\n", "y = \\frac{-540}{-90} = 6\n", "\\]\n", "Sam did not work \\(\\boxed{6}\\) days.\n" ] } ], "source": [ "print(format_distilled_answer(math_train[4]))" ] }, { "cell_type": "markdown", "id": "6ae68724-8a2a-4fc7-8ab5-46331cf20593", "metadata": {}, "source": [ "- As discussed in the previous chapter, `` tokens are not necessary\n", "- They can be useful to separate the reasoning trace from the final answer\n", "- Some providers and interfaces, like OpenAI's ChatGPT, hide the reasoning trace from the user, and using and teaching the model to use `` tokens can make it easier to parse out and hide these thinking traces" ] }, { "cell_type": "markdown", "id": "aef63a4d-221c-4585-8965-54b653393fd7", "metadata": {}, "source": [ "- In case you are curious about what accuracy the DeepSeek-R1 model achieved on this dataset, since the dataset contains ground truth labels, we can compute accuracy with the verifier from chapter 3\n", "- For convenience, we can use the [evaluate_json.py](https://github.com/rasbt/reasoning-from-scratch/blob/main/ch03/02_math500-verifier-scripts/evaluate_json.py) script from the supplementary materials (chapter 3) to calculate the accuracy directly from the JSON file" ] }, { "cell_type": "code", "execution_count": 8, "id": "0ff49d85-eacb-4891-9a49-dd462e7b5c84", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "evaluate_json.py: 3.3 KB (cached)\n" ] } ], "source": [ "from reasoning_from_scratch.ch07 import download_from_github\n", "\n", "_ = download_from_github(\n", " \"ch03/02_math500-verifier-scripts/evaluate_json.py\"\n", ")" ] }, { "cell_type": "code", "execution_count": 9, "id": "8602f25f-2ff1-406f-93dc-e2d94a3d4d7c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 90.6% (10871/12000)\n" ] } ], "source": [ "!uv run evaluate_json.py \\\n", "--json_path \"deepseek-r1-math-train.json\" \\\n", "--gtruth_answer gtruth_answer \\\n", "--generated_text message_content" ] }, { "cell_type": "markdown", "id": "42c3cdfa-71fd-4280-beea-73a0979d42d9", "metadata": {}, "source": [ " \n", "## 8.4 Building training examples" ] }, { "cell_type": "markdown", "id": "bcdb920c-9c90-47dd-ac08-4e687db00eae", "metadata": {}, "source": [ "- In this section, we prepare the dataset examples for training" ] }, { "cell_type": "markdown", "id": "0b5a6cb8-6082-4c74-9433-79c761cacebf", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "d4fe29bc-3bc9-49c9-9821-e36028b9f693", "metadata": {}, "source": [ "- This preparation is mainly about formatting and tokenizing the examples\n", "- In RLVR, we did this on the fly since we only used each training example ones\n", "- Distillation, which is a form of supervised fine-tuning, benefits from seeing the same examples multiple times\n", "- So, to avoid duplication efforts, we format and tokenize the samples once and store them for reuse" ] }, { "cell_type": "markdown", "id": "0553724f-77db-4b8b-bdbc-e20fb3991d91", "metadata": {}, "source": [ " \n", "### 8.4.1 Loading and understanding the tokenizer" ] }, { "cell_type": "markdown", "id": "ef3fffeb-bc4b-4ad5-a7ab-c8554c589826", "metadata": {}, "source": [ "- We start with the tokenizer\n", "- We use the \"reasoning\" variant as it supports `...` tokens, as discussed in chapter 7" ] }, { "cell_type": "code", "execution_count": 10, "id": "0dfcb257-d4f5-4e3b-bfda-cbc6d5dd64c9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✓ qwen3/tokenizer-reasoning.json already up-to-date\n" ] } ], "source": [ "from reasoning_from_scratch.qwen3 import (\n", " download_qwen3_small,\n", " Qwen3Tokenizer\n", ")\n", "\n", "def load_reasoning_tokenizer(local_dir=\"qwen3\"):\n", " download_qwen3_small(\n", " kind=\"reasoning\", tokenizer_only=True, out_dir=local_dir\n", " )\n", "\n", " tokenizer_path = Path(local_dir) / \"tokenizer-reasoning.json\"\n", " tokenizer = Qwen3Tokenizer(\n", " tokenizer_file_path=tokenizer_path,\n", " apply_chat_template=True,\n", " add_generation_prompt=True,\n", " add_thinking=True,\n", " )\n", " \n", " return tokenizer\n", "\n", "\n", "tokenizer = load_reasoning_tokenizer()" ] }, { "cell_type": "markdown", "id": "aaeae0f2-fbb9-427c-9138-7111c417458f", "metadata": {}, "source": [ "- Note that we set `apply_chat_template=True` and `add_generation_prompt=True`, which will apply additional formatting to the prompt, as we can demonstrate as follows:" ] }, { "cell_type": "code", "execution_count": 11, "id": "c1972c0a-6e93-4742-884e-a8922a547161", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>user\n", "Sam is hired for a 20-day period...<|im_end|>\n", "<|im_start|>assistant\n", "\n" ] } ], "source": [ "prompt = \"Sam is hired for a 20-day period...\"\n", "\n", "prompt_ids = tokenizer.encode(prompt)\n", "decoded_prompt = tokenizer.decode(prompt_ids)\n", "print(decoded_prompt)" ] }, { "cell_type": "markdown", "id": "7332395c-091d-4129-8965-aebf5f5e6e44", "metadata": {}, "source": [ "- Here, `<|im_start|>user` act as start tokens for the user provided prompts\n", "- `<|im_end|>` is the stop token for the user-provided prompt\n", "- `<|im_start|>assistant` are the start tokens for the LLM answer\n", "- This formatting is optional, but it is a common and useful convention for instruction/chat fine-tuning\n", "- Also, for the answer, we need to disable this wrapping via `chat_wrapped=False`, otherwise will have too many start tokens since the prompt and answer tokens are meant to be combined for training:" ] }, { "cell_type": "code", "execution_count": 12, "id": "133e7469-f385-4193-9cff-5e06d6805818", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Okay, let me try to solve this problem... \\boxed{4}\n" ] } ], "source": [ "answer = (\n", " \"Okay, let me try to solve \"\n", " \"this problem... \\\\boxed{4}\"\n", ")\n", "answer_ids = tokenizer.encode(answer, chat_wrapped=False)\n", "decoded_answer = tokenizer.decode(answer_ids)\n", "print(decoded_answer)" ] }, { "cell_type": "markdown", "id": "759bbff0-ced3-4e0b-8775-128c63b4d29d", "metadata": {}, "source": [ "- For training, we need the combined token IDs including an `<|im_end|>` end-of-sequence token at the end of the answer:" ] }, { "cell_type": "code", "execution_count": 13, "id": "494a48b0-a4f8-487b-869d-3b48177a3465", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>user\n", "Sam is hired for a 20-day period...<|im_end|>\n", "<|im_start|>assistant\n", "Okay, let me try to solve this problem... \\boxed{4}<|im_end|>\n" ] } ], "source": [ "token_ids = prompt_ids + answer_ids + [tokenizer.eos_token_id]\n", "decoded_token_ids = tokenizer.decode(token_ids)\n", "print(decoded_token_ids)" ] }, { "cell_type": "markdown", "id": "54cc86e9-a6df-4662-9366-bfe9fddf78f4", "metadata": {}, "source": [ "- Again, it's worth highlighting that we use the reasoning tokenizer for the tokens, but it does not really matter; we could also use the base tokenizer\n", "- The chat template is also optional; we just keep it because that's what Qwen3's own reasoning models use (only matters when working with agents or multi-prompt)\n", "- We need to remember to use `--which_model \"reasoning\"` in the evaluation script for consistency" ] }, { "cell_type": "markdown", "id": "593979f8-f009-4b16-a910-dc7f03aa95f2", "metadata": {}, "source": [ " \n", "### 8.4.2 Formatting and tokenizing the dataset" ] }, { "cell_type": "markdown", "id": "193aa285-6751-4513-8348-317d05b23fae", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "98db658b-f0cd-4900-ba31-1a6ff4b689e3", "metadata": {}, "source": [ "- Below, we develop a `build_examples` function that applies the required formatting to the whole dataset" ] }, { "cell_type": "markdown", "id": "7b5e256f-59a5-4838-a317-3da092f3f4fa", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 14, "id": "13bb403b-484e-46ee-b9d4-5baf7e20dd6d", "metadata": {}, "outputs": [], "source": [ "from reasoning_from_scratch.ch03 import render_prompt\n", "\n", "def build_examples(data, tokenizer):\n", " examples = []\n", " skipped = 0\n", "\n", " for entry in data:\n", " try:\n", " # Step 1: encode prompt\n", " prompt = render_prompt(entry[\"problem\"])\n", " prompt_ids = tokenizer.encode(prompt)\n", "\n", " # Step 2: encode answer\n", " target_answer = format_distilled_answer(entry)\n", " answer_ids = tokenizer.encode(\n", " target_answer, chat_wrapped=False\n", " )\n", "\n", " # Step 3: Combine prompt and answer\n", " token_ids = (\n", " prompt_ids + answer_ids + [tokenizer.eos_token_id]\n", " )\n", "\n", " if len(token_ids) < 2:\n", " skipped += 1\n", " continue\n", "\n", " examples.append({\n", " \"token_ids\": token_ids,\n", " \"prompt_len\": len(prompt_ids),\n", " })\n", " except (KeyError, TypeError, ValueError):\n", " skipped += 1\n", "\n", " return examples, skipped" ] }, { "cell_type": "code", "execution_count": 15, "id": "6a91fac0-e594-405f-8a16-8945a65b34ac", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of examples: 12000\n", "Number of skipped examples: 0\n" ] } ], "source": [ "examples, skipped = build_examples(math_train, tokenizer)\n", "\n", "print(\"Number of examples:\", len(examples))\n", "print(\"Number of skipped examples:\", skipped)" ] }, { "cell_type": "code", "execution_count": 16, "id": "7197a8b2-4c01-48bc-8429-3c21f944e37a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<|im_start|>user\n", "You are a helpful math assistant.\n", "Answer the question and write the final result on a new line as:\n", "\\boxed{ANSWER}\n", "\n", "Question:\n", "Sam is hired for a 20-day period. On days that he works, he earns $\\$$60. For each day that he does not work, $\\$$30 is subtracted from his earnings. At the end of the 20-day period, he received $\\$$660. How many days did he not work?\n", "\n", "Answer:<|im_end|>\n", "<|im_start|>assistant\n", "Okay, let's see. Sam was hired for 20 days. Each day he works, he earns $60, but if he doesn't work a day, they deduct $30 from his earnings. At the end of the 20 days, he got $660. I need to find out how many days he didn't work.\n", "\n", "Hmm, so let me think. Let's break this down. Let's say he worked x days and didn't work y days. So total days x + y = 20. That's straightforward. Now, his earnings. For each day worked, he gets $60, so total earnings from working would be 60x. But for each day he didn't work, they subtract $30. Wait, how exactly does that subtraction work? Is it that he loses $30 per day not worked from his total earnings?\n", "\n", "Wait, maybe. So if he didn't work y days, then he loses 30*y dollars. But wait, does he get a base salary and then deductions? Or is his total earnings 60x minus 30y?\n", "\n", "The problem says, \"On days that he works, he earns $60. For each day that he does not work, $30 is subtracted from his earnings.\" So probably, his total earnings are calculated by 60 times the number of days worked minus 30 times the number of days not worked. So total earnings = 60x - 30y. And x + y = 20. And we know total earnings is $660.\n", "\n", "So substituting x as 20 - y into the equation. Let's set that up.\n", "\n", "Total earnings: 60x - 30y = 660\n", "\n", "But x = 20 - y. Substitute:\n", "\n", "60*(20 - y) - 30y = 660\n", "\n", "Let me compute this. First, multiply 60 by 20 and subtract 60y.\n", "\n", "60*20 = 1200. So 1200 -60y -30y = 660. Combine like terms:\n", "\n", "1200 -90y = 660\n", "\n", "Now, subtract 1200 from both sides:\n", "\n", "-90y = 660 -1200\n", "\n", "660 -1200 is -540. So:\n", "\n", "-90y = -540\n", "\n", "Divide both sides by -90:\n", "\n", "y = (-540)/(-90) = 6.\n", "\n", "Wait, so y is 6. So he didn't work 6 days.\n", "\n", "Let me check this. Let's verify. If he didn't work 6 days, then he worked 14 days.\n", "\n", "Earnings from working: 14 * 60 = 840\n", "\n", "Deductions for not working: 6 * 30 = 180\n", "\n", "Total earnings: 840 - 180 = 660. Which matches the given total. So yes, that works.\n", "\n", "Alternatively, another way to think. Maybe he earns a base of something and then deductions, but the wording seems to be that he only gets paid for days worked minus deductions for days not worked. Which is how I solved it. Seems correct. So answer is 6 days not worked.\n", "\n", "Sam worked \\( x \\) days and did not work \\( y \\) days. We know:\n", "\\[\n", "x + y = 20\n", "\\]\n", "His total earnings are calculated as:\n", "\\[\n", "60x - 30y = 660\n", "\\]\n", "Substituting \\( x = 20 - y \\) into the earnings equation:\n", "\\[\n", "60(20 - y) - 30y = 660\n", "\\]\n", "Simplifying:\n", "\\[\n", "1200 - 60y - 30y = 660 \\\\\n", "1200 - 90y = 660 \\\\\n", "-90y = 660 - 1200 \\\\\n", "-90y = -540 \\\\\n", "y = \\frac{-540}{-90} = 6\n", "\\]\n", "Sam did not work \\(\\boxed{6}\\) days.<|im_end|>\n" ] } ], "source": [ "print(tokenizer.decode(examples[4][\"token_ids\"]))" ] }, { "cell_type": "markdown", "id": "56879f49-7262-4075-b615-f44ac82b6116", "metadata": {}, "source": [ " \n", "### 8.4.3 Filtering and splitting the dataset" ] }, { "cell_type": "markdown", "id": "8e784614-ff91-4c95-92af-dcc5eb42c27a", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 17, "id": "1047e390-d862-4eda-995b-df95d19765ca", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average: 2946 tokens\n", "Shortest: 236 tokens (index 10846)\n", "Longest: 42005 tokens (index 2529)\n" ] } ], "source": [ "def compute_length(examples, answer_only=False):\n", " lengths = []\n", " for ex in examples:\n", " total = len(ex[\"token_ids\"])\n", " length = total - ex[\"prompt_len\"] if answer_only else total\n", " lengths.append(length)\n", "\n", " avg_len = round(sum(lengths) / len(lengths))\n", "\n", " shortest_len = min(lengths)\n", " longest_len = max(lengths)\n", " shortest_idx = lengths.index(shortest_len)\n", " longest_idx = lengths.index(longest_len)\n", "\n", " print(f\"Average: {avg_len} tokens\")\n", " print(f\"Shortest: {shortest_len} tokens (index {shortest_idx})\")\n", " print(f\"Longest: {longest_len} tokens (index {longest_idx})\")\n", "\n", "compute_length(examples)" ] }, { "cell_type": "code", "execution_count": 18, "id": "0be72a04-6128-4c1d-bd3d-5260bb834978", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Original: 12000\n", "Filtered: 6695\n", "Removed: 5305\n" ] } ], "source": [ "def filter_examples_by_max_len(examples, max_len=2048):\n", " filtered_examples = [\n", " s for s in examples\n", " if len(s[\"token_ids\"]) <= max_len\n", " ]\n", "\n", " print(\"Original:\", len(examples))\n", " print(\"Filtered:\", len(filtered_examples))\n", " print(\"Removed:\", len(examples) - len(filtered_examples))\n", "\n", " return filtered_examples\n", "\n", "filtered_examples = filter_examples_by_max_len(examples, max_len=2048)" ] }, { "cell_type": "code", "execution_count": 19, "id": "2120f8e2-ca6c-462d-bf3d-0ab1f3cda96f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average: 1180 tokens\n", "Shortest: 236 tokens (index 5971)\n", "Longest: 2048 tokens (index 5587)\n" ] } ], "source": [ "compute_length(filtered_examples)" ] }, { "cell_type": "code", "execution_count": 20, "id": "139ce04a-dce3-4cb7-a170-b9613bcad32e", "metadata": {}, "outputs": [], "source": [ "import random\n", "\n", "rng = random.Random(123)\n", "rng.shuffle(filtered_examples)\n", "\n", "train_examples = filtered_examples[25:]\n", "val_examples = filtered_examples[:25]" ] }, { "cell_type": "code", "execution_count": 21, "id": "98d1c2d3-d90d-4a57-accb-72cb171697af", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of train examples: 6670\n", "Number of validation examples: 25\n" ] } ], "source": [ "print(\"Number of train examples:\", len(train_examples))\n", "print(\"Number of validation examples:\", len(val_examples))" ] }, { "cell_type": "markdown", "id": "a5237723-30f9-4b45-a7e5-4ce361485d7d", "metadata": {}, "source": [ "- Exercise: inspect the training and validation set length to ensure they are balanced" ] }, { "cell_type": "code", "execution_count": 22, "id": "849070a8-2694-4497-a1f1-77eeaf8e99d2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average: 1180 tokens\n", "Shortest: 236 tokens (index 5730)\n", "Longest: 2048 tokens (index 1319)\n" ] } ], "source": [ "compute_length(train_examples)" ] }, { "cell_type": "code", "execution_count": 23, "id": "64ce1488-1732-4a2f-bc7c-1ca39e0bf8ba", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average: 1106 tokens\n", "Shortest: 481 tokens (index 15)\n", "Longest: 1918 tokens (index 12)\n" ] } ], "source": [ "compute_length(val_examples)" ] }, { "cell_type": "code", "execution_count": 24, "id": "bfb63472-4065-4b5d-b2cc-bf3d8b92a687", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArIAAAGGCAYAAACHemKmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVi1JREFUeJzt3QncTfX6///rNk+ZMg+hKJlnkVJxolQ0HRwnkqE0HBJCIUqKEuIkx8nQRM6RSlIy1CkyEw2iRIMxs8z2//H+fH9r//e+731P7vu297q9no/Hdttrf/Zan73W2ntf+7Ouda24QCAQMAAAAMBnskS7AwAAAMC5IJAFAACALxHIAgAAwJcIZAEAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBTLAvffea+XLlz+n5z711FMWFxeX7n26UERaf9oW2ibJmTp1qnvuzz//nG790bw0T83bb7y+v/DCCxYLUrodo019zJcvX4a//iVLlrjto7/ReF/p/sMPP2znQ0a8N5E5EMjigqIPwpTczscXAzKXt956y8aMGWOxJBb7lFn8+eefLrjz42fFs88+a3PmzLFYFMt9Q2zKFu0OAOfT66+/HnZ/+vTptmDBggTTr7zyyjQt51//+pedPXv2nJ775JNPWv/+/dO0fITbtGmTZcmSJcODxo0bN1qvXr3CppcrV86OHTtm2bNnP++bJbE+IX0C2aFDh7r/X3fddVFZpddee63bt3LkyJHqYPGuu+6yNm3axNznUmJ9u+eee6xdu3aWM2fODO8D/IVAFheUv//972H3v/rqKxfIxp8e6UsrT548KV5OWoKWbNmyuRvSTzS//DTCnytXrqgtH5mXfpxl9L519OhRy5s3b9Q/l7JmzepuQHykFgDxaHSlWrVqtnr1ajfioQB24MCB7rH33nvPWrVqZaVKlXLB0WWXXWZPP/20nTlzJskc2dBcw0mTJrnn6fn169e3lStXpjgXTYfc1Dc9t2rVqjZ//vwE20+HOuvVq+e+4LScV199NUV5t5q/8voUtMfXvn17K1GiRPB1rlq1ylq0aGFFihSx3LlzW4UKFey+++5L9b6k9aF+bdu2LcFjAwYMcCNN+/fvd/f/97//2d13322XXHKJe/1ly5a1Rx991I1InUtu5TfffGM33HCD63+ZMmXsmWeeiTiKnpJtrn3mww8/dK/DS0/xtn9iObKLFi2ya665xgUJBQsWtNatW9t3330X1sbbblu2bHH9V7sCBQpY586dI26nUEn1SXbv3m1dunSx4sWLu32lZs2aNm3atGTXZSAQsO7du7ttM3v27OD0N954w+rWrevWZ+HChd3o2S+//JKgT9p/v/32W7v++uvde6t06dI2cuRIO1cHDhxwI87aH7R9KlasaM8//3zYtkzN+09mzZplVapUcetF/X333XfD3tOaX9GiRd3/NSrrrV9tr1C//fabG1nU+0rt+/Tpk+CzIrF1rP1R+6XWkdaV9tf4IuXIbt682e688073flX/NQ9ti4MHD7rH1V7Bqba112/vveHtb9o+f/vb36xQoULWpEmTsMciefPNN+2KK65wy9M+8Pnnn6fonIH480yqb4nlyP7zn/90n4XannqPPvTQQ26fONf97uWXX3bzUxu9fn2W6sgGYhfDPkAEf/zxh910003uC0Cjtfqy9z5M9aXUu3dv91fByODBg+3QoUM2atSoZNelPhAPHz5s999/v/tQ1gfpHXfcYT/99FOyo7hffPGFCxwefPBBu+iii2zcuHHuC2v79u128cUXuzZr1661li1bWsmSJd0XrL40hw0bFvzSTUrbtm1twoQJLvhRwOhRwPTBBx+4LxSNiCgAuvHGG908dahRwZW+XEKDmpT661//av369bN33nnH+vbtG/aYpmk5+jLxggv1pUePHu71rlixwn3p/Prrr+6x1Ni5c6f7Qjt9+rR7DQomFeAoCIsvJdv8iSeecIGC+vLSSy+5aUmd7PPpp5+6/evSSy91X+YKxvVarr76aluzZk2CL32tJ/1YGDFihHt88uTJVqxYMRewJSapPml5+nJXgKwfMJq31qG2sYKAnj17Rpyn9if9YJk5c6YL7hTgy/Dhw23QoEGun127drU9e/a416MfgtontY949MNE+6j2e7X/z3/+Y48//rhVr17drZPU0P7QtGlTFzDqPaUfOUuXLnU/gnbs2JEgPzgl7z/t/3ovqD9a3+qvAn4FPh7t+6+88orbF2+//XY3D6lRo0bYutKPvYYNG7oAWtv8xRdfdEG0npcU7V8KZG+++WZ30zbXe+HkyZNJPk+Pa5knTpywRx55xAWzWjdz585121U/gpRGpW3UoEED94NE1KdQev9XqlTJHeZXUJ2Uzz77zO0P//jHP1wwqcBS21fvTwWPqZGSvoXSe0efc82bN3frVClE2i76cfLll1+GfaamZL9TSpheh1Ib9B44fvy4ff3117Z8+XIX2CNGBYAL2EMPPaRP6bBpTZs2ddMmTpyYoP2ff/6ZYNr9998fyJMnT+D48ePBaZ06dQqUK1cueH/r1q1unhdffHFg3759wenvvfeem/7BBx8Epw0ZMiRBn3Q/R44cgS1btgSnrV+/3k1/+eWXg9NuvfVW15fffvstOG3z5s2BbNmyJZhnfGfPng2ULl06cOedd4ZNf+edd9xzP//8c3f/3XffdfdXrlwZSA+NGjUK1K1bN2zaihUr3DKmT5+e5LofMWJEIC4uLrBt27Yk15+2hbaJp1evXq7N8uXLg9N2794dKFCggJuu7ZXabd6qVauwbR5/20+ZMiU4rVatWoFixYoF/vjjj7DtmSVLlkDHjh0TvJb77rsvbJ63336725eSk1ifxowZ4+b7xhtvBKedPHnSbYt8+fIFDh06FNb3UaNGBU6dOhVo27ZtIHfu3IGPP/44+Lyff/45kDVr1sDw4cPDlrFhwwa334VO995bodv1xIkTgRIlSiTY7yKJvx2ffvrpQN68eQM//PBDWLv+/fu7Pm3fvj3V77/q1asHypQpEzh8+HBw2pIlS1y70HW5Z88eN03bKD71UY8NGzYsbHrt2rUT7OvxaT/Ue13bTu9Jz8CBA908Q1//4sWL3TT9lbVr17r7s2bNSnIZWmeh84m/v7Vv3z7Rx0Lpvm6rVq0KTtN7MVeuXG4fDV0fkfbDSPNMrG96/4S+N731dOONNwbOnDkTbDd+/HjX7rXXXkv1fte6detA1apVEywbsY3UAiACjSzo8G18oSN2GtnZu3evOzyskaHvv/8+2XWpkR5vhFH0XNGIUHI06hA6OqHRn/z58wefqxEgjfroUKYOsXl0qDUlI10aodJIzLx58+zIkSPB6Rpt0WiUd4jRG13TKM+pU6csrbROlMbx448/hi1T20CH2yOtex1+1Lpv3LixGzHSqF9q6DVeddVVbuQndJStQ4cO6b7N49NI4bp169zopw7Bh27Pv/zlL65v8T3wwANh97V8HTXQqPC50DI0WqeUEY9GrzQapW2vUbb4I33aN7TN9VyNDno0Eq/D+Brl0rrxbpq/RvUWL14cNi+NCofmpCtFQdshJe+B+DSKrHWh91TosvVe0fsh/iHu5N5/v//+u23YsME6duwYNqKuUV+N3KVWpO2W3OvUe1jrWyOqoYfdU3LCnkZc5eOPP0429SQ1/U5Ko0aNXDqBR6Piet+qDylJozhX3nrSegk9kbNbt27uc1Ej66nd7/TZpiMYkdJNELsIZIEIFLhFOhNYeWo6lKgvDH1YKvjxPhy9HLSk6EM+lPel6uWBpua53vO95+qQvw4ZK3CNL9K0SPRFr3m8//777r6CGgUuCmK8L1V9qSulQYf0lCOrL60pU6a4w5nnQvPWF5GCV1FgqgBFwbfWsUcpFF7w5+Ucqi8pXfehlDeqICs+5fml9zaPtOzElqVqGQrEFKin136T1OuPX8nBq9YRP2dZh9iVn63DsfHP0FdOpraZ5qd1E3pTzq/2y1DK2Yyfaxm6H6eGlq088fjLVSAr8Zed3Hr0Xnda3kMe5YvGT+lJyev0+hB//9S8QoPwSJQiohQYpZ7ovak0A6ULpXY/1XxSKtL76PLLL3eBtFJMMkpi7yN9bitlJ/4+nJL9TqkG+mxRgKvXpXxbpSggtpEjC0QQKVdSOWYKnBTMKO9Uo6P6slL+mj4AU1JuK7GzbpPLQ0vrc1NKo5TKz1R+qnLClBurwFYBrkdfBgpoVPFBj2vkRXmTyv/TtNQWgtfosUaqtEydVKd5KGgNzf/UyI5GK/ft2+fWdeXKlV1eq/L/FNyea6mz5KTHNk8P52PbJ0UBkQJG5ZQqkA09U17rQPvERx99FLGf8feH9HwtWrb2C+VZR6KAKqOWnZxonWGv96HeEzpJ8ZNPPnGj7PohoveVgrlz/fxLi8ROEsvIEdv4UrLt9UNOebY68qD9/b///a/L+VXOsldqDbGHQBZIIZ0ZrMO5OpSqk1g8W7dujYl1qJN/FGDoBJ74Ik1LjA4Rjx071h221iipAlsFuPFpmm460Ucn0eiw/IwZM9zJGqmlQFknselLRMvUGcO33npr8HEd7v3hhx/c2cw67OtR6bRzodquGs2LT8s/122e0quxadmRliVKVdBImoL09JBYn9QHncSiQDB0VNZLlfD66NF21uHmW265xY2g60QvrxSTgnsFAxrFix84ZjQtW0cNvBHYtPJed0reQxl19T2vD9o/NbLo0ehmSketlQahm2q/6uQ3nUQ4ceJEdwJZevc90vtI71W9h70RaY18xq8kIJGqlZzL+yh0PSndQO/Pc90n9N7T55FumpdODtNnnE4gpIxebCK1AEjlL/rQX/D6oNMv9ljpnz68dQhYuX6hX8AaLUspfYArTUBBo0YlFNiG0pdp/BGsWrVqub+h6QXKeQ3Ne02KUhXU/7ffftulFShgCg3mIq17/V8B97nQmeAaodKZ1aGBgsoInes2V39TcghXFSW0vrR+Q7/cdeECjaCpb+klsT5pGarc4KVziCo4qNKARlC9lI1Q2rf0Q0X7hIrTe6PR+qLXetKIVfz9Qvf1QyCjaN9ctmyZOyoQn9atXlNqjw7oTHtdKCU0T1w5w/oxFcqrKx0pQEsLrWflK2tbhK7PlFyhTT8+479mBbT6sRL63tR+kV791vrXEQqPSq5pNFh51N77Rz84tB/qx1Norrh+EMWX0r5pPSmNQNVbQtfTv//9b7csr6JGasTfVzV/lWHT/NPjfABkDEZkgRTSiUUaWejUqZM7XKeRA5WLOV+Hd1NC5WgUDGkERuVodOhu/Pjx7stZJxilRJ06dVw+oMo36csvNK1AFIApkFPeqL6gdAKUytbo8HtoENasWTP3NyXXRtdossphjR492s0v/jKVSqBlqQ6n0gm0LB32O9ccUR2K1rZTOR6V2fHKb3kjleeyzXXCiwJD5SiqPqkCwtBR5VAq26UcYJ0oo9JOXvkt5eHGr0WaFon1SaWNVF9Yh6B1op1G3ZUuonxABUwq7xaJTiRUPrRGxbUNNA9tF430acRK21pt9HyNiilQ0bK03TKCSrYpn1s/fPRa9HqVX6ygU69H/dEId2qo5JTyvvUe0gmf2se891BocKvD7wpytH41Eq3cbbVJbcmp+Lx6s0oH0OvSe0onM+rHaHKvRaXhVE5No+bqk4Ja7a8KKPVj0aP1pJOl9H5T8K7RdJUJOxd6vUo9CS2/JaGH4lXGUKk4+sxQO+XPqkyW+hgaBKemb1pP2ue0HL2Pb7vtNjc6q+VrX0/uIjeRKPjWSYra9iq5qBxvbXsFxYm9JxADol02AYjF8luJlWD58ssvA1dddZUrQVSqVKlAv379XCmi0BI4SZXfUhmj+OKX8EmszI36mlw5Ilm4cKEr86PSNJdddllg8uTJgccee8yVxEmpJ554wi2zYsWKCR5bs2aNK89zySWXBHLmzOnKSN1yyy1hJXi8vkUquZOYf/3rX26ZF110UeDYsWMJHv/2228DzZs3d+WhihQpEujWrVuwBFloaauUlN+Sr7/+2m1rrReVHVMpp3//+98Jym+ldJsfOXIk8Le//S1QsGDBsFJNkcpvyaeffhq4+uqr3Xzz58/vSqfpNYbyXotKPSVViigxifVJdu3aFejcubNbl9pXVHYqfh8T22//+c9/uul9+vQJTvvvf/8baNKkiSufpFvlypXdPrtp06Zk31uJlWeKL9J2VJmsAQMGuH1Vr0Ovp3HjxoEXXnjBlRRL6nVIpBJaM2bMcP3X/l2tWrXA+++/78o0aVqopUuXunJaWm7ofNRHrYP4Iu2bkaic1NChQwMlS5Z0+8d1110X2LhxY4LXH7/81k8//eRKtel9r/26cOHCgeuvv97ta6G+//77wLXXXuvmHVrSK7H9LbnPJZVxq1Spkltf+uwJfV94PvnkE7cuta6uuOIK95xI80ysb4nt8yq3pe2SPXv2QPHixQM9evQI7N+/P6xNSve7V1991S1bZdr0WrQe+/btGzh48GDE7YTYEKd/oh1MA8hYGiXT2feR8tkAJE/pIBoFPNe8bAAZgxxZIJOJf8lWBa8qoRW/bBKAhJQLGT/PVCf9rV+/nvcQEIMYkQUyGZ1MpHxBr5aictGU66o8u0g1HwH8/5RXqxOJlGOpHE1VctAZ/8pf1gl53uWgAcQGTvYCMhmd+KCz/3VWuk6+0AlFOoGFIBZInk7u0wlHuqiAKlnoRECd7PPcc88RxAIxiBFZAAAA+BI5sgAAAPAlAlkAAAD4EjmyUaQr4+gKTCq0nFGXOwQAAPATVYbVxXF0wmXoZbQjIZCNIgWxZcuWjWYXAAAAYpIueVymTJkk2xDIRpF3yTttKF3uEQAA4EJ36NAhN9CXkksDE8hGkZdOoCCWQBYAAOD/l5K0S072AgAAgC8RyAIAAMCXCGQBAADgS+TIAgCAmHfmzBk7depUtLuBdJA9e3bLmjVresyKQBYAAMR2TdGdO3fagQMHot0VpKOCBQtaiRIl0lxHnxFZAAAQs7wgtlixYpYnTx4uIJQJfpj8+eeftnv3bne/ZMmSaZofgSwAAIjZdAIviL344ouj3R2kk9y5c7u/Cma1bdOSZsDJXgAAICZ5ObEaiUXmkuf/bdO05j0TyAIAgJiW1jxKZN5tSiALAAAAXyKQBQAA8IHy5cvbmDFjot2NmMLJXgAAwHcGzN5wXpc34o7q6XbYfMiQIfbUU0+lug8rV660vHnzpvp5mRmBLADg/3zQM+1r4taxrE1c8Hbs2BFcBzNnzrTBgwfbpk2bgtPy5csXVo5K1RmyZUs+JCtatOgFv27jI7UAAAAgHanQv3crUKCAG6H17n///fd20UUX2UcffWR169a1nDlz2hdffGE//vijtW7d2ooXL+4C3fr169unn36aZGpBXFycTZ482W6//XZXBaBSpUr2/vvvX1DbkkAWAADgPOvfv78999xz9t1331mNGjXsyJEjdvPNN9vChQtt7dq11rJlS7v11ltt+/btSc5n6NCh9te//tW+/vpr9/wOHTrYvn377EJBIAsAAHCeDRs2zP7yl7/YZZddZoULF7aaNWva/fffb9WqVXMjq08//bR7LLkR1nvvvdfat29vFStWtGeffdYFxCtWrLALBYEsAADAeVavXr2w+wpA+/TpY1deeaUVLFjQpRdotDa5EdkaNWoE/68TwfLnzx+8/OuFgJO9AAAAzrP41QcUxC5YsMBeeOEFN7qqy7jedddddvLkySTnkz179rD7yps9e/asXSgIZAEAAKLsyy+/dGkCOnHLG6H9+eefo92tmEdqAQAAQJQpL3b27Nm2bt06W79+vf3tb3+7oEZWzxWBLAAAQJSNHj3aChUqZI0bN3bVClq0aGF16tSJdrdiXlxAlXgRFYcOHXL15Q4ePOiSswEgqrggAmLM8ePHbevWrVahQgXLlStXtLuD87RtUxMfMSILAAAAXyKQBQAAgC8RyAIAAMCXCGQBAADgSwSyAAAA8KWoB7ITJkyw8uXLuzPWGjZsmOz1gWfNmmWVK1d27atXr27z5s0Le1xFGAYPHmwlS5Z0V8Vo3ry5bd68OazN8OHDXXmLPHnyuMvAxTd16lR3ZYxIN++yb0uWLIn4+M6dO9NlvQAAACCGA9mZM2da7969bciQIbZmzRqrWbOmq5uW2DWCly5dau3bt7cuXbrY2rVrrU2bNu62cePGYJuRI0fauHHjbOLEibZ8+XJ3CTjNU2UePLrc29133209evSIuJy2bdvajh07wm6aR9OmTa1YsWJhbTdt2hTWLv7jAAAAyIR1ZDUCW79+fRs/fry7rytYlC1b1h555BHr379/xADz6NGjNnfu3OC0q666ymrVquUCV72UUqVK2WOPPeauWSyqQVa8eHE3ytquXbuw+Wlar1697MCBA0n2c8+ePVa6dGn797//bffcc09wRPb666+3/fv3RxzVTQnqyAKIKdSRRYyhjmzmddzvdWQ1Krp69Wp36D/YmSxZ3P1ly5ZFfI6mh7YXjZR67bVCdGg/tI1WhALmxOaZEtOnT3dpCHfddVeCxxREK43hL3/5i7tOclJOnDjhNk7oDQAAAOcmaoHs3r177cyZM260NJTuJ5ZnqulJtff+pmaeKaGRWF3zWDm3HgWvGgX+73//624aSb7uuutcikRiRowY4QJr76bnAAAAxKeYQkeNPTqfaMyYMUmuqLi4OJszZ06aV2Z6zed8yBbtDsQ6jeR+99139vrrr4dNv+KKK9zNo5PHfvzxR3vppZcStPUMGDDA5QR7NCJLMAsAQJRSYVLj1rEpb3rrrXbq1CmbP39+gsf+97//2bXXXmvr16+3GjVqpHieK1eudOf9pKennnrKBazr1q0Lm65zfgoVKmR+ELUR2SJFiljWrFlt165dYdN1v0SJEhGfo+lJtff+pmaeyZk8ebJLH6hbt26ybRs0aGBbtmxJ9PGcOXO6XI/QGwAAyFx0UvqCBQvs119/TfDYlClTrF69eqkKYqVo0aIuzfF8KFGihItZ/CBqgWyOHDlccLhw4cLgNJ3spfuNGjWK+BxND20v2lG89koY1soPbaNRT1UvSGyeSTly5Ii98847bodMCf2iUcoBAAC4cN1yyy0u8NRJ5fHjCpURVcUlVWHSieQKTlVO9O23305ynvFTC1Ra9Nprr3UnSlWpUsXFQ/E9/vjjdvnll7tlXHrppTZo0CA3Uizq29ChQ93IsFdC1Otv/NSCDRs22A033OBSLC+++GLr3r27ey2ee++9172mF154wcVBavPQQw8Fl5VpUwt0mL1Tp07ul4lGM7WBVJWgc+fO7vGOHTu6jazcUunZs6crgfXiiy9aq1atbMaMGbZq1SqbNGlScMUrn+SZZ56xSpUqucBWG02VDLSCPdu3b7d9+/a5v8rT9YbUK1asaPny5QsrD3b69Gn7+9//nqDv6qvmX7VqVXfmnUZuFy1aZJ988kmGrzcAABC7smXL5mIYBYZPPPGEi09EQaziDsUV+r8CTR2d/fDDD11VpMsuu8zFQ8nRwN8dd9zhzgHSYJ3O7g/Np/VcdNFFrg+KgxSMduvWzU3r16+fqwSl8qVKf/j0009de52/E5/iMp1YrwFBpTeoRGrXrl3t4YcfDgvUFy9e7IJY/dXRac1fR7S1zEwbyOpFqrSVLmCgk7H0grVCvZO1FGiqkkFoHupbb71lTz75pA0cONAFq/rFUK1atWAbbRytdP1aUFmtJk2auHmGlnbQ8qZNmxa8X7t2bfdXK1/J1aEneWlHiVReS1UXVObrt99+c790dIhAO4JKcgEAgAvbfffdZ6NGjbLPPvssGFsoreDOO++0cuXKBcuEisqOfvzxx+4ocEoCWcUb33//vXuOglR59tln7aabbgprp3gpdERXy9QgoGIlja5q8E5Bd1Lpl4q7NGCnCk5ejq7KpioP+Pnnnw/GbMqp1XSljerCVRpw1BHyTB3IiiJ63SJRrdb4dCED3RKjXz3Dhg1zt8ToF0T84f7ELsCQGO0EugEAAMSnYE4DcK+99poLZDVKqRO9FJ9oVFaBpwJXDYhpcEwlOlOaA6uT0HWyeKn/F8RKpBRKHVnWRaJ0MrpSAXSUObXn52hZumBV6IlmV199tRsV1kWhvEBWR6gVxHo0OqtR4Ex/iVoAAIDMSOfYqETn4cOH3WisUgeUIqmR2rFjx7rUAh0NVoqjDt8roE3PqksdOnSwm2++2V1ISldEVZpDei4jVPbs2RMMLCrYzWgEsgAAABngr3/9q0uR1OF5HZpXuoECPF1AqXXr1i5XVqOdOhHrhx9+SPF8r7zySvvll19cmSzPV199leCoslIYFLzqXCSlY27bti3BifcaHU5uWTohTGmbHvVfryu0DGm0EMgCAABkAOWg6nwg1ZFX0Kmz+0VBpaoMKNjUofv7778/QenQpOgKpqpG0KlTJxdkKmVBAWsoLUPnGiknVqkFSjF49913w9oob1ZXRdWIsC5UpfSG+DSqq/OMtCydHKYRZOX06uS0+BegigYCWQAAgAxML9i/f79LHfByWnUSVp06ddw05c/qZKvQ6krJ0WiogtJjx465k8NURWD48OFhbW677TZ79NFH3XlIOpleQbMqOYXSiWctW7Z0J6qrXFikEmDK29VJZar2VL9+fbvrrrusWbNm7sSuWBAXCAQC0e7EhUo1blXqQmUzuDgCgExxpaRUXP0ISI7OlteIocpdhlYfQubetodSER8xIgsAAABfIpAFAACALxHIAgAAwJcIZAEAAOBLBLIAAADwJQJZAAAQ087HFaLgz22aLV3mAgAAkM505SnVTP39999dnVPd15Wx4F+BQMBdJnfPnj1u22qbpgWBLAAAiEkKdFRnVFfFUjCLzCNPnjx2ySWXuG2cFgSyAAAgZmnETgHP6dOn7cyZM9HuDtJB1qxZLVu2bOkyuk4gCwAAYpoCnuzZs7sbEIqTvQAAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC8RyAIAAMCXCGQBAADgSwSyAAAA8CUCWQAAAPgSgSwAAAB8KeqB7IQJE6x8+fKWK1cua9iwoa1YsSLJ9rNmzbLKlSu79tWrV7d58+aFPR4IBGzw4MFWsmRJy507tzVv3tw2b94c1mb48OHWuHFjy5MnjxUsWDDicuLi4hLcZsyYEdZmyZIlVqdOHcuZM6dVrFjRpk6des7rAQAAAD4KZGfOnGm9e/e2IUOG2Jo1a6xmzZrWokUL2717d8T2S5cutfbt21uXLl1s7dq11qZNG3fbuHFjsM3IkSNt3LhxNnHiRFu+fLnlzZvXzfP48ePBNidPnrS7777bevTokWT/pkyZYjt27AjetCzP1q1brVWrVnb99dfbunXrrFevXta1a1f7+OOP02XdAAAAIGlxAQ1hRolGYOvXr2/jx49398+ePWtly5a1Rx55xPr375+gfdu2be3o0aM2d+7c4LSrrrrKatWq5QJXvZRSpUrZY489Zn369HGPHzx40IoXL+5GS9u1axc2P01TAHrgwIEEy9II7LvvvhsWvIZ6/PHH7cMPPwwLojV/zWv+/Pkpev2HDh2yAgUKuD7mz58/Rc8BgAzzQc+0z+PWsenREwAXsEOpiI+iNiKrUdHVq1e7Q//BzmTJ4u4vW7Ys4nM0PbS9aLTVa69R0p07d4a10YpQwJzYPJPy0EMPWZEiRaxBgwb22muvuUA5pX0BAABAxspmUbJ37147c+aMGy0Npfvff/99xOcoSI3UXtO9x71pibVJqWHDhtkNN9zg8mg/+eQTe/DBB+3IkSP2j3/8I8m+6FfEsWPHXH5ufCdOnHA3j9oCAADAZ4FsrBs0aFDw/7Vr13YpDaNGjQoGsudixIgRNnTo0HTqIQAAwIUtaqkFOmSfNWtW27VrV9h03S9RokTE52h6Uu29v6mZZ0opPeHXX38Njqgm1hflckQajZUBAwa4fA/v9ssvv6SpTwAAABeyqAWyOXLksLp169rChQuD03Syl+43atQo4nM0PbS9LFiwINi+QoUKLsAMbaPD96pekNg8U0qVCQoVKuRKbaWkL5HouQp0Q28AAADwYWqBSm916tTJ6tWr506oGjNmjDuE37lzZ/d4x44drXTp0u6QvPTs2dOaNm1qL774oit9pbquq1atskmTJgUrDagKwTPPPGOVKlVyga1SBFTJILT6wPbt223fvn3ur/J0FaSKasHmy5fPPvjgAze6qooIqlerAPXZZ58NVkKQBx54wFVb6Nevn9133322aNEie+edd1wlAwAAAGTyQFbltPbs2eMuYKCTp1RGS6WrvJOoFGiqkoFHFzF466237Mknn7SBAwe6YHXOnDlWrVq1YBsFlgqGu3fv7kphNWnSxM1TAalHy5s2bVpYDqwsXrzYrrvuOsuePbu7UMOjjz7qKhUowB09erR169Yt+BwFyQpa1Wbs2LFWpkwZmzx5sqtcAAAAgExeR/ZCRx1ZADGFOrIAYoAv6sgCAAAAaUEgCwAAAF8ikAUAAIAvEcgCAADAlwhkAQAA4EsEsgAAAPAlAlkAAAD4EoEsAAAAfIlAFgAAAL5EIAsAAABfIpAFAACALxHIAgAAwJcIZAEAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC8RyAIAAMCXCGQBAADgSwSyAAAA8CUCWQAAAPgSgSwAAAB8iUAWAAAAvkQgCwAAAF8ikAUAAIAvEcgCAADAl6IeyE6YMMHKly9vuXLlsoYNG9qKFSuSbD9r1iyrXLmya1+9enWbN29e2OOBQMAGDx5sJUuWtNy5c1vz5s1t8+bNYW2GDx9ujRs3tjx58ljBggUTLGP9+vXWvn17K1u2rJvHlVdeaWPHjg1rs2TJEouLi0tw27lzZ5rWBwAAAHwQyM6cOdN69+5tQ4YMsTVr1ljNmjWtRYsWtnv37ojtly5d6gLMLl262Nq1a61NmzbutnHjxmCbkSNH2rhx42zixIm2fPlyy5s3r5vn8ePHg21Onjxpd999t/Xo0SPiclavXm3FihWzN954w7755ht74oknbMCAATZ+/PgEbTdt2mQ7duwI3vQ8AAAAZLy4gIYwo0QjsPXr1w8GiGfPnnWjoI888oj1798/Qfu2bdva0aNHbe7cucFpV111ldWqVcsFrnoppUqVsscee8z69OnjHj948KAVL17cpk6dau3atQubn6b16tXLDhw4kGxfH3roIfvuu+9s0aJFwRHZ66+/3vbv3x9xVDclDh06ZAUKFHB9zJ8//znNAwDSzQc90z6PW8OPXgFARsZHURuR1aioRj516D/YmSxZ3P1ly5ZFfI6mh7YXjbZ67bdu3eoO7Ye20YpQwJzYPFNKK7Nw4cIJpiuIVhrDX/7yF/vyyy+TnMeJEyfcxgm9AQAA4NxksyjZu3evnTlzxo2WhtL977//PuJzFKRGau/lpXp/k2pzLpTSoDSIDz/8MDhNwatGgevVq+cC1MmTJ9t1113n0hnq1KkTcT4jRoywoUOHnnM/gHQdPcssI2exMIoYC31Ir35kBrHw3oiVfQLI5KIWyPqF8m9bt27t8nhvvPHG4PQrrrjC3Tw6eezHH3+0l156yV5//fWI81KerXKCPRqRVSoFAAAAUi9qqQVFihSxrFmz2q5du8Km636JEiUiPkfTk2rv/U3NPJPy7bffWrNmzax79+725JNPJtu+QYMGtmXLlkQfz5kzp8v1CL0BAADAZ4Fsjhw5rG7durZw4cLgNJ3spfuNGjWK+BxND20vCxYsCLavUKGCC1hD22jUU4f7E5tnYlStQCdzderUyZXrSol169a5lAMAAABk8tQCHWZXoKg8U41mjhkzxlUl6Ny5s3u8Y8eOVrp0aZdbKj179rSmTZvaiy++aK1atbIZM2bYqlWrbNKkSe5x1XFVFYJnnnnGKlWq5ALbQYMGuUoGKtPl2b59u+3bt8/9VZ6uAlCpWLGi5cuXz6UT3HDDDe5EMvXRy6/VCHLRokXd/9VXzb9q1aqutJdyZFXR4JNPPjnv6xEAAOBCFNVAVuW09uzZ4y5goGBRFQDmz58fPFlLgaYqGYTmob711lvuMP/AgQNdsDpnzhyrVq1asE2/fv1cMKx0AJXVatKkiZunLqDg0fKmTZsWvF+7dm33d/Hixe6Erf/85z+uX6ojq5unXLly9vPPPwerLqjM12+//eYurFCjRg379NNP3SguAAAAMnkd2QsddWTh+zOzY0EsnB0eC31Ir36kVSzsV7Hw3oiVfQLwIV/UkQUAAADSgkAWAAAAvkQgCwAAAF8ikAUAAIAvEcgCAADAlwhkAQAA4EsEsgAAAPAlAlkAAAD4EoEsAAAAfIlAFgAAAL5EIAsAAIALJ5D96aef0r8nAAAAQEYHshUrVrTrr7/e3njjDTt+/Pi5zAIAAAA4/4HsmjVrrEaNGta7d28rUaKE3X///bZixYq09QQAAADI6EC2Vq1aNnbsWPv999/ttddesx07dliTJk2sWrVqNnr0aNuzZ8+5zBYAAAA4Pyd7ZcuWze644w6bNWuWPf/887Zlyxbr06ePlS1b1jp27OgCXAAAACDmAtlVq1bZgw8+aCVLlnQjsQpif/zxR1uwYIEbrW3dunX69RQAAAAIkc3OgYLWKVOm2KZNm+zmm2+26dOnu79ZsvxfXFyhQgWbOnWqlS9f/lxmDwAAAGRMIPvKK6/YfffdZ/fee68bjY2kWLFi9u9///tcZg8AAABkTCCr1IFLLrkkOALrCQQC9ssvv7jHcuTIYZ06dTqX2QMAAAAZkyN72WWX2d69exNM37dvn0srAAAAAGIykNXIayRHjhyxXLlypbVPAAAAQPqmFugCCBIXF2eDBw+2PHnyBB87c+aMLV++3NWYBQAAAGIqkF27dm1wRHbDhg0uD9aj/9esWdOV4AIAAABiKpBdvHix+9u5c2d3Za/8+fNnVL8AAACA9K9aoBqyAAAAgC8CWV2KVhc50Cis/p+U2bNnp0ffAAAAgLQHsgUKFHAneXn/BwAAAHxRfkvpBBdddFHw/0ndUmPChAnuUrYq29WwYUNbsWJFku1nzZpllStXdu2rV69u8+bNC3tcJ6KpooKuOJY7d25r3ry5bd68OazN8OHDrXHjxq7qQsGCBSMuZ/v27daqVSvXRlcp69u3r50+fTqszZIlS6xOnTqWM2dOq1ixohuxBgAAQAzXkT127Jj9+eefwfvbtm2zMWPG2CeffJKq+cycOdOV9BoyZIitWbPGVT1o0aKF7d69O2L7pUuXWvv27a1Lly6ugkKbNm3cbePGjcE2I0eOtHHjxtnEiRNdObC8efO6eR4/fjzY5uTJk3b33Xdbjx49Ii5HpcQUxKqdljlt2jQXpCpA9mzdutW1uf76623dunXWq1cv69q1q3388cepWgcAAAA4j4Fs69atbfr06e7/Bw4csAYNGtiLL77opr/yyispns/o0aOtW7durgpClSpVXPCpEdDXXnstYntVSmjZsqUbHb3yyivt6aefdiOi48ePD47GKqB+8sknXV9q1Kjh+vn777/bnDlzgvMZOnSoPfroo25ENxIF5N9++6298cYbri7uTTfd5Jal0WMFt6K+6ipmet3qy8MPP2x33XWXvfTSS6lalwAAADiPgaxGT6+55hr3///85z9WokQJNyqroFGjoSmhgHD16tXu0H+wM1myuPvLli2L+BxND20vGm312muUdOfOnWFtlM+rlIXE5pnYchTkFi9ePGw5hw4dsm+++SZFfYnkxIkTbh6hNwAAAJzHQFZpBV6+rEYvVcVAQehVV13lAtqU2Lt3rzuEHxosiu4rGI1E05Nq7/1NzTxTs5zQZSTWRsGpUi8iGTFihAusvVvZsmVT3CcAAACkQyCrE5t0qP6XX35xOaE33nijm67cVi6SkLgBAwbYwYMHgzetPwAAAJzHQFYnPelStKo2oMP2jRo1Co7O1q5dO0XzKFKkiGXNmtV27doVNl33laoQiaYn1d77m5p5pmY5octIrI0CeVVLiETVDfR46A0AAADnMZDVSU0qT7Vq1SqbP39+cHqzZs1SfLJTjhw5rG7durZw4cLgtLNnz7r7XmAcn6aHtpcFCxYE2+vkKwWYoW10qF/VCxKbZ2LL2bBhQ1j1BC1HgadOSktJXwAAABCDl6gVBYzxRzlVvSA1VHqrU6dOVq9ePfdcVRw4evSoq2IgHTt2tNKlS7vcUunZs6c1bdrUVQpQ6asZM2a4YHrSpEnucV2wQWWwnnnmGatUqZILbAcNGmSlSpVyZbo8CsL37dvn/ipPV+WzvJSJfPnyuVQJBaz33HOPK+elfFhVQnjooYfcqKo88MADrlpCv3797L777rNFixbZO++8Yx9++OG5rlIAAABkdCCrYPO5555zI5IatdRIaqiffvopRfNp27at7dmzx6UqKFhUqSuN8HonUSnQ1ElkHl3E4K233nJB5cCBA12wqlzdatWqBdsosFT/unfv7kqDNWnSxM1TF1DwaHmqDevx0iEWL15s1113nUt5mDt3rqszqxFW1aJVwD1s2LDgcxQkK2hVGS+VBStTpoxNnjzZVS4AAABAxosLqPhqKumiBJ999pkbsdQVtLxL13o0corkKe1B1Qt04hf5ski1D9L4Prt1bOZY6WldD+mxLmKhD+nVj7SKhf0qFt4bsbJPAJk8PjqnEdmPPvrIjUZeffXV59pHAAAA4Pyf7FWoUCErXLhw2pYMAAAAnO9AVpdrVZ6pLowAAAAARMM5pRaoasCPP/7oTspSLdns2bMnuIQtAAAAEHOBbGgpKwAAAMA3geyQIUPSvycAAABARufIimq0qm7qgAED3MUFvJSC33777VxnCQAAAGTsiOzXX39tzZs3dzW+fv75Z+vWrZurYjB79mx3EYPp06efy2wBAACAjB2R1aVl7733Xtu8eXPYFbNuvvlm+/zzz89llgAAAEDGB7IrV660+++/P8H00qVLu0vNAgAAADEZyObMmdNdPiy+H374wYoWLZoe/QIAAADSP5C97bbbbNiwYXbq1Cl3Py4uzuXGPv7443bnnXeeyywBAACAjA9kdUGEI0eOuNHXY8eOWdOmTa1ixYp20UUX2fDhw89llgAAAEDGVy1QtYIFCxbYl19+aevXr3dBbZ06dVwlAwAAACAmA9mzZ8/a1KlTXaktld5SWkGFChWsRIkSFggE3H0A8JMBszek6fkjwq/SDQCIxdQCBarKj+3atau78EH16tWtatWqtm3bNleO6/bbb8+4ngIAAADnOiKrkVjViV24cKFdf/31YY8tWrTI2rRp4y6G0LFjx9TMFgCQDpZv/b+rLJ6rhhUKsx0AZN5A9u2337aBAwcmCGLlhhtusP79+9ubb75JIAsA5zm9Qdqw1gFcYLKk9tK0LVu2TPTxm266yZ38BQAAAMTUiOy+ffusePHiiT6ux/bv358e/QIAXIgnzt1RPd36AiDzS9WI7JkzZyxbtsRj36xZs9rp06fTo18AAABA+o3IqmqBqhPoErWRnDhxIjWzAwBkopPFnDLp0RMAyIBAtlOnTsm2oWIBgAsNASAA+CCQnTJlSsb1BAAAAMjoS9QCABCrZci40hpw4UjVyV4AAABArCCQBQAAgC/FRCA7YcIEK1++vOXKlcsaNmxoK1asSLL9rFmzrHLlyq599erVbd68eQmqKwwePNhKlixpuXPntubNm9vmzZsT1MTt0KGD5c+f3woWLGhdunSxI0eOBB9/6qmnLC4uLsEtb968YZfsjf+4+gQAAIALIEd25syZ1rt3b5s4caILYseMGWMtWrSwTZs2WbFixRK0X7p0qbVv395GjBhht9xyi7311lvWpk0bW7NmjVWrVs21GTlypI0bN86mTZtmFSpUsEGDBrl5fvvtt8FAU0Hsjh07bMGCBXbq1Cnr3Lmzde/e3c1P+vTpYw888EDYsps1a2b169cPm6ZAWH31KJgFcP5QMQAALlxRH5EdPXq0devWzQWSVapUcQFtnjx57LXXXovYfuzYse4yuX379rUrr7zSnn76aatTp46NHz8+OBqrYPjJJ5+01q1bW40aNWz69On2+++/25w5c1yb7777zubPn2+TJ092wXOTJk3s5ZdfthkzZrh2ki9fPitRokTwtmvXLhcIa+Q2lALX0HZJXfkMAAAAmSSQPXnypK1evdod+g92KEsWd3/ZsmURn6Ppoe1Fo61e+61bt9rOnTvD2hQoUMAFrF4b/VU6Qb169YJt1F7LXr58ecTlKui9/PLL7ZprrgmbrnSEcuXKWdmyZV3g/M0335zTugAAAICPAtm9e/e6y97GH8XUfQWjkWh6Uu29v8m1iZ+2oEvvFi5cOOJyjx8/bm+++WaC0dgrrrjCjRy/99579sYbb9jZs2etcePG9uuvvyZ65bNDhw6F3QAAAODT1AI/ePfdd+3w4cMJrmzWqFEjdyWzWrVqWdOmTW327NlWtGhRe/XVVyPOR3m9Gh32bhrFBQAAgA9P9ipSpIhlzZrV5Z+G0n3lm0bi5asm1t77q2mqWhDaRgGn12b37t1h8zh9+rSrZBBpuUor0IllyeW/Zs+e3WrXrm1btmyJ+PiAAQPciW0ejcgSzAJAbOGiDIB/RDWQzZEjh9WtW9cWLlzoKg+IDs/r/sMPPxzxORoF1eO9evUKTlPlAU0XVSlQMKo2XuCqgFG5rz169AjO48CBAy4/V8uXRYsWuWUrlzaUcm4XL15s77//frKvR2kSGzZssJtvvjni4zlz5nQ3AOkXNPzfJwcA4EIU9fJbGqHUIXudeNWgQQNXceDo0aOuioHo0H3p0qXdYXnp2bOnO4z/4osvWqtWrVylgVWrVtmkSZOCVQQU5D7zzDNWqVKlYPmtUqVKBYNlVTtQ5QNVS1CVBJXfUuDcrl071y6UcmA1snvTTTcl6PuwYcPsqquusooVK7rAeNSoUbZt2zbr2rXreVhzAAAAF7aoB7Jt27a1PXv2uAsY6EQrjaKqNJZ3GH/79u2umoBHJ1Op1qvKaw0cONAFqyqr5dWQlX79+rlgWHVhFWCqvJbmGXqxAp28peBVtWE1/zvvvNPVng2lEVpd9ODee+91KRDx7d+/3wXD6nehQoXc6K7q3KqMGAAAADJWXECFVxEVSnnQSV8HDx50F1YAUuWDnmlbYbeOzRypBb+OTHMf5pTp5/s+pFc/YuF1pNWI7JPT9PwBp7pGvQ+x9B4FYjk+omoBAAAAfIlAFgAAAL5EIAsAAABfivrJXgAApKflW/elbQZl0qsnADIaI7IAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC9RfgsAgFgrAWZmDdOlJ0DmxogsAAAAfIkRWQAAYtCA2RvS9PwRd1RPt74AsYpAFkBUtfl1ZNS3AH3IXOsys0j7unw97Z34oKdF3a1jo/86MksfMiFSCwAAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC8RyAIAAMCXCGQBAADgSwSyAAAA8CUCWQAAAPgSgSwAAAB8KVu0OwDAvwbM3pDmebRJl54AAC5EBLIAACCi5Vv3pWnNNKxQmDWLzJ9aMGHCBCtfvrzlypXLGjZsaCtWrEiy/axZs6xy5cquffXq1W3evHlhjwcCARs8eLCVLFnScufObc2bN7fNmzeHtdm3b5916NDB8ufPbwULFrQuXbrYkSNHgo///PPPFhcXl+D21VdfpaovAAAAyKQjsjNnzrTevXvbxIkTXRA7ZswYa9GihW3atMmKFSuWoP3SpUutffv2NmLECLvlllvsrbfesjZt2tiaNWusWrVqrs3IkSNt3LhxNm3aNKtQoYINGjTIzfPbb791AacoiN2xY4ctWLDATp06ZZ07d7bu3bu7+YX69NNPrWrVqsH7F198car6AgBANJD6gwtB1EdkR48ebd26dXOBZJUqVVxAmydPHnvttdcith87dqy1bNnS+vbta1deeaU9/fTTVqdOHRs/fnxwNFbB8JNPPmmtW7e2GjVq2PTp0+3333+3OXPmuDbfffedzZ8/3yZPnuyC5yZNmtjLL79sM2bMcO1CKXAtUaJE8JY9e/YU9wUAAACZNJA9efKkrV692h36D3YoSxZ3f9myZRGfo+mh7UWjrV77rVu32s6dO8PaFChQwAWsXhv9VTpBvXr1gm3UXstevnx52Lxvu+02NzKsYPf9999PVV/iO3HihB06dCjsBgAAAB8Gsnv37rUzZ85Y8eLFw6brvoLRSDQ9qfbe3+TaxE9byJYtmxUuXDjYJl++fPbiiy+6HNgPP/zQBbJKGwgNZpPrS3xKQVBQ7d3Kli2bzBoCAABAzObIxqoiRYq43F1P/fr1XdrBqFGj3CjtuRgwYEDYPDUiSzALAEAGl/n7NY3VF9LcA2TKEVkFi1mzZrVdu3aFTdd95aNGoulJtff+Jtdm9+7dYY+fPn3aVTJIbLmi9IQtW7akuC/x5cyZ01VJCL0BAADAhyOyOXLksLp169rChQvdYXs5e/asu//www9HfE6jRo3c47169QpOU+UBTRdVKVAgqTa1atUKjnwq97VHjx7BeRw4cMDl52r5smjRIrdsBauJWbdunSvpldK+AAAApEtNXlZjbKYW6FB7p06d3IlXDRo0cBUHjh496qoYSMeOHa106dIuv1R69uxpTZs2dfmrrVq1cpUGVq1aZZMmTXKPq9arAstnnnnGKlWqFCy/VapUqWCwrAoDqjagagmqkqDyWwqc27Vr59qJSncp0K5du7a7P3v2bFdJQZUOPMn1BQAAAJk4kG3btq3t2bPHXcBAJ0lpFFWlsbyTqLZv3+6qCXgaN27s6rWqvNbAgQNdsKqyWqF1W/v16+eCYdWF1cirTtTSPL0asvLmm2+64LVZs2Zu/nfeeaerPRtK5bS2bdvmTgTTRQ9U8/auu+5KVV8AAACQMeICKryKqFDKg6oXHDx4kHxZpN4HPdO21m4dGyMnYYxM8zyA9DSnTL9MsV/HwutIj0vUDjjVNfqv4x+vp7kPy8fdE/U+ZMb4KOojsgAAIHNKa16oUyY9eoLMKupX9gIAAADOBYEsAAAAfIlAFgAAAL5EIAsAAABf4mQvAACAjK7QwhrOEIzIAgAAwJcIZAEAAOBLBLIAAADwJQJZAAAA+BInewEXqPQ4eQEAgGgikAUAALgABh9G3FHdMhtSCwAAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC9RRxYAAOACMCCNtWhjsQ4tI7IAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC8RyAIAAMCXCGQBAADgSzERyE6YMMHKly9vuXLlsoYNG9qKFSuSbD9r1iyrXLmya1+9enWbN29e2OOBQMAGDx5sJUuWtNy5c1vz5s1t8+bNYW327dtnHTp0sPz581vBggWtS5cuduTIkeDjS5YssdatW7t55M2b12rVqmVvvvlm2DymTp1qcXFxYTf1CQAAABfAlb1mzpxpvXv3tokTJ7ogdsyYMdaiRQvbtGmTFStWLEH7pUuXWvv27W3EiBF2yy232FtvvWVt2rSxNWvWWLVq1VybkSNH2rhx42zatGlWoUIFGzRokJvnt99+Gww0FcTu2LHDFixYYKdOnbLOnTtb9+7d3fy85dSoUcMef/xxK168uM2dO9c6duxoBQoUcMv1KBBWXz0KZoHzYfnWfWmbQZn06gkAABfoiOzo0aOtW7duLpCsUqWKC2jz5Mljr732WsT2Y8eOtZYtW1rfvn3tyiuvtKefftrq1Klj48ePD47GKhh+8skn3YiqgtHp06fb77//bnPmzHFtvvvuO5s/f75NnjzZBc9NmjSxl19+2WbMmOHaycCBA928GzdubJdddpn17NnTLXf27Nlh/VHgWqJEieBNQS8AAAAyeSB78uRJW716tTv0H+xQlizu/rJlyyI+R9ND24tGW732W7dutZ07d4a10SiqAlavjf4qnaBevXrBNmqvZS9fvjzR/h48eNAKFy4cNk3pCOXKlbOyZcu6wPmbb75J9PknTpywQ4cOhd0AAADgw0B27969dubMmQSjmLqvYDQSTU+qvfc3uTbx0xayZcvmgtTElvvOO+/YypUr3cix54orrnAjx++995698cYbdvbsWTeC++uvv0ach9IhFFR7NwW/AAAA8GlqgR8sXrzYBbD/+te/rGrVqsHpjRo1cnmzOhGsadOmLu2gaNGi9uqrr0acz4ABA9yornf75ZdfzuOrAAAAyFyiGsgWKVLEsmbNart27QqbrvvKN41E05Nq7/1Nrs3u3bvDHj99+rSrZBB/uZ999pndeuut9tJLL7mgNSnZs2e32rVr25YtWyI+njNnTndyWOgNAAAAPgxkc+TIYXXr1rWFCxcGp+nwvO5rtDMSTQ9tL6o84LVXlQIFo6FtlIuq3Fevjf4eOHDA5ed6Fi1a5JatXNrQElytWrWy559/3lU0SI7SJDZs2OBKdgEAACCTl99S6a1OnTq5E68aNGjgKg4cPXo0mIuqUdDSpUu7/FJR9QAdxn/xxRddkKlKA6tWrbJJkyYFqwj06tXLnnnmGatUqVKw/FapUqVcmS5RtQNVIFC1BFVJUPmthx9+2Nq1a+faeekEKrOl5d15553B3FkF394JX8OGDbOrrrrKKlas6ALjUaNG2bZt26xr165RWZcAAAAXkqgHsm3btrU9e/a4CxgoWFS+qUpjeSdrbd++3VUT8OhkKtV6VXktlchSsKqyWl4NWenXr58LhjWKqgBT5bU0z9CLFejiBgpemzVr5uavYFW1Zz2qQfvnn3+6ANoLokVBtEZqZf/+/S4YVr8LFSrkRpdVf1ZlxAAAAJCx4gIqvIqoUMqDqhfoxC/yZZFay8fdk6aVNqdMv5hY6W1+HRntLgDp/t6Ihf2a15F+2yMW1mUsfGaPuKN6zMVHVC0AAACALxHIAgAAwJcIZAEAAOBLBLIAAADwJQJZAAAA+BKBLAAAAHyJQBYAAAC+RCALAAAAXyKQBQAAgC8RyAIAAMCXskW7A8CFaMDsDWmeR5t06QkAAP7FiCwAAAB8iUAWAAAAvkQgCwAAAF8ikAUAAIAvEcgCAADAlwhkAQAA4EsEsgAAAPAlAlkAAAD4EoEsAAAAfIlAFgAAAL5EIAsAAABfyhbtDgB+NGD2hmh3AQCACx4jsgAAAPAlAlkAAAD4EoEsAAAAfIlAFgAAAL4UE4HshAkTrHz58pYrVy5r2LChrVixIsn2s2bNssqVK7v21atXt3nz5oU9HggEbPDgwVayZEnLnTu3NW/e3DZv3hzWZt++fdahQwfLnz+/FSxY0Lp06WJHjhwJa/P111/bNddc45ZTtmxZGzlyZKr7AgAAgEwayM6cOdN69+5tQ4YMsTVr1ljNmjWtRYsWtnv37ojtly5dau3bt3eB59q1a61NmzbutnHjxmAbBZzjxo2ziRMn2vLlyy1v3rxunsePHw+2URD7zTff2IIFC2zu3Ln2+eefW/fu3YOPHzp0yG688UYrV66crV692kaNGmVPPfWUTZo0KVV9AQAAQMaIC2j4Moo0Alu/fn0bP368u3/27Fk3+vnII49Y//79E7Rv27atHT161AWfnquuuspq1arlAle9nFKlStljjz1mffr0cY8fPHjQihcvblOnTrV27drZd999Z1WqVLGVK1davXr1XJv58+fbzTffbL/++qt7/iuvvGJPPPGE7dy503LkyOHaqD9z5syx77//PkV9SY6C5QIFCrj+aWQY50dmKZ3V5teERwhSY06ZfpYZXgeQ3tLjvREL+zWvI7Y+L2OhD2k14o7qdj6kJj6K6ojsyZMn3WinDv0HO5Qli7u/bNmyiM/R9ND2otFWr/3WrVtd8BnaRitDAbPXRn+VTuAFsaL2WrZGcL021157bTCI9ZazadMm279/f4r6AgAAgEx6QYS9e/famTNn3GhpKN33Rj3jU5Aaqb2me49705JqU6xYsbDHs2XLZoULFw5rU6FChQTz8B4rVKhQsn2J78SJE+7m0S8N75cHUu6p979hdZnZ0eMn07QeTvwZnhPu19cBpLf0eG/Ewn7N64itz8tY6ENana94xVtOSpIGuLLXeTRixAgbOnRogulKpQBS66U0r7J3MsnrANLbO5lkv+Z1xNLnZSz0wW/79eHDh91R9ZgNZIsUKWJZs2a1Xbt2hU3X/RIlSkR8jqYn1d77q2mqWhDaRrmrXpv4J5OdPn3aVTIInU+k5YQuI7m+xDdgwAB3YptH+cBa5sUXX2xxcXGW0b9uFDD/8ssv5ONeoNgHwH4A9gH44TtBI7EKYnXOUnKiGsgq/7Ru3bq2cOFCd7a/F9zp/sMPPxzxOY0aNXKP9+rVKzhNlQc0XZQOoEBSbbzAVRtLua89evQIzuPAgQMuP1fLl0WLFrllK5fWa6OTvU6dOmXZs2cPLueKK65waQUp6Ut8OXPmdLdQytU9n7SzxtoOi/OLfQDsB2AfQKx/JyQ3EhsUiLIZM2YEcubMGZg6dWrg22+/DXTv3j1QsGDBwM6dO93j99xzT6B///7B9l9++WUgW7ZsgRdeeCHw3XffBYYMGRLInj17YMOGDcE2zz33nJvHe++9F/j6668DrVu3DlSoUCFw7NixYJuWLVsGateuHVi+fHngiy++CFSqVCnQvn374OMHDhwIFC9e3C1/48aNrp958uQJvPrqq6nqS6w4ePCgEk3cX1yY2AfAfgD2AWS274SoB7Ly8ssvBy655JJAjhw5Ag0aNAh89dVXwceaNm0a6NSpU1j7d955J3D55Ze79lWrVg18+OGHYY+fPXs2MGjQIBeIKkhu1qxZYNOmTWFt/vjjDxe45suXL5A/f/5A586dA4cPHw5rs379+kCTJk3cPEqXLu0C5PiS60usyCw7LM4d+wDYD8A+gMz2nRD1OrI4P1QtQSebKU83fnoDLgzsA2A/APsAMtt3AoEsAAAAfCnql6gFAAAAzgWBLAAAAHyJQBYAAAC+RCB7AZgwYYKVL1/ecuXK5erkrlixItpdQjp56qmn3MU0Qm+VK1cOPn78+HF76KGH3EU38uXLZ3feeWeCi3hs377dWrVqZXny5HGXbu7bt6+7QAhi1+eff2633nqrKxaubT5nzpywx3UO7+DBg91FYXLnzm3Nmze3zZs3h7XRxVg6dOjg6keqnnWXLl3syJHwS2B+/fXXds0117jPDhVOHzly5Hl5fUj7PnDvvfcm+Gxo2bJlWBv2Af8bMWKE1a9f3y666CL3+a2a/Js2bQprk17fA0uWLLE6deq4E8MqVqxoU6dOtVhAIJvJzZw5011NbMiQIbZmzRqrWbOmtWjRIsGVzeBfVatWtR07dgRvX3zxRfCxRx991D744AObNWuWffbZZ/b777/bHXfcEXz8zJkz7sPr5MmTtnTpUps2bZr7cFIQhNh19OhR917Wj9RIFHCOGzfOJk6c6C4GkzdvXve+1xeaR0HsN9984y7iMnfuXBcYde/ePfi4LiRz4403Wrly5dzFY0aNGuV+OE2aNOm8vEakbR8QBa6hnw1vv/122OPsA/732WefuSD1q6++cu9lXcRJ71vtH+n5PbB161bX5vrrr7d169a5C0F17drVPv74Y4u6aNf/QsZSXd6HHnooeP/MmTOBUqVKBUaMGMGqzwR0EY6aNWtGfEwX9dAFOmbNmhWcpgt36G2/bNkyd3/evHmBLFmyBC9AIq+88oqrrXzixInz8AqQVtqe7777blgd7RIlSgRGjRoVti+oHvbbb7/t7uviM3reypUrg20++uijQFxcXOC3335z9//5z38GChUqFLYfPP7444ErrriCjRbj+4Co/rouBpQY9oHMaffu3W5/+Oyzz9L1e6Bfv36uVn6otm3bBlq0aBGINkZkMzH9utJIig4rerJkyeLuL1u2LKp9Q/rRIWMdXrz00kvdCIsOEYm2vX6dh25/pR1ccsklwe2vv9WrV7fixYsH22jkTqNxGq2D/2jkZOfOnWHbXZd6VFpR6HZXOkG9evWCbdRenw8awfXaXHvtte5S4qH7hg5b7t+//7y+JpwbHQrWYWJdWl2XaP/jjz+Cj7EPZE4HDx50fwsXLpyu3wNqEzoPr00sxBIEspnY3r173SGD0J1TdF9fdPA/BSc6BDR//nx75ZVXXBCjnMbDhw+7bawgRAFLYttffyPtH95j8B9vuyX1vtdfBTihsmXL5r782DcyB6UVTJ8+3RYuXGjPP/+8O6R80003ue8EYR/IfM6ePesO+V999dVWrVo1Ny29vgcSa6Ng99ixYxZN2aK6dABpoi8mT40aNVxgq5zGd955x53kA+DC1K5du+D/Ndqmz4fLLrvMjdI2a9Ysqn1DxnjooYds48aNYedJXAgYkc3EihQpYlmzZk1wdqLulyhRImr9QsbRr+7LL7/ctmzZ4rax0ksOHDiQ6PbX30j7h/cY/Mfbbkm97/U3/gmfOkNZZ7Gzb2ROSj3Sd4I+G4R9IHN5+OGH3UmbixcvtjJlygSnp9f3QGJtVPUk2oMmBLKZmA4n1K1b1x1aCj30oPuNGjWKat+QMVQ+6ccff3Rll7Tts2fPHrb9ld+oHFpv++vvhg0bwoIanfmqD6cqVaqwmXyoQoUK7ksndLvr8J9yX0O3u77YlD/nWbRokft80Ki+10aVDJRfF7pvKN+yUKFC5/U1Ie1+/fVXlyOrzwZhH8gcAoGAC2Lfffdd9x7W+z9Uen0PqE3oPLw2MRFLRPtsM2SsGTNmuLOVp06d6s5S7d69e6BgwYJhZyfCvx577LHAkiVLAlu3bg18+eWXgebNmweKFCnizlyVBx54IHDJJZcEFi1aFFi1alWgUaNG7uY5ffp0oFq1aoEbb7wxsG7dusD8+fMDRYsWDQwYMCCKrwrJOXz4cGDt2rXupo/x0aNHu/9v27bNPf7cc8+59/l7770X+Prrr93Z6xUqVAgcO3YsOI+WLVsGateuHVi+fHngiy++CFSqVCnQvn374OM627l48eKBe+65J7Bx40b3WZInT57Aq6++ygaK8X1Aj/Xp08edla7Phk8//TRQp04dt42PHz8enAf7gP/16NEjUKBAAfc9sGPHjuDtzz//DLZJj++Bn376yb3/+/bt66oeTJgwIZA1a1bXNtoIZC8AL7/8stuJc+TI4cpxffXVV9HuEtKJyp+ULFnSbdvSpUu7+1u2bAk+rsDlwQcfdGWU9CF0++23uw+5UD///HPgpptuCuTOndsFwQqOT506xTaKYYsXL3bBS/ybSi55JbgGDRrkAlH9kG3WrFlg06ZNYfP4448/XOCaL18+V2anc+fOLgAKtX79+kCTJk3cPLR/KUBG7O8DCmIUlCgYUemlcuXKBbp165ZgAIN9wP8swj6g25QpU9L9e0D7XK1atdz3zaWXXhq2jGiK0z/RHhUGAAAAUoscWQAAAPgSgSwAAAB8iUAWAAAAvkQgCwAAAF8ikAUAAIAvEcgCAADAlwhkAQAA4EsEsgAAAPAlAlkAiEE///yzxcXF2bp1687rcsuXL29jxoyxWDB16lQrWLBgtLsBIIYRyAJABlEgmtTtqaeeYt3HYAANwD+yRbsDAJBZ7dixI/j/mTNn2uDBg23Tpk3Bafny5YtSzwAgc2BEFgAySIkSJYK3AgUKuFFY736xYsVs9OjRVqZMGcuZM6fVqlXL5s+fn+i8zpw5Y/fdd59VrlzZtm/f7qa99957VqdOHcuVK5ddeumlNnToUDt9+nTwOVre5MmT7fbbb7c8efJYpUqV7P3330/Vazhw4IB17drVihYtavnz57cbbrjB1q9fH3xco8rq++uvv+5GVfU627VrZ4cPHw620f87dOhgefPmtZIlS9pLL71k1113nfXq1cs9rv9v27bNHn300eBodaiPP/7YrrzyShf4t2zZMuwHAoALG4EsAETB2LFj7cUXX7QXXnjBvv76a2vRooXddttttnnz5gRtT5w4YXfffbfLl/3f//5nl1xyifvbsWNH69mzp3377bf26quvupzS4cOHhz1Xwe1f//pXt4ybb77ZBZT79u1LcT+13N27d9tHH31kq1evdoFzs2bNwubx448/2pw5c2zu3Lnu9tlnn9lzzz0XfLx379725ZdfuiB6wYIFru9r1qwJPj579mwX0A8bNswFqaGB6p9//unWkQLlzz//3AXxffr0SdW6BpCJBQAAGW7KlCmBAgUKBO+XKlUqMHz48LA29evXDzz44IPu/1u3bg3oI/p///tfoFmzZoEmTZoEDhw4EGyrac8++2zY819//fVAyZIlg/f1/CeffDJ4/8iRI27aRx99lGg/y5UrF3jppZfc/7Xs/PnzB44fPx7W5rLLLgu8+uqr7v9DhgwJ5MmTJ3Do0KHg43379g00bNjQ/V/Ts2fPHpg1a1bwcb0OPadnz54Rlxu6ztTfLVu2BKdNmDAhULx48UT7D+DCQo4sAJxnhw4dst9//92uvvrqsOm6H3rYXtq3b+9GKxctWmS5c+cOTlc7jXKGjsAq/eD48eNuFFOpBFKjRo3g4zq0r/QAjbCmhJZx5MgRu/jii8OmHzt2zI3CepRScNFFFwXvK33AW8ZPP/1kp06dsgYNGgQfV/rBFVdckaI+6HVcdtllEecNAASyABDDlA7wxhtv2LJly1x+qkcBptIG7rjjjgTPUc6sJ3v27GGPKf/07NmzKVq2lqHAccmSJQkeCy2LlZZlJCfSvP9vsBkAqFoAAOedRkVLlSrlRlSbNm0anK77oSOX0qNHD6tWrZrLn/3www+D7ZWrqgoIFStWzLB+ahk7d+60bNmyuVHXc6GT0BSMrly50uX2ysGDB+2HH36wa6+9NtguR44cbkQZAFKDEVkAiIK+ffvakCFD3GFznfU/ZcoUdzLXm2++maDtI4884oK8W265xZ101aRJE1fKS/cVHN51112WJUsWlwqwceNGe+aZZ9Klj82bN7dGjRpZmzZtbOTIkXb55Ze7lAgF1KqEUK9evWTnoZSDTp06uddbuHBhV61Br1v9Da1OoEBZJ3Op4oGqOBQpUiRdXgOAzI1AFgCi4B//+IcbmXzsscdczmeVKlXcWf0qkRWJSlXpcL1SDVSmS1UOVCFAZ/o///zzbtRTpblUKiu9KNCcN2+ePfHEE9a5c2fbs2ePKx2mkdTixYuneD4qM/bAAw+4wFuj0f369bNffvklLAVCr+P+++93gb2qNJA+ACAl4nTGV4paAgCQDo4ePWqlS5d25ce6dOnCOgVwzhiRBQBkqLVr19r333/v8n81Cq3RV2ndujVrHkCaEMgCADKcLmqgk9N0UlfdunXdRRHIgwWQVqQWAAAAwJe4RC0AAAB8iUAWAAAAvkQgCwAAAF8ikAUAAIAvEcgCAADAlwhkAQAA4EsEsgAAAPAlAlkAAAD4EoEsAAAAzI/+PwmZxYQqJa3RAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "train_lengths = [len(ex[\"token_ids\"]) for ex in train_examples]\n", "val_lengths = [len(ex[\"token_ids\"]) for ex in val_examples]\n", "\n", "# Normalize counts because the validation split is much smaller\n", "bins = range(0, max(train_lengths + val_lengths) + 64, 64)\n", "\n", "fig, ax = plt.subplots(figsize=(7, 4))\n", "ax.hist(train_lengths, bins=bins, density=True, alpha=0.6, label=\"Train\")\n", "ax.hist(val_lengths, bins=bins, density=True, alpha=0.6, label=\"Validation\")\n", "ax.set_xlabel(\"Token length\")\n", "ax.set_ylabel(\"Density\")\n", "ax.legend()\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "f0759c1a-6b46-40e8-a702-0893e9581fef", "metadata": {}, "source": [ " \n", "## 8.5 Loading a pre-trained model" ] }, { "cell_type": "markdown", "id": "ab49df92-847b-49fc-b1d3-dc98f9833dba", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 22, "id": "7a52ec3c-df1d-4bff-a45c-1a9fff83a5ee", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using Apple Silicon GPU (MPS)\n", "✓ qwen3/qwen3-0.6B-base.pth already up-to-date\n" ] } ], "source": [ "import torch\n", "\n", "from reasoning_from_scratch.ch02 import get_device\n", "from reasoning_from_scratch.ch03 import (\n", " load_model_and_tokenizer\n", ")\n", "\n", "device = get_device()\n", "\n", "model, _ = load_model_and_tokenizer(\n", " which_model=\"base\",\n", " device=device,\n", " use_compile=False\n", ")" ] }, { "cell_type": "markdown", "id": "af907c96-f5eb-4f6e-bfb4-ed01e66da175", "metadata": {}, "source": [ " \n", "## 8.6 Computing the training and validation losses" ] }, { "cell_type": "markdown", "id": "39da98d7-2d20-4d55-a40d-8259f25c2c80", "metadata": {}, "source": [ "- In this section, we implement code to compute the so-called cross-entropy loss, which we will use as a training signal during distillation\n", "- We will also apply it to the validation set for evaluation purposes during training\n", "- Readers with a background in deep learning may already be familiar with cross-entropy in the context of classification problems; this here is the same cross-entropy function, where the prediction target is the next token in the sequence\n", "- We can derive the cross-entropy loss from the logprob computation we calculated in chapters 5 and 6" ] }, { "cell_type": "markdown", "id": "cbb0a4f9-e7c5-4c4c-870e-4df21bb3cd12", "metadata": {}, "source": [ "- As explained in chapter 5, where the logprob computation was first introduced, the logprob calculates a score for the next-generated token\n", "- Higher logprob means the model assigned a higher probability to the correct token (better prediction); lower logprob means a lower probability (worse prediction)\n", "- The figure below reuses the example from chapter 5" ] }, { "cell_type": "markdown", "id": "d3d55ce0-002d-427a-9ba0-e5e595c43793", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "6cc37f1d-0bda-4fd6-981b-b2db0e2e787d", "metadata": {}, "source": [ "- Similar to what's shown in the figure above, the `sequence_logprob` function from chapter 6 computes the log-probabilities (logprobs) for each token that are then returned as a sum (i.e., the joint log-probability)" ] }, { "cell_type": "markdown", "id": "093cf6b6-69d1-4014-bc85-de1d3562174a", "metadata": {}, "source": [ "- Let's look into this using an example from the training set; (we choose the shortest one to reduce the computation time):" ] }, { "cell_type": "code", "execution_count": 23, "id": "11417094-1c44-4c07-8dc2-3da9dc8784cd", "metadata": {}, "outputs": [], "source": [ "# 5730 is the shortest example in train_examples\n", "token_ids = train_examples[5730][\"token_ids\"]\n", "prompt_len = train_examples[5730][\"prompt_len\"]" ] }, { "cell_type": "markdown", "id": "52c40026-839a-41f7-baaf-20a6eae4526e", "metadata": {}, "source": [ "- Note that instead of reporting the summed logprobs `sequence_logprob` returns, we average the logprobs over the number of answer tokens, which we can compute by subtracting the `prompt_len` from the total number of tokens, which includes prompt and answer tokens:" ] }, { "cell_type": "code", "execution_count": 24, "id": "e144c574-f570-4373-9552-cf9874141eb5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average logprob: 1.68\n" ] } ], "source": [ "from reasoning_from_scratch.ch06 import sequence_logprob\n", "\n", "tok = torch.tensor(token_ids, dtype=torch.long, device=device)\n", "\n", "with torch.no_grad():\n", " seq_logprob = sequence_logprob(model, tok, prompt_len)\n", " num_answer_tokens = tok.numel() - prompt_len\n", " avg_logprob = -seq_logprob / num_answer_tokens\n", "print(f\"Average logprob: {avg_logprob:.2f}\")" ] }, { "cell_type": "markdown", "id": "d0ac2a6a-b333-4d0b-b3d5-0f4b9c0654df", "metadata": {}, "source": [ "- We can compute the same using the `cross_entropy` function\n", "- Note that the function is developed for classification, and next to the model's logits, it expects the class labels as an argument; the class labels here are the \"correct\" tokens\n", "- Specifically, we compute the cross-entropy over the answer tokens only (ignoring the prompt tokens)" ] }, { "cell_type": "markdown", "id": "ceee0afe-3f8b-42d1-89ef-83953f129b6a", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 25, "id": "6beac5b8-3362-41e1-ae11-30ea85ba7b13", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cross-entropy: 1.68\n" ] } ], "source": [ "input_ids = tok[:-1].unsqueeze(0)\n", "target_ids = tok[1:] # Targets are inputs shifted by one position\n", "logits = model(input_ids).squeeze(0)\n", "\n", "# Remove prompt tokens\n", "first_answer_logit_idx = max(prompt_len - 1, 0)\n", "answer_logits = logits[first_answer_logit_idx:]\n", "answer_targets = target_ids[first_answer_logit_idx:]\n", "\n", "with torch.no_grad():\n", " ce_mean_direct = torch.nn.functional.cross_entropy(\n", " answer_logits, answer_targets\n", " )\n", "print(f\"Cross-entropy: {ce_mean_direct:.2f}\")" ] }, { "cell_type": "markdown", "id": "1564c69e-d686-4170-8df3-d525d2358f36", "metadata": {}, "source": [ "- As we can see, the result is the same as before when we computed the average logprobs over the answer tokens\n", "- In other words, the `cross_entropy` function implements the average logprob computation internally\n", "- We use the `cross_entropy` function instead of our own `sequence_logprob` function, as it is better optimized for faster training\n", "- Let's put it into a convenient function we can call:" ] }, { "cell_type": "code", "execution_count": 26, "id": "a4e6bed0-d2be-42f5-9013-6f2f3eebf5ab", "metadata": {}, "outputs": [], "source": [ "def compute_example_loss(model, example, device):\n", " token_ids = example[\"token_ids\"]\n", " prompt_len = example[\"prompt_len\"]\n", "\n", " input_ids = torch.tensor(\n", " token_ids[:-1], dtype=torch.long, device=device\n", " ).unsqueeze(0)\n", " target_ids = torch.tensor(\n", " token_ids[1:], dtype=torch.long, device=device\n", " )\n", "\n", " logits = model(input_ids).squeeze(0)\n", "\n", " answer_start = max(prompt_len - 1, 0)\n", " answer_logits = logits[answer_start:]\n", " answer_targets = target_ids[answer_start:]\n", "\n", " loss = torch.nn.functional.cross_entropy(\n", " answer_logits, answer_targets\n", " )\n", " return loss" ] }, { "cell_type": "code", "execution_count": 27, "id": "ec375b36-7182-4d0f-9377-312d2ad98260", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loss: 1.68\n" ] } ], "source": [ "with torch.no_grad():\n", " loss = compute_example_loss(\n", " model, train_examples[5730], device\n", " )\n", "\n", "print(f\"Loss: {loss:.2f}\")" ] }, { "cell_type": "markdown", "id": "300ecf57-fc03-49bd-85f6-26242976a129", "metadata": {}, "source": [ "- Note that it is also possible to compute multiple samples in parallel, called batching, but we omit that to keep the resource requirements lower\n", "- Appendix E discusses batching" ] }, { "cell_type": "markdown", "id": "f51fccc8-7f6f-4415-a7ff-53e50f3ab986", "metadata": {}, "source": [ "- We can also define a wrapper function that computes the loss over multiple examples:" ] }, { "cell_type": "code", "execution_count": 28, "id": "b2e29de8-8314-45e0-80aa-9a6c8a0ccce4", "metadata": {}, "outputs": [], "source": [ "@torch.no_grad()\n", "def evaluate_examples(model, examples, device):\n", " was_training = model.training\n", " model.eval()\n", " total_loss = 0.0\n", " num_examples = 0\n", "\n", " for example in examples:\n", " loss = compute_example_loss(model, example, device)\n", " total_loss += loss.item()\n", " num_examples += 1\n", "\n", " if was_training:\n", " model.train()\n", "\n", " return total_loss / num_examples" ] }, { "cell_type": "code", "execution_count": 29, "id": "0589589e-fbf5-4baa-bf36-37a1863f3baf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train loss (3 examples): 0.98\n" ] } ], "source": [ "train_loss = evaluate_examples(model, train_examples[:3], device)\n", "print(f\"Train loss (3 examples): {train_loss:.2f}\")" ] }, { "cell_type": "code", "execution_count": 30, "id": "2bbea2bc-8524-4a2f-92e2-20bfdad1ae31", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Validation loss (3 examples): 1.02\n" ] } ], "source": [ "val_loss = evaluate_examples(model, val_examples[:3], device)\n", "print(f\"Validation loss (3 examples): {val_loss:.2f}\")" ] }, { "cell_type": "markdown", "id": "b185db1e-e023-406f-bd95-bf3e2fe82b9a", "metadata": {}, "source": [ "- We will reuse this function to track the progress during training, where we want to see that the loss decreases" ] }, { "cell_type": "markdown", "id": "c10e094b-8be5-4f9e-b00d-f5413d1333c5", "metadata": {}, "source": [ " \n", "## 8.7 Implementing the training loop for distillation" ] }, { "cell_type": "markdown", "id": "582a9074-37bf-43b6-8926-2e923d9a3e80", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "09e33b80-c634-418a-aea2-fed3dacf7fe5", "metadata": {}, "source": [ "- The training loop is almost identical to chapter 6, except that we now loop over the training set multiple times (epochs) and calculate the cross-entropy loss instead of GRPO loss" ] }, { "cell_type": "markdown", "id": "d200a7a8-4223-4890-89b9-5024aad50937", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 31, "id": "c163a7c0-567a-4a92-98eb-0726d91cd1ac", "metadata": {}, "outputs": [], "source": [ "import time\n", "\n", "\n", "def train_distillation(\n", " model,\n", " train_examples,\n", " val_examples,\n", " device,\n", " epochs=2,\n", " lr=5e-6,\n", " grad_clip_norm=None,\n", " seed=123,\n", " log_every=50,\n", " checkpoint_dir=\"checkpoints\",\n", " csv_log_path=None,\n", "):\n", " # Step 1: initialize optimizer (model is already loaded)\n", " optimizer = torch.optim.AdamW(model.parameters(), lr=lr)\n", " model.train()\n", "\n", " total_steps = epochs * len(train_examples)\n", " global_step = 0\n", " rng = random.Random(seed)\n", "\n", " if csv_log_path is None:\n", " timestamp = time.strftime(\"%Y%m%d_%H%M%S\")\n", " csv_log_path = f\"train_distill_metrics_{timestamp}.csv\"\n", " csv_log_path = Path(csv_log_path)\n", "\n", " # Step 2: iterate over training epochs\n", " for epoch in range(1, epochs + 1):\n", " # Step 3: shuffle the training examples at the start of the epoch\n", " epoch_examples = list(train_examples)\n", " rng.shuffle(epoch_examples)\n", "\n", " # Step 4: iterate over training examples in epoch\n", " for example in epoch_examples:\n", " global_step += 1\n", "\n", " # Stage 5: reset loss gradient\n", " # (it's best practice to do this at the beginning of each step)\n", " optimizer.zero_grad()\n", "\n", " # Step 6: compute the cross-entropy loss for the current example\n", " loss = compute_example_loss(model, example, device)\n", "\n", " # Step 7: backpropagate gradients\n", " loss.backward()\n", "\n", " # Optionally clip large gradients to improve training stability\n", " if grad_clip_norm is not None:\n", " torch.nn.utils.clip_grad_norm_(\n", " model.parameters(), grad_clip_norm\n", " )\n", "\n", " # Step 8: update the model weights\n", " optimizer.step()\n", "\n", " # Step 9: periodically evaluate the current model on the validation set\n", " if log_every and global_step % log_every == 0:\n", " val_loss = evaluate_examples(\n", " model=model,\n", " examples=val_examples,\n", " device=device,\n", " )\n", " model.train()\n", " print(\n", " f\"[Epoch {epoch}/{epochs} \"\n", " f\"Step {global_step}/{total_steps}] \"\n", " f\"train_loss={loss.item():.4f} \"\n", " f\"val_loss={val_loss:.4f}\"\n", " )\n", " append_csv_metrics(\n", " csv_log_path=csv_log_path,\n", " epoch_idx=epoch,\n", " total_steps=global_step,\n", " train_loss=loss.item(),\n", " val_loss=val_loss,\n", " )\n", "\n", " # Step 10: save a checkpoint for this epoch\n", " ckpt_path = save_checkpoint(\n", " model=model,\n", " checkpoint_dir=checkpoint_dir,\n", " step=global_step,\n", " suffix=f\"epoch{epoch}\",\n", " )\n", " print(f\"Saved checkpoint to {ckpt_path}\")\n", " return model\n", "\n", "\n", "def save_checkpoint(model, checkpoint_dir, step, suffix=\"\"):\n", " checkpoint_dir = Path(checkpoint_dir)\n", " checkpoint_dir.mkdir(parents=True, exist_ok=True)\n", " suffix = f\"-{suffix}\" if suffix else \"\"\n", " ckpt_path = (\n", " checkpoint_dir /\n", " f\"qwen3-0.6B-distill-step{step:05d}{suffix}.pth\"\n", " )\n", " torch.save(model.state_dict(), ckpt_path)\n", " return ckpt_path\n", "\n", "\n", "def append_csv_metrics(\n", " csv_log_path,\n", " epoch_idx,\n", " total_steps,\n", " train_loss,\n", " val_loss,\n", "):\n", " if not csv_log_path.exists():\n", " csv_log_path.write_text(\n", " \"epoch,total_steps,train_loss,val_loss\\n\",\n", " encoding=\"utf-8\",\n", " )\n", " with csv_log_path.open(\"a\", encoding=\"utf-8\") as f:\n", " f.write(\n", " f\"{epoch_idx},{total_steps},{train_loss:.6f},\"\n", " f\"{val_loss:.6f}\\n\"\n", " )" ] }, { "cell_type": "markdown", "id": "295f2905-2379-41f1-9cc5-da69a23c3ced", "metadata": {}, "source": [ "- With the sequence length of 2048, the training run requires about 15 GB of RAM\n", "- To lower the resource requirements, you can filter out longer sequences, for example, in the `previous filtered_examples = filter_examples_by_max_len(examples, max_len=2048)` step, changing `max_len` from `2048` to `1024` or `512`" ] }, { "cell_type": "code", "execution_count": 32, "id": "994c65e5-2e70-48db-93f8-a106c8feb463", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Epoch 1/2 Step 5/20] train_loss=0.9648 val_loss=0.9082\n", "[Epoch 1/2 Step 10/20] train_loss=0.9844 val_loss=0.8871\n", "Saved checkpoint to checkpoints/qwen3-0.6B-distill-step00010-epoch1.pth\n", "[Epoch 2/2 Step 15/20] train_loss=0.8008 val_loss=0.8707\n", "[Epoch 2/2 Step 20/20] train_loss=0.7148 val_loss=0.8586\n", "Saved checkpoint to checkpoints/qwen3-0.6B-distill-step00020-epoch2.pth\n" ] }, { "data": { "text/plain": [ "Qwen3Model(\n", " (tok_emb): Embedding(151936, 1024)\n", " (trf_blocks): ModuleList(\n", " (0-27): 28 x TransformerBlock(\n", " (att): GroupedQueryAttention(\n", " (W_query): Linear(in_features=1024, out_features=2048, bias=False)\n", " (W_key): Linear(in_features=1024, out_features=1024, bias=False)\n", " (W_value): Linear(in_features=1024, out_features=1024, bias=False)\n", " (out_proj): Linear(in_features=2048, out_features=1024, bias=False)\n", " (q_norm): RMSNorm()\n", " (k_norm): RMSNorm()\n", " )\n", " (ff): FeedForward(\n", " (fc1): Linear(in_features=1024, out_features=3072, bias=False)\n", " (fc2): Linear(in_features=1024, out_features=3072, bias=False)\n", " (fc3): Linear(in_features=3072, out_features=1024, bias=False)\n", " )\n", " (norm1): RMSNorm()\n", " (norm2): RMSNorm()\n", " )\n", " )\n", " (final_norm): RMSNorm()\n", " (out_head): Linear(in_features=1024, out_features=151936, bias=False)\n", ")" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(0)\n", "\n", "train_distillation(\n", " model,\n", " train_examples=train_examples[:10],\n", " val_examples=val_examples[:10],\n", " device=device,\n", " epochs=2,\n", " lr=5e-6,\n", " grad_clip_norm=1.0, # Same as in chapter 6\n", " seed=123,\n", " log_every=5,\n", " csv_log_path=\"train_distill_metrics.csv\"\n", ")" ] }, { "cell_type": "markdown", "id": "c84f95ce-a2af-4518-816d-c19067ff81b3", "metadata": {}, "source": [ "- The run above is short for demonstration purposes\n", "- We train only on 10 examples (`train_examples[:10]`) and evaluate on only 10 validation examples (`val_examples`)\n", "- For the real training, we change it to `train_examples` and `val_examples`" ] }, { "cell_type": "markdown", "id": "5ff95716-1b26-4eea-81d8-15e6eb02bd4f", "metadata": {}, "source": [ " \n", "## 8.8 Evaluating the distilled model" ] }, { "cell_type": "markdown", "id": "ef25bd43-117a-4d34-8916-6f1a343c165d", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "801e4085-c9c3-4f46-9523-a95a387eed9a", "metadata": {}, "source": [ "- Instead of running the distillation training in this notebook, we can download the following convenience script" ] }, { "cell_type": "code", "execution_count": 33, "id": "2874147a-ef57-4f2e-9569-544a44ff201a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "distill.py: 12.9 KB (cached)\n" ] }, { "data": { "text/plain": [ "PosixPath('distill.py')" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# from reasoning_from_scratch.ch07 import download_from_github\n", "\n", "download_from_github(\n", " \"ch08/04_train_with_distillation/distill.py\"\n", ")" ] }, { "cell_type": "markdown", "id": "78c70bea-a400-4586-ae83-0c06d2b34492", "metadata": {}, "source": [ "```bash\n", "uv run distill.py \\\n", "--data_path deepseek-r1-math-train.json \\\n", "--validation_size 25 \\\n", "--epochs 3 \\\n", "--lr 1e-5 \\\n", "--max_seq_len 2048 \\\n", "--use_think_tokens \\\n", "--grad_clip 1.0\n", "```" ] }, { "cell_type": "markdown", "id": "45441981-f76b-4d96-86c7-f4e5d93ce120", "metadata": {}, "source": [ "- Using the settings above, the training takes about 3:05 hours on a DGX Spark using 15.02 GB of GPU RAM\n", "- If you don't want to run this yourself, below we will download the resulting metrics file for analysis" ] }, { "cell_type": "code", "execution_count": 34, "id": "7086cd62-eac9-499d-87ff-e75dabfb4de5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "deepseek-r1-2048_distill_metrics.csv: 10.3 KB (cached)\n" ] }, { "data": { "text/plain": [ "PosixPath('deepseek-r1-2048_distill_metrics.csv')" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "download_from_github(\n", " \"ch08/03_logs/deepseek-r1-2048_distill_metrics.csv\"\n", ")" ] }, { "cell_type": "code", "execution_count": 35, "id": "009896bf-2218-442e-9e5d-f3c03b9a9eef", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArIAAAGGCAYAAACHemKmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAA0FtJREFUeJztnQe4JUWV+M+9L+f3Zt7kPMwwzJCzBCWL6KKgrmH5BHXFRcXE6t9lVUAMuLqi7oq4ooC7qyvKCuJKEFBUECTnYQiT45uXc7z9/071re5T1VXdfdO79753fvPdeTd0V1d3V1edOnVCwnEcBxiGYRiGYRimzEgWuwIMwzAMwzAMkw0syDIMwzAMwzBlCQuyDMMwDMMwTFnCgizDMAzDMAxTlrAgyzAMwzAMw5QlLMgyDMMwDMMwZQkLsgzDMAzDMExZwoIswzAMwzAMU5ZUwiwjlUrB7t27oampCRKJRLGrwzAMwzAMwxAwV9fAwAAsXrwYkslwneusE2RRiF22bFmxq8EwDMMwDMOEsGPHDli6dGnYJrNPkEVNrLw4zc3Nxa4OwzAMwzAMQ+jv7xdKRymzhTHrBFlpToBCLAuyDMMwDMMwpUkcE1B29mIYhmEYhmHKEhZkGYZhGIZhmLKEBVmGYRiGYRimLJl1NrIMwzAMw5Q3U1NTMDExUexqMFlSVVUFFRUVkA9YkGUYhmEYpmzii+7duxd6e3uLXRUmR1pbW2HhwoU5x/RnQZZhGIZhmLJACrHz58+H+vp6TmxUppOR4eFh6OjoEJ8XLVqUU3ksyDIMwzAMUxbmBFKInTt3brGrw+RAXV2d+IvCLN7PXMwM2NmLYRiGYZiSR9rEoiaWKX/q0/cxV1tnFmQZhmHSDI9Pwh9f3g/buoaKXRWGYSywOcHMIJGn+8iCLMMwTJq+kQmYmExB5+B4savCMAzDxIAFWYZhGIZhmDJh5cqV8J3vfCcvZT3wwANCM1rOUSDY2YthGCaN43jvilsRhmFmFKeeeiocccQReRFAH3vsMWhoaMhLvWYCLMgyDMOkYfGVYZhihaTCqAyVldFi2bx586alTuUCmxYwDMOQwcT9W+yaMAwzU3j/+98Pf/zjH+G73/2uWMbH18033yz+3nXXXXD00UdDTU0NPPjgg/Daa6/B2972NliwYAE0NjbCscceC/fdd1+oaUEikYAf/ehHcP7554tIAGvXroU77rgj6/r+7//+Lxx88MGiTnisb33rW8rv3//+98UxamtrRT3f+c53er/deuutcOihh4rwWhgi7cwzz4ShocI6z7IgyzAMwzBMWTKVcoryygQUYE844QS4+OKLYc+ePeK1bNky8ds//dM/wde//nXYuHEjHHbYYTA4OAhvfvOb4f7774ennnoK3vSmN8G5554L27dvDz3Gl770JXjXu94Fzz77rNj/ggsugO7u7oyv5xNPPCHKec973gPPPfccXHXVVfDFL35RCN7I448/Dp/4xCfg6quvhk2bNsHdd98Nb3jDG8RveF7vfe974YMf/KA4H7S/ffvb3+4pCAoFmxYwDMOkkf0tK2QZpvRBgfIPL7nZoaab0w6aDxXJeOGjWlpaoLq6WmhLMSUr8tJLL4m/KBCeddZZ3rZz5syBww8/3Pv85S9/GW677TahYb300ktDtb7vfe97xfuvfe1r8G//9m/w6KOPCkE4E6699lo444wzhPCKHHjggfDiiy/CN7/5TXEMFKjRPvdv/uZvoKmpCVasWAFHHnmkJ8hOTk4K4RW/R1A7W2hYI8swDMMwDFMEjjnmGOUzamQ/85nPwPr166G1tVWYF6B2M0oje9hhh3nvUdBsbm72UsBmAh7rpJNOUr7Dz6+88oqw4UWhG4XU1atXw/ve9z746U9/KtLNIiiAoxCMwuvf/u3fwg033AA9PT1QaFgjyzAMo2tkWSXLMCUPakRRM1qsY+cDPfoACrH33nsv/Ou//iusWbNG2JqiDer4eHhs66qqKuUz2s2mUinIN6iFffLJJ4XZwO9+9zu44oorhPkBRlJAwRvr/pe//EX89u///u/w+c9/Hv7617/CqlWroFCwRpZhGEaj0DZdDMPkT6AsxitT0LQANZpRPPTQQ2IJHx23ULOJpghbt26F6WL9+vWiDnqd0MSgoqJCfMbICujE9Y1vfEPY5GL9fv/733sCNGpw0WYXbXzxvNE0opCwRpZhGCaNw9axDMMUAPT+R80kCn1oLmDTlmI0gF/96lfCwQuFQrRVLYRm1cY//uM/ikgJaJv77ne/Gx5++GH43ve+JyIVIP/3f/8HmzdvFg5ebW1tcOedd4r6rVu3TpwfOqm98Y1vhPnz54vP+/fvF8JxIWGNLMMwTBp29mIYphCgyQBqNDds2CDiwNpsXtHZCgXEE088UQizZ599Nhx11FHTVs+jjjoKfvGLX8DPf/5zOOSQQ4TpADqkoZYYQfMBFLRPP/10IaD+4Ac/gP/5n/8R4brQLvdPf/qTiJqAGtwvfOELInTXOeecU9A6J5xZtobW398vPAj7+vrERWcYhpFs6RyC1zoGobG2El63em6xq8MwDGF0dBS2bNki7C0xhilT3oTdz0xkNdbIMgzDaMyu6T3DMEz5woIswzBMmlm2QMUwzAznkksuETa5phf+NhNgZy+GYZg0Uoxlpy+GYWYCV199tbDPNTFTzCuLqpFFo2A0Zl68eLHwzrv99ttDt0cDYwzGi4bSeAMw5ds999wzbfVlGGZmwwpZhmFmEvPnzxfxaE0v/G0mUFRBdmhoSGSCuO6662ILvijIYrgHzAd82mmnCUEYY5UxDMPkDRZoGYZhyoKimhZgSIZMwjJ85zvfUT5jPuFf//rX8Jvf/MbL9cswDJM9LMEyDMOUE2Xt7IVBeAcGBmDOnDnFrgrDMDMAjiPLMAxTXpS1sxfmIh4cHIR3vetd1m3GxsbEi8YmYxiGCXX2YkmWYRimLChbjezPfvYzkcsXM1CEGSxfc801IqiufC1btmxa68kwDMMwDMMUhrIUZDF12oc+9CEhxJ555pmh215++eUiM4R87dixY9rqyTBMuZoWsEqWYZjSYeXKlQE/IRuJGFGgZhJlZ1qAOX0/+MEPCmH2LW95S+T2NTU14sUwDBMFC7AMwzDlRVEFWbRvffXVV73PmHP36aefFs5by5cvF9rUXbt2wX/+53965gQXXXQRfPe734Xjjz8e9u7dK76vq6sTZgMMwzD5gG1kGYZhyoOimhY8/vjjImyWDJ112WWXifdXXHGF+Lxnzx7Yvn27t/0Pf/hDmJychI997GOwaNEi7/XJT36yaOfAMMzMgQVYhmHyDcoumPgJIy1R3va2t4kV5tdee028X7BggUgde+yxx8J9992Xt+M/99xzcPrppwul39y5c+HDH/6wUCRKHnjgATjuuOOgoaEBWltb4aSTToJt27aJ35555hkRs7+pqUkkojr66KOF7FZKFFUje+qpp4bmNr/55puVz3ixGYZhCgWH32KYMntgJ4aLc+yqejRGjbXp3/7t38LHP/5x+MMf/gBnnHGG+K67uxvuvvtukeAJhco3v/nN8NWvflWYQuIqNCZ72rRpk1idzjXx1Nlnny0yoT722GPQ0dEhfIwuvfRSIWOhcvC8886Diy++WJhujo+Pw6OPPirsbJELLrhAKBivv/56qKioEKvmVVVVUEqUnY0swzBMoWAbWYYpI1CI/dri4hz7n3cDVDfE2rStrU0kf0LzSCnI3nrrrdDe3i60nclkUmQ5lXz5y1+G2267De644w4hcObCz372MxgdHRXCMWpcke9973tCUP6Xf/kXIZSiI/zf/M3fwAEHHCB+X79+vbc/rop/9rOfhYMOOkh8Xrt2LZQaZRm1gGEYppCErRQxDMNkCmo2//d//9eLa//Tn/4U3vOe9wghFjWyn/nMZ4QAiUv7aF6wceNGxbQyWzZu3CiEZCnEImg6gGYOqPFFn6T3v//9QmuLwi36IKFZpwRNPlGDixGivv71rwsziFKDNbIMwzBpWH5lmDICl/dRM1qsY2cACok4Qf7tb38rbGD//Oc/w7e//W3xGwqx9957r0jytGbNGmHL+s53vlMs808HN910E3ziE58Qpg633HILfOELXxD1ed3rXgdXXXUV/N3f/Z2o91133QVXXnmliBp1/vnnQ6nAgizDMIwGy7MMUwagHWfM5f1iU1tbC29/+9uFJhajNa1btw6OOuoo8dtDDz0ktKJSOEQN7datW/Ny3PXr1wtbWLSVlVpZPB5qgrEOEul4j9Gi0J4WTRJQkEUOPPBA8fr0pz8N733ve4XgW0qCLJsWMAzD6LAkyzBMAcwLULN54403ivcStDv91a9+JRypMEoAakD1CAe5HLO2tlaELn3++eeFwxk6nr3vfe8TURIw7CkKrw8//LCIVPC73/0OXnnlFSEAj4yMCBtddLTH31AARocxakNbCrBGlmEYJg2bFjAMUygwBBbapKJtKgqrkmuvvVaE4TrxxBOFA9jnPvc56O/vz8sx6+vr4Z577hFhStGkAT+/4x3vEMeUv7/00kvwk5/8BLq6ukRIUwxx+g//8A8iogF+d+GFF8K+fftE3VCr/KUvfQlKiYQzy7wasHFg8gT00sOYaAzDMJJnd/ZCR/8YJJMApx+0oNjVYRiGgN73qEFctWqV0DIy5U3Y/cxEVmPTAoZhmDSza1rPMAxT/rAgyzAMo8ECLcMwpQg6i2F4LtPr4IMPhtkI28gyDMOkYfmVYZhS5q1vfSscf/zxxt+qSizj1nTBgizDMEwa6TLAGlmGYUqRpqYm8WJ82LSAYRgmDcuvDMMw5QULsgzDMAZmWUAXhikb8hVjlZkZ95FNCxiGYdKw7MowpUt1dbXISLV7926YN2+e+JzA7F5M2SkJMP3u/v37xf3E+5gLLMgyDMN4OIpQy2Mkw5QOKPRgzNE9e/YIYZYpbzAZw/Lly8V9zQUWZBmGYdKwRpZhShvU3qHwg1mnpqamil0dJksqKiqgsrIyLxp1FmQZhmEMsEzLMKUJCj8Yamq2hptiVNjZi2EYJg0LrwzDMOUFC7IMwzAG0wKOWsAwDFP6sCDLMAxjEF5ZjGUYhil9WJBlGIZhGIZhyhIWZBmGYdKwFpZhGKa8YEGWYRjGaCNbzJowDMMwcWBBlmEYJo1DEyKwfpZhGKbkYUGWYRiGYRiGKUtYkGUYhpGwaQHDMExZwYIsw+RAz9A49I9OFLsaTJ5g2ZVhGKa8YEGWYbJkbHIKntjWA49u7i52VRiGYRhmVsKCLMNkyfDYVLGrwOQZNidgGIYpL1iQZZgsmUilvPecznQGRi3gW8owDFPysCDLMFkyMcWSzoyOI8sWswzDMCUPC7IMkyWTU1QjW9SqMAzDMMyshAVZhsmDRpbl2JkB30eGYZjyggVZhsmSCaKRZWYG1NaZtewMwzClDwuyDJMlk1Qjy1LPjIPvKMMwTOnDgizD5CNqQVFrwuQLvo8MwzDlRVEF2T/96U9w7rnnwuLFiyGRSMDtt98euc8DDzwARx11FNTU1MCaNWvg5ptvnpa6MozOxCSbFszsFLUs1jIMw5Q6RRVkh4aG4PDDD4frrrsu1vZbtmyBt7zlLXDaaafB008/DZ/61KfgQx/6ENxzzz0FryvD6Eym2J5ypsEhtxiGYcqLymIe/JxzzhGvuPzgBz+AVatWwbe+9S3xef369fDggw/Ct7/9bTj77LMLWFOGCXf2YgFo5sF3lGEYpvQpKxvZhx9+GM4880zlOxRg8XsbY2Nj0N/fr7wYJldSKUdx9mJmBqxZZxiGKS/KSpDdu3cvLFiwQPkOP6NwOjIyYtznmmuugZaWFu+1bNmyaaotM1scvRAWgGZgZi++pwzDMCVPWQmy2XD55ZdDX1+f99qxY0exq8TMADg97SyAbzHDMBbQGfSZHb3wyr6BYldl1lNUG9lMWbhwIezbt0/5Dj83NzdDXV2dcR+MboAvhsknHLFg5sFRChiGicvoRAr2D4xBRTIBaxc0Fbs6s5qy0siecMIJcP/99yvf3XvvveJ7hilmVi+Wgcof/R6yAx/DzG7CsjfK/oH7iVkuyA4ODoowWviS4bXw/fbt2z2zgAsvvNDb/pJLLoHNmzfD//t//w9eeukl+P73vw+/+MUv4NOf/nTRzoGZnYzrgix3ZmUP30GGYSQ7uofhj5v2w65es/8NKy9Kh6IKso8//jgceeSR4oVcdtll4v0VV1whPu/Zs8cTahEMvfXb3/5WaGEx/iyG4frRj37EobeYaYdtZGc+PFAxzOxl017X9nXjbnOkI9k9cD8xy21kTz311FC7NFPWLtznqaeeKnDNGCaclNZuuTObXkbGp2B4fBLmNubP/p1tZBmGybS/4G6j+JSVjSzDlK49JTOdPL+7D57a3gtDY5PG36dSjlgaHJ2Yil2mfg/5njLMzGFP3wj0j04UuxpMAWBBlmGyYLZr77Z1DUH30HjRjj+ejhphc8boGBgVS4Ob9w/FLnOW31ImhIHRCdi8f1AkQmHKj8GxSXhhVz+8aDETyAbaEmb7eFBsWJBlmCwIaO9mUUeGg/or+wbhpb3Fy5InL7ftsktBd1JLXJHZMWbPPWXCeW3/kJgUdQ6NFbsqTA7hEvOZjZGTp5QOLMgyTBbMZtMCXLanf4uBH/rGjKxaJgMMR55gbEylJ0Q5zIvKkt29I/D8rr6y10RLn4a8PuNUkM1fqUwWsCDLMFkwm4WeKG3odGLTmvoDVyZlaZ9zqRgzo5BtQ3fynOls7RyCvX2jMDBqtkUvFwoRYWA2jwGlBguyDJMFAaFnFvVpcjAv5qAedWjfozj7Os6me8rEFIRgdiHPt9wF+GwmtpnAZkjFhQVZhsmCQL81i/qxUhjUo+rgmRZMV4UYZiavvkCZ460ixT+TZDIDG9ls68XkBRZkGSYPy0qzaZnJ68Cd0o3h6GlgMrGR5RS1TKQ5zexqE54tepmfdzY28wlIhP5e3ldkZsGCLMNkQZn36zlRSjnGbXXwnXLi17EUzocpTWZr8PuZopHNytkrXI5VhPvZ1i5KDRZkGSYPzKaOrBScvaIOnY1GNuODMLOG2doUZkoa1mzOI5mIr5HlSXBxYUGWYbKg3J0fyl2QjTKSzaZus/iWMuXQ5ouAp4kuc0FNhg/LzLQgPrOtXZQaLMgyTBbM5lBNVIgvlu1cdBzZLMJvRXxmmHIX6GYravICJz8aWW4KJQMLsgyTBQGhZxb1asqSWpFPO7/OXrPnHjLZOj3BrMI7XWfmTEDi3sMIOZYnNSUEC7IMkwWzWeih514sE4uow/peyhxHlskDM8TpabaeN01MFvdcqBxrzGymaHmzrxuTO5V5KINhZh2zeRm6FOIn+t7UToRtXwZl5qNizIzEdxaaXa2k3DXRG/f0i781lb7Ozr2H0RawCaKSnXIcSGr7sLNX6cAaWYbJgtmc2Uu1NytV04Lw3+OUxYMTozPbWkTUhLGUmUo5sKtnRLwmpohpQZZl6RS772N8WJBlmKxQe7Fy7OjzYm9WhPNW4jfmM27kLJ6cMOHM3qgF6t9ygpo9UUE0rjlUJna15Xh9ZhIsyDJFBTuYsckpKDdmasc1MZUSr9j2Zk5peiB7g9UMvU/M9OILNbOzQZX7WVNBNnafRbZD04Lgz7lpeZn8wYIsU1Se2t4DD73aGSk8lRqBjitmT4aCV9/IhNl5oMhg3R7Z3AV/3dwdagtY7Iw2cQ6ZjRwbTDvMMOWvmcyWcrcHpprXST/VX2zo2Zu0uNmE9GIKAwuyTFEZHJsU6URHJ8pLK6sLonG7sf0DY/DYlm54bf8glBqTqB2fSIl7ESZnqx7Axe3AcfzoH50Qr9zDb+W7dsxMYTbqY8tdUKNVVmNfx9uf7mNSPJTfFZm5sCDLFBXZGZSggjKUbKs7khbYRydSZTxwFVkjq9m+Pbq5W7xUO7j0thncqUAkijIcvJnCMpuahDODzntScfZyshCETb+zaUGpwIIsU1y8JTtnVkQtyEbAmi7i2nwVO/yWo2mR/ff+5GA2pxBm8s9MSdWaCeXWJ+vQ6mdjI+tERC2wHYuZfliQZYpKucYplPVOJrMcEJ3yDaulamqyOxEcGF7eNwC9w+MZ72s7pEmjnEtmr1K8R0yRTQtmUZsoZY1sx8AovNoRbp5FJx3UWSt21IKIVNyldk329I3A1s4hmI2wIMsUFalEKzsNmqMGzY69XKX9LVXCzkexN8uy/K6hMdjeNQyv7c+t47U5nsl2lZmzF8NYmIXOXpRS00S/um9QCG0Dmm18LI0s2QZt65/d2QvD45PB/SE8aoF14yLxwq5+IdyHXZOZCguyTNEoZxsjWd9EpstV6Q1LUXCPrZGl26WmfwJjM4GQZVHHjHJfHmVKi1IT6GZL4hMdKViGBSOwmQbQc8FkCR39Y7Cvfyy0ALONLN20dC7Q+GTp+V8UGhZkmZLzKi0HZHWTnkY2s/1K8XTjdsaqRja7E5H7ZROGzNZuPEE2ywnSbM7WxsxME6hcKOU4qXHugzVMoEFTa8zcBVFRC0rtqpSn43Q+YEF2FtE9NA4PbOqAvX2jUAooz1u52siSfNyZdTIzRCOb5Wn4Tm+5QTUyvpaXbJCJjWwJ3pOZAAoUpRg3ORPitvMXdveJuNiTZRYXu9zCb/kmWvZ6ORll7HJC+5aoFLWldHmcUqrMNMGC7CziyW09IgzJ87v6cionXzFfVW0alKlGVn52Skazk+39UeW/zGx+Mz5WnkwsFK2JUSObkSRrLbsYGdRmCk9s64GHXussa2E27kpK1+A4jIxPwXCZxcWOotTuXBxnTpvZk2I76/UZ5vJDEyJY3hcDp4zH0nzAgiyTEehp/uArnbCrdyTnskrVxigOXsfmCbIx98vCCSkT0AEC78/mLBIuxM3YpQYXz9K0wFepZL+vxcyg1BUSKND9cdN+8Spn4S4u6FCDiTbGZoDtXlQ/5TXrGXBbS1XjiMTpPmz3KsocSeyr7Ro14S62FtQpYzO9fMCC7Cwkw9VwBfQ0R17ZN5BzPVRtGpQlmZoWFFojK0PSbM4iGgCtUlhnGBUoPBOy2d+khaVlZZPFxy1X+1ygezRONLE0Du5MxZ+8lee5ZpKS2dcUlue5Ukr6fsWykY3eVU4kA/bx2j5RCRGKjUPesyDLzAqqK3O/7RVyTX2G2mBl6uyV6X6leL5xIxBkY4Kg42lPs9g/SiObtSA7TbeEHieXSWU5MBOWPLOZDJXg450jpXVCvkIgZMKdwUQ8qJFVP5udwaKPVYqTrZkIC7KzkKqKPAiyeRiBS3npKr6zV2b19wRZKD3iasiVTj/LE5FF5K7RNS0T6seKdxBdqC7UPcpW0C5HynmyaiJS21XCz/dM7J9DTQsslTat6ERrZJ2Svj4Oec8aWWbGQr1o86GRTeZDI2v0Hi0PPBPZjKMWlK4dpy1ETdh22Z6Gr7nKQiNL3lOh1YuEEGHvVmxocPWSXr6d5Q6d2Wje4mgKy4VS0jhm43xn/cmJYVqgfQ6LV+sWWdwr5Bj6wdkEC7IlyLauIc8G9aW9/bCzx7VLNTle7eg2/6ZDHS0yXQ6fDtOCcptF+oIsZJnZq/TON67JQD6WsXJJ1asKCURQSlk0srHLDTtO/lATNsCMRjm9Mj3XTNpBKa+4zKTlaidfNrI20wLtDprHp9K5KA5NxzsLJdmsBNkdO3bAzp07vc+PPvoofOpTn4If/vCH+azbrAQ7j1f2DcK2rmGRT3pn94jRcQdz1KPj1aa9AxkLsvkYoPMhDJfyjD8Mev28hAgxT8DrEAt8wtncnrgDV1yBN14c2XxqZKUgq2tkM5tkFJrZNNDkOzIJriw9ub0nL1FT4qK0dyd3TWE5UkoTbzUjZMiE2/IbvTdydUTfUr9/phS1cVewpgOnjJVCRRNk/+7v/g7+8Ic/iPd79+6Fs846Swizn//85+Hqq6/OqKzrrrsOVq5cCbW1tXD88ceLcsL4zne+A+vWrYO6ujpYtmwZfPrTn4bR0dII8J8PJqaCMytTw8w0lM3YpB/XMKyZP7ezT+SejtIk5UMja7JvLAdoVTO9DNOlsclKkLW8L8wyVrpt5xiRibZJu3YlOwp1j2zpMmci+TYt6BuZgO7B8dgrUflAvUcxVyqKLdlMswBf6iZQOYffKnFnr9k6Uc5JkH3++efhuOOOE+9/8YtfwCGHHAJ/+ctf4Kc//SncfPPNscu55ZZb4LLLLoMrr7wSnnzySTj88MPh7LPPho6ODuP2P/vZz+Cf/umfxPYbN26EH//4x6KMf/7nf4aZAg2SjskLbA9k3LA9slFjHMeoBxy1Hfv6R0XuaVNmGnrMPMixJWUsnwm0qpmnqLVPTvJJpra7mTjm5DWObBZl2Gx0ZVvP1dzBdJx8MptsZCn5WAmyTVZk34kxlNHkqlCErlSUqmQzg/vnuBNu2z42E6egaUFpXx+HaplZkI3HxMQE1NTUiPf33XcfvPWtbxXvDzroINizZ0/scq699lq4+OKL4QMf+ABs2LABfvCDH0B9fT3ceOONxu1RWD7ppJOERhi1uG984xvhve99b6QWt2wFWTkwGx7XKaK5tYFB8f/4cofQYqgaXPO+tP2btpgk6rO8PLg5CDPFhNbVs5HNcPm60KebjelH3Nzq+ejAc9L2WJbRbJOEuOVPVxMsVY1svjL2FdIJxWujhrIGRyfFOXQOjOV+oGyeC/J+RsgSynhQOicUW2lgNS1wSOpk9Tt/G4iR2auUrolTlqubRRVkDz74YCF0/vnPf4Z7770X3vSmN4nvd+/eDXPnzo1Vxvj4ODzxxBNw5pln+pVJJsXnhx9+2LjPiSeeKPaRguvmzZvhzjvvhDe/+c0wU1CCpaffm5ZfqVBpyw6EAixuNoCZdahpQYwlF9M21OwhP3JsfsubLmhdEzK1V9x97eNwXslKYx5TQM1HB56LWYkt2oWvrbNvXwrQ57lUatbRPyq0mZgZLp9knS7YgizO1OX5sYnzS9yJWzlNxss5qkxsW34nfhuKchCNysBX7D7GmeUa2cpsdvqXf/kXOP/88+Gb3/wmXHTRRcIkALnjjjs8k4MoOjs7YWpqChYsWKB8j59feukl4z6oicX9Tj75ZNGYJycn4ZJLLgk1LRgbGxMvSX9/P5QyVFhUBEfHUZaLFa2OpSxvYE9pzl5ZChfU3CAfsz6To07Z2cimp4JOxuG3Cnu+mQrYiLoyGirJ5q6RjdD+x8Vo75bKl0a2MPdIXdkojXY/ODYp/g6Nu3/zRU6a91AhxF5YIfuSuAH4y6k/s1GqpxC3n3IyaENBjayTkbNXsa+VM8Pa3rRoZE899VQhUOKLmgF8+MMfFpraQvHAAw/A1772Nfj+978vbGp/9atfwW9/+1v48pe/bN3nmmuugZaWFu+FDmKlzAQROKmwqk+yFDs7S8OlS63jStQC87GjJnLqkmg+tCvk/FKlaeaB3tHU3IN2FDivkAJjpsKSo2nDUGue32gK2exPPxR2AqJqUjPUyFo0KbnayIYdJ5+U4kDjx+At7RBOUnAxL/Xm7zjZTLryNTkrFUr1HPKlIQ8bXwMaWUNRpfQYO8qEHmYdWQmyIyMjQsvZ1tYmPm/btk1EE9i0aRPMnz8/Vhnt7e1QUVEB+/btU77HzwsXLjTu88UvfhHe9773wYc+9CE49NBDhVYYBVsUVlMWSejyyy+Hvr4+74Whw0oZKjRNhGhupCNYWMN1iNBLHbVsD3ikaUHIg58NsTWARWLjnn7YuLsfntvVZ/xdCLIZRy1QJdmhsUl4dmcfvLA795UCek+ycvaKbQsYb7tCZbdStXzBcrK2kZ2mNkjnRaUy6BQqUUe+V13CBO442tpciWtyUyqa9pkYRzZ2zG4nflsPaGRTmdnIFvvyODC7TQuyEmTf9ra3wX/+53+K9729vSJs1re+9S0477zz4Prrr49VRnV1NRx99NFw//33e9+hMIqfTzjhBOM+w8PDwo6WgsJwWMeBTmnNzc3Kq3xsZO2CoyKY2py3UtSoPfqhow+vqUxqWpCPjq3UU3Vi9AYEw/1QZF3V5fuYnav311EmLrrWNxtoB5ZdHFnz+1CnwLxo5jPd3txurKF0shz4CtUklYGmRNq9b1+ab3Wm8W32xYXU08uqlYfjmMq1Hdevm/n9TKCUFA25ar4do/22vo2jmo5FmhYU9/o4tB9kQTYeuKz/+te/Xry/9dZbhV0ramVRuP23f/u32OVg6K0bbrgBfvKTn4hwWh/5yEdgaGhIRDFALrzwQqFRlZx77rlCUP75z38OW7ZsEY5mqKXF76VAW+5Qu1hFcNQetThCpaeRTcVzromK+UjrlhetR5l2/N69EKYFmdVf1wbkczk813sSV0OuxsvMx2Dk5CejjSfIqtv0j0zC87v6YGQ86JWPJh342/D45LQN1ap9e2k0fN97O7/l5ltL6WldTfM+T1ub35OIa3Iz01BXPqAkCbVZto2L6R+oeZ5tFaciLclieyu2sBqKE27PO9PJytkLNaNNTU3i/e9+9zt4+9vfLjSlr3vd64RAG5d3v/vdsH//frjiiitEYoUjjjgC7r77bs8BbPv27YoG9gtf+IJYMsW/u3btgnnz5gkh9qtf/SrMFFTTArvGUnUEM5clH050LonTGUfFB1VtivIwKFmOXSpg0zMNmL5GNhvTAu1vHs8/1/uTVWYvJx/hYjLd1xIBwJskqAViiufe4QlorKmEle0Nym97+kZhb98o1FVXBEKWFapJluJAY9Nm516u/z7vZgspB5LEGNxb7SicUrngYelKiXKw+Q1fOTL/6LUTpe8wb1ORSMAE2cbW3xf7fjt0Qj8LNbJZCbJr1qyB22+/Xdio3nPPPSK7FoKJDDJdur/00kvFy+bcpVS2slIkQ8DXTIU6e4VpXRXPZ1s34wRNFMjXAaJWPKmQnReFbIl3lCjYpIxLmEE71Lj11ycL+QwZpCTJcArp1JK7iioX8wT7xC1YNu3YTQKkrxGfPt1oKcaRLVzoqvyeq76ylCTmPXSSqEd5mXYb2ZLs0TKjVG1+49u8h+8frpF1P1NLxilLeysFHKe0lUIlaVqAGtTPfOYzIikBhtuSNq2onT3yyCPzXceyp390wpgpK8pGNqxxxhEq5XhJhd6w7ZVBxxi7Nr8a2VJPUWtLwyuvk/trhlELlHKIRjGD0+8YGIU/vrwfOgfVwO9x7KBD6xZTuFTbZRYH0srPXCNr3sGW0jmeg5DdTi7fhNnmZV1mjlqYgkUtUI6Re+Fhk61CxT2Nm3o2H89FSVGiiob4Nu9WUTbSR4NqZCVhtvfFHr4c8h7HlNlmJ5uVRvad73yniOWKWbxkDFnkjDPOEFpaxufJ7T3CWejgJc2wqKUudFsRG9eSsUttqH5GkvAlFCdghkC/11EVeoV39tKFulLDlh3LyUfUgnQ5YeGEbPQMTQjNfc/QOLQ3uhn2ojQMedU8KQKv+htmVqqtqsjw3jsFXR4Pi93rOQiFCLlRYLkYpznOeRcijB3a+f51czcsn1sPBy5wTb4yhWqmS1loDxOM9Yk/1Z7l65hxTW5mAqV6PnFNOJwM0hzr4538CU1XpIlZQNgtoQvkGPq8fLX/GauRRTBEFmpfMZvXzp07xXeoncU0tYxPc22V+Ls/RtpEqo3VoQ+dvl3UAxswLbBqcMNnmPnWyKrLjiXUK0QJsoaoBdl4xrumBcHvo/A1j2DXyDqFiSIRSDZAzvu1/YMiO9SO7uHIY+VkT+jEsz+Os2zu2yo7WQ9MGDoNzzvOMx43mUkmbN7vZuPa3hV93RF0etPT0frRAPKMJlzmXFwMLZr+PvdjxitXXWUovf4sU0rV5le9z/E05KbvFWWQNvTKcl0/iETkPS22KYlTBnb4JSfIYpisq6++WiQYWLFihXi1traKxAS2eK6zlXlNrsasa2g8Ut2va04ptF0qtpAWMwD64NF4tGL7rDN7ObPKRpaaFpiE7mQWUQv0c6ZtIu5ykE3zmKtG1sliG3qYLWmBatPegehycqhrVPvVnR1MGhi/HunfxCMSb8Kng85iyJaY6V3zbSNrm3DpYPt6aW8/PPRqJzy+tcdYp3yPf/l29gqLS1swQVKbfMbYrKQEv/xQOicU3+7a/KNtFcxULj5a8vkKS5pQ7PvtBJQaMKvIyrTg85//PPz4xz+Gr3/963DSSSeJ7x588EG46qqrYHR0dEZFEciV5tpKqKlKwthECrqH1aXgMEcvHfqQ6fa2Uc5eUwGNrJNVDngqEOfb3q0UbXqoiSxOHqoqNIFBmBbEt5E1pU7NRpj37To1QTZnG9loQSCYyjGLA+WoPbMd0xPGLNfdGIHCiz2KGtncluLiriooWfnyICDENW/Z2TMCO7tHxHtdI+uvDDgl7jAUopGdBg1iuEY27pZmsJ3uGxiFOQ3VUFNZ/HCSpWQDSsm0n7R9b5rwyi7e3zfhjQOhgi8UF0erwWzTyGYlyGLc1x/96Efw1re+1fvusMMOgyVLlsBHP/pRFmQJKOig8LqrZ0QsPYYKsiHTKNouAzavljarP3ioYcSHN45GNjhIOIpAnA/P4FLKjBIFaqOlCWTKmBAhmoCAhdc0oOlNZJ2FSWk+TmGES9M5ZINuz5gPbAJ+WLB/mzmCbfvQ48fcTpnQTKNGNmCSRJ5fP5117vVRj0He57m80KXevCpkY55EjoI0CrEv7OqHxa11sGFx8RP3lOqKWWxTD8jcBKlCOu9K0wKikdVXPkvpmoBWmdkWgisr04Lu7m6jLSx+h78xKlJ47RlWM0RlYlpAO209CoFde6Z+9mIuxhB8AylxDQ9GroNFvtNX5hvHFu4sXVfh7OVtG11/89I21YJCTqYFsUKyhaBqtCxa+zzdJ6WcDIu0OytKzauTgbDq/ybfZzs3i3NtsG75FhC0ZIdWwrTpckKVb1s/ZXKUh6JpEYGlXuW65u884pab68R8PL0iF+YrMZ3k+96VTEIEq2lBeKzw8O2Le4Ec7XMprnCWnCCLkQq+973vBb7H71Azy6jUVrmX2RaRIE7jo79MTGoPVMyyKtOCbLyoBdEzvNw940t0ym+4hqZle9rJxbkUpqXQqEgRxnp5mkf9eyIUZzUWRg/EYcu5GR1JmcRkv6/+fVg8WNN+dFJA7eLCjmOvWPQmAfv2adTIBgY70yQqz8+hYgOeh5ONStrib5fzofzj0Pch5eZq2iDrPFUiBo4l64wbc9iIUvDoy++mCBv4bPk2snp5uU1c8omja2RzuF+9w+Mi2+HYZDAT4owyLfjGN74Bb3nLW+C+++7zYsg+/PDDsGPHDrjzzjvzXceyRzoNRan75e+mjFL0IdJNEIyavpB62AUB2nGZ61ZRkfBMDJwCaldKAVonUyY1XJbNxLzAFIcwm4Dx/jKwuoOuyMnU9CPOQBwQZLNsBbkEjw/bGtup7TmT1wvtQ1/c0w/L2upJ1ALqqYzXLPPzitOGw2JR5sOWO+yeB+9dsF6FfAzzopENmQAVKoB/VhnvsloRcfcpEYWsQil1z9lEiDF9H67NdaKdvXKcuMRlV69rlnjI4maorEjGuiapHAbU7d3D0NE/Bm0N1bCkNTxkaFlrZE855RR4+eWXRczY3t5e8cI0tS+88AL813/9V/5rWeZUptf+hH1qSIuXsyi5vQLZLTBQh2ia1HqEC7JhS/1Sk1SVTFqXWjKlUEuB+YLeK1Uj63dyNEHFo1u6Y4dAcsvPzqvb5pijt4tMb48uYpm3yZNGMQeNbBhYlq1dyu9f2jsgYjs/s6PXq4hyLbM0LYjThnO9Ryao4Bp2LeMsjebbxEcVPHMvO2w5Nxctfz5QzIRSuWhkS6MvnC5BrVD1sj2PvmlBtI8ITmo9i7wiXQQcUzoHxqB3RCbLLaxGdspzji2hm14IjSyyePHigFPXM888I6IZ/PCHP8xH3WYMUoCUAiF6RtIc4RL5oFRVJD17KUmYwGNqbqZBQ8lLbtDchAUv9zSyyYQQ4NxlXMiJuPahuN0T23pE/Y9a3gbF0cgGQzpQ04K+Ecze5qacxcD0FNwXA9Y31aqPm2MRkKPrFd0RZxMUOzuNbEg9U46xnbt1o2VmqJENXVI2pOiS+6X/Do9PkrL8+sj3qIGZKpBGVh9g8jFU0CuM7UnPSCfvg02bLuIZp5t3vsfqfAvGRnOINMrHPB5WL8qm9c6fRrb0BIh8KhowTN3OnmE4ZsUcqKvOPDpD3Ccoqg8Lm1RSMyPZh4Xrj7K7PjjOowJkQXMNrLUkM4kjWDqWfbLBKWG/lbwnRGDigw+C7Pe6BsfhgZc7YKsh5qSUlQJhnrSOJEyzEvYdFajD7AXd33WNrFu5yopErADRcVA6/pCycFm/d3hCaNGms5On19xoI5uIjiSA9I9MiOVsPWC+a88ZT5g3h5OK0MjGK45sb5/IhAnLJl7eNyDS6FKh0XqsDCsaZacohUV9YcMkKCjTCDJBya5e0TvEDYWHxLVRc0LaAF5/vA94P2z9Rp6DKFjrln9tr/1a5vNYgQlAjEleNof3NLIlIkAo1chjlbAfxHCUmLo9G7Ixx1L2j9GX0f5JmhYEJqE53m9kR8+wGBu2hazkybE3rF04+rnkYJ5SqAgmhYQF2WlCakkwIDk2slc7BgPbyEHIZAcTtuwfFujdVAfxu6GOYQ+mfDCwDJvNUKbE7Qjo+eUqyOL+Q2Nm4Sp4XLOTjj9bJ0u6nkYrWD9blZ2IpdKoeunlhnW0edPIhuxDwYkHXuvBUYsgm4dBwAReA3lddCcokyZGTgYUywIvuEdmFYuzddx7hMuJf365E/b1j0Yf17GXj9cfzxdXDGz7xXWgyoY42eIyIcx8oEByV9CcJuZ2mWKLulEsCmXuJeOgZ9uXx93LPuGQ1xkiVz1dG9n07wENbu7XJ869tq3AqXUBhVwmQ6Z+odRhQXaaMNq9asiGQzWnmaZm9L9zQgVZUyMNyzgkZ4UVwoszWKdM8sJjiky9DnFTDeb6cP11Sxc8/FqX8MyMwhbyTE1fqO8TLMd2nfDrsKXSqHpF2chmnDFL0czZ66x9E1pH2zlFZZGLW09TubK8wBK7QZD1tTP+OceNAhCsV4xBKeZNHhhzBc94ky77RM+b9KT86+Lv5RQk01ihnl29jDB77UIOwtb7nOPkjGamm257TGwfXYNj1raQz9pMyBWlLM8xbr3sNrJgPr6hXNdG1uxbko+JU1R3IJ7blHk1J4ypHCZDhUqOUjI2sujQFQY6fTFm5KCKf2UYLrSdRHtYvfHR7+JpS4MNztSOTQJy3EGC2shmq5FFIRZtRWurKuDkte3qwJPKfIk/G4bHXCF6X/8YtNZXh26rZlMLCpwiaoGu9TN0adbZsebsFbc3tAmJuV6bOMv9cTN7eZrOiMFEf58rGLRc1hEnXUqdDOYf8r0w85D3Vfst9rGd/GlkTcv+NpTVGstkxglZIo8zgcme4KQhP6WZrk2BBPIQAcb2fa5xnPFZRjOu6WJP/yhs3N0PK9sbYM38Rrc+lrrligwvlu3yd1yzJP0337cjPWEIMy1Iv0f9k0yEUIgEHFHCfFTacVQK9Y9OWKMM5VKnElkYyL8g29LSEvn7hRdemGudZiSyU6JOQyhUtdQnAw3IaCObobbBNJuqIFrhqJBdeiOWS+vCRjbkGGFIOyCZHtMUMsTkHETrki/7MV1bZ8Kxmha4700lmB5+u2mBqiXLNKxMwMQknxrZCAHL+2wpayqqM8ywPau7OuGmBelHTG9LpsMYPZizjVoQR5CNOTp46XZjPetkP8vgTDXVpt9MZeWDbKJyxI8MYJ8U5DX8VqAOtroF36ONMtqELm2rj+xzFBMqx8neEzsLZJ9ss8vO1wQH27V8PvMSxSJsJU/7jP0BajXp5JViWh2jUQvi+KbkXZAlbdw09qGpYtfgOMxrUjOG6kmTMsE0wS11MnpWbrrppsLVZKby3K0Am/8AzUvOh776Q5VZ6ND4JLTUV8Hm/YPQXFdFBFmTjax9ZmZqcKbxEh/IsIgDdGCwLVujVtd39vJ/39s3KjwwdY99Wh5uE1Zv23OjBvqfHkGWauj0IOWe5o5k9jLV1fvOGtdUd1DJLbNXrjayyr4RJgPeZ1JpGv84ankqrlYFB1a0F13SVgf11ZWR29OOX7/Hxro4dueOQvTjgfTSEdc5Th0Uh0GLcCee+cB+Lmqa5NxTT5uO777Ph+BiLjvwW85HCjtOjDad/rt5/5Do97BPx9SzcY8z3TkRjNr5AkxwqCIna0E2w35SIqORyG9jRLF0bWTTvXxo3OIsW1zUJaAKFNM4IqMbjWlRjnKKWmAZX0qZ6Zz0zU5e+i3AC7+C5solAKsOVX7C2Xrn4Jjo7BAUZhHTkpKpk5ZCaVTqU/ogy31MhHkvy6V11OqaZqgb9/SLh2dBSw3UVAZDqnQOjocKigg6wNVXV8CyOfWhS265OhnEM7OITh8s0iFoxZidvSwDH3FMsu1r2sc2Yw5kjYosTS/b/D6sTOvkw+BEhbyyb0CE3Ikr4OzpHRWafBS2DlrYHHletH0EnL1M9ZRfksldIRd0dZtX+7MYfzCJo5E1TV69dhRIpJF9mt5A3fJtWhBzQl9YG9no7+XxpeAmNZ5h0HvnatQyD02VPcHntRATA8XRUruQr3YMCLMz1F6bQL8GTA7QXOuOk3p9dfTf0NRognwf5lNAV938TH92yTfb5haZJGkqXCPrC+WOokwwpZSPi15mOcDOXoVmkZuyt77rhcBPQ2NTSrxYX+sZrpH17XfsmiNTGxQxYNPDtFmLa5+Bezay6OylJVZAoUX+bkvDqztXuRoiddsd3cOwae+A4VzCH+a40Ic7aqAOZs0KXhtTGXG142JbbZadsT2k1iZkp2fteCPLzkxo0j/TLGcmQQwnbiiUvtIxGNthQ96zqPTOJo1P0NlL09AkVdMCP9GFbN/578gH04JsdWUyfCLgPVvRZdJq6tfJj3BhcCAivxm+zgu0/vlYTQlrN7mEdAs9ZlyTH8N38pRNE+HA/kXUyNJ2YqpPvp4Fk6+BFPS3drp9gw3sO3Biu3/QD2MYXq3g8x46STS0LbHqZsvsRd9neXmiIobQ8S4sTXwq/ZcmXwqjY2AUntzeYzQlCQslWaqwIFtoFh0u/tR1Ph/4CU0LqNbIT4gQHrVAtlHpzGJczrZoZKWsYeqc48SRrTDYyCoPm6X16zNEd2A1bmqwXcpPB0+FnKiHVP9d2ZdcOz1FrVyapdg6FVebntmAqQ808lh0oJRCUi5yg10jqws9iiQb2J+WI+uoe9+GO1Gk9wkxe6HI7UxmH7gb1crj6oJXTxpHtkAqWay3jKvbmE6OYTuXKGc5ZdtQLaU/0NkEv6BJSv5GsLxrZMn7fGhk0VP/kc1dxvBkpnJNn/3vg23Ufz6jO658TdizwX8OlG8N73KDpldX7D+JTTgKV5gkAJMmGO3ulagjYX2H+jng/ElCSorPiuAu3/k2svnMyvfczj54flefppgIbqdEyzH87iuTwOqLY+LZHX0iLvvLewftZZaRJMuCbKFZ6Aqy1f1boWJiMOBxSJVG4+nGhxrPYDD34Hv5AMbVAmKZYR7ZYdoOGhpMn6EqHZJF82AK1m97UHQv7Hx18FTYiypGrxtqEuTsNUrgCQ58FkFWCE/x6xQWXolONLzJUYzyVI1w9AAROHezHGsskwqRpjLDncbiDapywuRqUoJ1UbTyyjJa8DyyaWlhQuDIxJQYcPC5RROasGPI8860udvah3jebKG5CqiRzUa4DCPMjt/UR0bRMTAmYu2iiZcN2wQgsJ3h+LK6sm8Pg96eXJx1smE6HP+CGtngM42n3Tc8IZLI7LH4VMQ1L9O38gVW9Vyl8Gfq/9w4stEJgDKxkcVxBONDS7+SuCEx42hkq9L+NXGvkSlhTTlGLWBBttA0zAVoXireNvVuVH7C9qLYRUk7VBGr1RwD033vfvC3ceLFkSXhokxtNKwT821kg16curctzgajPOhNGiJ9W5y1/vnVTmUQyCSWng4VpKIGVTpZkKlle4ZcrY0TYiNrKjtu+K04A4e+jW+Ll+7Ikn5S2qhzxOU8zDKH9s3p6kTWRX5vStsYlbbT1rmG1dPk4Rx2WlIAEPbgurY8oFlXNWfefc1AJRu2eqAjk0M01FSGmvjQ7zM197BpZOn5BX9Tv8+nJkYpKg/FOjGvddxT8JdR41cu1nOqla+nHA+rS8k4eykb5Oc4tlUxk7IiqAENClhh90KfdHvPtfZsSU2t0rZk5BMyFgcUFFmastBt6dK+qYyoqAWecO+khfJ0vxzXRtbUJ+tttxxgQXYa7WSbel40pDYNbk5jtYalqPVnksEyTG3QHdzl76bZXYgwRqIW6F7d9KFB4eiPm/aL5TrT/rR+tudEHrtneBwmJlNKdqhcNLJUII4qRdYBT3VOgxtvtntoXHUEMCxfm8q2DUpOxLJwWL30z1JIpymEo0rb3j0s6rarZyR8EDMcz28D/pbGa0EnaiEmFhBxvBgKLfcYaYFeOjbqdaEaIcVxTruvUfWy1T1MEyLtYxtqKkjdwoX7eEITGews2YqwHJvmNZ9LpmF1y0exYVrXbLzIdc2peRvzdYtTNzmhj2NaoCsEcgH74TjH1K9XnCgjuWDTMCohFtMfbO017qRD9yXxVjAD42jaFMvQCBIG21q//FjVMNQrWkMtMWUhDJuIVUmNbEyFj1k4znxyV2xYkJ1GO9mmnhe8B8d7OEwPT8LgqEL7pPQudKbYMzSuzPqNgqwwC1CKiK+RNSVEkB0OOQdcqkOG09m7IHSwDBdsPAeyiOWVuER1GjaBrS0tyKJg7f4WtDk27Rt1LF24iHNmts7U08gKQVb+Fl5i8FpGXx/5rafFIJtFOb+F2QrbMGsm7TvItmLKzoXPELXRE5MpMoDLUsNWOgL1048RcjLo3Ik0Co2sXwcTtsHcWIeQaxwnakE+NC+4RIkJTxC0N/ViRVs8+eWksHBRC+KWZy7LVq5eB7uGTt4/yMBGNj+OcXgszF74+Nae2PvQCZ2pPvmSaajJhCJEKeeu/tXrEFsjm/4rtZS0zxIRYzQbWVNbDTMtyPaShI0HOlFjnyeUpyAQ8chmymUrX69HGcmxLMhOCwtdjWxL97PibxWJHmBqnK4JAIRoZNUHEO27ntjWI9Kv+ts45jiyIUuair2m9pjSiAq6xoqeg8273DTAWr350x2N3CcsZEv2y1rh5chf8Vxb66rEX7RpdtPr+p2crf4Uu1CoOrzFSnFqKVueG96fuJnXstEweFoOw4RIX8rX62DVyJJSUABC2zFvkiQFAkPUCBOy85Yxk0PtpMn1VwRyiE/cSQtNOysE2QgzZpNgEacOYfWRz5GcRHsRGwJa3MyfMRSa8IVZhh7b0g0PvtIZKAvf4v156NVOeHJbj5h8Z4qToZNgdHnmaxC3Duoxg9v47TdasFDsyXMQZFGJgPfaZP9oQ9cgFyoKhOqnYD6WZ1pgactxbPlpnZe01sHcxmpY0Fzj7YP26nIMrfEiiATLoilqA1ELQhQ/Ydh9Q5wsTAvU71wflvhtKBh32u8T2UaWUVn+OnASFdAwsBnqBrcLjaxsbKbGaTQtMMwW5QzTFa4AxibCl86pfau+AZ2h6sejQqVwGNNsZOk52DQPAc/okIcfy7UtQeWikaWdaFQxMrYm3ge8XzLGL2qbZL0TVuHNPAmI1MhmIEjq5yE7LVxa8oWkCGFdbwMhx9H38UOwhQ8qcZy96OVBb2X05t2ZNncwDWq2tq06ewVtZBE0VfGOK9LZ+ufl3dcCmBYIwUJqZIXNdbiNbEaZvcJiCZOP8hr64cXys2SKdcTVIKxzb9qO3FQ3PM7GPQPeZFeaWmR2LHs9s9HI2hzebOWaPnvfG7ah5UaF4DItr2eDfM5E/xKzHE9zn4UmOrO6mfvgWKYFTmZmF7K+aBp25PI2Eb9aliPbHjpdmgRV2hd4k/awNpKBftZ2S0xtULlehu4z+F3C6wvjtKHwZ6h8JFkWZKeD+jkwtewE8XbervsUO0a9o3EFRb8xhjUwQ7hZT6g1dWBKqC890EpIg6YPhKrxcwK/2wRZk9bHGnpI8y7Pl2lBJuG36NISUp22PcKlMbmryTPeVHZYx5Xp8p1NIysHL2Fa4BUYVZa9DVgH6/T3ur2ZbR/6nd1G1v9emsd0Do0Fwm/h4INB001tzBNkPRtZ870J8x73g6Bn4OylXWSbZk/aeddUJUXCkLDJRqaxhW3Pqv6bPD/dLCTXJVN6SNonYV2U+z/lCG9tSRxPfh2Tr4DpN3oWqJnERBwmhyubw1tEJcxfK5PSYLlR5xtmNpEJVGCOq9mVW4VNrPMh19jCb5nO3aQt1LcNq5PXT6efZzpuyecRJ5UmUyzF9j/PcWTt457pu5D2bignkfBjyer3Hle78DmQMoKpnEx9NkoFFmSniakD3+wJsuhZ7nv+q9vJh0ZPPGWypTSlWZV2ataECJaHMqgxDXaG0nZXn8HaTAtoR6SHkxEasGAVvXJTBXi4aB0ysZFVQrcQLZ5tETpOhyPLCtsvVtkpdfBytf1xTQvUz2pHbka2C5OWwrSPaWk7qh5u+eo5YDmPvNYlgqZjUHQdeX/kQEntwSlh3uM0Bq3tfCI1spZ7iEvuSFM6K1GYjWymWvow22aTJ7zuqBfWDmId3bI9TjjCyorjya+TykIj+9T2XhFM//ndfdlpZPVJv8223/Be1chGCbL++7AEIDihw1ULXSCRRIV0Mh87vR29vnkNxAahZmKOcTXC3L8oK4chx/J29aIW+PtQe/UwjawIhamtQMY6eAh2k7rgD4oSJ/BsB8tIkL5Q72+f3uE+B/g3jhN0GcmxLMhOF85BbxF/W7uehJrxLquNrNSWhKXX1DUrlP70TNPUCVFP7uAStS58GbJ6pR8QX36WM2fLORvK9+zzhI2sZUDQNbJT+TctiHpI5c/yPsi/2JnQGIMmUTYVd7k5C02YTVDxbWSpQ188Yd10fNv1kaeiL0+778OPZ9cOmdoqaA6FoUX7nTfVyBruTphWzD+38GMpNdeqbmvTA1IDVFNpvX6mdpEvZy+K74Vt3ibTR0xZIk6p1zqsqGwEWdrGUKDD7EReNBFlO/+9FPgwALyhxGhBNkRgtn0vhTD6XZQgG3fCjvbFaEf+1A6zMxdt43E1sn5bsG+TD7mGXgMl65tlwhvlHxFnud8Lv+VnAyIRRFCQDZZFZWC7s5dZEM+ns9cUVb7EeLYTwhROmlmp7U1qofU02arJXWZ9T6nAguw0Udm2HPraDoGEk4L2F39ineX5wqL7V7dHVTQrIRpZs5bLHuw9bECmEQvcOmkaWetSiROwv5XLHmLfEGFJyTFdAGev+BpZ9zMNwSJ3tco7gWtp3iyO0BFXiys7LWEjG3Fcr5ohkxe7jaw6iVL2MdU3QxtZSVyHNdA8kyXCRjZLjWxYopFo0wKbIOs+l83pmMT+c2ganDPV0tuPb6qOf+8sQlyGj5i6epNS7JHDqp+daYEqoKJwioKd+C2LQTgbAS5MdPLfBZ1ZJybDhGVV6A2bsMs2LG2uw/q5+HG3nUgHL6wjpjbF0IrZ2DeHRo6xtPkwDXwUej/t+aUQRzib46UpakHYhCaTVYzMnL3U40U5vSZCNLI2bNGO2NmLCYCNa9tBHxbvW5/5EVSO9xobm76UrQ+q9IExaWSl5kd/sKTtrU0TFKad0wf4gLOXRV0mi1RtbP1B1Jv1JqJsZKldFWSNGkM0ZieY0NIBE7s/ej31+od9tn0fpy+0ac9o+C2/440S1iFSw4Dnu6VzCP708n7Y3TsSiJihHMeJWC4NcXrTiXsO3vaaIGsKvxWlFZMTMtNzZUOvnmlSh9cQ01ErpgUhDmW2Ad5eBzLpC7SpYAF6nxIIc5ShJGsTvLFNhmmEs9HIWie/+Fxa6oR2ydbiDP1qZBViaNSE0KFtF2c1IB9xZJUVrJjlODEF+t29o0Kztz8dZjFf5l2m8UbfxqiRDTmWP74En2csCjWXtVVorx7UuPpCsJ9lM18aStvqkuna61rVqNWahLCRVf0FkLgmPtkK58WGBdlpAh+WrhVnw0DLOkhODMLS5683djRyPJbPntRgmrKamGxksVGiUbf+UOga3oAmKWS2KR8mWRc/jqz7u61/NkU1oHmtfXMD9Tzw+zhLTZmA+ynOBTGX3T2b5aTBtCBGWlbTZ7sdXPS52exvZVk0IkZUPxRI92nQfry4px9e6xgU7codvHxNhb5tVNSCqMxetD5y8Ig7EOsaWbr6EFuQTTdof7KX+f0wneLg+KR4VnDglJ7TYQ5ltkDxNmxCgG1/fWDP1bTANllxbWSD26PwkL2NrLlyNJqIvh0613n10+6/H9LJ/Yyaxj9s6hCTt8xDeanv9XsRdr5xI50gUfMs1bQgeEy8BrZ2EuXsNZzWxGaSbME7RkqNjGM7lk14NV2RsFujhwrUr5s089HHM3o0qpENdaSE+ISZ1OW6apeAhNHZa4y0Pf06qNkO7fe/lGFBdhqprKiAVw+9TLxftPFGaO56OrAUqJsW2LKRiG0MgqxstN4ScNpextOmyo2c+A5KeuBoP4ZoWlCN0LTR/b2wTSSGZ0ADhvEmLUso2c6K9Y43riNUwEaWaGRN2aPS1VeIm5o1Uw91uo90clI1sur5684h+rKdOhC7nzDnuen4ikbWUjf6WxyNLPVoloJeXKFK18Da7k2YrW0gQHqM4+rbmEwL5DWUIdyiUO5LhpObOJOogEY2hhY37vFpWxc2soay6qsrvW3jBG1X62b+vlskKzH/SLvJ4XT8UL08eQ5oe4rL8R0kukLgHlvqoGuz9LqGCX+ZCLImBYbtOHp7x3Ifeq1LhLlT6+7NRv3vAsoOP/5qNoIsfb71uqltSK1voI5qza3HcyL6CDmhMpn5+BpZs2lBmFlWFPYVOogWZC3XTNHIpsd8ulJKExTpu1EhVzFTKiNJlgXZaQQ7oK7Fp8HYhncKW9lD/vpZSIyqHoRS0FvQXAtNtZVeEGc5QNK2ZVsCxcHB0TRVQY0sGL3fvc9g0shKYTh8IPTLcJT9qQcoFZwCWcw0G1lKthpZfVkv6iH1QzGBIStMuGYkrslANqlBTYIKtg15veRsXJRH7iJ6qj68uVPL7a0KQErRwXHN3cYwKPi20BHL3lYTlOCEyPeoj3e/aUYbxK1eBl5bhCjTAlzxkFEI9OrhgIHaQYr83EIE2TCtuXpfYlSYDj6paDs6z0zGsMqjFRcLur0eis9UFoayk898JnayYQI2Jlewa/X8722hh+T28l5RgTegmbRGLaDtN9h2w4S/wAQ15MbTZ9wEPU7A4WdsUtguo8OPzL4mjmfoy/U64T7epDPd3+D1w2sfZ0IiNdL08fIiFFjTsvrfZyxXpbf3xj7t5+p0IgSjoCrfkDB+mUcTsVTLNhEytCt9jFBNC8zlVGgxtRE5AYk2LVDPsVzMC1iQnUZWzG2AeU01UPk3/wpjDYuhfnAbHHj/30NycjgwyGAQ5+NXz/UGP8dohG4+DrVN0zW8tkDsYcKXnLHJB19/sG2aNq9z9By9VBtdPbc9rYtpSSwX27G+YbNm0QbVuipL3cIWz9/ZtEQcdW0ltsw1YeiXWrcnVlPUqoMQXlI6kKuaP4uNrNZx+k5wiYzsBvVlRWWb9F96HnLbuB1pRUyNbBz8yBrm3zGcE2avEoKAts2O7mHxG82qFCbImh081AlGFPoWSvIIi9aG7peN06FyfItGFp2bjII0yaaUiXlBWLVwYmFzDKLno6fO9vZJ90dygoITw0xNH3SNXSY2svq5TeZJI6vfW/r8y3MVx5fOXiHXWIasosfY0T0iskrqIZ1M7Ot37Wpb692U37pDsP6d/t486XP/4gSdZgSk52QzqUHHWPf7YPm0/6eCrr0/cvKukaWrf76WNfrZrjTYyI6EZHlTnaDD61SqsCA7jWCqvMOXtUJFfRu8etZNMFHdAk37n4TD/vIJSKQmjJ2UHvpDEbCsgmzKEwTkgIECTriNrP2z7PzkUozUGntRC6xLxo4ysArhgpTvHcIgnFkf9iyfrB6x9CizKtk7JCwfhRHpNOc5e1Eb2fSu4jeTaQEdRLWg8GHnEkd+MGlxpdCPZiQ0o5VpydcWl1DXyHorjdrg7Lc/ch6GZVT/N/OyorpNup0YPJrjpg41PTdZyrGRgsLIhKuZwkx6tnYqB318FqV3eXPa0cvmgCJRBbIYgqxhADSVFTQtCAoR7heRh7TWN2BaYCgMjy8nxZkIi2ECNraTOE5CAY2sNI/CIPljk0ob9LeNd32CE1j1c1hmr4BpQci50tUHXcuLmlH6zOj9BZ1gyT7OrTypi0FLitBIBTJD3u4+NwNfr6Yo0ME2tie97bK2OiWCgPs7OT7tt7x+wDLGpP++1jEkYuvuGxgVE8dd6Jiavg5++C0Vm2KGvndNC2jd1OP65wexsQ1h+jnSeycT8kQlS0mQZEp0f30CRwlL3V4u5gVFF2Svu+46WLlyJdTW1sLxxx8Pjz76aOj2vb298LGPfQwWLVoENTU1cOCBB8Kdd94J5cbYnHXw9Mn/AVMVtdC+909w8KOfg0RqMqBF8j57D5CUogx2gUm6pOdu11ZfDSvbG2DN/Ea5m1sOaZ+4NLS1y9cK67/L5eg6KchqD37UDJNGPfCDsRMtgLafK5zZhOPMhFmcpaNg2pPuaNsbXW2ArYSuoXHYtHdAeOi755oIRi2IEEaiPG0hB02YqaORYX1kZ+dpFA3CtC1hhbiuTvjSvqM5wVFNhq3mfmQLs0Aj99eF3Uzvs77c6mb2sguLYdpaz4zE5LxGNMs0y5uOnDhKbSymwpQDJ8W0f6ZLmGGTUr1NKeGEbJOj6EPG08iiUGURpKU2jNrnRR4ng3rQbZWVCU0z5Qkmjh+DWzI8ISPAxKuHknVMJE5J93vptuCGI3OUVZJXOwbT10m7h2HOXuS9LsjqwnJQkJ0yCrL66oz469iF4In0DrboIDqdg+Ni4ldVmYT2xprAcj49lGlCYr3mjjpG4QQSV0Q27u73tpFTWr2qvnInaBPvKyvUdPEmDXJY/TJz9lI/03HTd5KmfUOwnATpCycjBFnZZdr8UUyfS5WiCrK33HILXHbZZXDllVfCk08+CYcffjicffbZ0NHRYdx+fHwczjrrLNi6dSvceuutsGnTJrjhhhtgyZIlUG5gg+trPwqeP/nfIZWohIXb/w8OffgTUDU5pG6nxdOkTkh6FyK9c7FhygaIDwAKsXI5Rx/EEFwaQmHW1imPTridZW06jI3eCUVqZL0Hksx+SR1N2uCwzDZxzQuwk8dZOgqmOIjgtWipqw4dKPSBQT7s8j64yz3+bD3K2ct0mDge+aj9eGF3MHuPXh4OEI9v6w61YabHkdpbPAd1mUo7B8MAQ+8ZCnv03KM6Z9PExNfsB4Vd/C4TMxLdbNDVTNvB6A72shKxU2xGpZv0zArqVUcvf0LpRAqymYZRCzNNEAs52lKq3DxutAsdmxBiy+xVSTWyGdnIhv9uM6mg1wBNjLC/kzad1Ea2N71qow/+pol2nPrJayFXxXRBc2vXEGztHHKXxB1V2xo2kaNf67Fp9etpSlOqxzZ2zzFaeKQaWeGH4ThW8zYdmbQCfT5o1j3/WOb7JR+3qEm+vNYmDb+nkdU6a2/ib7KBJZFpaH8QJVjHISw1O8VbaSOCrNrGg2Uk0tk3xbbk3tP7LpGTyUydRUuRogqy1157LVx88cXwgQ98ADZs2AA/+MEPoL6+Hm688Ubj9vh9d3c33H777XDSSScJTe4pp5wiBOByQwqDXYtOgWdP/HeYSlbD/F33wapfngXw2h+87XRTAN9GMfhgyg6Thr2xzZhtD5O3rOoJMo432/W8PL0BL12nCG90Lz6nSM0bdJoyLcmFOXXFdfiS5gQSdJ6j4b8wxM7GPf2hZQdS1BLto826Q13+McyajeFe1Ouws8dNxbq3f1Tkp39mR6+7bKiVR51cpDe4fn9M4ctM2mBatHyrDjCqjbYuMJvw6mCYmGhNLRAsPZNO1KyRDds+EW1vazi8ko5Zi11qOm+ZRYeaFSAhhwgNhRd2rDgOMlTA100L9IlFXExLwWHht5KajSwKVE9t7/FWQeIcx/x7uHC9bE69mPDgMyMnGLQPkoKrjC4hJ5FBjWy860PjEksBVXHESrd3KvCjcCH7GqoxRUEE7VA7B8eU66ALrvpEPEwjixpS2bebrp1+3nocbmz/dAzCiQDeRz1zFK2HVLbogpniLU9NI7w2Gigy/b28hk5AayyhTzrtE4LOXsEOUG6v283nornUn2/ZNoJaeUNq+aioBZAgNrL+ZM2kGJKCrKnfz+a8ZqUgi9rVJ554As4880y/Msmk+Pzwww8b97njjjvghBNOEKYFCxYsgEMOOQS+9rWvwdSU3f6jVPE1kwCdS86Ap065CUYalkLVwE6A/zoP4DefBJgYCWQ4UgI1a2Ox7CSos5c+mOva1KAGUj0eLv1hHXE3OfjooZHszl6qoEsdcsICO0dp4+IOtD1Dqt3WwpZaJR3hls5B2NUzonnvqmUnDJ7e9B6YULSYhrrKssKcvWTHg57+27uHRQxXDDFkO/clbXWwbmGTWUBUtGSO8Z65EQnoQCXvncVGVkuFa49ckT6uwdhVzzClOHuRiU42Gll3xcJ8f8TAEFJWmABMNWo4OEeZ1fiJKtQKhk0C4qSipDgGTT8KPS/vGwg6vJicvdLle8lKQo9mP74oS3PYM006qUYWn72Newaga3AcXtzdL1ZQMgGFAFPA+oGxSfjr5i6RiUp+vXxOvedwJ9saVRDIujbXVYbaFcbVyFJBRGr/FEcsMtGkK23oEEztT5GX9g5A58AYPL29VzlPLA+vIYbTQhvUMEEWBRupsZRJImTaUrMG23yi1ISN2pOjXSrexw5DsgTP+VgLZyij5YT5LKh1UvHacHo7xe43DRW2aTGes5fhNydglqAKm/pzlYl3v6zr8rn1cNSKNjEuuWVr23lKK7NG1mwjC75jmKFfpfbV0rQibAWHNbIRdHZ2CgEUBVIKft67d69xn82bNwuTAtwP7WK/+MUvwre+9S34yle+Yj3O2NgY9Pf3K69SQNeU9s47Fh5542+g79APuF88cTPAjWdD1TM/hZrhPUHtiSGXvOycaPgtXWurLx/qSzHSiF8+qDhrF2VX+hlQ9KWYqZidDHaApuDSpllgXI1smC2ZXM46dGkLHL96jnC2o0K4n9BBFU4ofkIE+rs/SYjK7BVmGxpw9jKcI9VOu3Z3ahkSHKR9G0y76YdfbvDYekcetMf177UrEAWvpe1amK6DF1PYq6fqdJCbRjbcPtZmP+sKuXaBjoYZEgOEVahRz1u/X/7hgwWY7k0oRAhCUCOGQg9q9fX74p63uX3oTmBxiWrvom7k9mA9m2pcgRIFH4yzK68Hrj5YT9NQtEjJTOzXvXpMOUKo2dfnazCpfTB9pmTZ8rvGdKB8qd0LCC22+mm/eM9KIiFsQ3UNqrc6Qto63oLFrXXiPa7GyG2kwKlXAIVJ7OfwGmLWLd3UgPbNMqQY1kVmmJNhmUznZGsG1RVphcmkalpAJ986umLFuw+eYGgmchLnqMvwpvZn6wn8qAWqYoaWK3eOSpmdyRMjzwmPj9GJfAWPWYjEa0x9NPxjms+1Qgu/Je8LnVCp6eINErxWh1Kn6M5emZBKpWD+/Pnwwx/+EI4++mh497vfDZ///OeFSYKNa665BlpaWrzXsmXLoFSZqmqAvlO/BnDhrwHq5wLseQZq7vwEnPTb0+Ggv1wG8PyvILn1T1Dfv9ldItTunmyk2FnShyAMm7OFbL+yo5P2sW6Z6kBot+VSBV13iUT9zS0nOLsNtZFNHw+XCP/48n7hzBU8rylviQsd3rDjptfMlipRP5VkRNQC0+WlRcjypMOH3I+eh6keshMS2hoifHqaDe3GUo1fYKKh2S3S8r06EyFZnkRgiZsksaA22sLe1jaZkQPNVLizl7jnmkY2A1+vtC0b+RxiWkDbYaAcX84zok4K7M5ejib4Su0Hra/YzsndCVAOaLINSGHJNLnAS25z1jQtNcbCJABo0IkDamTRZhiX+iXyvViyttjN2iIghF1L4ZBH2qwcvE32zfLZkIIstlmc6AeXkcPbukSa0+D1ltdW1+jL43j9dTIBbfVVIgMc7i9TwdIVDcVGlmTpwr/y3kulBm1L0lQCHQ+l9t0U5i6sveHlk20Z60QnjLJPNT23+mRJb4O2Q/q+IeGNMmy8iPJHC7ORlb/FrW8cPGVEumImQZoeS8RfVxQpajkK1JkzvYFsO6iNpeOG2UZWLS6TPriYuE9sEWhvb4eKigrYt2+f8j1+XrhwoXEfjFRQVVUl9pOsX79eaHDRVKG62o9PJ7n88suFQ5kENbKlIMzaHi7RYFefCvAPfwJ49IeQ2vYwJHc+Cgu3/QZg22+gGQBORG3j0jOhdslBsCCxGjqWvgmcZKVnw4oPtVOZngFro7IuSEqNLB4XbViXtNbC1s5h73e57C7LFtuSB0VxAEomjA+a14klEpCK4ZlNbTFNyPI27RsQ79GZiw6K1KwA7WKpt3jCaI9Jj60e19dCk3OWgiz+M9xH1VkhLSQkk54JjC3gvskkwRVe/e88p5BkAqQFMBZHBSUaq5deL/o+avnaZC7gLvf7HTxdok5ZBhJdmMZ9qGBBzz0XG1mpbZOiJXb+tgmWcFSzlKPXSUd6a+tOlTqy7lI4D4T00pb31X3Vz1GXQc9IFhZ4322zfvugURg804IMB684gx1e8ylyb5C18xuF1hOdSVe1N4gJqWgHKQdIVtnAcbCvonV275m5EnRyJIR4Gg9aFzzls1qRFH0G9o04kQ8Ksv577B/RBAG312tAl4blki5dAaMTVP+ZcJ9fnHyPjI94WmFTaDq3PAcqk7KtpTxhEvtrXE0zhWDC6DMBjahhMmK6okmiXcZ2RrsyuXpn0opS0wlE98K3TQ5M4bn0fioqGYNp9YV+pfeX9L3e/0f5g8RB106blDu0TKGRNZoWBA+aECaH/iSFKoVwEkdNC6or1clMriYTs1KQRaETtar3338/nHfeeZ7GFT9feumlxn3QwetnP/uZ2A7taZGXX35ZCLgmIRbBEF34KjVsS5/egNeyFOCsq2FicgqefuQPsGjbHbB88GmYHB+Fiq6XYc7O+wB23geHYgfV8G3Yu+JtUNv4XgCYl9ZCqKGyJPqyqTT2n99UC4csaREdJwqy8vdRzdFLlEFs8eiDhQPAVCqYEYfOxh2DsxF2rNQeTUQtCDMtIAOEDenMQYNv2/axhQ9yt1f3E5mTtI4orDy5rehAJsxe8XICQPeTnY8QNshgY9LI4iBKO2v5zqSRlQOdyUZWPQfTBIMI8cQGVSzL2rRU3nF9IUEO5vQc9HsujpWBOsDvwH1B27FOFv1UyWHaWmOmHSJQUE2adBCS6Whl1eW1100fwmxkM0l/SbeVgqi+vExxTZL886P3TdYxrjOTV4cY29N77WUbTCbgyOVt/vcVSWHaZA+95z/3dMISbtPsj9JUI4vHsAkOWFfUiGI7NXl707If3tzltQl9suKZVCXCbWTphEg+x74DUPC4uo3slONrX2W/IY+nryCI31Do1tqnaRLt2EKnpa+h3s7kWGEWZOX9Us/T0wgHD6XWxfI7/hA2VtiSoNGoJdRvQj+uvKO+sCm31asR/5nxJ2S6kAzW9qP7E5jq4E/owQOLoKtCqKyS0D4J+1ox+c9wEg2zXZBFUFN60UUXwTHHHAPHHXccfOc734GhoSERxQC58MILRWgtNA9APvKRj8D3vvc9+OQnPwkf//jH4ZVXXhHOXp/4xCeg3LAue+o2rZCAgTmHitey9fOhe2AMXnvhcVix506YVzUKznP/C/VDO2D1i98DePF78PraeTBZ2QCjcw6Cnub1UDNyCMDS9QBzVgNU+YGosZPGsC961i5P0HPMobcQmjmE2r9SYc0kSGEn6DuA+dfhsGWtwjEFzx2X0lzHi5AA+lIbQ2aX+IBTYc6UTUk5P0KYjayuQXCP73dyUZm9qFmFFFiD8X/TgiyYr53v2esLd7Qj1h2J9PKV8FvpgS4QfNswqAcHetWWj0ZHsGVi8x25UgFBlna4TsBGNsNlLaK1cMtOQCqRecQCm7mINfxW+j06QmKyE4wugW1YrlZ4gqwxha5/fWTK0LmNNRllfKM/yYExLPmEuvSoasp9p6mQCxBRBxu0TdoSTlQlE6L7sJsW+Pv7WiY1zqcO1fLj9ZGPiuvcaK8rTq77YEIsxwe1VOAlAaATm0A6UXJtPbMPopGlqyP+c2VeejddBz/5Dekr0u89fwnNXMcrm5QfXJ2R5xk8NvbRVZV+O1ME4JBQWXRVznR+USsbYc5eYf4UtrUXbGv6NrQU+V636bU5e2Uy99PN/mz3mrYJP/xj8HdKwqAgmCBjBu37pGYdwT4nCYmyTYhQVEEWbVz3798PV1xxhTAPOOKII+Duu+/2HMC2b9/uaV4RNAm455574NOf/jQcdthhQshFofZzn/sclBuRGllvOwjYDQ61rIU9iz8LbYua4ZHll8K8XffCgh13Qvu+h6BmdD/UwH5oGNwKc+FugOfl3gmAuWtgwZJToWayGlIVNbC/YRkMtK4DaFylxDqkD6rJtIDO9qVNFvWs9MuQ9aadtLpEgl+hTdpRy9tge5frne/mizdfGyrY0YcSBXJZR9xGxkhs1eN3Gk0B7A+u35H538nBwZ4QgVwDMnhIgVUfx7Fzx9rSQ08qy47yvV82PXc90L7u0GcabAMaWcP1NmmslM6Vfm9LP5suwtMUVSZhKO3UrAcaz4dpAf1sNR/AlQEnO9MCXbutp1mmQqLiLaw7e8ljAIig+BhPFEEvZv1ahgqy5L08Rli2LD11r2KHZ9D6xCHOfaLHtAmy7uRsypoBy3+W1AlL2MRDTqDkMCI1UlSbroPVk32J0bQgfdX7SRxWE4pGlizH679T7bA8Fyo40fshVrWoaYHQyAYnvTKCjSkUIA3yL8zQ9IpLQdZwTriv1OTZwquFaZF1xYDN+VS/RmFNLEwja2scVBEgVxhNTk+ydZkSEmQLXVmgfwPtjLR3U2xYs0Y24SWrkRNVOTHE/kFfGaHbYZMPrsJBWVBUQRZBMwKbKcEDDzwQ+A7Dbz3yyCNQ7ujPF3qqYmeHtlHKdsrA6odIks4s6CC2d+V54nXqimp47rmnIDXSB83dz0Fj/6swb3wnVPa8CjDaB9D1CszBl3bsqYoaSLWuBKiug+qKKjg4uRA6F58G0HgEpHqTkKya62b1Gh8WT31lVa0nVEoHAhq0WSI7UWonKDfxtRWq8BEWJB21WtT5iXZgKHDLwQeFWHwg8XpSAZweg6IuKam/yU5GdP5p2zyv7lbB2Dx4SIFVv06mTlJx8CKaDqNpgSW4vymqhJydx3EoMi0zybpQLYHQitg0Jmm7WlNweNmGpRmDGgYms0EDa0Ivq1s/J9JeU4eelwk9Dqi3JKxlD6LnQ+NAevVNf0ZvdOqRjl77mUQtUE0LgsKSDq0FNeeQg1rU8cx1CH6Hzo10AkW3sdmIm/LJq8dRhT3btaXQ9kqPTe1SA3VPmxZIQVZ/vuR+ctXHemzyrOrOXkpCGPJeF27wWlCHXCwL7WJpG6RmSHISIwVnabKE14heC/U6mPsCm2mBZyYxaZ4MGE0LUubldLm77V74v9v7lzAbWVvLoCskXjhEKiSScdatr3oewclNfHzzLPWvTSOLv9P7FSrIgqxvwnNMpiEAaR8pM21SB+agMF0ekmzRBdnZit75YhgOGU8uSiPrfh+0Dauob4XReYeLZcqeBegSBnD0ijbhBQtDnQDbHoTeF+6HgbEpqJgchvrBbdDYtwkqJ4ehomuTWy461eFr+28AHnEdyxCnqh5gAgXZCoAFG2B948EwDlVQt6kGWuaeAc0T+6Fhqh+gcjFMVjfDUOMqcKbqReSFxAi6qNUry4CmQclbGrV0TPrDTDseDBSO4Wcwi5nNrEBeNx0qhNlMC6htHl0yNKGYFpAlNVzuQ4Fb16DqqVqp4Ee1U3RpXzXa10wLtHigdMlevrctg5q+o841UsOFWgy6PG5KeECvB7WR1e1avQ5X0x5l2ofqpgVhcWRtzmlUZjFtQbXG1GHINChRb+G4Ayy2Yz15goy1aYLeNnmccA9uNbUwjfFs0wxFYdq+piIJw2nnRmw/dEC02Sd7Nr420wJZBNmdCuBheIKsvEYWRz2sK14jmY57dHwKqmr11SpQ7KHRPlq+N2tk/cmmPDc1214w7jftE+jEXhd28Hmk30mhl04YsR7YNuiSNnV6C0xYwQ6eBjUtMF1686TYP7b7V12Zsxl6xHH2CjUtsDQOaedL66IqEuT+QDTcE94KpX7ETJ4Z+cxRJYmpTMVGNkbMV0XwJquXss/GNqBMAklfYDPhYI0sE4r+fNnM9hSNLHqSk86RNktXu4Uzf7Ug8bCI9ft5AAefD/vbz4RtXSRclZMSAu2RrUNQl0zB1PgI7Hj2j9C2/6/QNNENMNQBydQEJFCIFdtPAex9DhbCc14Rx8KPvPdL039TySpwalsBhvfDYZUNsH/RqdDc3Aj1Uw5Uj05ARXUdNDQeCCPthwBMnCzsd+U1sA3ENLuWPuChg5q05ZVBsU2CrLnjpe+d0CVRWjeboGSaNeN92LCoWUwyXIHND3aux++02eziW8dkWhAIti+P7QTt5ITZRtCZztQRm5xraOcqj4NfebbAmiZOXg+pNaFCtxhLPC2gk1McWVdjTgXZkMggWv50vZzwpWpVAAlqbvxB0eYERLeXNNRUCmcZ9PzudVShaN/AKOzoGRYJL3R7aCoAmGI060iveHdfOlkJZvyKi0kIwUmbdODEkuMMiKYc8epxXOgdoiYRYegaWde+2bFu5wmyk1PQqA2TeH1QoMG+ADfHBAomQZaaIEnhTwqlyjNOnCh94QZ8jWzaT0HfD8FHRkl4khZk3fi6/iQS5TC6vK+EE7TELzXGKRVjjNTIphQ7S1sd5XGU8Fvacr6tyVH/ABO6CU+gvpbv6eTSq4tc+SLlee2BaOjd42Yv4en32m4jC9418/xSaB9pKDsBwTLlmEWFd7FNehVV+Lp491yva3lIsizIFolA5xvSF0uNGG1TeugnOYgHMggFctBrhSeSMNy0CirXzHOn2ykHXk0cL37asLgZXtzVB3MrR+DI9hRA/RzXvGDXE7Bv0yMwPDYJLRP7oGXr3TDetAwmm5dDonc7VE70Qe1IhxBioaIaKiaHYOGO33qHbEz/9VJh3JsEaF0O88dHoQUqYaqqCeoGt8NkVQMMN66C0YYlooyKuhbY13YUVNScClDTBrUdL0DD4B6oGd4LU1WN0Nt+NIwl68Hp2Q3tQ53QPlYL4BwM0L7WPfHJMUgaXNmVQOqpEEHWcM/MpgpmrSYKK/jSU3F6gqwxE5eqjfGXK4mzl66R9YQp8zmhJkXvoEymAXRpnC5fusdQBXAphNIwY971IGWpodAS3jK/yEZHHRksWaFsYE1UG1m7QCrOJxXHRjZ4fLqMSbVZAdMC4nSlP5N0ezqwttfUCNMCKYxI0HbcrY+b3INCNcI221ObVzPV/OdiWmC6TW7Q/LRwhyZQMQZEOQm3OntpWj293mEE4kEbNJH0d5wQY7l4C/VY245mW2xzHqRaSNkGcHJDnbu877zytHo6vh+Cew2CxzHZRLse6u5zqwuLQoCRfYRBM+19Nl0fYSPra7UrQ7SvapmqFlIP8G83LTALWN7vRPFBV44ketuor6mA4bEpWNziJp2gdQlbkZMTG1sCiUyiFujt2GabTrfz7d/DNbIg2zm5vnIMwT6G7iHMAbX7YDNvmEwn3sB90CG11GBBtkjoXV9oJiKxtdvh0CUofdnb6B0deWRzuj5ExDDEGXjjHIC56QG0rg2gZQl0t50q0rviTHXk6G/AnMZqmNdYI2K6olaupudVWFndCwsPOQ2efehOaOjZCEvn1AntAjp01U4NQl3X89DS8wJUjXUD9GwF7CpoNNjKySFXIN7vf9cOvwB41H1/FMQEk0vMOQBg91Pi/F7XuAKGmg+A/rZDYLKqGZq29ABsOBNg56Ow9uVHoGuOLNmB6spTAKoPBxgbgKaujTBRtRBSyWpwklXG7GruXsHORlzd3h1u2uEqNY6xF5zcoH1QHKCIcBeqkfV+Cmp45Wdd620yI6C54qV2x2RLKjSyKX8A1X1gqC2mGu/Wj4crw8DZ4qNGCbXUjsytn31y6CYFMP9Il9tM6NpyfUmYDkp6EHi1wsE6zW2sDs1shb8dCqogazt/G0rYNBrKjlywzE0LgjvosZvjaLCkA47N2cu3LUx4y/kLmmuFOUYUsq1SQdbmlS+3R/t69AGQmmWTQoGWrUNXL6iGVDhoae3ZE8YCDkBuUgYTsj6m36XWVdjQSrt4xbSAXof4wqOwqyT9ldGxS6uOG8HDr5cpCYDdtCBdZkj7kVpKDLWISXKw7Ukttt4/H7dyjpiYoEKBnpN1RS79l5qauOekn6O1esFzUp65YGgvvy7pOpDYvYpGNsS0IEGur2cHj4Kso/d12n2wnBdes2d39okyTl03H0oNFmSLRAYKWcWORbGRJdvIZ1EXagKhniLGOrq57MB1hylxnPSDJZ298LiYlADzhKPpwo6pA6CzuQY2bRmAqQUnQ+eCk2HxAXNhaGQCtu3uFw8map7qqpJw0oJJgO7N0DNZDa/u6oCqiX4YblwODTAKFT2vQd3QLpisaoS2qS6o2fkXaO7BUAwOjNXOg7G6hTBWtwBqRvZCU88LQsM8XjMXJurnQxN2Vh0vAgx3ua/0dUYnOHwt2HmPf0KPfUf8mYuvHb/zv3/mX7y3GLOX4tTNAaishZPwEiQwIHpCaI+dlmUAFSMAw92wMJWExlQNtPW9ADDkpl5un7sONjQdAgOt66FjyZlQ07IE6vtfg6axvQBOAzipSmjumoKpynpwEgmYrG4V54Txk2FqEhIpB4Q86DhQP7gV6nf0A8xZJKJSiBBrsn7ptqIvvaGgYHP2cjX96UmT7DPTgh/VqlENILXDotphkyNXDUlmQrW63sBDBGZqYxopyKK9LWn6YdpJm42mPL7EdERqx0sFkIBTCEnwYNLYBZ7/RAIaa6O7Yxyo2xqqzZ7oMfI0CnHVoJF1bWTT32eokzVqZKkJCbbhsJBgaeR1sjp7ETOOo5e3iXbtHieRN40sbRtSkJUacnfQx52ok599UqTbQsrQcxj2UI+mIrVmARvSlHmSJxPYYJ9gEmSph7o0cfH9EnxNnNm0IKjtp+/pio8p1nMwfFz4faDH0vEEyxCNrSwD+/tDFreI6/XXzd1efZXrgiGoAhN/1XvftNIpTQvQ1CRXB6igaYGqFZXQibA0C7Al8pEkDA7Esm2J1TLFVt0UBk3XCrt/fafQ0kwGy4JskdA7vzDPW9mBii6UaIBMy94y7Iq/r6ksleVz640PtZcJJv0QU3SBuTE9w8XOXz4cNM4izoBxVitD1nixWIWR2SLxSg2OQd94r1dmVX0V9DUf4n2ebK2FPb2jsLylAlbOa4YHX+2xXjPU1Ihl2Mkx4XAGXa8CLD1WCJ5PP/VXqO95GVq6n4GKqRGoaWyFpq33ATTOh61L3gKNnc8Ks4aKqXGY2/sMJIc7XQG5th2qRvZjtD237iNuZ+kvUoGI6QsdflQNvLLe1UVHuWQlVHZtgsVp57p1T39F/F1LymhAzQFEs0gMSXSUqAKYfxC01C2AY/q7IFlZDTBvKawaGII1/TuhcmIAxtEkY+GBsBAaoXV0FMZr50LCcaC1cgzqhwYhVVknzn2yohFqu+fB/KFKqKpvgrFUEhJj/ZCqqIOh5tWQ3LcX6rsHoWlwHGqGuqGtaxtUDY9CxfLjYDCxEKaqGwCqGrw88l7WpMoEVI90iMlJZdVSqErOgeTkGEx27Ibq0WqoTw1AYmAvTNS0QdVECpr7J8CZvwFqO58X5iU4eRmvmweJlGsHPV47D+oHtkCycz8kncXiu4qJQUju2QHVA91QN9wmJgF4n8er2wCSFZ7WNYG236lJcc6yxxYTxJBxSl/29gUQ+Vz5Ws2wzt+0ItNQHd0d7+kbVQRZKlDFMS3Qw6aZIivEkDk1IjSy6SX6KORqkjUOLhUe0Xve84CPLlsKJJ6wbPDWd8vyC8P+ivYwYjJHhFj33HyhUEfef3n75zfXwM7uERFurb1JXZ7VQ/pRAYPayHrHBXeJ3xY3AYU1Kqy6Zclr4Tu9mZ29pEbWfyaojavvwW+e8gTMlsgBbClqbRpXf8nbfJ6OFhUH2x0tK0bTcOuDqz5pMwyMp+p+5/+OznOKqUngmsXH5vjmhGxHI3rI5AVGy4KEdn3J6ptw9iJNCduHJ/DKCYWlDt6krDTlWBZkS4Wwzpg+9GrUAl/olB11Q01FuMCslX3y2nZjiCpXkJ1UllUoetxZulQjHyaphcFO+4hlrenjqzNxKljrddVtC7086clqmHQX60WHPL+pRtQXHTBQeJapad2dagCWHee+0vQsroHOBa/3Pq9sr4emeY3ixLe81KF0vMeunAMtiUGAihp4Ye8YdPcNCuGxGsbg5PnjMDkxDk9s7YKEkxKCUcPAZmhJ9cGSRYuEGcburl4Y6OmEppVHwuINJwKkJqHvubugc+sLMHfvg9Da9aR7ThV1MNK4HBob6mFybAQmRvpFZAk8VuV4vyqwyuuMDlLJakjOXQ2JwX0AIz3CEa8G8JVm31/Bz50EwrEPup6GeYHSIBCWDXHvmpkDTV8+C3BA+i0m5kCwDz4NM745KUhWVMEbJob8a092XWUobjmEg06F6IyIrEtWw+rqFqge7RTXBmOAnES3TVQKzX5FVQ2kIAnrhzsg6UyJa59MjQEOYVBRBcnJETg9WSW+h9pGgGqciuDAkYJjnDoRlWOquhkmKhuhqr4VFna9DM3D2wDaV8G8htVQNZaCucObIVU/DyqnaqC5YhygYgwAnR+FnfkQ1A73w2G93aLdDLQcBHWNTVDxWhJWD1bBUEULTNS0Qqq6ASrGBmG8Zg4MthwITkW1mz1JNPZegJoWcCbHoL5/MzSO7oa64QpYtHs31A7vhdH6hTBR3QrJ1DhUj3aJSRh+1ziYguoeB+bt7YKG6gRUNc+FxpFaqK1oh8qxaqgZHoGKqjqA7i5h7gN1rQDNSwEmRwFwcrL3WYDdTwMsOASgaYGY4CWG66ECJ73i+U2IVYQapxqSU3hNEyK8Xxw8Z68YpgVxlQB2TaA5BirtcmQKT+846b+uTpYeP14GwnULmkT/h6l4u2UwZcuEiAoYpnCEYTbRUkjVo1hQDbHi9Bblqa4pTKhDozFCgWW1R0aEoOcZoXCNjGqAX+vZ8+JM6HSkrb4oS8s+Jt9LDT2+9HueiZY2GEfWHrdbbkdXdXCiV5OssCZEQJKm1TLRuFMWMxEIt5FljSwTZ3kzzFeaprPUDcU9QTb9UEQtT+oPoGlZTNrkSsWISZDVQz5JjSw9hp7DndbbVJ+gIKt+VpdL/GXbgxe7doMv7e0PCrIGpGZFIsYJ0rGrFUzbBYv9xoUwgVtU1DQALJgrdh7o9414++YdA8P1VbBkpSsW9uzuE1pkDAsmBAHUIh50PmxpOA22HHwpJKbG4dCFtfBsxyRUVFTAaQfNh+7+UWGP5FVhahyqxvugqaEG+kZSoq5HLGmCZ7d3wUR1C5xx6DK3EfRuA9i/CQY7d8DmwWqoS07C2oYR2Nw9DgPV82GqthUqR7pgpbMDBgf6YXQqCXUT3TDpJKG2oQkGJqug0hmHiokBSI4PQmtyBEYHe6E6NQIwNQETlQ1QNd4rHPES9XNgMoXe6ClwmpfCcP1iSI0OwNx9f4EERrZI2zh7906+SY2Dk0jCaN1CqBnvEUKjuO5VjZCcGBLaURS4qsZ63es1PgjVYz0wjuYVdfOFNrd6vFeUIe5JakIISiggY50xIQjiNKJWuAmSvVs9QTfpTIp90SiXtmjU1rq3Gl3A3ckb7iP2m+hXmoLZQjVN/zZogQeUbTDwnAlcWJaWZvN2/977fnVI8ZOV9ZCqbgZnaggSYwOiLTZMTcCJpC2HTTwodDtXj+2iWm/HAycucvJCOT39d6KqWQjm7sjsANyD9y5tc1JVC9C4AGDOKmjr2ASHpxpgaNHrAF7sBOjYKNodLDlKCNWtE5Owtmop1NTWA7zW4K62DHfB0rEKaJtKQsXkqLiX2DZQ0y7aTU2bMNFpSPUDjO6EGseBo1P1MFHVBPVNTbBhFIS9e6qiGiarmqCqeT5AzyKA/S/Bgv07oH4sBU7SnQDhRCQ1PgJNjQ1Q2dwGi/uTUO2MQcu2KVjT2QFNPc+LiYNbXo2wv8d227ytHZewRJ2WTAIkR6qgenQ/VJIJHQqd41ANjfNXADQ3Q83EFCzrGhbfY3/V7JkPOFAzsg8ah7bDVONCmBwbFn0DrlTUjuwVxxhtOxBgSw2s6u2GhWMTULNlGUBLCyzdsxsq+ndB48vzoWLeWljYXwVz9z0I1a8ugiVTC8X1wmvXsLsRoKkZ2vsqYKSyBTBJWGX/DnEdWvYmoRL6Yd3uHTBVUQ9TTYtgsHqemNAkp8bFZB6vQaonAcmxAeFbkKxqguX92H+mAIaWiPveNDgEiwcB2qY6AV6YgIWjVdDg1It7gNfafTVBorYFoH8Ckh27oX5gUkwuWzsfdydJuAJVWQNVbUugYaIWaqvrACoXQEWiAmqG90HCmYSqUQegs0coEAAn0w3tAA3zAEZ6AXBFraYJoGkhmThomlJso2lH4XoYhRGoEg5fckwUfR1q9yuqACaxY6lSBzX8DpUMTYvciD9jg1Ax0gOpqmZ335F+SKTcsJu6XEodCkUiioq0zfOUAzjcisdntFuYl+GkVayupdURNOGFCDXoCcO+sI+fKicGoW5wL0B3PUD9Ae75T40JvxVsD9C4BiBVD9C9BZq6d0JzTQJg1RlQarAgWyQC4mMMjaybECG9uTSaTwudUpOApgWywbv7hgvMUaGB8D1NT2sSZFEDQLcJ2uXSArVj0ffaYaiDBK2rWLb1spX4OzWQpdkmLRan7Zim5RPlN8OyGE0BadIG0WLkfaD7KnaYFdVQVd8CkOgJzH7pNricPlJVAZPYGaMgMm8BLEw1Qr00+8BC21aK18SScejY1iM8dNce0A47X94v7Oikx27rgibY1TsiQoHJ79C2GZ3w8LykM9gB8xvhtY5BaGuoEqmKpT00ajvP2LAQNu8dENqlle0N0DU4JsKeHb64AbpHUtCUGIZdu3YA7rJmQRO80jEs7GePX94Ef9oNMFHRCGvmNcDkaD/s7BqERH0bTI6Pw9zmOugdmRTHR+enroExWFrVBzvHm4RZgDj+FAoLlWLQrh3eDaP1i+H1By2CPTteg717dotB/cQjNkD/6CQ8tXmfEEgrUfgZ2CuEh7XtNTA0OgLbxlvEYFw93iOEYdSOLmuphIVz2+CxzfuhKjUMr1uajp+c9lh+aesuqJkagLqpQZga7oO6qQEYqp4LDSuOgiWwH/p2PAf9A0OQXHQIVIx2w/DQELS1zYG2lhZhMw1j/UJAH6uoh819uNQ3Cc09L0B9dQJamxphsHc/jA90QdVYD1RNDonJAwooKKxgvGfAl2xHU+Oe5nu0cRnU1jVCX6pa2I2jsIOmGE6iAiZq5ggbchyYElW1UFldB4NTlSJrYs3kgBjQ8VhCuExNCVOTBF7r1uUwPtjpCjgV1e7xUKu84iSA/RuFZlkImmkTGxto844vK2j2s+0hMRjhSsG8PVoinJ2udycO9ysMu7enX3HAR6/NIsTrNKVfNjbEmLAIXgUlYsvaGNuiudI6yJK9f1JXWLa4f5bIzzvcP57R1laA9YZiTN9lM+Gppqs3T/sTKdtkysQCGuUmAuyZ/fW2GLQsg6OnMLTGOFQnpkQ7P7miPi1odrgDk5OCI1EJUdMGycoaSFRWw/FQJ8yaKlLj7qrD1BhAy3Lx3IjVjIpK0eeI1Qw0K0tP8E8Rq0MV7iqbkxIT31MrG2CyugWgsQ1gfED4QqxL1MDqRLU7tjqTcPzYGIxXNUPtAw7AwB5Yk6yF9UN7lNU6544qIaivq26FNZOTUDveDUtGe0QfWfHoCqgc7oY3jI8LpQH8sgsOS/chkvWJJGwggatxRQWF/Xlj/eLZRN8TOPYVKDVYkC0SGTlheYKsGrVA/esXUF/txzUM04DaBVlXOEZwOcUkrFEbWdTGKsswEYHqKcmQ32QCBT0lrWJzSbS2MpA82vTqGmPlmNpxTBmwJNR5ie4nTStMt40uz8iYtorpheUYcjebs4vuBX/gAvMw61VTlpcuGCc5KLTiUpN//dy0oKojl3v//XqonvyJ9PFp3nrvHlVVw7pWHLqaYfNApTD3GGtsgtGhAUii0D23HZzODqyUKCdR2wyT+D1qnNB+uAJtrKf84+MyaMtiSHQP+8uQFdXe35Em1zghgROalqUwMNqq2ILiNlMV1cLJbLh+EeCCbmphC4wPT8B4OqzVSLV/HZ021EbVwlh9JUzg+S32h0/cp2u8WzwTOLig9l86La6e1wAwrxH6Vw6LyB1oo43XuWtwXISxa2ulltQ4Zk7CrlddB0QEI3q0LmyGof5ReC6tjcfVFZH1y3GgarxHaE8anEEYTVXCcMMyqB7rhsNWLoRHO5JQl3Z0eWpruFC5qLUWFrXUwZPbekSbxBiocsUA2xXWHW05D1vqXsc/vYAOiClY0NogVg6EHba2vPjizm7Y0zPkDcx4745d2QqPb+kWgnrNaIewz0YRA80OjlvVnn4IEiKKB/RuB+h+DcZbD4Adr70ATX2bYP6K9QDzD3YH/91PAcxZDb1jDvTtegXQT2peQ6Vbl4a50NHVA8Ojo0Jbl6pAcRcn/ElhZ1091iU0THVNc2DxqvVCY/bca9uFuc6ixgR09AwKDSK+8LvmVC+0Ov1iQthTtwI6B0bEOVRO9IvJy1iiBpoqU9CYGIbenm5IVNZAQ3MbdI4mYaBtg1ghEdp81GqN94kJycLaCairxutWAePjY9DVtR/Ga9qFiYoQFKQT1cQQzIduaKh0Iw10Dox58cHphBqPMdG2GprGO2FgskKYnuBKBE7gnEQltI1shYVzmmHQqYW9/ePQPL4P5tcnYfdwAgZrF8Pyhgmo6XwBBjp3Qvf8E2FB7TgM7t8pykFaagAak+PQ09UBlWM9eNdguH6puLcNdTVQ17YItg7XCtOn2pE9YpUE762IHY7a6GQVzJkzR2hT+1K1UD3RD/39fVCRrID51RiVA217K6FvoB8m6+fBwgWLoKOzU2SkxHaC7dz9m34/NQJTtSjgDUNyahQG2g52tY+pCaiYHIE61ERPjrh9B5rcOFNuXRIV4lVZide+0hVI0d9BCmr43KNA17dDTJIotdBnzEgitJRpqxDa+1agEIv0bXdfFCEIqw57aNJEwdUrsYI1vJvUwX1JsAehvYi0lB+tX+RdK7wmMLAb6sAvx/PdwBfZj670iEgWE8NiMo/g9cOVlJqxLjH5dipqYKxmDqQa5ovVpFKDBdkiEccJS6LYJBnDmKAXOxVkKzxBNqxcmy0R/dYmEApngnRYpAbNSUUXFKndmX5E+lmvDmpbTR6kKNBI73HqgdpSXyWEBio0mtCP44Wb8ux2AY5c3iaEMGoyoWhkNac6ihS4cH98CX82Yuqg3xNqekG9cHV0D3kbUuuOznpobiEd7qTwTXOye5MDYj8nzVeUTEAGW2Y/dAvxfqeBxtNvpY2f7+jhtlmsZ5V2M6QWHqGpFQ9Z0iKO8eJuu2ZPqaMWX5ROmMJCbNHq6PMamiFHtju5MqDfU5pxyBi1QHsSZN1p2/X2w9TGNXPEa6oyKSZqqeEJGK1cAuNow5roTTtAmc9JPw6NTkBDnPnRDGhFUZKqgIGxCYBKs2EFapekyY23W00TTFWlNcY1ra52fXBcaP4hLST7uJbMyakUbKl1TUPQxMZ73o74O/FnoHsYXpnnCtrzSBn70+Y7YWDWxMVL3Pp3TXWIZ6l2fiNs7xhUtlvSVgeti1z96lDPMGzbgwI4KBMLTCeOE49nNruTmoMWNcGm7b6TKl1FQpqXt0JdOvZmamIKXnil01rPauy/WutgcmIKnn+l03Pq1f3fcCVltK46EJMaGcYskSva0CsJtr7WJdrr/HXz4dX0ysyi1XOgtrYKHn1xn9i+dkkLPL/LF9xwUtPY3gBPb+oQfQeOJ9LxF891aVs9bN64LzTk1Elr2qF3ZBxe2OVHqME6zz8grTufSsHTm9x73b5uHmzd3mtMKoGcvm4e7OodhZf39osJQqpSFTvxHmA/K/wZ6lBgTcDv0+eG1+6M9USXi+Yoo/2u7TeaAWDq9n0vwsZ9Q9A/noA1i+dAVVW1u/JSXQmHb9iQXsOvg32Dk7DlleehuaYCljQlYPOOXTDesgpGk41QlRqBEw9aDrDnaTeLJkaQQaFQhH5c5ZoXVNYKjeqfN+0TS/evX9suTLSGB3vhqU1boWaqH45ZUOEK2BWVsHFHB4wMDcKq+Y3Q1tgIm/aPwEhvByxvb4A5i9fAKzv2wl5oh7H6heLU8NqcuNiB2tEu2L5jO3QOp6BuzkLonKiDOSNb4ODWtGkFCvVottXQDhv7a2DXIMAB8xpgVeMkbNy+D/aNJCBR1yr63xXVvbC2cQK2JpbCa11jou3TlYhSgQXZIhG0VQ3ZNv3XMXo8yr9+AXQwDLVJtQmy5GtTMHcq5OJyMxX29P31ugWF3BCNbNo+DM9cDxUihTNdSMAHLRJdkNUFt2RCpAzWoZfC08gaLqEUBGWEBrwfVOBWhHe0P1PiGNqDx4fGJaWnR+qJXtISaf7hJh+gGlniSENtnKknPy1fOqQQu2Iaxkk/UT8gd1IThINtEO+5nvsct0cNpy11qTyUl8fd0GTpNaMe+jo09bN+G6hgKu8ZjRrgHlsK96o3ddznv57Yo5uePZFulJSH4bhkBeJkuBKCkZ9OzRPOlcxehv1Qk58JIsYyEehwIoKmK+iYaQPbh9wH7zVq8IznoD3A4bnYZH3U++/aGgbbU+jqkeddrkYuMG2nhDkiv4f1p+622rFoNj/NZMyU+lhsly5EtiXcB0NS0edZnqswI9KkZK/mBuc6GjIqLMkFzSglw5fp1wHNmDAiw9CYXzcTT+3sc/tb1ExrQqw4P5oKWp9Q6sWi8y9muZSgDe6KE2Ak0QMDg+MwPrcZEpUV0D/Q7o6jTXP9XafGYLB1PUBtJcyf3whdk2vFGIjnN4W3Fctde5b5JJoXexMZNIvCJfpEsytgJxvaYaQJYAzLOMAXuvtGu2CwaRJWLG/FZU8Yr+yDzoZRmLuwCebMqYeh8SUwNuA7DaJdNjS3A8xdASMVa6G7e1j4ioyNTsLg3OUAq/1zkSRHcaI2jAtkQrgfr8f2MgZ1FUmYmJqCifpFAAubYarDndDZ2lyxKU0XtFlAJp0xDUCtb++lpCN3soGEywrz8LWFjaEdjh5miyIFWNSEqsdQt7PZh0Z5QUrTAllvea4oH4TlsY/CZlqgx/fTodcuzEZW3qW+tOOZniqX7oLnr2eTioqZGhX43varm21JFTxpWBe5czCyhG4CEhxs/YxjwUHPE2Q1QRN/DmhkhRbe/c4XstTyjOdMPLFV+/HgvlRoC5ZjfxZpIHNdODV5IPthb6K7Wc+bO5mA1x0wF45bPUcReuTKCL13SFdakBWxQS0THPqIuSmrZR39+0YzZNlsxqWNtI5JCKFpUBE8F5xkRl0LPb4ohcYytZ2fDVOMYVPihbBJt++rQIW84PMYrJ8qRIfVV49aQKHtQb++FD8NbMILnYgTEeoFT+ulR4nQU9TS85FlR82ZZNg9U72oCRwyND4Zqt3FyRrG3rVh8kPIFC/DlcGhWuKvwvgTGT36QhT6fqbfJPrEwwtPl+5PjVELErJ8dTXM9txR/xvx1/telTnknC/OZLkYsCBbJKJsV03b4rhus5GlDUzGmfQcgehxaLmWu0+30UPQUA5e3AzHr54TENTCnL10wS9MA0JjFia0h84PNp95E9bPyH9YDVpFWp9EPNMCWV7fiEWQJTXQU2zinlGCbFhAf1G+of4iHWG6IxwnKWTl9aPB0j2BhnRe9JD+QOh+Rs0jXaL2zxOUjlc1LUgLnwaNrDw+1cjS8vzrQM7ZJPAaBmA9o41bD/N28rr4phZ++fpKgH9If2DzNEWxNLIJZYKI9t50G2riQwPgCxtaOdmwtVtyoXALT+MszB/849O6I7rGTbZnHXM8y+yeTSmsmUJw6X2fJM7gqmhktYmSuh15H0jvTSacUsgzOKlG+T+EaWV1QYR+r/aV5nalT+7lGIDCon79/OugOZc60cJ9lNCIk6BgulezIgSdTvXwWtlEebJdjzjQpAR+vF1tkk3iHOtCflxB1tSGVe27YxV6vRTOIUkkktokRfYVtmtDw3TR+sk2JI8R1peVAmxaUCSCmlL7trLjQ4FAf8jkbrr245R184x6JWXQth2UDqAkE5MOzvKaDJ1yuEOXXkbYb752TsbN9TSympYvE/QOyouhFxH02eTshdAlVFkeliUdvXRBll5feo7uvtEa2aiB23RJsEyp/aS542mAeHkuMiGAsgJgEPx0jSuipInVNF/yXntaHYOmQJgWaMv2eufs1SMdsUPWSdeK2to6nSDJcqWpCjUtkNft0S3dUF9TCXPTE0QaSFyHTrZ0rXew7sH91O8SRkF2zLAkHqqhQxtbeVwtIYIcoHBf/36rEzsqyKKtqY45nqW9PmHI9mhKiuBYV7OiodfS12wF661PeJTjWA4UGqXF8Bn7Z1MmO7qtzNTmTSa1VQBdsKXQPhGXx9E2GZfvg8+T+zlgWiC1c972tH7mc/J+T9fZlGhB38cXsjGEFVif7bjkopGV14LG1U1YxmHsK3STorjoWlb3Pf3djbtNt5UTJTkxlMJpWOzapKYptq1c+gK8+1lvI/7qTO7XuJCwRrZI6M0hzLTACxRueMj0TB70oTMtJ9Dj2AdjIhSHaGRt6H1cRZYaWde0wP9NsZENERKiCDp7qbNRWydNNVTU5ELfHO8RhmrCOuL56ZpxenxZf9mR4iEiTQuyfGrltZYdoSvQaWYE3n9qQPcwrYwsT9gxGyYt/qRD2sj65egTEeHgp9WbbpKwvvdNC0waXH15XXkOLNpZpHt4XDi6oAc51abo2kbvvL1z9gc7Y2avOPGcE+EpR/VtbWOMLgD5tp7qAKVrl/R22C3tcTVMrRUPKc1vMoEu3waOY9HIKjac1kkofe9+MNnIhpsW+MKB1xUYTAv0+2ASZE3v08UZz4v2hfI3m8abauBl3zOIznpa/WwJKLzpjGc+ERwzbEI9tbnXJzj6daAa2UD47kRMfweyfZzEGDZoum1b4g3aV8k+T9HSx1DLmszX6Ht6zfQVKRndRmpkQ1PUJoL9arhJhZy8+KZG9Hs64S1FWJAtEsmMNLKJoEbWm7mby7OhLGlYGiX9NsxGNs4x3OOQ99pvSrIE7UeaeQT/0Fmimq0k0/qpx5ECm+7Jr0M1j2HL+1gMds5SIxLWwcpjqUvSuWlkMVg3mpdgqCVqv6wPXNSznw5ani0y0VArQqHW9qQdVrBefrul57qwpU54gGO+eX0iIhz8Asv25k7fs4VNf4WODc11VbAorTU0DcDyvWlpT/ymObZRYY4OLDQ6g3+mREgiWi6js5f22dSedOErbBBJEC2ejj5Z9O+5H73CDZAuny/3d+qshvtgm0bvcB1bqtf1C90IIhhJJC60r9Ox2bDrzQ7jH4t20Fpr1sgSxUBmzl7u3339o/Dinv6AKY63ncXsxNSn6v2XtBsVddHbpfYc2HwPaP8lTaBomlt/IgrG64D9K72npjFDvzaoqccMjjJRgGllSW+/0ikZba91rTCeK7abwzDNeAxytd1UTQvkc65uI5MSqJnY/N/jmBeY7G9pe6FlyPeyHcgVtclQG9lE+m/MCEUJ3S5a3V7ewrAILKUAmxYUixgaGZNpgf4gUC/SWIeNYVpAO0waRzUugaQLVPiIkZDBO3YFOkJ5O/rnKEwL/EE28/qpyP5A9qU2wdMmNOvLYNgpiOUyi52yaVD1T82c9pESGbUgkYCjMfwOAHQOjsHT23thTmN1QPgS6SaTQccuv3Pzz8+0FOZFF7DcC2rbTX9f0lonXvS41NnLZCvtn1ywfLk5avOOW+Un2qWl6KYF6uBMy1SPTbVVVLDTndTkbomYDih6GzNtpmjkiMmFuTy//mjBR9HtlumkCTOziW00AZdOZFCzim0fnRfRuUzeuygbWXQ0OuGAoKd0GJ7gRcxfyJGUczVObiABq9obxOuVfQMWZz+wCsuKIBti+0od32yOf7bPdJULTUZktloMIyWds4J1Vp/BhCZUYDlSWNUnbeJcyXOmC6OYrpeCH21RGWw2shiVAnlye0+6zGA/pl8HrDP2Sdh/6Fphb3yL2b+blCcG6xQrvpIk6PBEwedgEr35vVCImY0/3mRYbzPp+tJrptvTytUKeWzjcweyPK3NWVYudedKGobRPYZqZhT3fkw3LMgWicBsL2RbmjPba7vaIJ5VfumIWVq2GtlknqIW0DBJQpPkBeD3paysbGQjEiLYrsuytjqhlcJQUApep+sPAlIjK5fPlM0TwXsr4wGLsE2WXPO2+ofR3lgjHPJQU6JrXnRNqzyVREADGVzGN10n/XNgac7QmUovbi+dMTp7RWgmZXrhqLBLJk2SvzQaHJzle/obFbT90EeGcwVdOAjv+IPnmIiY8AQFWfxZX/6VoaXEpCVtpqI/f1QL42l9DOG3PME9kRD2wUKQHRwLCLKmZeFsl3mlLTB1aJPY7BKVz4Z7LusksS2pi30sWnwbuHmUc1cypM+jtvaYQc+2n25aoK8KYD8tBVnaJ3oOXVKDGCN6Q8DxihzXN98hv9MyyaqZLkgaVwu0dOES2RqjIrSETR70CV0YVKDThTldkB2BKbNGNsZxrKsKaWWIbOOuk6m6rScHpC+saSUkIeWBMGWARYCnf+WxykUjy6YF5ebsZfHkjjtuhNmAmQbv7EwL9NmmXZCNGiyoxpmaFtBQSJnXD0KdvWydJ86IUfMggrqH1BWRjl4YBDxwfC1qgfsdeJ2TF8oqwkA/LpiuVyawUM7HoP2kQoivETBrZAP2zhEDuj1cUDJCYCPtx1C+TWgyhd+ipiqmeunCPRX+vcFLxJHVbGTlZCbimvjnFP7ZrT+ELiXTFYKErjEj2nbFfIcciI6D1F5aj6uMg9rcdEB/1MjqA6j+OZdlXk+QNWhLPbPUEI0n/cXkeCi+D+kz4sSR1TEtQYftZzMt0GNXK/bmukZWi1qgpAyn56p5n5smcPqqiLABttTfD51H6xK8trp2UfxmuH5RTpPZamQzVexQAdwW5o0KeHLFgG4Tx0bWFtqLTi7dv/5vcltcraL9kOloCcNEI0xhJL/WQ675pgWqkqdUNbIsyBYJZbU0QoPhefJiIPuAjWx64Ir7wJP3FTnagtoIaiTsg4NtyUMvy9V8QDCTVCIPGlnN2SvTIhOG6xlbI+s5QPkzYCk82SYR2Rrc67NpLD54rqpTnayvafndFDpLLUm715bOVC7Tu6G3gudG24hy72TbMJaqnpvUfMlrao0xqx2fagYnQ2xkvboElqNtdYsWknQTFH3JmN5PunKh/x5HIFPMLTQtDG6PmelEAP0px8vyhL+jzaxJI5st8l6bNLK259O0WiDPSWKbuIRtpx/H1HwTEWYi2E4Cy7zEtID+1FZfba0XjaltmthQQZYKLcHJZfD66CYWehgo0/6quVjwdxR89PBtpmfbOulJfx1XA6iXc9iSVmEWF9c+m0YtCBsHdGFSDckWjc2RzDaJpPfJ05J6USGkcAmB+pr8TUzokxl5WNlPyvOcLHGNLJsWFAk6kEZ1/L63YiqwtING9tjY9E7QelylIzNvE+VslMkxwjo9tw5Rgqx/vtS2T585Zla/iDiyGZYpzQICAkPSdbwKwzctSH+RDt1Fl7HyFbVAxm2lAopJKNC91nUvf3pPKFFOODYNs9dRa0K9KWav0llnIDE1VFcKxxFpg6iWYy+TDvLe8qwhhqefoEQl7kTQdC70K7x2gRBimu2r/F6WJzdXNc7Be+6HQ0s/X6AvJ7oh4tA5B9OI4iQN3z+xrceYVjQX73F5rzMyLQjObQRqBA1y7QwrRqZnX99OCvDqsf3nV18KRmoN8aapFhZXS7xttb5Cb5dK3GQtcgYKHrIOSprokImVbh8pEVpJ7bO3jzZhCrw3hLHSjxf1HS0ztoJG2wydXE85kGTwioAK4GHRa3Rnr0xbus2RTF/iV30WEkr6bixCd/xOYcrvkH7MaiOrCdDylsmII/gcilXCEHOLUoAF2SKhdr7hjYPOApPpjlHuQx1nYh2XvLeaFkTYaEYeI0STEerIE1KW8KZPv6fLP9k8WPr1lhoIU3aqTNDrgh7IpkFd0chKQTb9eWzKDdul28+p+2ffmVBB1g11pQtkBtMCi6ZKP9/gcrv+u7ne8hrocWbldzYb60QG1wM1pfOJbbMiJFpsSAOCLNXIaudiNS0IaUuq8GP+3StHO6aukdUnF1TDGnT2sgiy2jKjb+frft9QUyEE14GxSZiPKZgtCRJyUdrINk+TdkhMy+P6Z/2aGb/XI2Uk/Ti7Ydr59oYa6B40hyATttsGAYg6b5kEWYzccdSKtminUC9dt/+bvMdCeE1n8UINOe03wjSyvkOPelzH4j0v9pFtxSI0yeO5cZQhhmlB4Cu3fEPZYeQqYMlnwB0H7GVSMz99G6oAQFtyNBXRQ2DahGT5zARj+Ab7E7TtphOFivR3qrLBfH46ehuQ9ZMTSvyIZhR+KMHSFGTZtKBIqMth4dvSJQVvJpi1Vs5/X6iYcGF2grqGyh4LUZ35JywdbzanYDokdl7ZpuHzBSq17EaDWYG7fbDjl4NxbzqtLdrW2qIkZGNOYQ53Zrbv8yYMNOuTYQA0aaDVstTPNlMJ2Qak1oDup2dQUzprzb4YYmrqAuVomk56rlQzqNrImoWpwCQu5F6pfUBwO71e6lKzamrgTS7S3+GllnXUTTNswrb8Vj5f/nKie3+aalztIWpkxybN6WqjzjkK2UZoXxetzbK1D/Mzo0ecsDl46drr1oYq4Tg5t9Ff/fJ+t/TnphUZ+hzgsVHg0bWxgXoFsuup9cW/GKnkdavnKv2GyUHQbmqUIBrZcGWGyWZevJdaXkOKWlOzsE1CTfbdceuTDf4Su92OlT5LfoQb/zd5zTbtHYBnd/bBxj1+1AxvG5uzl6aRtZk3qDHltbadgIzHWapBd88hXSb2celzpSH3OI4sE2JaEN445JKC6jmdXYOyDeD5JGrJmWLrqHRtkjsAB7fL5sEy1QfNNrIN+kz7EXp9UdsStj3tYOQxMfC+zAZmd47K/r7pWr2AdsuydKsOVuZ6hGlksVO05fsOMy3Qg+qrQotfv0zbus2cIGAjq5gW+HbZenpd+S4jjWyIBkX8Tr7TBVlXIxu8nnTpd3V7IyyfW684J+oaZ1pn2RbkYOavUPgaWWRwbBKGx+yCbC69ishgFWIn65avt9mgQB92X03Z5Ezb6Z8TaVMAao9qkGNV0wJDUggUhuSkN0xQC3tWdd8IPCZO+mgcWrmdzYEtsKpAtJJUBjWFA1T2NVxz6v1vO574zvLwes9TbEEWcoKaWdicCk1th8YRlpdpd++IF29YT7xBTQZM9ddtZG323KqNbCJdF1qe1jdHZfbSjotFygkXJveRx85lNbCQsCBbJEyz6zCCD1CWaINjGNkKTPrAEtYZ2X7zO2i7Jkl4mGfxYPmdpN+h/PnlTtjTO5pTp0ht7RBbdAOlw0nf1ra00Cvt8FCQtdUjF89R2qHpWh6b46H+nc1GVpdTaTEmjZNEapAC9sIGG0PTYG4zzQlb9YhyCtInjhRdG2YqX9/WXMHwZy1cIxu0maTl4F+0EzxwQVMguoF+f6WWSV/e9GPhJgMB7FGYtZ5WjoNdTbq+uiBr8ya3yFTqSpA2qVL3jxbuKKYJhMlO1KaRldFPDl7cEvos6xMZPWoBrW9YX22L2BBwCDKseOFXJpcJ20SQlqE7e9mcDE0oz2eM7I25tjkqSIY6e+kOrYaJIeXVjkF4YNN+LyuefG70/tCP9pAWKC2rg/L4KBzry/0Jw32wfQ4c14v75R9XTthkzORSjViAsCBbJNSQQtENRF8Oy1rIjCHIYvB8ZNmc+pyPkW3j17VEpg4j287LLzMoHNNjxy7P4OiDna++LK4fH5FCxgIthz0Ksrbzy8W0gIa6Ms2wXccuFV0b7gt7qjBFy6bbRQmycrCXf2NrZDWhU4d+bbIFDp5PTHvb9DZhUQP8skMElYjt9GVg3dmLClR6HFtdk6jXUylbi+Sg28vJ4+I9RCEQv+5IrxyYyHW8s4XgsiYKUW+0Xw96f5L26BmqrTGEtDcpNNLv1L/B9mtu9xiLOhCPOkIjaxIevUlgmGZXEejN34vfaJzuNLiJOV6peZKgev+r+5hWt23PhzJOxXomISdkW6Hxb402siFxz3VwDNjaOeSZG+B1xAQ1SDsxT8nEtCAswUVCKU/dxxrPWgrwqeAzJtuWVK6Uqn0sws5eRcI0IIfhDjbpBlWRCBUMwogaPJFDl7QIW00Mgp7VMTLshOKEeZF2mjR4frZle9qMRAKcRNBrN9NiTYMZOobYwI4FJwnYacj72FxbJRw+sNPADrCxphI6kzbHEsgaJVi6wYNdPw/3eLo2SH9vC4UWvsQqWdpWJxxVZBxNWo7u8GYSGGyXw+YERPcV7zV7bLm9TW4yacHkrkEtdYggS66d8T5QW2qDaYGqXVP/2paTaVuV56fbJsvFVS8hAtFCYbvEvqEnrWEykavmxpYUwZawRBXa1WsURyPrrf4k40VcUSdssg3629F2ExW1JAxdU2zyDcBnB3/DlNQ21HYabDP68ahpAZ6Xk8GKorf0TTzdTedjqptt2zhKm5ydvRKGJXtDkaYlej2ZgQSX5kfSTotYZv/opFjlwDJQUaEcn1w3ub3pvOTkdYI8GxWGyaneV0Sdtzx3asMrn0NpI1uq9rEIC7JFImyQNUGXNGjIllyOa2uYOBOzLYvHIdNOyFyG+xe1FphcYElbXSAdbK7RBXB/k5YnLNtY1LXFAQUH+VXzGkK3XbewKfAdamW37B8SQq2rLSZ1JgJ8LoKCPrgHnQ4AEo7hO4vGiWbm0ZcAaTVt2mmxXTIRsOOU6BM2Y3IEy+WwaSj0H6WWJW57kptRcx+5Z0AQil1m+HYm0wLVRjdhdJxz6xQ8Dn2OdKdCPfwaHexwAJYOiYFzSLfRXIc7OYDqaWqlYBQMfRY9iU7GcHIya8WD19E0gaZFUkc4ak+baz+qBsl3f1vUUideYdgEet35Ul4XN02rH+tMF0j1Mk3tC/eJlxDBXGe6ZTxBFnJCcXSLEbXAqyduQiaA1EGRtrPJdCQDBCfsttjDeN2e2dEL+9MrHjYbWXqcCk8ja5m4hZmdkO2kj4g8L8+0gAVZJgypFYllWkAeIAxOni1hS0v5Isx5RtnO0ImtbG8QyzFS0ENN3aFL3TzeiE2gyqx+flkYgErSXFclbFXnZyjEe0IMABy+rFV0MtlozFegljbliNjAwYEsCSnpjJbDfdM1snrfJFJdOqoAIa6TRSgw2cf5+8XTyOpQQT2gkQWbUBZE1DuZDk9kGThkhIjV8xo8G1C/TLNKNixjVnBp2n6v6JKtaTv6O9Zfv3eqaYP7F0PxoQC6pLU+EInC7WvU7WnfQqOCuOGT1PBbCK4kbOsaNp6PH88yt37FFkvW0xDrgmyM5Wl9RQEnXTLMYEN1pYjVbepXjbbLBtMC2gRH06lic0Xvq6cSfnvI5BJT2UsR6DWhzOTshd+YbWTV66l/T73qTfuY9rVta7ejtUcByBRfAA9PjBMQZEnv85dXu5SIFhTs16WdrMySZzo+mtNIIdb9Xj++v52+b0Jp7/77uPbTdGVSaGTT5ypTH7MgyxjBxoJLCfFMC/KjkVWOXyALaVXYCDm+4cTXzG+EVe0NEUtO9qWfePVLeIMDHXIWNNfAirkN2RSoeAhXJLNbTsSBZe2CJmMnU2WJdZkpulZPFwJxFq4nxMBtbMKrKXe7vx2EBoaPg67JtS2Tm8D7gU41ODgFnCU1ze7qeY1ZafSDdbFfA52oMHKKBk6zcwuE30r/xYmfSdvvCbJKPVUNJ626cNbRwm8hOEGb31wDHf1BG1kZz7JQNrKm+gTMtLT6+NuolUIt/FR62RczbZ28pt1Yb7XsRCxnr7DQZJmgh1ujz2UmkwVqCkHrqTsueY5a2v5mG9lwodPoJGkYb9TQjOawVsmQbcLSyWYCvc9SM2kzhbDVE+kicYbp84ttVzp66WYF9Fj6dbOt8GCWT91hOaHtJ+tpixZDy8T6Ke3LsJpQKMVXPmBnr2KSiC+Y0M6zKSeNbPRMN1fi5jW3nXfcfXJ1eNM1kmHL37HKhfxim1XnMgGh7ciU5Qmvga51wEPT7aIGMW87xUY2/rWl9l96Z2qrh435TbWehlutW3g7tIWrcbcPbmONXxrSRmmsTpNgoi/NKhrZQKKIRGwHR7e+PtRhSBYzkcLMQWbb54MWNovQcnoQf1PWp2yo0bylJSYNsXsu5uuA5yMTaujLq/q9s4UWUhzGEgaNrPYXkUv90mk2W3QBzqZxi8KWGEI3LaChmGgIqkiNrPK9334C9TBUWm/Dpsmp3d7b/PxlAz1GVIjLxnQ8ZVlP27OnP7+y3FqDuYnJZCDURtbLLEbMwxLRTqkm5O40ERK2e719lLJGlgXZImJaErBBG7gpC0wuxy8EfqefuSAbhmo3ml39PeceTctly6QVhe4xni/oAKSEzcrFRlYLv6VXGQVHveNzHe3Uz6a66AIPtbnK5Nrqnblel6i0sHEwaXYpYdny5L6mpX33vXng1bE5k9niECsTRD2ObcRFSBjMaiS+s5cf6xc1PiZnL9lGjlk5R1k9ENsZNLvZ0Jg28Rgen1SWO20aWRqRQL8PmCTg6JVtgb5CTRJhr4tRK2cQpmj5q9sbhInRYUt8k6h8KATimmwFyyHvtborMXTT20nTErGNJTtZVGxaKRRRAdwYIUZbVfBlMnVCEnXcXFcXaQQW2e5tlxhX7pR9LWXqTsRh8bTlV0FBVttOpshNT/ZdQRrS9UgY22nUeCEF1p7hcSVSDJrbUNB3o1RhQbaI6INLGFQQyFdQ4kLOsPSoA8Ztsmh9YbHy4tKQ7pgbqitVjWwGdpxqndS/+ULVLpmXBzNF1+rpZWE707WRuIltEFUGMW0/6qyTyaRDalBNCSUUgTp9SbKKJUwFHsNQtDwk9JxvWpC0lBdTIxshyGJw+xMOmAtvSOeMD7WRjSEc4j5y1UERFMjAKsP8YZxK6eRje84ycZrKBNTe47OI12dg1DWooTa7Qc2/+b00wzINwIowHFJd07mrYc+CRUjnxagl3UxXZKImXzbC+g6qdVMcnshxDl7cLJxuj105x3h8dVIU1PKiudiKufXGVRl9UuBPDNQyTNvbzJ2yhdr3uvUxb0fDptmSdiB6HF0kKiSjNBnQ66Q/n9LshjoFJ7T66iHabDSmV3h3p+Ooy+dF18iiSVGpwjayRcRveNEPIQ6s+IBl6ogUbpdXOEGWLt/byOb4tiWnTGitr4aT17YLoe2JbT2RKVSLBe2gwuz9MkHXoOolobYt6JmrBuFRBxBStlYvk0YiDih84P0x3Q9PC5tQ3+eEYX8UQg5a1CTiP6JAiWlZA6YFiiCp35/cMvBJqAOa7omsLnuHH+fI5W1CQy5NNeh9o9dZaGHGp8RAaRMcJYEoFXnSyMrBdP/EGPSPTIrnlbYl04qBJO6x4z5PJoHNNAEvRFeqxA3WophkspRuc/ai95seD680Db+FAih1uHXLMV8/0+QmzO9AFVjNzlu2JA5RmfEyRT63MhOXrV1QgXxobMp67+lSfZSyRF57mgXMVAfdBtm93gnL+aj7hD1raPMuw2xJwZauWuAqcLYhP6eD0hq5ZxmZmBbgwIpZerBTzwX0Hhfe+Q3VBdXI2gLEU7I5frZaCR18KLGO1KEkWw2KrEbeTQuIhitfts3BqAXajL/CZFqgnZtFE6LXC532cKBEjUw298ekxfW1sGYnh7jYojBQlrbVw6nr5sPi1trYcWT19/l0qFQmIRWZaWSpNhahQyzVwMsypaeyflylTG1pNyqKRCZgH4X0pzWyVEumtwu1Ccc7tp7tLBNTljhOfvlAD5VIzzuTLsBmV6pfBxq1wIu+lcjM7hb7UKX9R1wXNU6v3x/RvWwxURXBPg/XX5YdJxLCEctbRd+2dkFjRu3d5vRKoxYo32v9h25Wg+N5MmGurym6Stizpvvg0GvanqMCbVYIstdddx2sXLkSamtr4fjjj4dHH3001n4///nPxcU+77zzoBwplAAUesxEAo5bNUfYjhWSOLPBXG1k8yGIm7xrM8WULSkf0HJ1zUW26JooRcOdFmhMHv625b2wwNuoTXzD2nYRUi1fmAaNbK67KniGT7b0yZPnRKE4DMULSZYLlWFxZDM8DPVQpuVKwUZqZ/TVAIoex9brz/IwqshQWP0jriAbph3OZnKr2IbGNH8KW2UqRA9OJ0z6M5hJ36maFqi/Ua0bTS9LHRFNhNVFEY4j6hl8tuT7aOHVNqHOloAgGFJke2MNnHLgPPGXgiseJpOoqDCEnn2uNh7pPnO62RdOThOW8ccUXcWE7jxOTXEwHjqeDyolSpmiC7K33HILXHbZZXDllVfCk08+CYcffjicffbZ0NHREbrf1q1b4TOf+Qy8/vWvh3LF5EE8U/AdoIK/YQ74KIcaa7l5FhLCHIsyJd+CLHZ6WCZmB/LilBqyD2UCHdRcL/VEYIlZD8uD91B1qAkOYjaBJ9+aKs/cQvVVz6KczLSZ0fZ65vLyueqB1xIdb7BMvFf0+FH2tjpTSvBzkyCbXl4NqT8VgFHw8dpoHu651BJhpjsUqm1ZvXTiHpmmGk3k4Owl5wOF0MjqygBVA5lBORZ7U5uNLF7qVzsG3eNYyqT9gX5L1ElCVN3U58a0Sml/zuIfJw5hUVfCoHVFcyR0hLRhS1lsa9d6KDd9O6qRTVjKjLKRxd8xlrZ4X5lUTAgOmNcozieqjGJT9Npde+21cPHFF8MHPvAB2LBhA/zgBz+A+vp6uPHGG637YPy/Cy64AL70pS/B6tWroVzxbK6mUSM7XXjLHYYH9KjlbXDsqjnGsEhxy7WVnSmZCgAm/CXu/N5HnG2/bvVcYd9oWz7KvMyk6KxQINKvnxRMAl7hNMRL4B7Iv9PThr1qKNqbLMoxlWlB9Y4mWlG6PK0IGebt8wGupBy/eo44Nh3UTBnqwjBER1KEkNH0ABo2WVSOjxm9vDYKOYMDp9QUPbmtB4bH7NmFwpbObSja9JB9TApAWgcvJFgBmr/7jPp20tmcZ2Q8XdKGsW+QfbLM3ubkaK4QNfEIhN+S7+mxDOHsqPbW3T6Rf0E2pnRE94rKEmgKvRXWr+vZ7XQzAaGRhYTVrwbvpy1Jg0kLm0toz1kryI6Pj8MTTzwBZ555pl+hZFJ8fvjhh637XX311TB//nz4+7//+8hjjI2NQX9/v/IqRxvZskNqE0wajWQw13Rcsl1esyG1w2HLQVHIWhRiMMNBDJer5GCRq4YPBasTVs+F41cFtQYy3EogBaiwkaWfg1rJKDuswqxi0P8zLYe8jyjBZtahht+yCPp5frhRW4LOZ/oxMxVkbVRrpgVh9zWY/MEkhmTPYUtbhTCHWtnN+wetyQjUewlZCLL27dQUtYnAd36Yqvy3fxRSTlrTDkcuaw3UM5PHLcwERQmxlACRQOQw4tgl20GgTEt7D3suosoRbw0TdlPUBb3YfDxmwZWomIXSPiFdhm0Ca9PI2o6lR9zQ+390Hkt4E3r1N4ycgfczjjZVRmKgERnKiaIKsp2dnUK7umDBAuV7/Lx3717jPg8++CD8+Mc/hhtuuCHWMa655hpoaWnxXsuWLYNSYUYKsBna52Rbrig7Dxfw0CUtIj2p7pVbavczTjizuIhYsYbOTQox2AkHbGkty3iegD1Njdl0lOzCb8UTZMLanDvBCNqt5dv8JQ5ZBogIIM9FOnvpsSTDqK5M5DX6BwqxMgwaCrP5tJGlpgWhNrIRQrLUyBbqNqMw68W9znISrzrl6dpTcu3SZzi/uVZo/rF9Y0piE7YViEw1snoUNJOgapoY6jkJ87HyEZYxLi5+umfIUCOrfm5rqBLjEjqTUWR6ZdVpGdK/Qdag0Hv6QfOzMvcrBcpKjzwwMADve9/7hBDb3t4ea5/LL79c2OBKUCNbKsJsvpaLSxHfpjO/56Z0cHkYL7EjyCQ9aXidCncfPTuoArYV3YOZDtLqwEIENTnITpNG1h/oSBagLMta1Forlu6ikjWYAuDL64VaQ5NNsfd+mlQF2YY60+una2/QPjsuy+c0iOdpYR61OoE0mRGOVnE1ozZHvUDZEUKytN2dji4824gtYUlL6KSDXlqMavP6te3W/iYs6ofq7BVeN1oOXslEhOmCdy5aRq1kMTWyjsFBz7QSaUmGQPej2cNs4xJqvGVoL+y7ksoqVfZMVx8+4wRZFEYrKipg3759yvf4eeHChYHtX3vtNeHkde6553rfpdLGXpWVlbBp0yY44IADlH1qamrEqzQxL5PMBOQ55Vsjla8wVOV2Hwul4Q7zYJ4gx3YsgqyszvSZFqT/5qHjPnhxPC18WNgj1GQE61gMjWx2gixN6qBrKsGS1SlM6MSQZflE1+6a2pkpjWwUcROMRDVraWuMwvue3tGMrlemZGuyYvX618OQaWWGTZrD6hI3IoSom2KmEYxNrB9LVlc8/1leDxvZhl80RQAxCYW20Fumax1WFWy7Y5ASk1B8PuY0VAsN7uIy1aaWvWlBdXU1HH300XD//fcrgil+PuGEEwLbH3TQQfDcc8/B008/7b3e+ta3wmmnnSbel4qmNS6+c0SpCGT5Q6bRld6Q+YJequkSEkrhPnoTgwI8sTIANnW+U0MrhcRJLZDm3QZdeixU2LPgMf33cSYS+R5g45CJ5jRMMKxKmwfkWu50amSRTC8zFbbCQk1FOVjJCcTcxho4bvUcEdqwUODxsV/FumfiRa47VFGy9UYPs7tVtbzhN4b2G2hv7GlkLQ5qyooMKScfj1lYoo24qyEmXwb0v8C+FU0FbFRkcGzZdmXordqqCjh6xZyytW+dEaYFuOx/0UUXwTHHHAPHHXccfOc734GhoSERxQC58MILYcmSJcLWFePMHnLIIcr+ra2uIbz+fTkwk5291i9sFqE78p0NJKxTLjaFrI0ccEzOLrmCqScxDBm9V3o2HSdl7mTRGQE/Zuu8lw8K4WiTS5ujz3WhBXy0Zdw/OBaaPSkMfQDV25eckBYLXdCyaf7xmptSgtqozCKzlwl6zOnIRY+CMspNmazMhJsB+F9g5re4SNMePH39maCThEzqKUwLDMvkpolhogB9QDC9bvZmPfSaY7nodBWGfqyw6yZ/izKJmk0UXZB997vfDfv374crrrhCOHgdccQRcPfdd3sOYNu3bxeRDGYisqmWljiWH3AAr03mfxCk16pUTAu8+1jA6mDg7QPmNxqXsnMFr2OFdq90RQ1ug8tXGHaXDn4Y9QEDg+eaVz4uXhpUMnQVXiNrFwRMmLRKodvnUH+0ZcRXpmCWwM2dg3Dw4uZQO8GoiSh2zRnIPxmD9ZECk3s8y8XKcFUEBabDlrWIAPRh5xg1cUnly8suJtk8Z2Hht9QIDJmVKycPwagF1Nkrfnmo3Z7bUA0DoxPKxJjeH3VFhtYFimcjm4dVnGAyBvs+0vynlFPGzjpBFrn00kvFy8QDDzwQuu/NN98M5QpdJmHiUQz7wyh8LULh6oOd4XRmVzF1pLh8ZWK6hFiECq9y2Vlffs43mdplZ5oYoBjNePncelg2p85oF4kaNRTwcLyM0vq4+xdOmMPyUaM1no6nGaqRzbAe85uil2Kjbneeop4VlCjt84q59dA9NB7IUpWtSZUSCSGDxo3XErMAYn3ofiiwHb7MdaocGJ2kNSiojWwuRYZNHqK2F/uwRrb8BNnZiue4UhryWFlQDI/wuMyk+1gIE4Z8QAcFdLBBoautPnONZCEHpUwdHQttGmE9rqV+KDhOTk2lQ/uE160iCwEyU3CiIgVZq42s/JvnS1mse5NPRHrbtObcFOR/7YKmrMrFe4ETHv2aK+G3Mrgh0t7Y1ObkStTg2KS/ImMwOcgFVQDPTcGkxME1OLCFbR913Ra11MLw+GRWCYVmKizIFhHqgc2Ub9QC2emUSn3yQYnKscozg1qLOFq16fYW9zRVca9hojQ1eDLxQhjT4cxGNe62CVah/A1myiON1208lcprdBEa0zXbqAWUONMhm4Nn3CxcYdC2lU07UsKc5ayRtW+LpkTHNBTOqbAcYUG2iKBzwN7EKDRNg5PATCHfmb3ytUyL3t6FsF8tFqUqlHu3PFEcjVYc4TTT5BWldqVlEoQ4EQumo5koXvDJqAlOfitUa3F2Q9vwvuEJL+JHqYM20UPjk7EmJ5mbFoQkm8hAwJRZ0kKPSf7Sw+bFtICcSDZ23zYnt7h9KbU3LxWzuXKhPJ7CGQpmTcFlgum0MSx3ihHaKAp0TCim134hKNWOVAoq073kK21B47S5TGP+ZhsCqVBI27uGGKHzhNCbzrpVTI1socy0UNmwbmFTID4spnHd1TNSNrE7C7EMjTa1e6dGA8K8yAxY4QbtzyhqQQyVrPf8FSAhQq4RRmxa2LjF4j6ptF66VBUJpQoLskWGhdjsOxt+2AsHxigsRfwMZ9N7XGkLmolpQZSNHQpDr3YMwiF5SI+cTzBsHgpwi1qihbQNi5rhmR29sCokRmauxEl56k9w8o8pTSs+H7lmBCx3UMu7dn6jsZ2jVhbTz2dkWhBDkKWa90SJ2TKrSTb87+NeA3c7FmSzgQVZpqxQl5OKWJEZzpK2OugcGoP2htIyl0DNN9qIzZ9mMw4/EkGcbSHWYIQ57fFVaqCTl0l4M9FQUwknromXLjwfGtmohAglupAwY7E7DCZgdCL/q2Y2p75SuO805nI2pgXZ7MO4sCDLlBV+JikOW1ZIsCM9ankblOIKBiYBmG6ksiXeAJOZjSwT30Y2LPyWC1/zUgDDaHX0j4n0qXnFcnuLaWZ2xPJW2N07IjTUpvrEFUpL0WyuXGBBlikrZJ/ADzoznWQSG9Zro2w1lHdB1iYUFMvkhDGD6VILkTJVLt8H08nmp3xsX6ZMXVG2wnoM3kyjFoh9StCRuVzgrpYpK0y5rBmm0Mj2FschpLW+Wji75F0bNUtRTAssAzyHMixfZGa6OY3Rz0tbfZXIcLh2fpMSritfgl++UkqzacH0whpZpqxgjSxTDDKxkUUB9tQD57HpS56orUoKYQc1szZBw48vyte83ECnx719o7EiK+D9lRkOU52+KJuv246a3ok8lEPrEzsMXwnGSC8XWJBlygqZX5rzTDPTyZLWOhHnMm4WMRao8gdeyyi7aLaQLV8wKkVc58JCP2/5UpAoZgIx172l8MpCbOawIMuUFeglfdzqOVBbouGhmJkJaos4JWTpUqjMXkzpEieBQqbkS4jMKiGCZ1vPjThT2EaWKTswxiW1m2MYZnZTqMxeTOlSADlWxMWV2RpzgQqjmcWR5Wgn2cAaWYZhGKaskUkTaJpQZmZTADlW2GKfsm5eztn2qDAaXyPrh5ZkMoMFWYZhGKasQQegxprKgoR8YmaPRjZfKaOVOLJxw29JG1nWyGYMC7IMwzBMWZNJNjJmZlAIG9l8gaZvKJjiCkFcm1cpP7OzV+awIMswDMMwTFlRumKsK4wet2pORkKpjLzAzl6Zw4IswzAMwzBlRQkrZL0IO5kgTQrYtCBz2KyYYRiGYZiywilpnWzmVKUj8eTDRne2wRpZhmEYhmHKi5klx8LC5lpIpRyY11RT7KqUHSzIMgzDMAxTVswwOVbY07LDYnawDpthGIZhmLIiVepGssy0wYIswzAMwzAMU5awIMswDMMwTFmRYoUsk4YFWYZhGIZhyopSTojATC8syDIMwzAMU1awGMtIWJBlGIZhGKasOKC9Ufxd3FpX7KowRYbDbzEMwzAMU1Ysn1sPcxurob66othVYYoMC7IMwzAMw5QdmaaBZWYmbFrAMAzDMAzDlCUsyDIMwzAMwzBlCQuyDMMwDMMwTFnCgizDMAzDMAxTlrAgyzAMwzAMw5QlLMgyDMMwDMMwZQkLsgzDMAzDMExZwoIswzAMwzAMU5bMumjCjuNmaO7v7y92VRiGYRiGYRgNKaNJmS2MWSfIDgwMiL/Lli0rdlUYhmEYhmGYEJmtpaUFwkg4ccTdGUQqlYLdu3dDU1MTJBKJgs8oUGDesWMHNDc3F/RY5QRfFzt8bczwdTHD18UOXxszfF3M8HUprWuDoikKsYsXL4ZkMtwKdtZpZPGCLF26dFqPiTeeH4wgfF3s8LUxw9fFDF8XO3xtzPB1McPXpXSuTZQmVsLOXgzDMAzDMExZwoIswzAMwzAMU5awIFtAampq4MorrxR/GR++Lnb42pjh62KGr4sdvjZm+LqY4etSvtdm1jl7MQzDMAzDMDMD1sgyDMMwDMMwZQkLsgzDMAzDMExZwoIswzAMwzAMU5awIMswDMMwDMOUJSzIFojrrrsOVq5cCbW1tXD88cfDo48+CjOJa665Bo499liRIW3+/Plw3nnnwaZNm5RtTj31VJE9jb4uueQSZZvt27fDW97yFqivrxflfPazn4XJyUllmwceeACOOuoo4TG5Zs0auPnmm6FUueqqqwLnfNBBB3m/j46Owsc+9jGYO3cuNDY2wjve8Q7Yt2/fjL4mEnwe9GuDL7wes6m9/OlPf4Jzzz1XZKzBc7z99tuV39H/9oorroBFixZBXV0dnHnmmfDKK68o23R3d8MFF1wggpO3trbC3//938Pg4KCyzbPPPguvf/3rRR+EWXm+8Y1vBOryy1/+UrRP3ObQQw+FO++8E0rxukxMTMDnPvc5UceGhgaxzYUXXiiyNEa1sa9//etlfV3itJn3v//9gfN+05veNKvbDGLqb/D1zW9+c0a3mWtijM/TORYVXB7CqAVMfvn5z3/uVFdXOzfeeKPzwgsvOBdffLHT2trq7Nu3z5kpnH322c5NN93kPP/8887TTz/tvPnNb3aWL1/uDA4Oetuccsop4tz37Nnjvfr6+rzfJycnnUMOOcQ588wznaeeesq58847nfb2dufyyy/3ttm8ebNTX1/vXHbZZc6LL77o/Pu//7tTUVHh3H333U4pcuWVVzoHH3ywcs779+/3fr/kkkucZcuWOffff7/z+OOPO6973eucE088cUZfE0lHR4dyXe69916MmOL84Q9/mFXtBev9+c9/3vnVr34lzv+2225Tfv/617/utLS0OLfffrvzzDPPOG9961udVatWOSMjI942b3rTm5zDDz/ceeSRR5w///nPzpo1a5z3vve93u943RYsWOBccMEF4hn9n//5H6eurs75j//4D2+bhx56SFybb3zjG+JafeELX3Cqqqqc5557zim169Lb2yvu+y233OK89NJLzsMPP+wcd9xxztFHH62UsWLFCufqq69W2hDtk8rxusRpMxdddJFoE/S8u7u7lW1mW5tB6PXAF47JiUTCee2112Z0mzk7xvg8XWPRdMhDLMgWAOxgP/axj3mfp6amnMWLFzvXXHONM1NBIQU7kj/+8Y/edyiYfPKTn7Tugw9GMpl09u7d6313/fXXO83Nzc7Y2Jj4/P/+3/8TgiHl3e9+t3hQS1WQxcHCBA7G2Ln98pe/9L7buHGjuG44MM/Ua2ID28YBBxzgpFKpWdte9MEXr8XChQudb37zm0q7qampEQMoggMG7vfYY49529x1111igN61a5f4/P3vf99pa2vzrgvyuc99zlm3bp33+V3vepfzlre8RanP8ccf7/zDP/yDU2xMQonOo48+Krbbtm2bIpR8+9vftu5T7tcFsQmyb3vb26z7cJtxwWt0+umnK9/NhjbToY3P0zkWTYc8xKYFeWZ8fByeeOIJsRwoSSaT4vPDDz8MM5W+vj7xd86cOcr3P/3pT6G9vR0OOeQQuPzyy2F4eNj7Da8HLsEsWLDA++7ss8+G/v5+eOGFF7xt6LWU25TytcRlYFzqWr16tVjKw+UZBNsFLpHS88GlqOXLl3vnM1Oviek5+e///m/44Ac/KJbyZnN7oWzZsgX27t2rnAPmG8flONpGcGn4mGOO8bbB7bGf+etf/+pt84Y3vAGqq6uV64DLiz09PTPiWmGfg20HrwUFl4VxufTII48US8h0KXQmXxdc4sXl33Xr1sFHPvIR6Orq8n7jNgNi2fy3v/2tMKnQmeltpk8bn6drLJoueagybyUxgs7OTpiamlJuPoKfX3rpJZiJpFIp+NSnPgUnnXSSEEAkf/d3fwcrVqwQQh3aGKGNGz78v/rVr8TvOGCbrpP8LWwbfJhGRkaEDWEpgQIH2gjhYLJnzx740pe+JGyrnn/+eXEu2BnqAy+eT9T5yt/K8ZqYQFu23t5eYds3m9uLjjwP0znQc0SBhVJZWSkGKbrNqlWrAmXI39ra2qzXSpZRyqB9H7aP9773vcLmU/KJT3xC2OvhtfjLX/4iJkP4HF577bUz+rqgPezb3/52cW6vvfYa/PM//zOcc845QlioqKjgNgMAP/nJT4TNKF4nykxvMynD+DxdYxEK+tMhD7Egy+QMGoyjoPbggw8q33/4wx/23uPMDp1XzjjjDNHRHnDAATATwcFDcthhhwnBFoWzX/ziFyUvRE0nP/7xj8W1QqF1NrcXJnNQk/Sud71LOMVdf/31ym+XXXaZ8vzhYP0P//APwvmlVNNr5oP3vOc9yrOD547PDGpp8RliAG688UaxQoYOR7OpzXzMMj7PJNi0IM/gsijOgHXvP/y8cOFCmGlceuml8H//93/whz/8AZYuXRq6LQp1yKuvvir+4vUwXSf5W9g2qIUpB8EQZ7wHHnigOGc8F1xqQU2krW3Mhmuybds2uO++++BDH/pQ6Hazsb3I8wjrP/BvR0eH8jsuhaJXej7aUSn3U1KIxTZ07733KtpYWxvCa7N169YZfV100KwJxyL67MzWNoP8+c9/Fqs7UX3OTGszl1rG5+kai6ZLHmJBNs/gbO7oo4+G+++/X1Ht4+cTTjgBZgqoDcGH5LbbboPf//73gaUXE08//bT4i5o2BK/Hc889p3SwcnDasGGDtw29lnKbcrmWGN4GNYp4ztguqqqqlPPBzhVtaOX5zIZrctNNN4llTgzrEsZsbC/4HGEHT88Bl+nQjpG2ERyA0PZMgs8g9jNS+MdtMDQRCn70OqDJCy6FluO1kkIs2qDjRAhtGqPANoQ2eXJZfSZeFxM7d+4UNrL02ZmNbYauAGH/e/jhh8+KNuNEjM/TNRZNmzyUN7cxRgk3gV7GN998s/AW/fCHPyzCTVDvv3LnIx/5iAgR9MADDyhhS4aHh8Xvr776qghpgmE9tmzZ4vz61792Vq9e7bzhDW8IhPd44xvfKEKEYMiOefPmGcN7fPaznxVeldddd13JhVOi/OM//qO4JnjOGJIFQ5dgyBL0GpUhTzAMyu9//3txbU444QTxmsnXhIIeq3j+6PVLmU3tZWBgQISzwRd2wddee614L73vMfwW9hd4DZ599lnhaW0Kv3XkkUc6f/3rX50HH3zQWbt2rRJKCb2SMWTQ+973PhGCB/skvC56yKDKykrnX//1X8W1wogbxQwZFHZdxsfHRRiypUuXintP+xzpQf2Xv/xFeJ/j7xhe6b//+79F+7jwwgvL+rpEXRv87TOf+YzwNsdn57777nOOOuoo0SZGR0dnbZuh4bPwPNDjXmemtpmPRIzP0zkWTYc8xIJsgcB4athIMH4ahp/A2H0zCew0TC+MXYds375dCCFz5swRjRhjFmJjp3FBka1btzrnnHOOiMuHAh8KghMTE8o2GGf0iCOOENcShRt5jFIEQ48sWrRI1HXJkiXiMwppEhRGPvrRj4pwLtgBnH/++aKDmcnXhHLPPfeIdrJp0ybl+9nUXrB+pmcHQyjJEFxf/OIXxeCJ1+KMM84IXK+uri4hhDQ2NopwOB/4wAfEoE7BGLQnn3yyKAPbIgrIOr/4xS+cAw88UFwrDKPz29/+1inF64ICmq3PkXGIn3jiCRHyCAfw2tpaZ/369c7XvvY1RZgrx+sSdW1QOEFhA4UMFJ4wnBTG6tQFhdnWZiQocGJ/gQKpzkxtMxAxPk/3WFRoeSiRPmmGYRiGYRiGKSvYRpZhGIZhGIYpS1iQZRiGYRiGYcoSFmQZhmEYhmGYsoQFWYZhGIZhGKYsYUGWYRiGYRiGKUtYkGUYhmEYhmHKEhZkGYZhGIZhmLKEBVmGYRiGYRimLGFBlmEYpgxIJBJw++23F7saDMMwJQULsgzDMBkKlGGvq666yrrv1q1bxTZPP/103uu1f/9++MhHPgLLly+HmpoaWLhwIZx99tnw0EMPKXVnYZhhmJlEZbErwDAMU07s2bPHe3/LLbfAFVdcAZs2bfK+a2xsLEq93vGOd8D4+Dj85Cc/gdWrV8O+ffvg/vvvh66urqLUh2EYZjpgjSzDMEwGoKZTvlpaWoSWU36eP38+XHvttbB06VKhFT3iiCPg7rvv9vZdtWqV+HvkkUeK/U499VTx+bHHHoOzzjoL2tvbRZmnnHIKPPnkk7Hr1NvbC3/+85/hX/7lX+C0006DFStWwHHHHQeXX345vPWtbxXbrFy5Uvw9//zzxbHlZ+TXv/41HHXUUVBbWyuE4C996UswOTnp/Y7bX3/99XDOOedAXV2d2ObWW2/1fkcB+tJLL4VFixaJMvD411xzTU7XmWEYJg4syDIMw+SJ7373u/Ctb30L/vVf/xWeffZZsbSPguQrr7wifn/00UfF3/vuu09odn/1q1+JzwMDA3DRRRfBgw8+CI888gisXbsW3vzmN4vv44BaYHyh2cDY2JhxGxSWkZtuukkcW35GAfjCCy+ET37yk/Diiy/Cf/zHf8DNN98MX/3qV5X9v/jFLwqt7zPPPAMXXHABvOc974GNGzeK3/7t3/4N7rjjDvjFL34htNM//elPFUGZYRimYDgMwzBMVtx0001OS0uL93nx4sXOV7/6VWWbY4891vnoRz8q3m/ZssXBbvepp54KLXdqasppampyfvOb33jf4X633XabdZ9bb73VaWtrc2pra50TTzzRufzyy51nnnlG2cZUxhlnnOF87WtfU777r//6L2fRokXKfpdccomyzfHHH+985CMfEe8//vGPO6effrqTSqVCz4thGCbfsEaWYRgmD/T398Pu3bvhpJNOUr7Hz1JzaQPtWS+++GKhiUXTgubmZhgcHITt27fHPj5qS/H4qBl905veBA888IAwF0DtahioYb366qs9rS6+sC6otR0eHva2O+GEE5T98LM8r/e///3CgW3dunXwiU98An73u9/FrjfDMEwusLMXwzBMkUGzAnTKQtMEtC9F+1oUFNH2NBPQPhVtbfGFpgAf+tCH4MorrxSCpg0UmNEm9u1vf7uxvDigwLxlyxa46667hNnEu971LjjzzDMVO1qGYZhCwBpZhmGYPIBa1MWLFyvhrhD8vGHDBvG+urpa/J2amgpsg5pMtIs9+OCDhSDb2dmZc53wuENDQ97nqqqqwLFRCEW71jVr1gReyaQ/RKDtLgU/r1+/Xjn/d7/73XDDDTeIaA7/+7//C93d3TmfA8MwTBiskWUYhskTn/3sZ4UG9IADDhARC9CxCpfc0fkJwagG6PWPkQwwsgFqPNGUAE0K/uu//guOOeYYYaKA5eB2cUFt7t/+7d/CBz/4QTjssMOgqakJHn/8cfjGN74Bb3vb27zt0AELQ3KhuQMKy21tbSJ82N/8zd+I+LPvfOc7hfCK5gbPP/88fOUrX/H2/eUvfynqd/LJJ4vzQce1H//4x+I3jNSAEQswGgPuj9tiFIfW1ta8Xl+GYZgAebe6ZRiGmaXOXuikddVVVzlLlixxqqqqnMMPP9y56667lH1uuOEGZ9myZU4ymXROOeUU8d2TTz7pHHPMMcJRa+3atc4vf/lLZ8WKFc63v/3tWM5eo6Ojzj/90z85Rx11lKhPfX29s27dOucLX/iCMzw87G13xx13OGvWrHEqKytF+ZK7775bOIjV1dU5zc3NznHHHef88Ic/VI593XXXOWeddZZTU1PjrFy50rnlllu833HbI444wmloaBD7owMZnhPDMEyhSeB/QfGWYRiGYfw4srfddhucd955xa4KwzCMAtvIMgzDMAzDMGUJC7IMwzAMwzBMWcLOXgzDMEwobIHGMEypwhpZhmEYhmEYpixhQZZhGIZhGIYpS1iQZRiGYRiGYcoSFmQZhmEYhmGYsoQFWYZhGIZhGKYsYUGWYRiGYRiGKUtYkGUYhmEYhmHKEhZkGYZhGIZhmLKEBVmGYRiGYRimLGFBlmEYhmEYhilLWJBlGIZhGIZhyhIWZBmGYRiGYZiyhAVZhmEYhmEYpixhQZZhGIZhGIYpS1iQZRiGYRiGYcoSFmQZhmEYhmGYsoQFWYZhGIZhGKYsqYRZhuM4MDAwUOxqMAzDMAzDMCE0NTVBIpEI22T2CbIoxLa0tBS7GgzDMAzDMEwIfX190NzcHLYJJBxUUc4iWCM7O+jv74dly5bBjh07Ih8ChilnuK0zswlu77OLJtbIBsELwo1/9oD3mu83Mxvgts7MJri9MxJ29mIYhmEYhmHKEhZkGYZhGIZhmLKEBVlmRlJTUwNXXnml+MswMxlu68xsgts7A7Pd2YthGIZhGIaZGbBGlmEYhmEYhilLWJBlGIZhGIZhyhIWZBmGYRiGYZiyhAVZZkbxpz/9Cc4991xYvHixiBl8++23F7tKDFMQrrnmGjj22GNFwPD58+fDeeedB5s2bSp2tRimIFx//fVw2GGHefFjTzjhBLjrrruKXS2mBGBBlplRDA0NweGHHw7XXXddsavCMAXlj3/8I3zsYx+DRx55BO69916YmJiAN77xjeIZYJiZxtKlS+HrX/86PPHEE/D444/D6aefDm9729vghRdeKHbVmCLDUQuYGQtqZG+77TahqWKYmc7+/fuFZhYF3De84Q3Frg7DFJw5c+bAN7/5Tfj7v//7YleFKSKzLkUtwzDMTKSvr88b3BlmJjM1NQW//OUvxeoDmhgwsxsWZBmGYcqcVCoFn/rUp+Ckk06CQw45pNjVYZiC8NxzzwnBdXR0FBobG8WK24YNG4pdLabIsCDLMAxT5qCt7PPPPw8PPvhgsavCMAVj3bp18PTTT4vVh1tvvRUuuugiYUrDwuzshgVZhmGYMubSSy+F//u//xMRO9AhhmFmKtXV1bBmzRrx/uijj4bHHnsMvvvd78J//Md/FLtqTBFhQZZhGKYMQT/dj3/842J59YEHHoBVq1YVu0oMM+0mNWNjY8WuBlNkWJBlZhSDg4Pw6quvep+3bNkilqLQAWb58uVFrRvD5Nuc4Gc/+xn8+te/FrFk9+7dK75vaWmBurq6YlePYfLK5ZdfDuecc47oxwcGBkTbxwncPffcU+yqMUWGw28xMwrs2E477bTA92hLdfPNNxelTgxTqPByJm666SZ4//vfP+31YZhCgiG27r//ftizZ4+YrGFyhM997nNw1llnFbtqTJFhQZZhGIZhGIYpSzizF8MwDMMwDFOWsCDLMAzDMAzDlCUsyDIMwzAMwzBlCQuyDMMwDMMwTFnCgizDMAzDMAxTlrAgyzAMwzAMw5QlLMgyDMMwDMMwZQkLsgzDMAzDMExZwoIswzDMLM4Odvvttxe7GgzDMFnDgizDMEwRwDSyKEjqrze96U3FrhrDMEzZUFnsCjAMw8xWUGi96aablO9qamqKVh+GYZhygzWyDMMwRQKF1oULFyqvtrY28RtqZ6+//no455xzoK6uDlavXg233nqrsv9zzz0Hp59+uvh97ty58OEPfxgGBweVbW688UY4+OCDxbEWLVoEl156qfJ7Z2cnnH/++VBfXw9r166FO+64YxrOnGEYJj+wIMswDFOifPGLX4R3vOMd8Mwzz8AFF1wA73nPe2Djxo3it6GhITj77LOF4PvYY4/BL3/5S7jvvvsUQRUF4Y997GNCwEWhF4XUNWvWKMf40pe+BO9617vg2WefhTe/+c3iON3d3dN+rgzDMNmQcBzHyWpPhmEYJicb2f/+7/+G2tpa5ft//ud/Fi/UyF5yySVCGJW87nWvg6OOOgq+//3vww033ACf+9znYMeOHdDQ0CB+v/POO+Hcc8+F3bt3w4IFC2DJkiXwgQ98AL7yla8Y64DH+MIXvgBf/vKXPeG4sbER7rrrLrbVZRimLGAbWYZhmCJx2mmnKYIqMmfOHO/9CSecoPyGn59++mnxHjWzhx9+uCfEIieddBKkUinYtGmTEFJRoD3jjDNC63DYYYd577Gs5uZm6OjoyPncGIZhpgMWZBmGYYoECo76Un++QLvZOFRVVSmfUQBGYZhhGKYcYBtZhmGYEuWRRx4JfF6/fr14j3/RdhbNASQPPfQQJJNJWLduHTQ1NcHKlSvh/vvvn/Z6MwzDTBeskWUYhikSY2NjsHfvXuW7yspKaG9vF+/RgeuYY46Bk08+GX7605/Co48+Cj/+8Y/Fb+iUdeWVV8JFF10EV111Fezfvx8+/vGPw/ve9z5hH4vg92hnO3/+fBH9YGBgQAi7uB3DMMxMgAVZhmGYInH33XeLkFgU1Ka+9NJLXkSBn//85/DRj35UbPc///M/sGHDBvEbhsu655574JOf/CQce+yx4jNGOLj22mu9slDIHR0dhW9/+9vwmc98RgjI73znO6f5LBmGYQoHRy1gGIYpQdBW9bbbboPzzjuv2FVhGIYpWdhGlmEYhmEYhilLWJBlGIZhGIZhyhK2kWUYhilB2OqLYRgmGtbIMgzDMAzDMGUJC7IMwzAMwzBMWcKCLMMwDMMwDFOWsCDLMAzDMAzDlCUsyDIMwzAMwzBlCQuyDMMwDMMwTFnCgizDMAzDMAxTlrAgyzAMwzAMw5QlLMgyDMMwDMMwUI78f+H0eLXvS/eoAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import csv\n", "import matplotlib.pyplot as plt\n", "\n", "def plot_distill_metrics(csv_path=\"train_distill_metrics.csv\"):\n", " total_steps, train_losses, val_losses, epoch_bounds = [], [], [], {}\n", "\n", " with open(csv_path, newline=\"\", encoding=\"utf-8\") as f:\n", " for row in csv.DictReader(f):\n", " step = int(row[\"total_steps\"])\n", " epoch = int(row[\"epoch\"])\n", " total_steps.append(step)\n", " train_losses.append(float(row[\"train_loss\"]))\n", " val_losses.append(float(row[\"val_loss\"]))\n", " epoch_bounds.setdefault(epoch, [step, step])[1] = step # first/last step for each epoch\n", "\n", " fig, ax = plt.subplots(figsize=(7, 4))\n", " ax.plot(total_steps, train_losses, label=\"train_loss\", alpha=0.3)\n", " ax.plot(total_steps, val_losses, label=\"val_loss\")\n", " ax.set_xlabel(\"Total Steps\")\n", " ax.set_ylabel(\"Loss\")\n", " ax.legend()\n", "\n", " # Epoch axis\n", " epoch_axis = ax.secondary_xaxis(\"bottom\")\n", " epoch_axis.spines[\"bottom\"].set_position((\"outward\", 45))\n", " epochs = sorted(epoch_bounds)\n", " epoch_axis.set_xticks(\n", " [(epoch_bounds[epoch][0] + epoch_bounds[epoch][1]) / 2 \n", " for epoch in epochs]\n", " )\n", " epoch_axis.set_xticklabels(epochs)\n", " epoch_axis.set_xlabel(\"Epoch\")\n", "\n", " plt.tight_layout()\n", " plt.show()\n", "\n", "plot_distill_metrics(\"deepseek-r1-2048_distill_metrics.csv\")" ] }, { "cell_type": "markdown", "id": "59799c27-db40-4237-b1f5-8e8fb22978c5", "metadata": {}, "source": [ "- Training loss is noisy because it's based on a single sample\n", "- We could run the evaluation on multiple samples, e.g., 25, as in the case of the validation loss, but this would increase the training time\n", "- Validation loss is what we are interested in; we want to see it decrease\n", "- It sharply decreases but then stops making significant progress\n", "- We could increase the learning rate to train more aggressively, but overall, the validation curve looks good" ] }, { "cell_type": "markdown", "id": "fe80c3d1-cc79-497f-9e09-19892cbbc2be", "metadata": {}, "source": [ "- Similar to chapter 7, we can evaluate the resulting model checkpoints with the evaluation utilities from chapter 3" ] }, { "cell_type": "code", "execution_count": 36, "id": "d3883209-ad24-4f4a-898f-171e4be23f19", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "evaluate_math500.py: 3.5 KB (cached)\n" ] }, { "data": { "text/plain": [ "PosixPath('evaluate_math500.py')" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "download_from_github(\n", " \"ch03/02_math500-verifier-scripts/evaluate_math500.py\"\n", ")" ] }, { "cell_type": "markdown", "id": "ecae5e64-b840-4e9d-a00e-2a23351a2679", "metadata": {}, "source": [ "```bash\n", "uv run evaluate_math500.py \\\n", "--dataset_size 500 \\\n", "--which_model reasoning \\\n", "--max_new_tokens 4096 \\\n", "--checkpoint_path run_11/checkpoints/distill/qwen3-0.6B-distill-step05746-epoch1.pth\n", "```" ] }, { "cell_type": "markdown", "id": "9d0ebf3c-b088-4500-9d63-831676cbe1c8", "metadata": {}, "source": [ "| | Teacher data | Epoch | MATH-500 Acc | Final val loss |\n", "| ---- | ------------------------------------ | ----- | ------------ | -------------- |\n", "| 1 | Base (chapter 3) | - | 15.2% | - |\n", "| 2 | Reasoning (chapter 3) | - | 48.2% | - |\n", "| 3 | DeepSeek R1 distillation data | 1 | 30.6% | 0.5436 |\n", "| 4 | DeepSeek R1 distillation data | 2 | 32.4% | 0.5349 |\n", "| 5 | DeepSeek R1 distillation data | 3 | 33.6% | 0.5343 |\n", "| 6 | Qwen3 235B A22B distillation data | 1 | 45.0% | 0.4043 |\n", "| 7 | Qwen3 235B A22B distillation data | 2 | 43.8% | 0.3963 |\n", "| 8 | Qwen3 235B A22B distillation data | 3 | 44.2% | 0.3948 |" ] }, { "cell_type": "markdown", "id": "ec1dfffe-34e2-4bc9-bd2b-768c00d41c57", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "id": "e94e63f4-c4c9-4b56-a53a-2e5ce7020c20", "metadata": {}, "source": [ " \n", "## 8.9 Future directions for reasoning models" ] }, { "cell_type": "markdown", "id": "cc733c9e-dee3-423e-8a09-84f998986f17", "metadata": {}, "source": [ "- No code in this section\n", "- Also see appendix A for more references" ] }, { "cell_type": "markdown", "id": "609d8301-bb8c-4e55-bb05-4d2d02ce1cad", "metadata": {}, "source": [ " \n", "## 8.10 Conclusions" ] }, { "cell_type": "markdown", "id": "74c0c52c-5e28-4996-85f3-93b5f2c26a6b", "metadata": {}, "source": [ "- No code in this section" ] }, { "cell_type": "markdown", "id": "3621e48d-a039-442b-b19d-5c28c356e524", "metadata": {}, "source": [ "- I hope you enjoyed this book!\n", "- We are at the end, but there are more \"appendix\" chapters you might find useful (covering LLM architecture details, running the code in batched mode for higher throughput, discussing different evaluation methods, and more)\n", "- Also, you can find additional bonus material in the bonus [#bonus-material](https://github.com/rasbt/reasoning-from-scratch?tab=readme-ov-file#bonus-material) section" ] }, { "cell_type": "markdown", "id": "b9505b63-6522-4db6-bc9f-207532eca14a", "metadata": {}, "source": [ "- I regularly write about AI and LLM topics on my blog at https://magazine.sebastianraschka.com, which you may find useful" ] }, { "cell_type": "markdown", "id": "492424cc-1cf8-4581-b441-48e435dda9b5", "metadata": {}, "source": [ " \n", "## Summary" ] }, { "cell_type": "markdown", "id": "95375d44-6652-4105-b38d-4719d328c7ee", "metadata": {}, "source": [ "- No code in this section" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" } }, "nbformat": 4, "nbformat_minor": 5 }