{ "cells": [ { "cell_type": "markdown", "id": "cell-0", "metadata": {}, "source": [ "# Supervised Fine-tuning (SFT) with Unsloth\n", "\n", "Fine-tuning requires a GPU. If you don't have one locally, you can run this notebook for free on [Google Colab](https://colab.research.google.com/github/Liquid4All/cookbook/blob/main/finetuning/notebooks/sft_with_unsloth.ipynb) using a free NVIDIA T4 GPU instance.\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Liquid4All/cookbook/blob/main/finetuning/notebooks/sft_with_unsloth.ipynb)\n", "\n", "### What's in this notebook?\n", "\n", "In this notebook you will learn how to perform Supervised Fine-tuning (SFT) using Unsloth for efficient, memory-optimized training.\n", "We will use the [LFM2.5-1.2B-Instruct](https://docs.liquid.ai/docs/models/lfm25-1.2b-instruct) model and fine-tune it on the FineTome-100k dataset using LoRA adapters and Unsloth's optimizations for 2x faster training with reduced memory consumption.\n", "\n", "We will cover\n", "- Environment setup\n", "- Data preparation\n", "- Model training\n", "- Local inference with your new model\n", "- Model saving and exporting it into the format you need for **deployment**.\n", "\n", "### Deployment options\n", "\n", "LFM2.5 models are small and efficient, enabling deployment across a wide range of platforms:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Deployment TargetUse Case
📱 AndroidMobile apps on Android devices
📱 iOSMobile apps on iPhone/iPad
🍎 Apple Silicon MacLocal inference on Mac with MLX
🦙 llama.cppLocal deployments on any hardware
🦙 OllamaLocal inference with easy setup
🖥️ LM StudioDesktop app for local inference
⚡ vLLMCloud deployments with high throughput
☁️ ModalServerless cloud deployment
🏗️ BasetenProduction ML infrastructure
🚀 FalFast inference API
" ] }, { "cell_type": "markdown", "id": "cell-0b", "metadata": {}, "source": [ "### Need help building with our models and tools?\n", "Join the Liquid AI Discord Community and ask.\n", "\n", "\"Join\n", "\n", "And now, let the fine tune begin!" ] }, { "cell_type": "markdown", "metadata": { "id": "heTG3uw03baO" }, "source": [ "### Installation" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "2sA8ZNSO3baO" }, "outputs": [], "source": [ "%%capture\n", "import os, re\n", "if \"COLAB_\" not in \"\".join(os.environ.keys()):\n", " !pip install unsloth\n", "else:\n", " # Do this only in Colab notebooks! Otherwise use pip install unsloth\n", " import torch; v = re.match(r\"[0-9]{1,}\\.[0-9]{1,}\", str(torch.__version__)).group(0)\n", " xformers = \"xformers==\" + (\"0.0.33.post1\" if v==\"2.9\" else \"0.0.32.post2\" if v==\"2.8\" else \"0.0.29.post3\")\n", " !pip install --no-deps bitsandbytes accelerate {xformers} peft trl triton cut_cross_entropy unsloth_zoo\n", " !pip install sentencepiece protobuf \"datasets==4.3.0\" \"huggingface_hub>=0.34.0\" hf_transfer\n", " !pip install --no-deps unsloth\n", "!pip install transformers==4.57.3\n", "!pip install --no-deps trl==0.22.2" ] }, { "cell_type": "markdown", "metadata": { "id": "6g6_GHNTjqJR" }, "source": [ "### Unsloth" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 348, "referenced_widgets": [ "a13d396c32264bdcac472747446b0b85", "47770a78256b4117a06efd3c63f1ff0a", "6a74bf1e4ac648f7a16b1248f37ada8e", "7253617b3b63429f9fcac85965ec79bd", "43ee612dde3c4839950543d031ddc9dd", "649b1cb7f97d42e980e00dae6a1d8f13", "71160c58763a4dc59524eb0df42f2c28", "bb8c77d065b8473eb8c2da420c66a092", "e6e2aa192331401a83cb152fc24a178b", "40e6033b12d04e60a416451d957c34c3", "fef1381ee0904ce096a9387c123e4027", "3a2615fc2b5d475db2db432402eb37b9", "460416873c1c4f1f95e97d08fd07984e", "3dbbecfbb6104dafb78eaab86e4dd786", "90b4e2cb07ef40e699f54e97bdd0b7e4", "d94a2677bd664e499144ac9b2654ddb7", "a3c0891945fd4248bf14dad61a6bbafd", "ee23aea525a445a1a3c3f12ae35fa822", "0d118fec9e4744539606cea352b3af74", "b539ef4f2d2e4f798feb788a117f1d6e", "714a61a7a4424ad1bf3f918b66354cad", "d15b4c07efaa40abb194a25c1b9cdc13", "b6de5b5690e44ff6b2393c3e4f1670d1", "db1a21003fed47a99585b21cb360deaa", "cffb5ffd975548669ddadea2282d819d", "4d25ad050dda427b8d5ba24b0d2bc956", "4732aad6d3ed4d65bd58b069e3110851", "64f29ad5f4fa4b04967c6750fe21b97d", "649da39b80e04cb68bb2573c21af4bfe", "ce0b453e581e431084f58c79c289dea1", "db8a9c9b49c44456a517d0d301a2fcd0", "00f523c5508e4f1aaa2f7fd6dc1af4a7", "e5e7c7f7a9f14e18b2c2cdbf561dadbc", "02192fcd872547f0a86fab653451d81b", "b02f78769a04437a91a137dafdc784f0", "cfe8d77906204c419c1a0fbf4a310e1c", "814e94c8c20047199691f6ae736da0af", "925a277c4ac248f3b734db7f28cedf2d", "71436f80de1e4f3086e30167efe22e7f", "5b73ad58426e43b2842a340671bbfd9d", "febfcbf1669c49fbb72ea1fce63769b8", "25d5da66c7c24c1184f356d85cf39ef9", "0f4007a1da8b44cd90f0bae8fb88839b", "ecc6c6c5c5584fa38a9de733f85190bf", "45adf26de469485098438f8896e0abb9", "d358a15ca6bc41478dc0bd552c9c816f", "7fbb89b3ed74468f9c53baa0acca5397", "27566c12a1e14241888f92d053f025a3", "20b005b9f503440481fe0f89dff9cfd4", "b196d74a081b49e7b50daf542957bf0a", "70cf844603a34f7a8dfca853ed83b8f4", "f420c237a73f4f59a87494cb2132fcc0", "dfca706373024a679e7e5093a722b37c", "34c119ed677e4091b2b092b98e09bc9e", "2e7ae466652a415fa25248447c61399e", "840b1bf0aadb4a59a9b740c292086c94", "dbe5b17435b34738a4367a3e70410ec8", "a825d050630e4512b770ab4633e5cc41", "7324d3ec42a84742ab5c09a832cbc03a", "63e885bfa5f1448ea55a0a3b759077d1", "93d28ef4de6b486dbe280b2991042810", "f2ef6d3675434b6dabaa5b3cb3aaca1e", "6fed2bf745bc47bbb6be188cde4b4966", "d7095d3533b843e486f81df13b746d40", "6af24a0566e04b65953b662ad9591128", "b1dee64c15cc4b97a2c5a9ab4e9589f7" ] }, "id": "QmUBVEnvCDJv", "outputId": "ab25804e-31b1-49c4-e2a8-018423d3cb9a" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n", "🦥 Unsloth Zoo will now patch everything to make training faster!\n", "==((====))== Unsloth 2026.1.2: Fast Lfm2 patching. Transformers: 4.57.3.\n", " \\\\ /| NVIDIA L4. Num GPUs = 1. Max memory: 22.161 GB. Platform: Linux.\n", "O^O/ \\_/ \\ Torch: 2.9.0+cu126. CUDA: 8.9. CUDA Toolkit: 12.6. Triton: 3.5.0\n", "\\ / Bfloat16 = TRUE. FA [Xformers = 0.0.33.post1. FA2 = False]\n", " \"-____-\" Free license: http://github.com/unslothai/unsloth\n", "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a13d396c32264bdcac472747446b0b85", "version_major": 2, "version_minor": 0 }, "text/plain": [ "model.safetensors: 0%| | 0.00/2.34G [00:00 0 ! Suggested 8, 16, 32, 64, 128\n", " target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"out_proj\", \"in_proj\",\n", " \"w1\", \"w2\", \"w3\"],\n", " lora_alpha = 16,\n", " lora_dropout = 0, # Supports any, but = 0 is optimized\n", " bias = \"none\", # Supports any, but = \"none\" is optimized\n", " # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n", " use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n", " random_state = 3407,\n", " use_rslora = False, # We support rank stabilized LoRA\n", " loftq_config = None, # And LoftQ\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "vITh0KVJ10qX" }, "source": [ "\n", "### Data Prep\n", "We now use the `LFM` format for conversation style finetunes. We use [Maxime Labonne's FineTome-100k](https://huggingface.co/datasets/mlabonne/FineTome-100k) dataset in ShareGPT style. LFM renders multi turn conversations like below:\n", "\n", "```\n", "<|startoftext|><|im_start|>user\n", "Hello!<|im_end|>\n", "<|im_start|>assistant\n", "Hey there!<|im_end|>\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 35 }, "id": "TigGJ1Ug7Ruo", "outputId": "dcbae7e4-86a1-4f7c-8064-8546637224b4" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'<|startoftext|><|im_start|>user\\nHello!<|im_end|>\\n<|im_start|>assistant\\nHey there!<|im_end|>\\n'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.apply_chat_template([\n", " {\"role\" : \"user\", \"content\" : \"Hello!\"},\n", " {\"role\" : \"assistant\", \"content\" : \"Hey there!\"}\n", "], tokenize = False)" ] }, { "cell_type": "markdown", "metadata": { "id": "oHdjT5Xs8fcU" }, "source": [ "We get the first 3000 rows of the dataset" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 113, "referenced_widgets": [ "06b25f5dca9f486e886004023e7484d4", "b761459a7ee14b23a7a25283a350d4e0", "0151d6cab2264f459906c7b45a4ca70e", "651f7fb447334c20aa5caf6a8b500cbf", "1312e721aa734a60a48707b84cb51f11", "30cd762e60974afd9368c1640a336f10", "0854bf6f6b734fff8c021353404fb60a", "ddfdfcea8dc545939bf8df9f0acb3238", "0dd1da12b16e48c18ee5ea05311f0018", "e714bd4f171e4301a6750deb6ee322f9", "f41166a6c613429e983248b67ed003b3", "9689d16a48834818887964192c822671", "81e7776f13d442c29de30e65169e372c", "dacac937b9cf476d812daeba571e613c", "72b841c26e9c4c779caf6d6225b905b3", "030cd2f1ddc64dbaaf671cd0adaac5f5", "324fed58a93c406785910a3b5447313f", "6acf85c1e7564845b86a3f15287fe9b6", "f9397a07dcc04875bb6568d4bc5b57f6", "592dc6b4e8034383806c7a51a6c8ae81", "f1f6a0b2d1cc4513a0bee092e2ff3697", "d6cb8e277896466cb018e1d65be103b6", "d7c0eeeab9204ed9b3a536883890ff5a", "a2862b6705e94f68abbe9cc9fab73d72", "81f870db408b4958a0f537e6a6e96de0", "02ad7e3226944f77a04ba2e244a334b7", "f50eae161c11463fbad3ca9752d10e2d", "7a387cbdc797470f8b43dfa0a89992a5", "8eaae5d970f84284916bfade3ff5111e", "6045615ca53f43d59a3ded815e347c94", "f3cd79679089422da605bc2904363b28", "d003f9b4771049e191715f5a41659f9a", "1dc3adc542f644ccaa74823066b8fecd" ] }, "id": "LjY75GoYUCB8", "outputId": "82009ff0-6f8f-45ab-dfc8-34d77c8210d9" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "06b25f5dca9f486e886004023e7484d4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "README.md: 0%| | 0.00/982 [00:00user\\nExplain what boolean operators are, what they do, and provide examples of how they can be used in programming. Additionally, describe the concept of operator precedence and provide examples of how it affects the evaluation of boolean expressions. Discuss the difference between short-circuit evaluation and normal evaluation in boolean expressions and demonstrate their usage in code. \\n\\nFurthermore, add the requirement that the code must be written in a language that does not support short-circuit evaluation natively, forcing the test taker to implement their own logic for short-circuit evaluation.\\n\\nFinally, delve into the concept of truthiness and falsiness in programming languages, explaining how it affects the evaluation of boolean expressions. Add the constraint that the test taker must write code that handles cases where truthiness and falsiness are implemented differently across different programming languages.<|im_end|>\\n<|im_start|>assistant\\nBoolean operators are logical operators used in programming to manipulate boolean values. They operate on one or more boolean operands and return a boolean result. The three main boolean operators are \"AND\" (&&), \"OR\" (||), and \"NOT\" (!).\\n\\nThe \"AND\" operator returns true if both of its operands are true, and false otherwise. For example:\\n\\n```python\\nx = 5\\ny = 10\\nresult = (x > 0) and (y < 20) # This expression evaluates to True\\n```\\n\\nThe \"OR\" operator returns true if at least one of its operands is true, and false otherwise. For example:\\n\\n```python\\nx = 5\\ny = 10\\nresult = (x > 0) or (y < 20) # This expression evaluates to True\\n```\\n\\nThe \"NOT\" operator negates the boolean value of its operand. It returns true if the operand is false, and false if the operand is true. For example:\\n\\n```python\\nx = 5\\nresult = not (x > 10) # This expression evaluates to True\\n```\\n\\nOperator precedence refers to the order in which operators are evaluated in an expression. It ensures that expressions are evaluated correctly. In most programming languages, logical AND has higher precedence than logical OR. For example:\\n\\n```python\\nresult = True or False and False # This expression is evaluated as (True or (False and False)), which is True\\n```\\n\\nShort-circuit evaluation is a behavior where the second operand of a logical operator is not evaluated if the result can be determined based on the value of the first operand. In short-circuit evaluation, if the first operand of an \"AND\" operator is false, the second operand is not evaluated because the result will always be false. Similarly, if the first operand of an \"OR\" operator is true, the second operand is not evaluated because the result will always be true.\\n\\nIn programming languages that support short-circuit evaluation natively, you can use it to improve performance or avoid errors. For example:\\n\\n```python\\nif x != 0 and (y / x) > 10:\\n # Perform some operation\\n```\\n\\nIn languages without native short-circuit evaluation, you can implement your own logic to achieve the same behavior. Here\\'s an example in pseudocode:\\n\\n```\\nif x != 0 {\\n if (y / x) > 10 {\\n // Perform some operation\\n }\\n}\\n```\\n\\nTruthiness and falsiness refer to how non-boolean values are evaluated in boolean contexts. In many programming languages, non-zero numbers and non-empty strings are considered truthy, while zero, empty strings, and null/None values are considered falsy.\\n\\nWhen evaluating boolean expressions, truthiness and falsiness come into play. For example:\\n\\n```python\\nx = 5\\nresult = x # The value of x is truthy, so result is also truthy\\n```\\n\\nTo handle cases where truthiness and falsiness are implemented differently across programming languages, you can explicitly check the desired condition. For example:\\n\\n```python\\nx = 5\\nresult = bool(x) # Explicitly converting x to a boolean value\\n```\\n\\nThis ensures that the result is always a boolean value, regardless of the language\\'s truthiness and falsiness rules.<|im_end|>\\n'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dataset[0][\"text\"]" ] }, { "cell_type": "markdown", "metadata": { "id": "idAEIeSQ3xdS" }, "source": [ "\n", "### Train the model\n", "Now let's use Huggingface TRL's `SFTTrainer`! More docs here: [TRL SFT docs](https://huggingface.co/docs/trl/sft_trainer). We do 60 steps to speed things up, but you can set `num_train_epochs=1` for a full run, and turn off `max_steps=None`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 66, "referenced_widgets": [ "d7527424345b41dda0264ec73f009bfa", "1ac058b33cbc4cfba16f9fe5af3a26b5", "8a5dbd321e0645d8896503c841af8a66", "7898e35c5a8e441d8f1f992bbb976f02", "100c8e4290b24fdeadf25ce7adb022da", "88d0e0bc0de8432e97da0deb22a83fd9", "1f4c9d7c9c1b42039f26d9d89cde61b4", "d44f5de67f064ccda3f6f7fbb4b754e9", "f20cd4e3fdaf420ea36bc09000109ed2", "674cfc0874504b68a431b34ca590f1bb", "ad7a1adf7352425b989937661c93e119" ] }, "id": "95_Nn-89DhsL", "outputId": "c30cdec4-0b2e-4826-e71d-c4526c1481f0" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d7527424345b41dda0264ec73f009bfa", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Unsloth: Tokenizing [\"text\"] (num_proc=16): 0%| | 0/3000 [00:00user\\n\",\n", " response_part = \"<|im_start|>assistant\\n\",\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "mPmWWLYxAa_4" }, "source": [ "Let's verify masking the instruction part is done! Let's print the 100th row again." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 105 }, "id": "zDJcRw2OAcIb", "outputId": "04154710-6040-4aa1-cba1-7c0604148014" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'<|startoftext|><|im_start|>user\\nWhat is the modulus operator in programming and how can I use it to calculate the modulus of two given numbers?<|im_end|>\\n<|im_start|>assistant\\nIn programming, the modulus operator is represented by the \\'%\\' symbol. It calculates the remainder when one number is divided by another. To calculate the modulus of two given numbers, you can use the modulus operator in the following way:\\n\\n```python\\n# Calculate the modulus\\nModulus = a % b\\n\\nprint(\"Modulus of the given numbers is: \", Modulus)\\n```\\n\\nIn this code snippet, the variables \\'a\\' and \\'b\\' represent the two given numbers for which you want to calculate the modulus. By using the modulus operator \\'%\\', we calculate the remainder when \\'a\\' is divided by \\'b\\'. The result is then stored in the variable \\'Modulus\\'. Finally, the modulus value is printed using the \\'print\\' statement.\\n\\nFor example, if \\'a\\' is 10 and \\'b\\' is 4, the modulus calculation would be 10 % 4, which equals 2. Therefore, the output of the above code would be:\\n\\n```\\nModulus of the given numbers is: 2\\n```\\n\\nThis means that the modulus of 10 and 4 is 2.<|im_end|>\\n'" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.decode(trainer.train_dataset[100][\"input_ids\"])" ] }, { "cell_type": "markdown", "metadata": { "id": "IeE-Ly0fAe17" }, "source": [ "Now let's print the masked out example - you should see only the answer is present:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 105 }, "id": "cB5Cq_3bAfKr", "outputId": "b3f914b2-e2f2-44b1-8d30-f67d202f60c0" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "' In programming, the modulus operator is represented by the \\'%\\' symbol. It calculates the remainder when one number is divided by another. To calculate the modulus of two given numbers, you can use the modulus operator in the following way:\\n\\n```python\\n# Calculate the modulus\\nModulus = a % b\\n\\nprint(\"Modulus of the given numbers is: \", Modulus)\\n```\\n\\nIn this code snippet, the variables \\'a\\' and \\'b\\' represent the two given numbers for which you want to calculate the modulus. By using the modulus operator \\'%\\', we calculate the remainder when \\'a\\' is divided by \\'b\\'. The result is then stored in the variable \\'Modulus\\'. Finally, the modulus value is printed using the \\'print\\' statement.\\n\\nFor example, if \\'a\\' is 10 and \\'b\\' is 4, the modulus calculation would be 10 % 4, which equals 2. Therefore, the output of the above code would be:\\n\\n```\\nModulus of the given numbers is: 2\\n```\\n\\nThis means that the modulus of 10 and 4 is 2.<|im_end|>\\n'" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.decode([tokenizer.pad_token_id if x == -100 else x for x in trainer.train_dataset[100][\"labels\"]]).replace(tokenizer.pad_token, \" \")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/" }, "id": "2ejIt2xSNKKp", "outputId": "d3241883-5c21-42bf-bb84-a7f4f183e4bd" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GPU = NVIDIA L4. Max memory = 22.161 GB.\n", "2.23 GB of memory reserved.\n" ] } ], "source": [ "# @title Show current memory stats\n", "gpu_stats = torch.cuda.get_device_properties(0)\n", "start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n", "max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)\n", "print(f\"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.\")\n", "print(f\"{start_gpu_memory} GB of memory reserved.\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "yqxqAZ7KJ4oL", "outputId": "a0111cc7-264a-4a8a-b73c-04332cefcad0" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "The model is already on multiple devices. Skipping the move to device specified in `args`.\n", "==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1\n", " \\\\ /| Num examples = 3,000 | Num Epochs = 1 | Total steps = 60\n", "O^O/ \\_/ \\ Batch size per device = 2 | Gradient accumulation steps = 4\n", "\\ / Data Parallel GPUs = 1 | Total batch size (2 x 4 x 1) = 8\n", " \"-____-\" Trainable parameters = 11,108,352 of 1,181,448,960 (0.94% trained)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unsloth: Will smartly offload gradients to save VRAM!\n" ] }, { "data": { "text/html": [ "\n", "
\n", " \n", " \n", " [60/60 01:12, Epoch 0/1]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StepTraining Loss
10.863600
20.912900
30.840000
40.534600
50.774800
60.785900
70.837100
80.824400
90.922100
100.972900
110.793100
120.875400
130.559400
140.604100
150.652200
160.688400
170.772600
180.917400
190.717700
200.807300
210.786400
220.572400
230.833100
240.641900
250.935800
260.621100
270.642300
280.922300
290.634500
300.811700
310.698200
320.650500
330.735700
340.646300
350.794400
360.704900
370.829200
381.035000
390.839700
400.639900
410.886400
420.608500
430.530600
440.693500
450.567800
460.762600
470.769500
480.767300
490.676400
500.616500
510.648600
520.485100
530.537600
540.682300
550.675400
560.687300
570.632200
580.699200
590.663700
600.684200

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "trainer_stats = trainer.train()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/" }, "id": "pCqnaKmlO1U9", "outputId": "1e90cdd2-929e-4010-9a06-3c49eded85e3" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "121.0305 seconds used for training.\n", "2.02 minutes used for training.\n", "Peak reserved memory = 3.453 GB.\n", "Peak reserved memory for training = 1.223 GB.\n", "Peak reserved memory % of max memory = 15.581 %.\n", "Peak reserved memory for training % of max memory = 5.519 %.\n" ] } ], "source": [ "# @title Show final memory and time stats\n", "used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n", "used_memory_for_lora = round(used_memory - start_gpu_memory, 3)\n", "used_percentage = round(used_memory / max_memory * 100, 3)\n", "lora_percentage = round(used_memory_for_lora / max_memory * 100, 3)\n", "print(f\"{trainer_stats.metrics['train_runtime']} seconds used for training.\")\n", "print(\n", " f\"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.\"\n", ")\n", "print(f\"Peak reserved memory = {used_memory} GB.\")\n", "print(f\"Peak reserved memory for training = {used_memory_for_lora} GB.\")\n", "print(f\"Peak reserved memory % of max memory = {used_percentage} %.\")\n", "print(f\"Peak reserved memory for training % of max memory = {lora_percentage} %.\")" ] }, { "cell_type": "markdown", "metadata": { "id": "ekOmTR1hSNcr" }, "source": [ "\n", "### Inference\n", "Let's run the model! You can change the instruction and input - leave the output blank!\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "kR3gIAX-SM2q", "outputId": "e0b276e0-f96a-4cef-d7e3-3b6660fb94fa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters the Earth's atmosphere, it interacts with molecules and small particles in the air. Shorter wavelengths of light, such as blue and violet, are scattered more than longer wavelengths like red and orange. This scattering occurs because the shorter wavelengths have a higher frequency and interact more strongly with the air molecules. As a result, the blue light is dispersed in all directions, making the sky appear blue to our eyes.\n", "\n", "However, violet light has an even shorter wavelength than blue light, which means it is scattered even more. In fact, violet light is scattered about \n" ] } ], "source": [ "messages = [{\n", " \"role\": \"user\",\n", " \"content\": \"Why is the sky blue?\",\n", "}]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", " tokenize = True,\n", " return_dict = True,\n", ").to(\"cuda\")\n", "\n", "from transformers import TextStreamer\n", "_ = model.generate(\n", " **inputs,\n", " max_new_tokens = 128, # Increase for longer outputs!\n", " # Recommended Liquid settings!\n", " temperature = 0.1, top_k = 50, top_p = 0.1, repetition_penalty = 1.05,\n", " streamer = TextStreamer(tokenizer, skip_prompt = True),\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "VMbUhAumnufn", "outputId": "2f01ea05-f016-4977-8a41-449372ad006d" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "In the jungle, where the trees sway,\n", "A sloth slips by, in a slow parade.\n", "With a snout so wide, and eyes so bright,\n", "It's a sight to behold, in the dappled light.\n", "\n", "Its fur is soft, like a cloud in the sky,\n", "And its tail is long, like a snake in the night.\n", "It moves with a lazy grace, like a leaf on the breeze,\n", "And spends its days in a peaceful, relaxed ease.\n", "\n", "But don't let its slow pace fool you,\n", "For this sloth has a heart that's full of love.\n", "It loves its jungle home\n" ] } ], "source": [ "messages = [{\n", " \"role\": \"user\",\n", " \"content\": \"Write a poem about a sloth.\",\n", "}]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", " tokenize = True,\n", " return_dict = True,\n", ").to(\"cuda\")\n", "\n", "from transformers import TextStreamer\n", "_ = model.generate(\n", " **inputs,\n", " max_new_tokens = 128, # Increase for longer outputs!\n", " # Recommended Liquid settings!\n", " temperature = 0.1, top_k = 50, top_p = 0.1, repetition_penalty = 1.05,\n", " streamer = TextStreamer(tokenizer, skip_prompt = True),\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "uMuVrWbjAzhc" }, "source": [ "\n", "### Saving, loading finetuned models\n", "To save the final model as LoRA adapters, either use Huggingface's `push_to_hub` for an online save or `save_pretrained` for a local save.\n", "\n", "**[NOTE]** This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "upcOlWe7A1vc", "outputId": "f01072cd-b772-4331-8a91-ae064ad2dc0f" }, "outputs": [ { "data": { "text/plain": [ "('lora_model/tokenizer_config.json',\n", " 'lora_model/special_tokens_map.json',\n", " 'lora_model/chat_template.jinja',\n", " 'lora_model/tokenizer.json')" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.save_pretrained(\"lora_model\") # Local saving\n", "tokenizer.save_pretrained(\"lora_model\")\n", "# model.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving\n", "# tokenizer.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving" ] }, { "cell_type": "markdown", "metadata": { "id": "AEEcJ4qfC7Lp" }, "source": [ "Now if you want to load the LoRA adapters we just saved for inference, set `False` to `True`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "MKX_XKs_BNZR", "outputId": "6e8e017b-05ad-4a57-fb66-52bf984146c9" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "To code up a transformer, you can follow these steps:\n", "\n", "1. Define the input and output data structures: Determine the shape of the input data and the expected output data structure. This will help you understand how to transform the input data into the desired output format.\n", "\n", "2. Implement the transformation logic: Based on the input data structure, write the code that performs the necessary transformations. This may involve using built-in functions or custom functions to manipulate the data.\n", "\n", "3. Handle edge cases: Consider any special cases or edge conditions that may arise during the transformation process. For example, if the input data contains missing values, you\n" ] } ], "source": [ "if False:\n", " from unsloth import FastModel\n", " from transformers import Lfm2ForCausalLM\n", " model, tokenizer = FastModel.from_pretrained(\n", " model_name = \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n", " max_seq_length = max_seq_length,\n", " dtype = dtype,\n", " auto_model = Lfm2ForCausalLM,\n", " load_in_4bit = load_in_4bit,\n", " )\n", " FastModel.for_inference(model) # Enable native 2x faster inference\n", "\n", "messages = [{\n", " \"role\": \"user\",\n", " \"content\": \"How do I code up a transformer?\",\n", "}]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", " tokenize = True,\n", " return_dict = True,\n", ").to(\"cuda\")\n", "\n", "from transformers import TextStreamer\n", "_ = model.generate(\n", " **inputs,\n", " max_new_tokens = 128, # Increase for longer outputs!\n", " # Recommended Liquid settings!\n", " temperature = 0.1, top_k = 50, top_p = 0.1, repetition_penalty = 1.05,\n", " streamer = TextStreamer(tokenizer, skip_prompt = True),\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "QQMjaNrjsU5_" }, "source": [ "You can also use Hugging Face's `AutoModelForPeftCausalLM`. Only use this if you do not have `unsloth` installed. It can be hopelessly slow, since `4bit` model downloading is not supported, and Unsloth's **inference is 2x faster**." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "yFfaXG0WsQuE" }, "outputs": [], "source": [ "if False:\n", " # I highly do NOT suggest - use Unsloth if possible\n", " from peft import AutoPeftModelForCausalLM\n", " from transformers import AutoTokenizer\n", " model = AutoPeftModelForCausalLM.from_pretrained(\n", " \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n", " load_in_4bit = load_in_4bit,\n", " )\n", " tokenizer = AutoTokenizer.from_pretrained(\"lora_model\")" ] }, { "cell_type": "markdown", "metadata": { "id": "f422JgM9sdVT" }, "source": [ "### Saving to float16 for VLLM\n", "\n", "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `merged_4bit` for int4. We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "iHjt_SMYsd3P" }, "outputs": [], "source": [ "# Merge to 16bit\n", "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_16bit\",)\n", "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_16bit\", token = \"\")\n", "\n", "# Merge to 4bit\n", "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_4bit\",)\n", "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_4bit\", token = \"\")\n", "\n", "# Just LoRA adapters\n", "if False:\n", " model.save_pretrained(\"model\")\n", " tokenizer.save_pretrained(\"model\")\n", "if False:\n", " model.push_to_hub(\"hf/model\", token = \"\")\n", " tokenizer.push_to_hub(\"hf/model\", token = \"\")" ] }, { "cell_type": "markdown", "metadata": { "id": "TCv4vXHd61i7" }, "source": [ "### GGUF / llama.cpp Conversion\n", "To save to `GGUF` / `llama.cpp`, we support it natively now! We clone `llama.cpp` and we default save it to `q8_0`. We allow all methods like `q4_k_m`. Use `save_pretrained_gguf` for local saving and `push_to_hub_gguf` for uploading to HF.\n", "\n", "Some supported quant methods (full list on our [Wiki page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n", "* `q8_0` - Fast conversion. High resource use, but generally acceptable.\n", "* `q4_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K.\n", "* `q5_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K.\n", "\n", "[**NEW**] To finetune and auto export to Ollama, try our [Ollama notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "FqfebeAdT073" }, "outputs": [], "source": [ "# Save to 8bit Q8_0\n", "if False: model.save_pretrained_gguf(\"model\", tokenizer,)\n", "# Remember to go to https://huggingface.co/settings/tokens for a token!\n", "# And change hf to your username!\n", "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, token = \"\")\n", "\n", "# Save to 16bit GGUF\n", "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"f16\")\n", "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"f16\", token = \"\")\n", "\n", "# Save to q4_k_m GGUF\n", "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"q4_k_m\")\n", "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"q4_k_m\", token = \"\")\n", "\n", "# Save to multiple GGUF options - much faster if you want multiple!\n", "if False:\n", " model.push_to_hub_gguf(\n", " \"hf/model\", # Change hf to your username!\n", " tokenizer,\n", " quantization_method = [\"q4_k_m\", \"q5_k_m\", \"q8_0\"],\n", " token = \"\",\n", " )" ] }, { "cell_type": "markdown", "metadata": { "id": "gXYiQs93jqJU" }, "source": [ "Now, use the `model-unsloth.gguf` file or `model-unsloth-Q4_K_M.gguf` file in llama.cpp or a UI based system like Jan or Open WebUI. You can install Jan [here](https://github.com/janhq/jan) and Open WebUI [here](https://github.com/open-webui/open-webui)\n", "\n", "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n", "\n", "Some other links:\n", "1. Train your own reasoning model - Llama GRPO notebook [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb)\n", "2. Saving finetunes to Ollama. [Free notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)\n", "3. Llama 3.2 Vision finetuning - Radiography use case. [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb)\n", "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://docs.unsloth.ai/get-started/unsloth-notebooks)!\n", "\n", "

\n", " \n", " \n", " \n", "\n", " Join Discord if you need help + ⭐️ Star us on Github ⭐️\n", "
\n" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "L4", "machine_shape": "hm", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.9" } }, "nbformat": 4, "nbformat_minor": 0 }