{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "aSPH_2aycMCO" }, "source": [ "# πŸ’§ LFM2.5-VL - SFT with TRL\n", "\n", "This tutorial demonstrates how to fine-tune our LFM2.5 vision models (e.g.[`LiquidAI/LFM2.5-VL-1.6B`](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B)).\n", "\n", "Follow along if it's your first time using trl, or take single code snippets for your own workflow\n", "\n", "## 🎯 What You'll Find:\n", "- **SFT** (Supervised Fine-Tuning) - Basic instruction following\n", "- **LoRA + SFT** - (Optional) using LoRA (from PEFT) to SFT while on constrained hardware\n", "\n", "## πŸ“‹ Prerequisites:\n", "- **GPU Runtime**: Select GPU in `Runtime` β†’ `Change runtime type`\n", "- **Hugging Face Account**: For accessing models and datasets" ] }, { "cell_type": "markdown", "metadata": { "id": "fOu_2kKKmMqC" }, "source": [ "# πŸ“¦ Installation & Setup\n", "\n", "First, let's install all the required packages:\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true, "id": "fmOBSWrCAw5S" }, "outputs": [], "source": [ "!pip install -qqq transformers==5.1.0 datasets trl \"torchao>0.16.0\" --progress-bar off" ] }, { "cell_type": "markdown", "metadata": { "id": "ovEzw3MamRsu" }, "source": [ "Let's now verify the packages are installed correctly\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "046FXiTJNZjh", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "21c5b2e1-81bf-4ee9-84ff-4895b1329a96" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "πŸ“¦ PyTorch version: 2.11.0+cu128\n", "πŸ€— Transformers version: 5.1.0\n", "πŸ“Š TRL version: 1.9.2\n" ] } ], "source": [ "import torch\n", "import transformers\n", "import trl\n", "import os\n", "os.environ[\"WANDB_DISABLED\"] = \"true\"\n", "\n", "print(f\"πŸ“¦ PyTorch version: {torch.__version__}\")\n", "print(f\"πŸ€— Transformers version: {transformers.__version__}\")\n", "print(f\"πŸ“Š TRL version: {trl.__version__}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "qay-sfdfNy2N" }, "source": [ "# Loading the model from Transformers πŸ€—\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "x5I6sA6NBMxF", "colab": { "base_uri": "https://localhost:8080/", "height": 239, "referenced_widgets": [ "53758b5111f649f18322c777492c1f8c", "f35d284aa0984482ab66f787ca46950b", "cbbf0dbd13c144a78f7baa88fe84db87", "1e77d4eee9f94bf2be9fced6f1590667", "2c3d13d2343d4c3284bbca1b5f9ac8fc", "100791ae62f0452eaac81de222516dde", "77a6a77bfac448fca161a034154ed18c", "b5345f636230436e956116ebf6516760", "34980b6eb5dc4859b168cf6d069f4d7d", "0ba74d3ea65a49bb884292990cd6aeb1", "87b93c50d9de42f38a7eac97558625e3" ] }, "outputId": "10efd287-2d56-4b45-b96f-89daf4d0655b" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "WARNING:torchao:Failed to load /usr/local/lib/python3.12/dist-packages/torchao/_C_mxfp8.cpython-310-x86_64-linux-gnu.so: Could not load this library: /usr/local/lib/python3.12/dist-packages/torchao/_C_mxfp8.cpython-310-x86_64-linux-gnu.so\n", "WARNING:torchao:Failed to load /usr/local/lib/python3.12/dist-packages/torchao/_C_cutlass_90a.abi3.so: Could not load this library: /usr/local/lib/python3.12/dist-packages/torchao/_C_cutlass_90a.abi3.so\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "πŸ“š Loading processor...\n", "🧠 Loading model...\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "`torch_dtype` is deprecated! Use `dtype` instead!\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "Loading weights: 0%| | 0/707 [00:00" ], "text/html": [ "\n", "
\n", " \n", " \n", " [100/100 14:20, Epoch 1/1]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StepTraining Loss
1015.547905
208.262823
301.327862
400.542546
500.443762
600.280686
700.235280
800.219503
900.185039
1000.161899

" ] }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": [ "πŸŽ‰ SFT training completed!\n", "πŸ’Ύ Saving to: lfm2-vl-med\n" ] } ], "source": [ "from trl import SFTConfig, SFTTrainer\n", "\n", "sft_config = SFTConfig(\n", " output_dir=\"lfm2-vl-med\",\n", " num_train_epochs=1,\n", " per_device_train_batch_size=1,\n", " gradient_accumulation_steps=16,\n", " learning_rate=5e-4,\n", " warmup_ratio=0.1,\n", " weight_decay=0.01,\n", " logging_steps=10,\n", " optim=\"adamw_torch_8bit\",\n", " gradient_checkpointing=True,\n", " max_length=512,\n", " dataset_kwargs={\"skip_prepare_dataset\": True},\n", ")\n", "\n", "print(\"πŸ—οΈ Creating SFT trainer...\")\n", "sft_trainer = SFTTrainer(\n", " model=model,\n", " args=sft_config,\n", " train_dataset=train_dataset,\n", " eval_dataset=eval_dataset,\n", " data_collator=collate_fn,\n", " processing_class=processor.tokenizer,\n", ")\n", "\n", "print(\"\\nπŸš€ Starting SFT training...\")\n", "sft_trainer.train()\n", "\n", "print(\"πŸŽ‰ SFT training completed!\")\n", "\n", "sft_trainer.save_model()\n", "print(f\"πŸ’Ύ Saving to: {sft_config.output_dir}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "pkmnNHBSzFCn" }, "source": [ "## (Optional) Save merged model\n", "\n", "In case you used LoRA, merge the extra weights learned back into the model to obtain a \"normal\" model checkpoint." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "id": "QHB1ACVMzFZ5", "colab": { "base_uri": "https://localhost:8080/", "height": 83, "referenced_widgets": [ "5450293fdd7b463f8f280c180ec8d9d3", "0b2420d265164eeba8ca434499a65011", "16632b0e3d1e45bc88e4acb651f057cb", "b44b00ef378a4fd69c932616b797d56d", "c80fddbbf60148a59cf41eea9af06f7c", "e8f847fd469b459cb9896e95eca4a3a4", "d28ad49ce00c48e4b86e01d8f4f50f58", "0bb4c0d6cb9b4897a8c95658e76e0f6f", "4eb5219031854313a1d9b800e7bdec73", "87f399fca863425db6e20b7d72429617", "9066e3b77fd64665bcdb88fa8be09640" ] }, "outputId": "71c0c908-e8ed-4773-9b7a-8a2b4839e9f7" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "πŸ”„ Merging LoRA weights...\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "Writing model shards: 0%| | 0/1 [00:00