How do I grasp fundamentals and set up environment?

# Understand Key Concepts Fine-tuning adapts pre-trained LLMs to specific tasks by updating weights on custom data. Learn Supervised Fine-Tuning (SFT), LoRA/QLoRA for efficiency (train ~1% parameters, 70% less VRAM). ## Installation Use Google Colab (free GPU) or local setup: ```bash pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" pip install --no-deps trl peft accelerate bitsandbytes xformers datasets ``` Login to Hugging Face: ```python from huggingface_hub import login login() ``` **Why?** Builds foundation to avoid errors later. 71 72

How do I prepare high-quality training dataset?

## Load and Format Data Use Hugging Face Datasets: ```python from datasets import load_dataset dataset = load_dataset("json", data_files={"train": "your_data.jsonl"}, split="train") ``` Format for chat/instruct: ```json {"messages": [{"role": "user", "content": "Prompt"}, {"role": "assistant", "content": "Response"}]} ``` Or prompt-completion. Apply chat template. Split 90/10 train/eval. Aim 1000+ examples. **Tip:** Start with `trl-lib/Capybara` for testing. 73

How do I configure model with peft for efficiency?

## Load Quantized Model ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/llama-3.1-8b-bnb-4bit", max_seq_length=2048, load_in_4bit=True ) ``` Add LoRA: ```python model = FastLanguageModel.get_peft_model(model, r=16, target_modules=["q_proj", "k_proj", ...], lora_alpha=16) ``` Supports Llama3, Mistral etc. 4x less VRAM. 72

How do I train model with sfttrainer?

## Setup and Train ```python from trl import SFTTrainer, SFTConfig trainer = SFTTrainer( model=model, train_dataset=dataset, tokenizer=tokenizer, args=SFTConfig( per_device_train_batch_size=2, gradient_accumulation_steps=4, warmup_steps=10, max_steps=60, learning_rate=2e-4, output_dir="outputs", optim="adamw_8bit" ) ) trainer.train() ``` Monitor loss dropping to ~1.0. 73

How do I evaluate, save, and deploy model?

## Inference Test ```python FastLanguageModel.for_inference(model) inputs = tokenizer(["Your prompt"], return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=128) print(tokenizer.decode(outputs[0])) ``` Save LoRA: ```python model.save_pretrained("my_lora_model") ``` Merge/push to HF Hub. Deploy: vLLM, Ollama (GGUF export). 74

Master Fine-Tuning LLMs in 5 Practical Steps

🧰

Instant Toolkit

2 artifacts

📋
Step-by-Step Guide

1

Understand Key Concepts

Fine-tuning adapts pre-trained LLMs to specific tasks by updating weights on custom data. Learn Supervised Fine-Tuning (SFT), LoRA/QLoRA for efficiency (train ~1% parameters, 70% less VRAM).

Installation

Use Google Colab (free GPU) or local setup:

pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes xformers datasets

Login to Hugging Face:

from huggingface_hub import login
login()

Why? Builds foundation to avoid errors later.

Why this step matters:

-Establishes core knowledge for effective customization
-Enables hands-on experimentation on real hardware

1-2 hours

Google Colab, Hugging Face Hub, Python 3.10+, Git

$0

Definition of Done

•Explain SFT vs. LoRA in own words
•Successfully install Unsloth and login to HF

Common Mistakes to Avoid

❌ Skipping HF login for gated models

→Run login() early and accept model licenses

❌ Using incompatible CUDA/PyTorch versions

→Use Unsloth's [colab-new] for latest compat

2

Following along, or just reading? 👀

Spin up a personalized “learn fine-tuning LLMs” plan you can save, check off, and return to anytime — unlimited on the free trial.

Start free trial →

3

4

5

🎥
Suggested Videos

Fine-tuning Large Language Models (LLMs) w/ Example Code

Finetune LLMs with HuggingFace & PyTorch

📚
Learning Resources

📖 Online Courses

Finetuning Large Language Models

beginner

DeepLearning.AI

Generative AI Advanced Fine-Tuning for LLMs

intermediate

Paid

Coursera

📖 Recommended Books

A Hands-On Guide to Fine-Tuning Large Language Models

beginner

by DVG

Build a Large Language Model (From Scratch)

intermediate

by Sebastian Raschka

🌐 Helpful Websites

Hugging Face Transformers Training Docs

documentation

Official guide with Trainer API examples

Unsloth Fine-Tuning Guide

documentation

Efficient QLoRA tutorials and notebooks

OpenAI Fine-Tuning Guide

documentation

API-based fine-tuning for GPT models

🛠️ Useful Tools

Unsloth

Fine-tuning Library

2x faster fine-tuning with 70% less VRAM

Hugging Face Transformers

ML Framework

Core library for loading/training LLMs

TRL (Transformers Reinforcement Learning)

Trainer

SFTTrainer for supervised fine-tuning

Master Fine-Tuning LLMs in 5 Practical Steps