Continuing the training of a pretrained model on new data so it follows instructions, adopts a domain or style, or reasons better. Full fine-tuning updates every weight; parameter-efficient methods train only a small number of added parameters.

How many parameters does LoRA train compared with full fine-tuning?

In the simplified example from a public LinkedIn post, a model of about 70 billion parameters has 175,000 weight matrices of 800 x 500. Rank-4 LoRA trains about 910 million parameters (1.3%), LoRA-FA about 350 million (0.5%) because it also freezes one of the two adapter matrices, and QLoRA the same 910 million as LoRA but with the frozen base stored in 4-bit. Real models differ, so treat these as illustrative.

A method from the paper "Learning to Reason in 13 Parameters" that scales low-rank adapters down to a handful of parameters. The authors report 91% accuracy on GSM8K for the 8B size of Qwen2.5 with 13 trained parameters, and say this needed reinforcement learning; supervised fine-tuning required 100 to 1000 times larger updates.

How are the techniques on this page chosen and ordered?

They are grouped by role: parameter-efficient methods, instruction, preference and reinforcement objectives, tools and guides. They are not ranked and the list is not complete. Check each paper and library for current details.

Fine-Tuning Techniques (2026): LoRA, QLoRA, TinyLoRA, DPO, GRPO

What Fine-Tuning Is and How Many Parameters It Updates

Fine-tuning continues training a pretrained model on new data so it follows instructions, adopts a style or domain, or reasons better. Full fine-tuning updates every weight. Parameter-efficient methods freeze the base model and train a small number of extra parameters, which cuts memory and storage and lets one base model serve many tasks. The table below is a worked example from a simplified model described in a public LinkedIn post (linked under Guides & Sources): 175,000 weight matrices of 800 x 500, about 70 billion parameters in total. Real models differ, and LoRA is usually applied to selected matrices, so treat the figures as illustrative.

Method	What is trained	Trainable parameters	Share of full
Full fine-tuning	Every weight	about 70B	100%
LoRA (rank 4)	Two small matrices A (800 x 4) and B (4 x 500) per weight matrix	about 910M	1.3%
LoRA-FA (rank 4)	Only B; A is frozen	about 350M	0.5%
QLoRA (rank 4)	Same adapters as LoRA; the frozen base is stored in 4-bit instead of 16-bit	about 910M	1.3%
TinyLoRA	A tiny vector projected through fixed random tensors; with full weight tying the whole model can share one vector	as few as 13	negligible

Check on the arithmetic: each rank-4 adapter has 800 x 4 + 4 x 500 = 5,200 parameters, and 175,000 of them give 910 million. Freezing A leaves 2,000 per matrix, or 350 million. QLoRA changes how the frozen base is stored, not the number of trainable adapter parameters. For TinyLoRA, the paper "Learning to Reason in 13 Parameters" reports 91% on GSM8K with 13 trained parameters on the 8B size of Qwen2.5, and says such results required reinforcement learning, with supervised fine-tuning needing 100 to 1000 times larger updates. Entries below are grouped by role, not ranked, and the list is not complete.

Parameter-Efficient Fine-Tuning Methods

12 entries

Papers behind the methods that train only a small number of parameters while the base model stays frozen.

01Adapters: Parameter-Efficient Transfer Learning for NLPSmall bottleneck modules inserted into a frozen transformer and trained for each task. 02Prefix-TuningOptimizes continuous prompt vectors prepended to the model's activations for generation tasks. 03Prompt TuningThe Power of Scale for Parameter-Efficient Prompt Tuning: learns soft prompts, with results that improve as models grow. 04(IA)3 (T-Few)Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning. 05LoRALow-Rank Adaptation: freezes the pretrained weights and trains two small low-rank matrices per adapted layer. 06LoRA-FAEfficient and effective low-rank fine-tuning that also freezes one of the two low-rank matrices to cut activation memory. 07QLoRAEfficient finetuning of quantized LLMs: LoRA adapters trained on top of a frozen 4-bit base model. 08DoRAWeight-Decomposed Low-Rank Adaptation, which splits weights into magnitude and direction. 09rsLoRAA rank-stabilization scaling factor for fine-tuning with LoRA. 10VeRAVector-based Random Matrix Adaptation, which shares frozen random matrices and trains small vectors. 11LoRA-XSLow-rank adaptation with an extremely small number of trainable parameters. 12TinyLoRA: Learning to Reason in 13 ParametersIntroduces TinyLoRA. The abstract reports training the 8B size of Qwen2.5 to 91% on GSM8K with only 13 trained parameters (26 bytes in bf16), and says strong results needed RL: supervised fine-tuning required 100 to 1000 times larger updates.

Instruction Tuning, Preference & Reinforcement Fine-Tuning

4 entries

Fine-tuning objectives that teach a model to follow instructions, match preferences or reason.

13FLAN: Finetuned Language Models Are Zero-Shot LearnersInstruction tuning on many tasks improves zero-shot performance. 14InstructGPT (RLHF)Training language models to follow instructions with human feedback. 15Direct Preference Optimization (DPO)Optimizes a policy directly from preference data without a separate reward model. 16DeepSeekMath (GRPO)Introduces Group Relative Policy Optimization, widely used for reinforcement fine-tuning of reasoning models.

Libraries & Frameworks

9 entries

Open-source tooling for running fine-tuning jobs.

17Hugging Face PEFTLibrary of parameter-efficient methods including LoRA, QLoRA, IA3, prompt tuning and TinyLoRA. 18Hugging Face TRLLibrary for supervised fine-tuning, DPO, GRPO and other post-training methods. 19bitsandbytesQuantization library used for 8-bit and 4-bit training and inference. GitHub lists 0.50.2 as the latest release, from August 2026. 20UnslothLibrary for faster, lower-memory fine-tuning of open models. 21AxolotlConfiguration-driven framework for fine-tuning LLMs. GitHub lists v0.21.1 as the latest release, from October 2026. 22LLaMA-FactoryUnified framework for fine-tuning many open models. GitHub lists v0.9.5 as the latest release, from May 2026. 23LitGPTPretrain, fine-tune and deploy LLMs from Lightning AI. GitHub lists v0.5.13 as the latest release, from June 2026. 24torchtunePyTorch library for post-training and fine-tuning LLMs. Its README says torchtune is no longer actively maintained after development wound down in 2025. GitHub lists v0.6.1 as the latest release, from April 2025. 25MLX LMRun and fine-tune LLMs on Apple silicon with MLX. GitHub lists v0.32.2 as the latest release, from October 2026.

Guides & Sources

4 entries

Where the parameter comparison on this page comes from, and further reading.

26How many parameters do we need to fine-tune? (LinkedIn)Public LinkedIn post comparing the trainable parameters of full fine-tuning, LoRA, LoRA-FA, QLoRA and TinyLoRA on a simplified model; the worked example above follows its numbers. 27PEFT: TinyLoRA documentationHugging Face PEFT reference for the TinyLoRA method. 28LoRA Without Regret (Thinking Machines)Blog post on when LoRA matches full fine-tuning. 29OpenAI: model optimization guideOpenAI's guide to optimizing models, including fine-tuning.

Related Pages on This Site

16 entries

LLM building, optimization, quantization, reasoning, reinforcement learning and open-model pages that relate to fine-tuning.

30LLM Building StepsThe six-step pipeline for building an LLM. 31AI Model Optimization TechniquesPruning, quantization and distillation. 32AI Model Quantization TechniquesGPTQ, AWQ, SmoothQuant and related methods. 33LLM Model Formats ExplainedGGUF, GPTQ, AWQ and EXL2 compared. 34LLM Training & Inference OptimizationKV cache, batching and other techniques. 35LLMOpsOperating LLM applications in production. 36Reinforcement Learning GuideFrom MDPs to deep reinforcement learning. 37Large Reasoning Models17 large reasoning models compared. 38Reasoning Models for Edge DevicesSmall reasoning models and on-device runtimes. 39Small Language Models (SLMs)Phi, Gemma, Qwen, SmolLM and other small models. 40Open-Source LLMsDeepSeek, Qwen, Llama, Kimi and GLM. 41Data Sets for Generative AIDatasets used to train and tune generative models. 42Linear Algebra for AIMatrices, SVD, attention and LoRA. 43Linear Algebra for Deep LearningTensors, attention and LoRA. 44Generative AILLMs, assistants, platforms and datasets. 45LLM HubThe hub for large language model directories.