Skip to main content

COMPEL Glossary / peft-parameter-efficient-fine-tuning

PEFT (parameter-efficient fine-tuning)

A family of fine-tuning techniques — most prominently LoRA, QLoRA, and adapters — that update only a small fraction of model parameters while freezing the rest.

What this means in practice

Achieves most of the quality benefit of full fine-tuning at a fraction of the compute and storage cost; widely used as the middle rung on the customisation ladder (RAG → few-shot → PEFT → full fine-tune).

Synonyms

parameter-efficient fine-tuning , LoRA , QLoRA , adapter tuning

See also

  • Distillation — The training of a smaller "student" model to imitate a larger "teacher" model's behaviour — typically on a shared dataset of prompts and teacher outputs.
  • Quantization (AI cost) — Representation of model weights (and sometimes activations) at lower numerical precision — INT8, INT4, or mixed-precision — to reduce memory footprint and accelerate inference.
  • Fine-Tuning — Fine-tuning is the process of further training a pre-trained AI model on a specific dataset to adapt it for a particular task or domain.