You're fine-tuning an 8B-parameter open model, the kind in the Llama-3-8B class, so it always answers a support ticket in one JSON shape: category, priority, a one-sentence summary. You have one GPU with 24GB of VRAM. The optimizer tries to allocate state for every one of the model's weights, and the process dies with an out-of-memory error before it has read a single training example.

That's strange, because the same GPU runs this model for inference without complaining. Loading 8B parameters at half precision takes about 16GB, comfortably under 24GB. Training is a different problem: on top of the weights, you need gradients and optimizer state for every parameter you update, and that stacks up to several times the model's own size. Call that gap between what a model costs to run and what it costs to train the training tax. Full fine-tuning pays it on every parameter. This post covers four ways to shrink the bill: full fine-tuning as the baseline, then LoRA, QLoRA, and (IA)³, compared on memory, quality, and complexity, ending with a rule for picking one.

Parameter-efficient fine-tuning (PEFT) is a family of methods that adapt a pretrained model by training a small add-on, not the model itself. The base model's weights stay frozen: no gradients, no optimizer state, nothing to update. You train a much smaller set of new parameters, an adapter, bolted onto or threaded through the frozen model, and at inference time the frozen weights plus the adapter together produce the fine-tuned behavior.

Full fine-tuning is the alternative these methods are all reacting against: every weight gets a gradient and optimizer statistics, whether it needed to change or not. With the Adam optimizer that's on the order of 16 bytes per parameter, roughly 128GB for 8B parameters, before a single batch is loaded.

PEFT methods make a bet: most of what a pretrained model knows is still useful, and adapting it to a new task doesn't require moving every one of its weights. It requires moving a much smaller, carefully placed set of them. How small, and where, is what tells LoRA, QLoRA, and (IA)³ apart.

LoRA (Low-Rank Adaptation) freezes the base model entirely and adds a pair of small matrices alongside a handful of its layers. Instead of learning a full table of edits for a weight matrix, LoRA learns a short recipe for them: two thin matrices, A and B, whose product approximates the full update. Multiply a tall, thin matrix by a short, wide one and you get something the same shape as the original weight, but built from far fewer numbers.

Rank (r in the diagram) is the one knob that matters most: the inner dimension of A and B, and the width of the bottleneck the update has to squeeze through.

A rank of 4 or 8 is common; higher ranks give the adapter more room to represent a bigger behavior change, at the cost of more trainable parameters. Target modules are which of the model's weight matrices get an adapter at all: a few key layers, or most of them. More target modules means more capacity and more training tax, in exchange for a shot at matching full fine-tuning more closely.

Because the base model never moves, LoRA's training tax applies only to the adapter matrices, typically well under one percent of the base model's parameter count. For the 8B model on 24GB, you still need the frozen base loaded in half precision (about 16GB), but the gradients and optimizer state are now sized to a few tens of millions of parameters instead of eight billion. Training fits in the remaining headroom, where a full fine-tune did not.

The adapter itself is cheap to keep: saved on its own, without the base model, it's typically tens of megabytes rather than the many gigabytes of the full checkpoint. In practice, for format and style tasks like the ticket-triage JSON, LoRA gets close to full fine-tuning's quality; the gap tends to widen as the behavior you're asking for gets further from what the base model already does well.

LoRA still assumes the frozen base model fits in memory at half precision. That assumption breaks down with a bigger model or a smaller card: 16GB for an 8B model is fine on a 24GB GPU, but the same math on a 70B model needs well over 100GB just to hold the frozen weights, with no room left for adapters, gradients, or activations.

QLoRA answers that by quantizing the frozen base model to 4-bit precision before training starts, while keeping the LoRA adapters themselves in higher precision. Quantization here means representing each weight with far fewer bits than the 16 or 32 the model was trained in, which shrinks the base model's memory footprint by roughly four times over half precision. The adapters still train the way they do in ordinary LoRA; only the frozen part underneath them gets smaller. So QLoRA doesn't shrink the training cost. It shrinks the cost of holding the frozen base, which is the part that didn't fit.

A minimal config, using Hugging Face's transformers, bitsandbytes and peft libraries, looks like this. The first block is what makes it QLoRA: it loads the frozen base in 4-bit. The second is the ordinary LoRA adapter:

lora_alpha scales how strongly the adapter's output is added to the frozen layer's output. The effective scale is lora_alpha / r, so 16 with rank 8 doubles the adapter's contribution. Setting it to about twice the rank is a common starting point, not a rule.

The quantization step costs something: every forward and backward pass has to dequantize the relevant weights back to a workable precision on the fly, which adds compute overhead that a plain LoRA run doesn't pay. For the 8B model, that overhead is a fair trade if the alternative is not training at all; on a model that already fits comfortably in half precision, QLoRA usually isn't worth its added complexity and slower steps.

(IA)³ (Infused Adapter by Inhibiting and Amplifying Inner Activations) skips low-rank matrices entirely. Instead of adding a matrix to a weight, it learns a vector that rescales the signals (the model's activations) flowing through a layer, one learned multiplier per dimension. Nothing is added to the weight matrices themselves; those signals are stretched or shrunk before they reach the frozen weights.

That makes (IA)³ the lightest of the three PEFT methods here. It trains fewer parameters than LoRA does, since a per-dimension scaling vector is far smaller than a pair of rank-r matrices, and it has fewer knobs: no rank to choose, no decision about how many target modules to adapt, just which layers get a scaling vector at all. Training memory sits close to the model's inference footprint, since there's no separate adapter matrix and no quantization step to manage.

That simplicity is also the ceiling. Rescaling those signals can sharpen or soften behavior the base model already knows how to do; it struggles to teach the model something structurally new. For a small, format-only nudge, such as always closing an answer with a fixed field order, (IA)³ is often enough. For a bigger shift, such as reasoning through a support ticket's category before writing the summary, it tends to underfit where LoRA has the capacity to actually learn the pattern.

Put the four methods side by side, and the trade-off is really three questions: how much memory does training need, how close does it get to full fine-tuning's quality, and how much do you have to configure to use it well.

Walk the ticket-triage example through it. Start with GPU headroom: the 24GB card rules full fine-tuning out immediately, and it rules QLoRA in as a fallback only if the base model itself stops fitting, say if the team later swaps in a larger model than 8B. With an 8B model at half precision, that headroom problem doesn't exist yet, so QLoRA's extra complexity isn't buying anything.

Next, size of the behavior shift. Getting the model to reliably choose a category, set a priority, and write a coherent one-sentence summary is more than a fixed format nudge: it's a judgment call on every ticket. That argues for LoRA over (IA)³, since (IA)³'s ceiling is exactly this kind of shift. If the task were narrower, purely "always emit valid JSON with these three field names," (IA)³ would be worth trying first, since there's nothing to configure and it's the cheapest thing to test.

One 24GB GPU is a common training budget, not the only one. The rule doesn't change on an 80GB card, only the numbers do: full fine-tuning becomes possible for smaller models, and the memory wall that forces a PEFT method moves to a bigger model size.