Improving Structured Output Generation by Fine-Tuning a 350M-Parameter Model with Just 100 GRPO Steps

Structured output is one of the everyday jobs people actually ask large language models to do, but most benchmarks bury it inside broader reasoning or extraction scores instead of measuring it directly. What often decides whether a model can be plugged into a real system is simple: can it reliably return valid, parseable output in exactly the format and shape you asked for? This guide is for developers and tinkerers who want to make small, local models better at that specific skill — schema compliance — and shows how a short fine-tuning run on free-tier hardware can get a 350M-parameter model close to models many times its size.

One important note up front: the training pipeline described here is not the one used to train the RL model described in the IFStruct blog. The goal is not to recreate the IFStruct benchmark score, but to demonstrate how task-specific fine-tuning of smaller models can improve performance.

What You Need Before Starting

This tutorial has two halves that run in different places:

• Fine-tuning runs on a GPU. The accompanying notebook is sized for a free-tier Colab or Kaggle GPU.
• Evaluation runs locally on a MacBook (here, a MacBook Pro with an Apple M5 Max and 36 GB of unified memory) through llama.cpp, which exposes an OpenAI-compatible server that the IFStruct evaluator connects to.

You'll need uv for Python tooling and llama.cpp for serving. Following the Liquid AI llama.cpp deployment docs, install llama.cpp with Homebrew and confirm that llama-server is available.

Baseline: Evaluating LFM2.5-350M on IFStruct

First, let's evaluate LFM2.5-350M on the IFStruct benchmark to see whether we can reproduce the reported score of 21.1%.

IFStruct is a benchmark for testing the validity of LLM outputs and schema adherence. It's open source in the Liquid4All/ifstruct repo, with the public dataset on Hugging Face at LiquidAI/ifstruct-v1.0.

For the evaluation, we serve the model locally on the MacBook using llama.cpp and the BF16 GGUF (LiquidAI/LFM2.5-350M-GGUF). Then we start the base-model server with these flags:

• `--alias`: the model name IFStruct sends to the OpenAI-compatible endpoint
• `-ngl 99`: offloads all layers to the GPU when available
• `-np 4`: serves four requests in parallel
• `-c 32768`: prompt context size

Once the server is running, we run the full benchmark with 2000 samples.

The IFStruct release blog reports 21.1% for LFM2.5-350M. Our local llama.cpp/BF16 setup measures 22.6% — close to the published number. We use this local result as the baseline so both runs use the identical serving stack.

GRPO Fine-Tuning with TRL

The full, runnable pipeline lives in the accompanying notebook. Here are the key pieces.

Training data. We use nvidia/Nemotron-RL-instruction_following-structured_outputs, which pairs each prompt with a target JSON Schema and an expected field count, using about 500 samples. Because the Nemotron data distribution differs from IFStruct's evaluation, we augment the prompts to close two gaps:

1. 40% of prompts get a "return the output inside a fenced code block" instruction appended, so the model learns to follow format instructions rather than always emitting raw JSON.
2. A disjoint 20% are converted into top-level-array tasks (the schema is wrapped in an array with a required item count), training bare-list output and item-count compliance.

Model and LoRA. We load LiquidAI/LFM2.5-350M and attach a LoRA adapter. Since LFM2.5 uses a hybrid attention/convolution architecture, we target the LFM-specific module names. This trains roughly 6M parameters — about 1.66% of the model.

Reward functions. Three reward functions, each on a [0, 1] scale, score every completion on whether the extracted structure is correct:

• `json_format_reward`: Is the output parseable, and in the requested form? Full credit (1.0) for the requested form (fenced vs. raw), 0.2 for the wrong-but-parseable form, 0.0 for unparseable output.
• `field_count_reward`: Does the object have the expected number of top-level fields? An exact match earns 1.0, and the score decays linearly with the miss.
• `schema_validation_reward`: Does the output validate against the row's JSON Schema? It counts constraint violations and gates partial credit on required-key coverage.

These are combined as a weighted sum with `reward_weights=[1.0, 0.5, 2.0]`.

Training. We train for 100 steps with 8 generations per prompt group, sized for a free-tier 16 GB GPU. Over the run, all three reward components climb, the KL from the reference model lifts off zero after warmup, and the truncated-completion fraction stays near zero.

Merging and saving. Finally, we merge the LoRA adapter back into the base weights and save a single self-contained checkpoint, ready to convert to GGUF for serving.

Re-Evaluating the Fine-Tuned Model

After GRPO fine-tuning, we rerun the IFStruct evaluation. First, the merged checkpoint must be converted into a BF16 GGUF — the converter script ships with the llama.cpp source, so we clone the repo once and install the converter's gguf package. Then we serve the merged model with the same setup and run the full IFStruct evaluation again.

Comparing both runs on the identical serving stack, the gains land exactly where the training aimed: the JSON pass rate rises by nearly 14 points (18.0% → 31.9%), while YAML stays mostly the same. This is still below the Qwen3.5-2B score of 33.15%, but it shows that even light task-specific fine-tuning can bring a small model close to a larger one.

Common mistakes

• Skipping the baseline run. Without a baseline measured on your own serving stack, you can't tell whether gains come from fine-tuning or from differences in setup.
• Mixing up evaluation locations. Training runs on a GPU notebook; evaluation runs locally through llama.cpp. Serving the wrong checkpoint or the wrong GGUF precision will give misleading numbers.
• Wrong LoRA target modules. LFM2.5's hybrid architecture means you must target the LFM-specific module names; standard attention-only targets will miss most of the model.
• Ignoring the augmentation splits. The fenced-code-block (40%) and top-level-array (20%) augmentations are disjoint and intentional — changing the proportions shifts what the model learns.
• Overlooking reward weights. The combination `reward_weights=[1.0, 0.5, 2.0]` heavily prioritizes schema validation; changing these weights changes what the model optimizes for.

To reproduce or extend this work, see the original IFStruct v1.0 blog post, the Liquid4All/ifstruct benchmark repo, and the LiquidAI/ifstruct-v1.0 dataset.