Create and Train a 25M-Parameter LLM from Scratch Using Only Your CPU
A tutorial shows building, pre-training, and fine-tuning a 25M-parameter LLM on CPU, covering architecture, loss, and reinforcement learning.
small language models
Less Size, Stronger Recall: How Qwen2.5-1.5B Beats the 3B Model (With Four Hypotheses)
SRRM shows Qwen2.5-1.5B beats Qwen2.5-3B on long-context recall: 86.7% retrieval and 45.7% reconstruction, challenging size-equals-memory assumptions.
Training Small Language Models Using GRPO with Verifiable Rewards
GRPO trains compact models by comparing outputs, while verifiable rewards define the learning signal. Unsloth makes local experiments practical.
Improving Structured Output Generation by Fine-Tuning a 350M-Parameter Model with Just 100 GRPO Steps
This tutorial shows developers how to improve schema compliance in small local language models by fine-tuning LiquidAI's LFM2.5-350M using GRPO with TRL and LoRA. Using ~500 Nemotron structured-output…