GPU Optimization

Tutorials

I made local LLMs run as much as 2.24 times faster on a 4GB NVIDIA GPU.

llama.cpp tuning on a 4GB NVIDIA GPU can double local LLM speed. Full GPU offloading improved Qwen models; KV cache reduced memory with minor cost.

Oct 24, 2026