Gemma 4 emerged as the winner among four local AI models I tested on a literary translation, but with a catch.

A Windows laptop with an NVIDIA GTX 1650 Ti and 4 GB of VRAM, four GGUF models at Q4_K_M quantization, and the first 20 non-empty lines of an English novel: that was the setup for a practical test of which small local model can be trusted to translate literary English into Serbian. Gemma 4 E2B finished fastest and passed every automated check, yet it still invented words and bent Serbian grammar. The model built for translation, TranslateGemma 4B, did not escape those problems either.

The test matters for anyone running AI locally on modest hardware. Speed and a clean pass rate are easy to measure; literary quality is not. This pilot shows how far apart the two can be.

The setup: one novel, four models, fixed settings

The source was The Adventures of Ferdinand Count Fathom. The prepared TXT file held 3,694 non-empty source lines, or 158,396 words. Only the first 20 non-empty lines went to each model, totaling 943 English words.

Those "lines" were not guaranteed to be grammatical sentences. Some packed in several sentences, and the text preparation had joined boundaries such as "impracticable.Indeed" and "ostentation.I have". Both defects make translation harder and limit what the results can tell us.

All four runs used the same laptop and local llama.cpp server, with a 4,096-token context, temperature 0, seed 0, a 512-token output cap, sequential calls, and cache_prompt=false. Each model used its own server-side chat formatting, so this compared practical local configurations, not isolated neural-network weights.

Models tested:

• TranslateGemma 4B — translategemma-4b-it-Q4_K_M.gguf
• Gemma 4 E2B — google_gemma-4-E2B-it-Q4_K_M.gguf
• Qwen3 4B — Qwen3-4B-Q4_K_M.gguf
• Qwen3.5 4B — Qwen3.5-4B-Q4_K_M.gguf

The English instruction asked for fluent Serbian in Latin script, preservation of literary style and meaning, and no commentary around the translation. Its text never changed: the recorded prompt SHA-256 matched across all runs.

Gemma 4 E2B won on speed

Gemma 4 E2B reached 59.65 output tokens per second. TranslateGemma reached 41.76, Qwen3 40.06, and Qwen3.5 34.62. Gemma 4 generated roughly 43% more tokens per second than TranslateGemma and 72% more than Qwen3.5.

Prompt processing followed the same pattern: about 325.39 tokens/s for Gemma 4 E2B, 209.74 for TranslateGemma, 180.91 for Qwen3, and 171.15 for Qwen3.5.

Elapsed time told a clearer operational story. Gemma 4 finished the 20 passages in 38.95 seconds; TranslateGemma took 55.48, Qwen3.5 117.16, and Qwen3 147.59. These are wall-clock figures including failed requests and retry overhead, not clean inference benchmarks.

Mean latency for accepted translations was 1.81 seconds on Gemma 4, 2.63 on TranslateGemma, 3.47 on Qwen3.5, and 3.83 on Qwen3. The 95th-percentile accepted-response latencies were 2.81, 4.06, 6.47, and 7.51 seconds. Those percentiles rest on very small samples and should not be overread.

One caveat: a 20-line batch cannot rank models across all workloads. The Qwen models also produced failed responses and extra requests, so elapsed time measures end-to-end usefulness in this workflow rather than architecture-only speed.

A 100% pass rate is not quality

TranslateGemma accepted 20 of 20 lines. Gemma 4 also accepted 20 of 20. Qwen3.5 accepted 19 of 20, and Qwen3 only 16 of 20. These were technical checks: empty output, invalid scripts, truncation, formatting leaks, and line-alignment problems. They said nothing about whether the Serbian was elegant, grammatical, or faithful.

Reading the outputs side by side made that gap obvious.

The opening passage. The novel's first sentence weighs how hard it is to write a dedication flattering a patron without inviting public ridicule. Near the end, the English calls the task "altogether impracticable". TranslateGemma returned a phrase resembling "bili saglasni da je ova zadata posta"; Gemma 4 returned "složili da je zadatak potpuno nemoguć." On that clause, Gemma 4 keeps the meaning and TranslateGemma does not. But the same Gemma 4 passage contains "samopoželju patrona", an unnatural and seemingly invented expression. A better local result is not a publishable translation.

A false friend. Where the source reads "In dedication, as in poetry, there is no medium…", with "dedication" meaning the literary dedication and "no medium" meaning no middle ground, TranslateGemma produced "U čast, kao i u poeziji, nema sredstva…" and Gemma 4 produced "U posvećenju, kao i u poeziji, nema sredstva za to…". Gemma 4 read the dedication context more closely, but both misread "no medium". A generic script validator cannot catch this: the words look like Serbian and the passage is still wrong.

Grammar that flips a relationship. In a passage about the narrator's affection and sincerity, TranslateGemma reversed the intended relationship with wording resembling "you are credible and sincere to me." Gemma 4 came closer: the addressee is convinced of the narrator's affection and sincerity. Yet it still used stiff constructions such as "ovaj adres". Elsewhere Gemma 4 produced "Osmešila sam se … i tremeo sam …", switching grammatical gender and using a malformed verb in one sentence. On "disgrace and discomfiture of vice", neither Gemma model expressed the moral meaning well. The failures went beyond spelling into interpretation, register, and coherence.

Serbian variety: Latin, Cyrillic, Ekavian, Ijekavian

Serbian is written in Latin or Cyrillic script, and both Ekavian and Ijekavian are recognized pronunciation standards. The practical target here was consistent Serbian Latin in the Ekavian variety, a tighter constraint than producing something merely recognizable as Serbian.

TranslateGemma's accepted outputs included Ijekavian or regionally different forms such as "svijet", "mjeri", and "posljedice". Gemma 4 generally leaned Ekavian in these examples but made numerous grammatical mistakes. Qwen3.5 mixed forms such as "cijenimo" and "nedosljednost" with Ekavian forms elsewhere.

Ijekavian is not "incorrect Serbian". The problem for this use case was inconsistency, sometimes combined with vocabulary drifting toward other closely related standards. An editorial workflow should pick the desired variety up front, test for consistency, and still rely on human review. Mechanical spelling normalization cannot repair a misunderstood sentence.

Cyrillic was a real but mostly fixable problem. The first TranslateGemma prototype rejected every response containing Cyrillic, and with temperature 0 and seed 0, retrying the same request three times tended to reproduce the same Cyrillic. That burned inference time without improving the answer.

The pipeline was changed to keep each raw response and, when safe, transliterate Serbian Cyrillic into Latin as a separate postprocessing step, handling љ→lj, њ→nj, џ→dž, ђ→đ, ћ→ć, ч→č, ш→š, and ж→ž. The original output stays in the attempt log so native script behavior remains measurable.

In the 20-line TranslateGemma test, five outputs needed transliteration (25%), and all 20 were accepted without extra inference. Gemma 4 and Qwen3.5 needed none. Qwen3 accepted seven transliterated passages but generated some Cyrillic with unsupported non-Serbian characters; three lines failed the script check despite repeated attempts. Transliteration changes the writing system, not the meaning: an incoherent sentence stays incoherent in Latin letters.

The Qwen3.5 "ništa" loop

The most memorable failure came in passage four of the Qwen3.5 run. The source discusses an author expressing genuine gratitude while the public dismisses that praise as self-interest or flattery. Instead of finishing, the model repeated the Serbian word "ništa" ("nothing"): "…ništa ništa ništa ništa ništa ništa…"

All three Qwen3.5 attempts hit exactly 512 output tokens and ended with finish_reason=length, repeating essentially the same degeneration because prompt, seed, and decoding settings were unchanged. The passage did not need 512 useful tokens; the model was stuck in a generation loop, and raising the cap to 1,024 would not fix the repetition by itself.

Qwen3 showed a similar pattern on passage five: three truncated responses at the token limit plus three separate lines failing script validation. Qwen3 ended with 16 accepted passages after 28 HTTP calls; Qwen3.5 finished with 19 accepted after 22 calls.

The engineering lesson: a retry policy should understand the error. Transport failures may deserve retries. Deterministic script violations and repetition loops need different treatment, such as model-specific decoding changes, bounded repetition detection, or explicit human review, not two automatic copies of the same failed request.

What the numbers actually say

Under these exact local settings, Gemma 4 E2B was the strongest overall practical performer: highest observed generation and prompt-processing throughput, shortest end-to-end run, all 20 responses accepted, no transliteration or retry overhead, and better meaning preservation than TranslateGemma on several passages.

It did not win literary translation in the editorial sense. It still coined strange words, mishandled idioms, lost grammatical agreement, and rendered nuanced prose awkwardly. TranslateGemma, the model with translation in its name, did not escape these problems. Qwen3.5 delivered 19 technically acceptable lines but was weakened by quality defects and a repetition loop. Qwen3 was the least reliable configuration in this batch.

None of this proves one model family is "better" at Serbian, or that a different prompt, tokenizer configuration, decoding setup, or cleaned dataset would not change the ranking. The sample was small, contiguous, and literary, with text-preparation artifacts, and the models used different native chat templates. There was no independently curated Serbian reference translation, no BLEU, chrF++, or COMET evaluation, and no blind multi-rater human scoring. These results are an honest pilot, not a publication-grade multilingual benchmark.

What a production pipeline should change

Clean the source text first, restoring lost spaces and separating real sentences, since joined boundaries and multi-sentence lines distorted the input. Fix the target variety up front and test outputs for consistency rather than normalizing spelling after the fact. Keep raw responses and transliterate as a separate, logged step. Treat repetition loops and script violations as distinct error classes with their own handling. And budget for human review: every model here passed technical checks while still producing Serbian that no editor would publish.

The broader takeaway for local-AI users is that a fast model with a perfect acceptance rate can still be the wrong tool for literary work. Automated checks measure whether output is well-formed, not whether it is faithful. On a 4 GB GPU, Gemma 4 E2B is the configuration to start with; it is not the configuration to ship without a reader.