Your Runtime’s Parser Determines How Local LLMs Call Tools

If you run agents on your own machine, local LLM tool calling depends on a hidden parser that converts model text into a structured call. Decoding AI's Local Agent Lab part 1 of 3 read the source of Ollama, llama.cpp, vLLM and SGLang on 6 October 2026. The audit found 41 model families with a dedicated tool-call parser in at least one runtime, but only 8 families covered in all four. For readers, this means model choice alone does not guarantee agent-ready tool calls; the runtime, parser path and documentation all need to line up.

Why the runtime decides whether tool calls work

Models emit text, not executable calls. Each family wraps calls in its own format: a `<tool_call>` block, a `[TOOL_CALLS]` marker, or Harmony channels used by GPT-OSS. A runtime parser cuts that text into a function name and JSON arguments. If the parser is missing or wrong, calls arrive as plain text, so an agent sees a model describing a tool instead of using it.

The runtimes choose parsers differently. vLLM requires `--tool-call-parser` and refuses `--enable-auto-tool-choice` without one. SGLang lets you name a parser or pass `auto`; auto reads the chat template and has detection rules for 28 of its 42 parsers, excluding Llama 3. Ollama uses the parser named in the model config, or a generic parser that takes the tool-call tag from the template. llama.cpp compares the chat template with 16 dedicated handlers, then builds a template-derived parser if none matches.

What the parser census found

The four runtimes ship 136 named tool-call parsers: vLLM 53, SGLang 42, Ollama 25 and llama.cpp 16. Ollama's switch lists 27 names, but passthrough and olmo3-think report no tool support, so 25 count. Of 41 families, coverage is: all four, 8; three, 7; two, 10; one, 16, including Granite, Nemotron and Jamba. The eight with all four are Qwen, DeepSeek, Gemma, Mistral, GPT-OSS, Cohere Command, LFM2 and Muse Glimmer. GPT-OSS works in Ollama because its Harmony format is detected from the template.

The seven popular families chosen before counting were Qwen, Llama, Gemma, Mistral, DeepSeek, Phi and GPT-OSS. Five have dedicated parsers everywhere. Llama has one only in vLLM and SGLang; Phi only in vLLM, as phi4_mini_json. Llama 3.x still works in llama.cpp: chat tests run three Llama 3 templates through the template-derived parser. The status line is 6 October 2026, 12:45 UTC, using main branches at Ollama 8a971df, llama.cpp 4f54067, vLLM 3e18218 and SGLang 1291f31. Parsers are added most weeks, so an installed release may have fewer.

Where the documentation falls behind

The code is ahead of the docs. vLLM's tool-calling page names 29 of 53 parsers; six more appear elsewhere, and 18 appear nowhere, including phi4_mini_json, kimi_k3, minimax_m3 and llama4_json. SGLang's tool-parser page names 14 of 42, while its server-arguments reference lists 41. llama.cpp's function-calling page lists 15 native formats, including Llama 3.1, Hermes and Qwen 2.5; today only 3 have dedicated handlers: Functionary v3.2, GPT-OSS and LLM-jp. The other 12 use the template-derived parser, and 13 of the 16 handlers belong to formats the page does not list. The page also mentions Chat format: Generic, a log line no longer present in source. Ollama's tool-calling page lists no parsers because they are set per model.

None of this means features are broken; it means the source tree is the reliable inventory. For vLLM, run `grep -c '^ "' vllm/tool_parsers/__init__.py` to re-check.

How to check your model's format

To pick a parser, identify the model's tool-call format. Decoding AI's toolfmt.py checks a tokenizer_config.json or .jinja template with plain Python. Run `python3 toolfmt.py tokenizer_config.json`. It maps markers: `<|channel|>` to Harmony (GPT-OSS style), `<|tool_call>call:` to Gemma 4, `tool▁calls▁begin` and `DSML` to DeepSeek variants, `<|tool_calls_section_begin|>` to Kimi K2, `[TOOL_CALLS]` to Mistral, `<function=` plus `<parameter=` to Qwen3-Coder XML, `Environment: ipython` to Llama 3.x JSON, and `<tool_call>` to a generic wrapper used by Hermes, Qwen, GLM, Granite and others.

The script was run on all 71 chat templates in llama.cpp's repo and checked by hand. Meta's Llama 3.1 template printed FORMAT Llama 3.x JSON. The most common answer, 19 of 71, was the generic `<tool_call>` wrapper, where the family name matters: GLM, Granite and Qwen put different payloads inside the same tag.

What this does not prove, and what comes next

The audit read registration and selection code, not READMEs, using shallow clones of main branches at the status commits. It used vLLM's lazy parser table, SGLang's parser-name list, Ollama's parser switch and llama.cpp's specialised-template dispatch. Names were mapped to families by hand; generic formats such as Hermes, pythonic and JSON were left out, and unclear names were checked in model docs. A doc mention counted only as a parser argument or code formatting, not as an ordinary sentence. Counting scripts stayed with the draft. One tested-parser measurement was dropped after a hand check found false "untested" results.

The audit does not prove tool calls succeed. No models were run, so a dedicated parser is not proof of good tool calling and a generic parser is not proof of bad. A family counts if any runtime has a parser for any member: Ollama's Mistral parser targets Ministral 3, and llama.cpp's DeepSeek handler targets the V3.2 format. The audit covers main branches only, not older releases, and excludes LM Studio and LocalAI.

For practical checks: Ollama supports tool calling when the model template describes tools. vLLM needs the family parser via `--tool-call-parser`, with 18 of 53 undocumented. llama.cpp supports tool calling by starting llama-server with `--jinja`. If a model prints the call as text, check the parser or chat template.

Local Agent Lab part 2 tomorrow measures the same ten tools rendered through each family's chat template and the tokens spent before an answer. Part 3 will compare default context windows and how long an agent runs before hitting one. A day-21 re-probe on 13 October will examine GPT-6 Sol support across AI frameworks three weeks after launch. Every family counted here emits tool calls in some format.

This article is based on Decoding AI's Local Agent Lab part 1, originally published in Towards AI on Medium. It builds on Decoding AI's "We Timed 50 Model-Support Commits Across 3 Local LLM Runtimes" and "Why Your Claude Code Token Usage Is So High," and uses vLLM and SGLang documentation read on 6 October 2026, llama.cpp function-calling.md and common/chat.cpp at commit 4f54067, and Ollama tool-calling pages and model/parsers/parsers.go at commit 8a971df.

Meta description: Local LLM tool calling depends on runtime parsers. A source audit of Ollama, llama.cpp, vLLM and SGLang maps model support and documentation gaps.

Tags: local LLM tool calling; parser audit; Ollama; llama.cpp; vLLM and SGLang

Featured image: abstract friendly illustration of four local LLM runtimes sorting tool calls into structured JSON, no real people, no logos, no text