Coding with AI

LLMON is an LLM-native markup language now available in Granite 4.1 models with 3 billion and 8 billion parameters.

Granite 4.1 LLMON preview models use LLM-native markup to separate prompt instructions, reducing injection risk while maintaining capability.

LLMON is an LLM-native markup language now available in Granite 4.1 models with 3 billion and 8 billion parameters.

Key Takeaways

  • LLMON is an LLM-native markup language added to Granite 4.1 3B and 8B preview models to make prompt structure explicit for language models.
  • It improves security against indirect prompt injection by clearly separating instructions, data, and requirements, with attack success rates dropping to nearly zero on some benchmarks.
  • The LLMON preview models maintain general performance comparable to Granite 4.1 Instruct models while reducing contamination from irrelevant context.
  • LLMON is not just JSON or a chat template: it uses reserved special tokens, explicit tags, local nesting, and includes lossless JSON conversion tools and support for custom tags via LoRA tuning.

Meta Description: Granite 4.1 3B and 8B LLMON preview models add explicit structure to prompts, improving resistance to indirect prompt injection while keeping general capability

LLMON is an LLM-native markup language now available in Granite 4.1 models with 3 billion and 8 billion parameters.

Tags: LLMON, Granite 4.1, prompt injection, LLM security, JSON

Featured image: abstract friendly illustration of a clean prompt structure diagram with blocks, tags, and data flowing into an output block; no real people, no logos, no text.

For developers using Granite 4.1 3B or Granite 4.1 8B, the new LLMON preview models provide a way to make prompt structure explicit before a large language model (LLM) generates the next tokens. The release includes granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, experimental models derived from Granite 4.1 base models and trained to consume and generate text marked with LLM Object Notation, or LLMON, pronounced "lemon." It also adds a GitHub repository with a toolkit, workflow examples, templates, converters, Mellea support, and an attention visualization demo.

Why LLMON Matters for Long Contexts

LLMs generate the next set of tokens from whatever appears in their context. Over a long trajectory, that context can accumulate many different pieces: a system prompt, past conversation turns, instructions that act on retrieved documents, tool responses or user data, requirements constraining the allowed output, and reasoning chains. Much of that structure is implicit to the model or encoded in formats such as JSON, which were not designed to be digested by LLMs.

That implicit structure is one of the main reasons LLMs, especially smaller models, can be brittle and sometimes fail in unexpected ways. It also creates openings for attackers. Prompt injection is one example: a malicious instruction can be hidden inside data and treated by the model as if it were a legitimate command.

LLMON is an LLM-native markup language intended to communicate structure and semantics directly. Semantically coherent components are marked with meaningful tags wrapped in reserved special-token delimiters. Data schemas, including nesting, are stated in a way that is unambiguous to a causal LLM, meaning a model that predicts text token by token from earlier context. Unique identifiers make it possible to point to a specific instruction that may appear anywhere in the context and bind that instruction to the data payload it should act on, similar to how functions and arguments work in programming languages.

What the Preview Models Change in Practice

The two preview models are designed to read and write LLMON-marked text. The release also includes a GitHub repository with tools for working with LLMON and examples of workflows and templates.

The main practical effect is clearer separation between instructions and data. Indirect prompt injections hidden in user data are mostly ignored by LLMON models because the model can see which text is the command, which text is the material, and which text states requirements. On some benchmarks, attack success rates drop to nearly zero, while regular instruction-tuned models can be hijacked with high probability.

The release also reports that this robustness comes without a general capability penalty. Across standard benchmark categories, the LLMON models perform on par with Granite 4.1 Instruct models. For teams already experimenting with Granite 4.1 3B or 8B, the message is that swapping in the LLMON preview can preserve comparable general performance while offering much stronger resistance to prompt injection.

Another benefit is reduced contamination from irrelevant context. Unrelated passages, old chat turns, and tool outputs are less likely to be pulled into the answer because each instruction is bound to the exact data it applies to. Instructions, data, and requirements are clearly marked, so the structure and meaning that already exist at the application’s code level can be leveraged by the model.

Developers can try the approach in Mellea, which has native LLMON support enabled by automatically LLMONizing its LLM calls. For existing JSON pipelines, the release includes converters that turn existing JSON into LLMON without loss. Teams can also define their own structure and associated tags, then tune a low-rank adaptation (LoRA) on top of the LLMON models so the model understands that custom vocabulary.

Why LLMON Is Not Just JSON or a Chat Template

JSON was built for parsers, not for causal language models. In a JSON prompt, the tokenizer is not constrained to respect the grammar, and structural markers such as curly braces, quotation marks, and colons can also appear inside ordinary text. To a model, a quotation mark inside a document and a quotation mark closing a field may be the same token. Escaping rules that are clear to a parser become statistical patterns to the model. Nesting depth is represented by counting brackets, and transformers are known to be weak at tracking such structure over long spans.

LLMON is an LLM-native markup language now available in Granite 4.1 models with 3 billion and 8 billion parameters.

LLMON takes a different approach. It is an XML-like markup language whose structural delimiters are reserved special tokens rather than ordinary characters. That means structure and content can use separate parts of the vocabulary. Tags appear at the beginning of each span, boundaries between spans are explicit, and nested structures are stated locally so the model does not need to maintain a hierarchy over long distances.

Because many teams already use JSON, the release includes conversion tools and example notebooks that translate JSON to LLMON and back in a lossless manner.

LLMON does not replace chat templates. It enriches them. A regular chat template tells the model which part belongs to which role, such as system, user, or assistant. LLMON tells the model what each component means and how components relate to each other within and across roles.

The LLMON preview models ship with their own chat template. In practice, a developer builds the same messages list that would be passed to any chat model and calls apply_chat_template as usual. The template renders the LLMON structure. The difference is that a turn can now name its parts and mark each one as exec: True or False. A part marked True is bound into the exec block at the bottom of the rendered output, which states which instruction to run and what it operates on. A part marked False is still rendered into the context but is referenced by nothing.

Block identifiers are two-character labels drawn from a fixed set; the tokenizer encodes the relevant pieces as single tokens, so addressing a block always costs two tokens no matter which identifier is drawn. In one example, the role is still a single user turn, but its components reach the model as five separate addressable blocks. A sixth block, the exec block, binds them together and states: run this instruction, on this data, subject to these two requirements, and put the answer in the block named here, which is the same assistant block the model then fills. A conventional template would flatten all components into one string and leave the model to infer which part was the task, which was the subject, and which were constraints.

How the Models Handle Prompt Injection

Indirect prompt injection is a type of attack where a malicious instruction is hidden in a data payload, such as a retrieved document, a tool response, or user-provided data. LLMON’s instruction/data separation is designed to address exactly this problem.

The team compared the LLMON previews against several open-source and closed-source models using the OPI benchmark, which places the injection in the data of classic natural language processing tasks. Several attack strategies are used. The reported metric is robustness, defined as 1 minus attack success rate: the share of attacks where the model does not carry out the injected instruction.

The release states that all non-LLMON models tested were highly susceptible, carrying out injected instructions with high probability. On the OPI benchmark, LLMON practically eliminates the problem, avoiding 47% more attacks relative to the best frontier model tested and 95% more than the best open-source model. Similar trends are reported on the ASB and StruQ benchmarks.

The release is explicit about the limits: these benchmarks test robustness to static indirect prompt injections. Instruction/data separation directly addresses that attack type. The team says it is actively working on improving robustness to adaptive attacks that unfold over many chat turns.

The resulting behavior is described as a privilege rule that falls out of the training layout rather than an added guardrail. An instruction reached through an execution block is authoritative. Anything reached as input, such as data, documents, tool schemas, or tool results, is material to read, not a command to follow. Instructions and data spans that are not called on in the execution block are ignored. This is a learned behavior, not an enforced guarantee.

Training, Benchmarks, and Custom Vocabulary

The models start from Granite 4.1 base checkpoints rather than the released instruct models, so LLMON behavior is learned during post-training rather than layered on an existing chat format.

LLMON defines six delimiters that mark the start and end of tags and allow nesting and anchoring. These are installed in the tokenizer as reserved single tokens. The team re-serialized a conventional instruction-tuning corpus into LLMON: every prompt component becomes its own addressable block, and each turn carries an execution block naming which instruction to run, which inputs it operates on, and where the output goes. The task mix is the one a normal instruct model sees, including chat, multi-turn interactions, retrieval-augmented generation (RAG), and tool calling, which is why benchmark performance stays close to the conventionally tuned model.

LLMON is an LLM-native markup language now available in Granite 4.1 models with 3 billion and 8 billion parameters.

The training data is also augmented with decoy instructions and data blocks that are present in context but deliberately left unbound, while the target output remains unchanged. This teaches the model to follow the dependencies dictated by the execution block and ignore the rest.

Reinforcement learning follows. The release describes five sequential Group Relative Policy Optimization (GRPO) stages spanning multi-domain reasoning, reinforcement learning from human feedback (RLHF), knowledge calibration, and math. A final Direct Preference Optimization (DPO) stage uses a thousand on-policy preference pairs built from the model’s own responses to injection attempts.

To help users inspect what is happening inside the model, the release points to an LLMON Attention Visualization on Hugging Face Spaces, which shows how the instruction/data separation plays out in the attention layers.

LLMON is not tied to a fixed schema. Developers can choose tag names and structures that fit a domain and fine-tune an adapter to read and write them. The release demonstrates this on ScienceWorld, a text-based benchmark where an agent carries out elementary science tasks, such as boiling water, by issuing one command at a time and receiving a score from the simulator.

For ScienceWorld, the team defined a vocabulary capturing the structure in the planning trajectories involved in solving tasks. The vocabulary includes tags such as instr.task.goal, req.planning_policy.rule, data.state.room, data.state.inventory, and assistant.next_action. Every step of ScienceWorld’s reference solutions was rendered with these tags, and a LoRA was tuned on top of an LLMON model. The trained model then plays held-out games and writes its next action in the new vocabulary. A notebook walks through the full workflow, from converting reference trajectories to playing test games and scoring the adapter against the untrained base model. Another example for tool-calling agents is also provided.

How to Try the Preview

This is an experimental preview, ready for exploration and evaluation but not for production.

Models: granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview

Visualization demo: LLMON Attention Visualization

LLMON repository: https://github.com/generative-computing/llmon

Mellea: https://github.com/generative-computing/mellea

Papers: https://arxiv.org/abs/2610.38120 and https://arxiv.org/abs/2603.22519

The release asks users to report what worked, what broke, and which use cases they would like to see next in GitHub Discussions.

Comments (0)

  1. No comments yet. Be the first to share what worked for you.

Leave a comment

Comments are reviewed before they appear. Your email address is not published.