Nokia Releases AnyJev as Open Source: A Training-Free Layer That Transforms Any Open LLM Into a Calibrated Decision Model
Nokia's applied research team open-sourced AnyJev, a Python library that converts any open LLM into a structured, calibrated decision model without any training. Supporting choice, yes/no, and score questions, it corrects prior and position bias via cyclic option rotations and batch calibration (L0), plus temperature scaling (L1). Benchmarks on Qwen3-8B with BANKING77 show reduced order flips (0.23 to 0.073), higher accuracy, and much lower calibration error, letting far more decisions be auto-decidable. It installs from PyPI, supports transformers and vLLM backends, and suits classification and routing tasks.
Nokia's applied research group has released AnyJev, an open-source Python library that converts an open large language model into a structured decision model — and it does so with no training whatsoever. If you're a developer who needs an LLM to reliably pick one option from a fixed list rather than generate free-form text, AnyJev is worth a close look. It's a practical answer to a very common production problem: classification and routing tasks where you want a clean decision plus a trustworthy confidence score, not a paragraph of prose.
Is It Actually Ready to Deploy?
Yes. AnyJev installs straight from PyPI, ships under the Apache-2.0 license, and supports both a transformers backend and a vLLM backend with shared-prefix scoring. That means you can prototype on a laptop and move to a fast serving setup without changing your code structure.
What Is AnyJev?
AnyJev borrows its interface from Jev, the System One decision model that TypeSafe AI launched in September 2026. The idea is simple: you hand the library a typed question, and you get back a decision accompanied by a probability you can threshold against. That probability is read directly from the model's next-token distribution — nothing is generated, nothing is parsed, and nothing is fine-tuned.
The library supports three question types:
- Choice questions pick one option out of K possibilities.
- Noul questions are simple yes/no decisions.
- Score questions place an answer into one of several ordered bins.
Why Reading Logits Directly Falls Short
Plenty of open-source projects already restrict the next token to option labels and read the resulting scores. Nokia's team points out two flaws in that shortcut. First, the answer can shift depending on how the options are ordered. Second, the raw probabilities are poorly calibrated.
The team's documentation identifies two root causes:
- Prior bias — the model favors certain labels regardless of the input, for example leaning toward "Yes" over "No."
- Position bias — the model favors certain slots in the option list, no matter which label sits there.
How AnyJev Works: The L0 and L1 Levels
Every decision AnyJev produces carries a level field describing how much correction was applied.
L0 (zero labels, enabled by default) applies two fixes:
- Cyclic shifts. For a question with K options, the option list is presented in K different rotations, so every option appears in every position exactly once. The results are then combined in log space as a geometric mean. If position bias is additive in logit space, this cancels it out exactly.
- Prior correction. By default, AnyJev uses batch calibration: it maintains a running mean of predicted distributions on real inputs and divides it out at a strength of 0.75. The correction kicks in after 8 items.
L0 costs K prefills per decision, batched over a shared prefix. In practice that's roughly 0.25 seconds per decision at batch size 32 on a single H100, with K = 20.
L1 (100 to 500 labels per question) layers temperature scaling on top of L0. The fitted values are stored as a small JSON artifact. Note that L1 reshapes confidence levels but does not change the ranking of answers.
Benchmark Results
On Qwen3-8B with the BANKING77 dataset (20-way classification, 300 test items), the results look like this:
| Metric | Raw logits | AnyJev L0 | AnyJev L1 |
|---|---|---|---|
| Labels required | 0 | 0 | 100 to 500 |
| Flip rate when options reversed | 0.230 | 0.073 | 0.077 |
| Accuracy | 0.747 | 0.803 | 0.807 |
| Calibration error (ECE) | 0.240 | 0.184 | 0.095 |
| Auto-decidable at 5% error | 7.7% | 46.3% | 52.0% |
A few additional findings from the repository:
- L0 reduced order flips on all 9 model-and-task rows tested.
- On a typed-decisions set, Qwen3-32B with L1 reached an ECE of 0.036, compared with 0.144 published for Jev. On raw accuracy, however, the fine-tuned Laya still leads.
- The full ablation table spans Qwen, OLMo, Granite, Phi, and Mistral models.
- According to Wu, the team tried AnyJev on an internal Nokia routing problem and saw promising results.
Getting Started With AnyJev
```python
pip install "anyjev[hf]"
from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Question.choice("Which team should handle this?",
["billing", "technical", "sales", "other"], name="route")
r = d.decide({"conversation": [...]}, [route])
r["route"].distribution # probabilities per option
```
For production serving, you start vLLM with prefix caching enabled and point a `VLLMBackend` at it.
Key Takeaways
- AnyJev turns open LLMs into Jev-style typed decision models with no training required.
- L0 combines cyclic shifts and a batch prior to remove position and label bias.
- On Qwen3-8B BANKING77, the order-flip rate drops from 0.230 to 0.073.
- Auto-decidable traffic at 5% error jumps from 7.7% to 52.0% with L1.
- It's Apache-2.0 licensed, available on PyPI, with both Hugging Face and vLLM backends.
Who Should Use It / Who Should Skip It
Use it if you're building routing, moderation, or classification pipelines on open models and need decisions you can actually trust — especially when you can't afford fine-tuning or labeled data beyond a small calibration set (or none at all with L0). It's also a strong fit if option-order sensitivity has bitten you before, or if you want to auto-decide on a large share of traffic while escalating only low-confidence cases.
Skip it if your task genuinely requires generation — summarization, writing, extraction of arbitrary text — since AnyJev is strictly about picking from fixed options. And if maximum accuracy is your only metric and you have the budget for a fine-tuned model, the benchmarks show a dedicated fine-tune like Laya can still come out ahead on raw accuracy, even though AnyJev wins on calibration.
Comments (0)