Construct a Jev-style decision model using an LLM
This method uses logit bias and logprobs to make an LLM score fixed labels in one prefill pass, reducing agentic routing cost without generating text.
This method uses logit bias and logprobs to make an LLM score fixed labels in one prefill pass, reducing agentic routing cost without generating text.