EdgeWise brings long-horizon agents to the edge through selective cloud delegation.
EdgeWise routes long-horizon agent steps from local SLMs to cloud LLMs only when critical, cutting inference costs by up to 62% on SWE-Rebench.
Key Takeaways
- EdgeWise splits long-horizon agent tasks between a local small language model and a cloud LLM, using the cloud model only for critical steps.
- This selective cloud delegation reduces inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1 while maintaining or improving task performance.
- The framework uses step-level reinforcement learning and judge-based delegation to learn when, how, and what to escalate to the cloud LLM.
- The main practical insight is that frontier cloud models can be treated as limited resources, making agent workloads cheaper when many steps can be handled locally.
Meta description: EdgeWise splits long-horizon agent tasks between a local small language model and a cloud LLM, cutting inference cost by up to 62% on SWE-Rebench.
Tags: EdgeWise, agent tasks, cloud LLM, inference cost, local small language model
Featured image: Abstract friendly illustration of a small local model passing a task packet to a larger cloud LLM, with no real people, logos, or text.
EdgeWise Uses Small Models and Cloud LLMs to Cut Agent Costs
The proposed EdgeWise framework addresses a cost problem in long-horizon agent tasks: it runs a local small language model for most steps and escalates only critical steps to a cloud LLM. According to the reported experiments, EdgeWise matches or outperforms pure-LLM execution while reducing inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1.
Why Full Cloud LLM Runs Get Expensive
Long-horizon agents often need frontier cloud models to perform well, but calling a frontier model at every step can make inference expensive. In a long-horizon trajectory, the agent may make many decisions before finishing a task. If every decision is sent to the same frontier model, the cost grows with the number of steps, even if the task remains solvable with less capable handling.
The capability required by each step is not always equal. Some steps may need the strongest model, while others may be handled by a smaller one. That mismatch is the starting point for EdgeWise: use a local small language model as the default, and reserve the cloud LLM for the steps that most likely need its strength.
How EdgeWise Splits Work Between Edge and Cloud
EdgeWise is an edge-cloud collaboration framework. It uses a small language model on a local device as the main solver. When a step is judged critical, the framework sends that step to a cloud LLM instead of running it locally. The local model keeps the trajectory moving, and the cloud model is called only where its higher capability is expected to matter.
Because the cloud model is called less often, the system can keep strong capability available without paying for it at every step. This pattern changes the cost question for agent builders. Instead of selecting one model for an entire task, the system decides how to route each step.
The framework’s reported finding is that a frontier cloud model can still achieve strong results even if it is used only for a small subset of key steps. That finding matters because it separates task quality from the assumption that every model call must be the most powerful one available.
Learning Which Steps Deserve the Cloud LLM
The method uses a step-level reinforcement learning approach that optimizes task solving and delegation together. It assigns credit to each individual delegation decision with fine granularity, so the system can learn whether a specific cloud call improved the outcome.
To examine the routing decisions, the framework introduces an external judge-based delegation strategy. It explores four practical design questions: when delegation should happen, how delegation decisions should be made, what context should be passed to the cloud LLM, and how frequently the cloud LLM should be used.
These questions are central to edge-cloud agent systems because too much delegation raises cost, while too little may weaken performance. The judge-based strategy provides a way to study those trade-offs without relying only on a hand-written routing rule.
What the Benchmarks Show
Experiments show that EdgeWise performs at the same level as, or better than, pure-LLM execution while reducing inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. Those benchmark names are the specific evaluation points given in the source; no broader claims about other tasks are needed. The result suggests that selective escalation can be an effective cost-control pattern for agent workloads.
The approach was also deployed on a local workstation, where it produced comparable cost savings in a practical edge-cloud setting. That deployment detail moves the method from a benchmark exercise to a local-to-cloud workflow. For readers, the practical question becomes whether their agent workload has enough steps that can be handled by a local small language model.
What Comes Next for Agent Deployments
The EdgeWise results point to a narrower, more operational path for long-horizon agents: treat frontier cloud models as a limited resource, not as the default. The source does not provide pricing figures, model versions, or deployment limits, so the main public evidence is the reported cost reduction on SWE-Rebench and Terminal-Bench 2.1.
The next step for practitioners would be to evaluate whether the delegation pattern works for their own task mix, especially where cost and routing are practical concerns.

Comments (0)