Enhancing the Speed and Energy Efficiency of AI Agents

Meta description: MIT and Microsoft's Murakkab can design and run agentic workflows with about 35% less computation, lowering cloud energy and cost while keeping performance.

Tags: agentic workflows, cloud computing, AI efficiency, Murakkab, MIT

Featured image: Abstract friendly illustration of AI agents, cloud servers, and optimization arrows; no real people, logos, or text.

For developers, cloud operators, and business teams that pay for AI inference, researchers from MIT and Microsoft have built Murakkab, an intelligent system that makes agentic-workflow design easier and automatically optimizes how those workflows are implemented. In agentic workflows, several AI models and external tools work together to complete tasks such as video understanding and answering user questions. In tests, Murakkab met user requirements while using about 35 percent of the computation required by traditional approaches, reducing energy use and cost.

The Cost of Hand-Built AI Pipelines

Agentic workflows are different from a single chatbot prompt. They can include autonomous AI agents that call databases, Python programs, or other services, then decide what to do next. These workflows often sit behind user-facing applications and handle tasks such as data processing or code generation. A video question-and-answer app might need to inspect frames, make a transcript, and answer queries. A code-generation app might need to retrieve documentation, run a test, and revise an answer.

Because these systems bring together black-box models and diverse tools from different vendors, developers usually hard-code choices before launch: which agents to use, which model to call, what order to run tasks in, which hardware will host the workflow, and how to trade speed against cost. That creates two problems. First, the number of possible setups is too large for a human to explore fully, so the best combination is unlikely to be found by hand. Second, if a better model or GPU accelerator appears, the developer may need to rebuild the application from scratch.

The cloud data center also has a visibility problem. It may know the customer's request and the available machines, but it cannot see inside the workflow to allocate resources efficiently at the moment of execution. Gohar Chaudhry, an MIT electrical engineering and computer science graduate student and lead author of the paper, said the issue matters because agentic workflows are becoming central to cloud services and energy use is a major concern. He argued that over-allocation wastes energy and money, and that helping cloud providers make workflows more resource-optimal benefits everyone involved.

How Murakkab Decides What to Run and Where

Murakkab, whose name comes from an Urdu word meaning a composition of things, tries to remove much of that manual guesswork. A developer can describe the intended application in plain, high-level language instead of listing every component and setting in advance. If the app should answer questions about a video, Murakkab can choose which existing models and tools to combine, and decide whether parts should run one after another or at the same time.

The system also adapts over time. If a new model or GPU accelerator becomes available, it can incorporate that option without requiring the developer to rewrite the whole workflow. When a cloud provider runs the application for a customer, Murakkab can tune the workflow around user priorities, such as keeping accuracy high while meeting a latency target or reducing cost while staying fast enough.

For cloud operators, the design gives another capability: visibility across multiple workloads. Instead of treating each application as a black box, the provider can see enough of the workflow to share computational resources in a way that satisfies customer constraints. The system then generates a configuration ready for execution, adjusting hardware allocation and deployment schedules in real time.

What the Results Mean

In tests covering video question-and-answer and code generation workloads, the researchers reported that Murakkab met user requirements without hurting performance, while using only about 35 percent of the computational units needed for deployment compared with other methods. It also consumed only about 27 percent as much energy, at less than 25 percent of the cost. The paper will be presented at the USENIX Symposium on Operating Systems Design and Implementation.

The results suggest two practical benefits. One is a lower cloud bill for applications that already use many AI models and tools. The other is flexibility: users can choose the tradeoff they care about. In one example, Murakkab lowered the energy consumption of an agentic workflow by more than an order of magnitude, while accuracy for the customer fell by only about 2 percent.

The system also found a surprisingly effective setup for a model that selects video frames, improving performance in a video question-and-answer task. Chaudhry said that kind of optimization would be very hard to discover by hand, because the choices interact in too many ways.

Coauthors, Support, and What Comes Next

The paper's coauthors include Adam Belay, an MIT associate professor of EECS, or electrical engineering and computer science, and a member of the MIT Computer Science and Artificial Intelligence Laboratory; Ricardo Bianchini, a technical fellow and corporate vice president at Microsoft Azure, who is the senior author; and other Microsoft Azure team members. The research was supported in part by the Semiconductor Research Corporation and the U.S. Defense Advanced Research Projects Agency.

The next step is to test Murakkab on workflows with more steps and on larger clusters of machines, and to look for ways to make new agentic applications more efficient. Chaudhry said there is broad potential to make these systems consume far less energy, but the industry needs to evaluate the approach as large as major cloud platforms.

For readers, the main takeaway is not that one system will automatically fix every AI deployment. It is that the efficiency of agentic workflows is becoming a first-class engineering problem. As more products chain models and tools together, the teams that can tune those pipelines automatically may gain an advantage in cost, sustainability, and responsiveness.