Operating an AI Agent Harness with a 30-Billion-Parameter Mixture-of-Experts Model for Agentic Coding

Meta description: A September 2026 setup shows serving Qwen 3 Coder 30B and Cline on a 32GB Ryzen 9 3900X node, no NVIDIA GPU, with memory-bandwidth tradeoffs.

Tags: AI Agents, Local LLMs, Ollama, Cline, Edge AI

Featured image: Abstract friendly illustration of a garage computer linked to a laptop by a network cable, with token streams and code files; no real people, no logos, no text.

Developers who want to test AI coding agents without paying cloud inference fees can learn from a September 2026 project by Robert David Hernandez, a software engineer based in the San Francisco Bay Area. He described serving a 30-billion parameter AI model as an inference endpoint for an AI Agent Harness on a low-cost edge device, using no NVIDIA GPU. The setup pairs Cline v4.1.19 in VSCode with qwen-3-coder:30b running through Ollama on a garage node that has 32GB of RAM and a Ryzen 9 3900X.

Why a garage node can replace a GPU for experiments

Hernandez frames AI Agents as part of everyday software development. He says developers at his office already use private frontier AI models and private frontier AI Agent Harnesses inside IDEs. The garage experiment asks a narrower question: can a local coding agent do useful agentic work when the model is served over a network link, while keeping incremental spend near $0?

The problem space includes cost, data privacy, memory, and compute resources. Cloud inference has recurring cost: Claude Code Pro costs $20/mo, while an NVIDIA H100 on Lamba Labs costs $3-$4 per hour, or about $120/mo at 10 hours / week. Hardware prices are also high, with GPU and RAM prices at all-time highs as of this writing in Sept 2026. On the privacy side, depending on cloud AI model providers requires sending data to third party companies.

The local hardware has its own limits. Hernandez’s M3 MacBook Air has 24GB of RAM. He says it has plenty of compute, but it cannot load models large enough to be useful; loading a 9B model causes noticeable stutter. The solution uses three choices: keep using lab hardware that already exists, select an open-weight MoE coding model that can run locally through the open-source runtime Ollama, and use the open-source AI Agent Harness Cline directly inside VSCode on a development laptop. That combination aims to control cost, avoid vendor lock-in, and make self-hosting and fine-tuning possible.

How the model and harness split the work

The architecture separates two roles. The AI model makes decisions. The AI Agent Harness executes those decisions, manages the agent loop, handles safety checks, and interacts with tools. In the request flow, the harness sends a POST request to the model endpoint and receives a chunked JSON token streaming response.

Cline v4.1.19 is the harness in this project. It is an open-source project, licensed under Apache 2.0, with a GitHub repository. Hernandez chose it because it supports both Model Context Protocol (MCP) and native tool use. It has file read/write access, terminal execution, browser use, and natural conversation. It also provides human-in-the-loop safety guardrails through a strict Plan-and-Act loop, requiring explicit approval before destructive actions.

The model is Qwen 3 Coder 30B, listed as qwen-3-coder:30b in the Ollama Library. Its license is Open Weight, Apache 2.0. After comparing several models, Hernandez selected this Mixture of Experts (MoE) model because it is built for advanced code generation and tool-driven agentic behavior. It supports a native context length of 256K tokens. Running it with standard Q4_K_M quantization needs about 18.6GB of RAM. On a 32GB node, that leaves room for context windows and the operating system without forcing heavy swap use.

The speed limit is memory, not the CPU

The bottleneck is not whether the machine has an NVIDIA GPU. It is memory bandwidth. Each generated token requires reading all model weights from RAM, so the speed limit is memory bandwidth rather than GPU compute.

The garage node’s RAM is rated for 3200 MT/s with a 16-byte bus width. That gives a theoretical bandwidth of 51.2GB/s. In practice, the setup produces about 2.75 tokens per second. That is modest, but it costs no incremental capital expenditure.

The cost comparison makes the tradeoff clearer. The Ryzen 9 3900X can beat the cloud H100 on token-per-dollar efficiency, even though it is slower during interactive use. The H100 buys raw speed, but each token carries a premium cloud markup. In production, a team would likely choose higher throughput for customers. In the garage lab, the goal is cost-efficient experimentation.

The system gains lower token generation costs and $0 incremental capital expenditure. It gives up faster response times. Memory bandwidth remains the limiting factor, so the experience depends on how much patience an agent workflow can tolerate.

Setup, firewall, and the coding test

The server node runs the model version qwen-3-coder:30b through Ollama version 0.34.2 on win-11, using an available TCP Port 11434. The client node runs Cline version v4.1.19 and VSCode version 1.138.0, with a trailing 600 listed in the setup notes. The physical layout is simple: the server hosts the model, while the laptop runs the harness.

Network configuration is also lightweight. The garage network has a 1Gbps link, which can comfortably handle the kilobyte-scale bandwidth needed for a small number of tokens per second. A DNS service exists in the lab, but the write-up does not configure DNS. The key server setting is the OLLAMA_HOST="0.0.0.0:11434" environment variable. At startup, Ollama uses that setting to bind to all interfaces on the server node, so the client node can reach it. A firewall rule must allow inbound TCP Port 11434 from the garage subnet.

To test the system, Hernandez asked Cline to perform a multi-step coding task: create a Python Hello World project using uv, use FastAPI to serve responses on /api/v1/hello, include a Docker container with an Alpine base and uvicorn, add unit tests, and add Kustomize files. The model weights began streaming into memory, and memory usage rose to about 80% and stabilized. The agent then executed terminal commands, created files, and wrote code while staying inside the Plan-and-Act approval loop. The resulting containerized FastAPI service returned 200 OK when GET /api/v1/hello was called.

What this means for developers

For readers, the value is not that this setup replaces a production AI service. It shows that a capable 30B coding model can be served locally over a network link if the machine has enough RAM and the workflow accepts slower generation. That matters for private coding tasks, low-cost experimentation, and teams that want to avoid sending code to third-party cloud providers.

The next obvious upgrade is GPU hardware. Because token generation is limited by memory bandwidth, while the PCIe bus has plenty of room, Hernandez points to an RTX 4090 with 24GB of VRAM as a strong next step. Until then, qwen-3-coder:30b and Cline form a low-cost weekend project rather than a high-throughput production path.

The project was originally published in Towards AI on Medium. Robert David Hernandez is a software engineer based in the San Francisco Bay Area, currently a member of TikTok's Traffic Infrastructure Global Engineering team and previously at Microsoft. He holds an M.S. in Computer Science from the University of Illinois Chicago and lists connect links on LinkedIn and GitHub.