I ran one local agent for 22 hours on a single DGX Spark, and it timed out 1,330 times.
A DGX Spark ran Qwen3.8 27B through vLLM for 21 hours, finishing large research despite 1,330 five-minute timeouts and heavy concurrency.
Key Takeaways
- A single DGX Spark ran Qwen3.8 27B through a 21.5-hour multi-agent research workflow, completing 2,874 model steps and generating 3.77M output tokens across 91 subagent sessions with only three human inputs.
- The run produced a full research deliverable: 93 per-firm files, an 1,819-row classified workbook, and a shortlist of 122 strong matches, despite 1,330 client-side five-minute timeouts.
- The reported 48.5 tokens per second was end-to-end wall-clock throughput, not a vLLM decode benchmark; the real issue was extreme client concurrency, with up to 30 outstanding steps against a server configured for only four active sequences.
- Timeouts were strongly correlated with queued or stalled requests rather than slow long generations: 1,304 non-timed-out steps ran longer than 300 seconds, while most timed-out requests produced little or no initial data.
- The likely bottleneck was requests waiting to start or prefill very large prompts, showing that local agent workloads need concurrency, queue, and timeout tuning to avoid wasted client-side waiting.
Meta description: One DGX Spark ran Qwen3.8 27B through a 21-hour research agent, logging 2,874 steps, 3.77M output tokens, and 1,330 five-minute timeouts while finishing work.
Tags: Local AI, agent workflow, vLLM, DGX Spark, Qwen3.8
Featured image: abstract friendly illustration style of a small AI server sending many agent threads, with no real people, logos, or text.
For readers evaluating local AI agents, this session shows what one DGX Spark can do with Qwen3.8 27B: it ran one agent for 21 hours 36 minutes, logged 2,874 model steps, produced 3.77M output tokens, and still delivered a large research package after 1,330 client requests were aborted after five minutes of silence.
The setup and workload
Earlier tests ran Qwen3.8 27B through DeepSeek Harness, or DSH, on the DGX Spark, an RTX 5090, and a Radeon R9700. A follow-up isolated the RTX 5090 video memory pressure problem. Both used real agent sessions, not synthetic benchmarks. This run is the smallest: one machine, one long session, one workload.
The DGX Spark served vLLM with the first-article configuration: Unsloth NVFP4 checkpoint, FP8 key-value cache, 0.80 memory budget, 262,144-token model length, four concurrent sequences, and multi-token prediction speculative decoding with three draft tokens. DeepSeek Harness ran client-side against an OpenAI-compatible endpoint. Nothing was retuned. Serving flags come from the first article, not the export; the export records model, context window, steps, attempts, and retries, but not the server process.
The workload was company research at scale: take 93 private equity groups, find every portfolio company, score each against a written profile, write notes, and produce a shortlist. The parent agent read the profile, converted the source Portable Document Format file with GenOffice, and built a rubric. It dispatched 33 research subagents in waves, private equity group batches first, then company-level batches. Those dispatched 58 more, for 91 subagent sessions. Each researcher pulled the firm's website, searched portfolio companies one by one, and grepped customer relationship management exports for prior contact. Across 91 sessions: 3,847 tool calls, including 1,944 shell commands, mostly curl and grep, 1,186 web searches, and 105 file writes.
Human input was three messages: the task at 18:32 Coordinated Universal Time, a one-line scoping correction at 18:50, and one continue at 00:45 Coordinated Universal Time. More than 15 hours ran unattended. Context compaction fired three times in the parent and seven times across subagents.
The time gap that was not real
The run produced 93 per-firm research files, a 1,819-row workbook with one row per portfolio company classified against the rubric, and a Word review with the shortlist: 122 companies flagged as strong matches and 17 already in the customer relationship management system.
Definitions: a step is one model response between step/start and step/end; an attempt is one request, which can be several per step; a retry follows a timeout; a timeout is an abort after 300 seconds without bytes. Token counts are cumulative usage, not unique context, and input includes history resubmitted every step.
The reported 48.5 tokens per second is output tokens divided by full wall clock, including orchestration, tool execution, waiting, and retry overhead. It is not a vLLM decode benchmark. At roughly 0.75 words per token, 3.77M output tokens is about 2.8 million words, the length of 35 novels, produced in under a day by one small machine across thousands of separate requests, not one long generation. At a hypothetical constant 240-watt draw, 21.6 hours would consume about 5.2 kilowatt-hours, but actual power was not measured.
Summing client-observed duration of all 2,874 steps gives 384.9 hours, while the session ran 21.6. The gap is concurrency: 384.9 step-hours over 21.6 wall-clock hours means an average of 17.8 steps outstanding, peaking at 30, against a server configured for four active sequences. vLLM max-num-seqs caps executed sequences, not waiting requests. The client had no limit on in-flight model requests. The parent dispatched subagents in waves, those dispatched their own, and nothing between agents and endpoint held them back.
Timeouts rose and fell with outstanding steps: 107 to 127 per hour during first waves, when 26 to 27 steps were outstanding, and 134 to 173 per hour at the second peak. They fell near zero once fewer than 12 steps were outstanding. Across 22 hourly bins, r = 0.86. Caveats: a timed-out step stays outstanding for at least five more minutes, so timeouts raise concurrency as well as follow it; hour seven averaged 20 outstanding steps but had only 34 timeouts.
Measured from send to abort, the 1,330 timed-out attempts total 114.4 cumulative client request-hours, about 30% of all client step-time. That is client-side overlapping time, not idle server time, and none produced usable output because received data was discarded on abort. Long steps were not the problem: of 2,114 steps that never timed out, 1,304 ran longer than 300 seconds. They never went 300 seconds without data. A timeout is 300 seconds without a byte.
The export has no server timing, so it cannot show queue, prefill, or decode directly. But timestamps bound the silence. Assuming the client idle timer resets whenever a byte arrives, a timed-out attempt ends 300 seconds after the last byte, so logged duration minus 300 is the latest point it could have heard anything. For all 1,330 timeouts, that point falls within 28 seconds of the attempt starting; median about 8 seconds; 454 under one second. None received data for more than about 28 seconds before silence. The export does not say what arrived first, whether headers or an initial chunk. Leading hypothesis: requests were waiting to start, either queued behind roughly 16 other requests for one of four slots, or prefilling a prompt averaging about 70,000 tokens. Server metrics are needed to separate those; no claim that vLLM was the bottleneck.
The retry budget is five retries after the initial attempt, for six attempts maximum. 732 recovered steps completed after one to five timeouts; 15 worst timed out five times and succeeded on the sixth. The 28 lost steps each show retries one through five, six timeouts, then no response. Those account for the gap: 1,302 timeouts were followed by a retry, and 28 were final. Agents were not traced after the 28 lost steps. The two firms missing from the workbook were not lost to these steps; their research files are complete.
What the research output actually contains
Throughput and retries do not prove correctness, so the deliverables were checked directly. The source Portable Document Format lists 93 firms, not their portfolio companies, so it confirms coverage but not existence. Coverage came up short: only 91 firms reached the workbook. The other two were fully researched, had 32 current portfolio companies, and three scored strong matches, but never made it in. The aggregation script matched the section heading "## Portfolio (current)" exactly, while the two research files used slight variants. The workbook firm summary sheet shows both firms with zero companies reviewed, and nothing is flagged. That is a pipeline bug, not a model failure, but exactly the kind of silent loss long unattended runs make easy to miss.
Across the workbook, 302 of 1,819 rows have no numeric score: 279 excluded as out of scope, such as financial services and pure services; 21 have no verdict; 2 carry a verdict with no score. Fifteen companies appear more than once. The full company sheet carries 130 strong-match verdicts and 19 customer relationship management hits, compared with 122 and 17 on the shortlist.
A desk review checked the first 25 rows of the full company sheet against research notes and the rubric. They are not random: the sheet sorts strong matches first, so all 25 are companies rated 7 or higher, where errors cost most. The five errors: three companies the notes label as exited, still scored 8/10 with every other field empty; one row whose business description and revenue do not match the company; one company listed under two sponsors with conflicting descriptions. The three exited companies did not carry to the shortlist sheet, part of the 130 versus 122 gap. None looked invented, though several are 2026 formations that could not be independently verified.
Five clear errors in 25 rows is a 20% error rate for that subset. Because it is the top of a sorted list, not a random draw, it is not an estimate across the workbook. The strong-match list is a useful starting point, not a finished answer. The tester has worked from it since the run and finds its overall usefulness higher than the error rate suggests.
Next tests and what readers should watch
Two experiments are planned. The first reruns a comparable workload with vLLM's Prometheus metrics scraped throughout. The metrics endpoint exposes queue time and time-to-first-token, separating queue wait from prefill for attempts that went silent. The second adds missing client backpressure: a semaphore on in-flight model requests, capped at two to four, roughly matching the server's four active sequences, while agents keep doing tool work in between.
The test should record timeout count, median step duration, and completed-work rate. If timeouts fall and completed work holds, the cap helped. If both fall, agents used the concurrency given. Neither result would make 48.5 tokens per second a hardware ceiling, since prompt length, caching, tool latency, and workload mix move that number. Neither test has been run, so no optimization result is claimed.
The broader point is that local inference should be judged on sustained agent work, not peak tokens per second. Timeout overhead was real: about 30% of client step time went to aborted attempts. But the retry policy recovered 732 of 760 affected steps, and the workflow finished. On a configuration already published and never retuned, one DGX Spark carried the session through to delivery.

Comments (0)