Our minimal telemetry map for an AI agent covers 12 fields and 4 places.
The telemetry map links 12 fields to four AI agent components, but a harness logged only three, exposing gaps in tool wrapper observability.
Key Takeaways
- Decoding AI proposes a minimal AI-agent telemetry map of 12 fields organized across four places: orchestrator, tool wrapper, file reader, and write step.
- Each field exists to catch a specific failure type, such as partial data, stale input, outages, wrong-scope edits, or silent completion.
- The team’s own retry-test harness recorded only 3 of the 12 fields, all from the orchestrator, exposing a major observability gap in tool execution.
- The most missing data came from the tool wrapper, including exit codes, HTTP status, row counts, completeness flags, and call arguments.
- The map is not a universal schema, but teams can audit their traces to identify missing fields that force them to guess at failures.
Meta description: Decoding AI's telemetry map assigns 12 fields to four agent components, and its own harness recorded only 3 of 12, exposing a tool wrapper gap.
Tags: AI agents, telemetry, observability, debugging, Decoding AI
Featured image: abstract, friendly illustration of four pipes feeding a 12-field log map, no real people, no logos, no text.
On 10 Oct 2026, Decoding AI published a method note for practitioners building AI agents. It shows a telemetry map that assigns 12 fields to four places in the agent stack, and it reports that the team's own retry-test harness recorded only 3 of those 12 fields. This week's tests produced the map. The note is useful for engineers who need to catch failures such as partial data, stale input, outages, wrong scope and silent completion, because it explains which component must write each record and which failure that field exists to catch. The note says 12 fields, written from four places, would have proven every failure in its test series this month. It was written with AI assistance, and every number comes from a measurement run and log with the draft. It is the method note behind Flagship 3 and the six trace fields.
Why the map is organized by place
Telemetry means recorded operational data. An AI agent is software that plans work and calls tools. In this note, the places are the orchestrator, the tool wrapper, the file reader and the write step: the orchestrator runs the agent, the tool wrapper calls external tools, the file reader loads input files, and the write step changes records.
The map is organized by place, not by field name, because a field is only useful if the right component writes it. The note says the agent cannot be trusted to report what its tools returned: in tests, agents summed half the data and said done. The record has to come from the code that saw the fact. Each field is there to catch one named failure, and a field without a failure is noise.
The 12 fields, by place
The orchestrator records run_id, final_status and evaluation_result. Run ID joins everything else; all 4 log-based debuggers needed it to find their run. Final status is the agent's own claim, so it should be kept but never alone: status-only debuggers found 2 of 4 root causes, leaving half the root causes unknown. Evaluation result is the only field that notices a wrong answer by itself; retrying on it fixed 4 of 4 silent failures in the retry test.
The tool wrapper records call, exit, http_status, rows and complete. Call with arguments was cited in every log and trace diagnosis. Exit code and HTTP status proved the outage, which involved six 503 responses. Rows and completeness proved the partial-data failure through "rows": "10" and "end_marker": "false"; a success exit code alone is how that failure stayed silent.
The file reader records path and generated_at. The stale-input failure was proven by two dates: 30 September for the file the agent read and 9 October for the one it skipped. The note says a read without a date is a guess about freshness.
The write step records records_changed and keys. The wrong-scope edit produced valid JSON with three changed records where one was asked for, and the count was the whole diagnosis.
The audit: 3 of 12 fields were recorded
The harness behind the retry test was checked against the map on 10 Oct 2026. Decoding AI charted the audit and diagrammed the map. The harness recorded 3 of 12 fields, all from the orchestrator. For all 40 attempts, it recorded a run ID and final status, and it recorded an evaluation result per policy. For the tools, it kept only a call count per attempt: no exit codes, no row counts, no completeness, no file dates and no write counts.
That matches the team's experience. It could grade a run but not explain one. It could say which runs were wrong in seconds, but to explain why it needed the agents' own reports and the answer key. A production system has neither. The tool wrapper is the gap that matters most: 5 of the 12 fields live there, and 0 of its 5 fields were recorded.
Limits, next tests, and how to check your own map
The note does not claim a universal schema. The 12 fields came from five failure types, and timeouts, costs and multi-agent hand-offs will need more. It does not claim that 12 of 12 is enough; it says this is the minimum that would have proven the team's own failures, nothing more. Field names also do not matter as much as places, because teams should use whatever their tracing system calls them.
To check a stack, list the fields that traces actually contain and compare them with the map. The note shows the map as four groups: orchestrator with run_id, final_status and evaluation_result; tool wrapper with call, exit, http_status, rows and complete; file reader with path and generated_at; write step with records_changed and keys. It suggests feeding a day of trace records as dictionaries into a missing-field check, such as a missing(trace_lines) function, and asks how many of the 12 your agent records today. Every missing field is a failure that will be diagnosed by guessing. It asks readers to say in the comments which place their stack is missing.
The next tests include Flagship 4, where a tool times out halfway through a task. The note invites readers to follow Decoding AI for it. Scheduled follow-ups are Friday 23 October for tool timeouts injected halfway through agent tasks and what the agents report; Monday 26 October for a compact evaluation set for agents and what it catches; Tuesday 27 October for why 10 regression cases beat a vague 1,000-case claim. The note says it drew the map and failed its own audit, and tomorrow a tool goes quiet mid-task, revealing what the map misses. The next step points to tool timeouts halfway through, on Friday.
The note cites OpenTelemetry's Observability primer and earlier Decoding AI pieces: Retrying a Failed AI Agent Fixed Nothing Until We Checked Its Output, Better AI Agent Telemetry Didn't Find Root Causes Faster. It Found Them., and Five Ways a Valid-Looking AI Agent Output Still Fails. The original article, Our Minimum Telemetry Map for an AI Agent: 12 Fields, 4 Places, was published in Towards AI on Medium, where readers can continue the conversation by highlighting and responding.
Who should use it / who should skip it
Use the map if you are building or debugging AI agents and need to know which component should record run status, tool output, input freshness and write counts. Skip it if you need a universal schema for every agent failure, because the note says the 12 fields are only the minimum for the failures tested and not a complete standard for timeouts, costs or multi-agent hand-offs.

Comments (0)