Prepare an AI codebase archaeology evidence packet before your agent touches legacy code
AI agents need an evidence packet before editing legacy code, mapping change boundaries, execution paths, Git history, tests, and unknown risks.
Key Takeaways
- Before AI coding agents edit unfamiliar legacy code, use a read-only “codebase archaeology” workflow to create a concise evidence packet that explains what the change touches, where claims come from, and what remains unknown.
- A clean patch plan can still be unsafe because hidden risks may come from call paths, Git history, production behavior, external contracts, tests, or undocumented decisions.
- The evidence packet should include six parts: narrow change boundaries, one concrete execution path with exact code references, relevant Git history, characterization tests, assumptions and unknowns, and a change contract with permission gates.
- Run the workflow in two passes: first explore and produce the packet without editing tracked files, then make only one bounded change after human review.
Meta description: Use AI codebase archaeology to build a six-part evidence packet before an agent edits legacy code, reducing hidden assumptions and risky patches.
Tags: AI coding agents, codebase archaeology, legacy code, Git history, characterization tests
Featured image prompt: Abstract, friendly illustration of a magnifying glass scanning code branches, commit dots, and test checks, no real people, no logos, no text.
AI codebase archaeology is a read-only workflow for teams that use AI coding agents in unfamiliar repositories. The core idea is simple: before an agent is allowed to edit, it must produce a small evidence packet that explains what a proposed change touches, where each claim comes from, and what remains unknown. This is for developers, maintainers, and reviewers who need to prevent confident but thin plans from breaking retry rules, compliance logs, external client integrations, or hidden side effects. The source frames the need alongside OpenAI's planned acquisition of Pierre Computer Company, describing a broader push toward tools that help developers build, understand, and review code.
Why a clean patch plan can still be unsafe
A coding agent can scan thousands of files and still invent a plausible reason for an odd branch. That is the failure pattern: a developer asks the agent to clean up a payment flow, gets a tidy plan, and the change may even pass tests. Later, something small breaks because nobody proved what the strange code was protecting. The gap is not reading ability. A safe change may depend on call paths, Git history, production behavior, test coverage, external contracts, and decisions that never reached documentation. When an agent fills those gaps with a story, the report can sound more certain than the evidence supports.
What the evidence packet should be
An evidence packet is not an architecture document and not a raw transcript of agent notes. It is a short artifact tied to one change. It gives a reviewer the same material they would want before approving the first write: the behavior in scope, entry and exit points, sources for each claim, uncertainty labels, and the checks needed before a merge. The packet also changes the prompt. "Explain this repository" invites a broad summary. "Trace this checkout failure through one real input" invites a bounded investigation.
Six parts of the packet
The source recommends six sections.
- Change question and boundaries. Begin with one narrow question, such as what happens when a cancelled subscription is retried after its invoice is archived. List in-scope and out-of-scope areas, current branch and commit, environment assumptions, and whether the task is read-only. A code claim without a revision can already be stale.
- One concrete execution path. Ask the agent to follow one real input from trigger to durable side effect. For a request, that may include route, authorization, handler, domain rule, persistence, event publication, and background consumer. Each hop should attach an exact file path plus symbol or line range. A diagram can help, but the reviewer must be able to jump to code.
- History that explains odd code. Current files show what exists; history can hint why. A redundant guard may follow an incident, two files may change together, or a temporary adapter may protect an old client. Targeted Git checks are enough, such as git log -S archiveInvoice --all -p -- src/billing, git log --follow --date=short --format='%h %ad %s' -- src/billing/retry.ts, or a co-change count over src/billing. The output should be treated as a lead, not a proof.
- Behavior probes and characterization tests. Static reading may miss dependency injection, feature flags, retries, date handling, and data shape. A characterization test records what the system does today rather than what the team wishes it did. For an archived-invoice retry, the test might assert that the result status is skipped, the payment gateway charge is not called, and the audit log records retry_skipped. A green test proves the test ran, not that it captured the right promise.
- Assumptions, unknowns, and disconfirming checks. This section stops a polished report from becoming a hallucination. The agent must list what it could not establish: missing production config, undocumented vendor retry policy, absent fixture, or a database trigger outside the app repository. For each important assumption, name the cheapest check that could disprove it. Unknown is not blocked; it means the next irreversible step needs selected evidence.
- Change contract and permission gate. End with a narrow write proposal: files allowed, files off limits, tests required, and checkpoints needing human review. Safe writes may be limited to a named module and focused test on an isolated branch. Human checkpoints can cover schema changes, access control, payment behavior, public contracts, or unresolved high-impact assumptions. Stop conditions can include a new call path, unmodeled side effect, failing characterization test, or evidence that contradicts the packet.
Run the workflow in two passes
Most teams do not need a new platform. They need a repeatable split between exploration and modification. In pass one, the agent gets read-only access where tooling permits it. It maps the smallest behavior, inspects relevant history, locates tests, and writes the packet. It may run safe local commands, but it should not edit tracked files, change infrastructure, call production services, or fix a failing test on the way. The review question is practical: could another engineer reproduce the claim from the cited evidence in a few minutes? If not, the claim needs a better source or a lower confidence label.
In pass two, the agent makes one bounded change only after the packet is reviewed. The packet becomes the task brief. The final response should include a diff summary, tests run, current commit, and a packet changes note explaining which assumptions changed during implementation. Implementation is new evidence; a failed test or unexpected call path should update the model rather than be hidden.
A reusable read-only prompt
Use a prompt that rewards useful uncertainty instead of a confident answer.
Enter read-only evidence mode. Do not modify files, make commits, or give a final implementation. Task: [specific behavior or question]. Commit: [SHA]. Scope: [folders, service, or component]. Deliver a six-part evidence packet: question and boundaries, one concrete input trace, file and symbol citations, relevant Git history signals, safe behavior probes, assumptions and unknowns with disconfirming checks, and a narrow write proposal. Label each statement as confirmed, likely, or unknown. If a statement is likely, name the proof still missing before making it a design recommendation.
What to measure
Do not count packets or agent summaries as the main metric. Measure whether the practice makes changes easier to review and safer to finish. Track how often a packet exposes a missing requirement before editing, how often a listed uncertainty becomes a real issue, and how often an agent stays inside its approved file boundary. Match the packet to the blast radius: a one-line fix may need a small packet, while a billing migration may need a deeper trace, production review, and more than one owner.
Common mistakes
The first mistake is treating a long context window as evidence. More files can give an agent more clues, but they do not reveal runtime state, undocumented operations, or missing external contracts. The second mistake is saving a sprawling agent transcript as documentation. Transcripts mix discoveries, dead ends, and stale guesses; distill the packet instead. The third mistake is demanding perfect understanding before progress. That creates analysis paralysis. The goal is not perfect certainty. It is a supported decision about the next safe step.
Comments (0)