AI Agents Should Not Code From Memory

A research-first architecture for building more reliable AI agents

Most AI coding agents begin with a model and a prompt. The model reads the request, predicts what the user probably wants, and starts writing code.

That workflow is convenient. It is also built on a dangerous assumption: that the model already knows enough.

In real software work, the required information is often current, repository-specific, version-dependent, or hidden inside documentation. A model can produce code that looks reasonable while using an old API, misunderstanding the task, ignoring a repository convention, or inventing a function that does not exist.

My proposal is simple:

An AI agent should learn, understand, verify, and then implement.

This does not mean that every request requires a long web search. It means the system should not silently act from memory when the task depends on information outside the model’s reliable internal knowledge.

The important question is not only, “Which model should we use?” It is also:

Which parts of the agent’s workflow should be left to model judgment, and which parts should be enforced by the system?

The architecture

The architecture has six stages:

1. Interpret the task.

2. Ask for clarification when the request is ambiguous.

3. Decide whether external evidence is required.

4. Search and read relevant sources or documentation.

5. Implement through bounded tools.

6. Verify the result before claiming completion.

The final stage is important. A successful tool call is not the same as a successful task. A file can be edited incorrectly. A command can return an incomplete result. A generated patch can compile and still fail the user’s actual requirement.

Why model memory is not enough

Large language models store a great deal of information in their parameters, but that knowledge is not a live database. It can be outdated, incomplete, difficult to inspect, and disconnected from the exact environment in which the model is operating.

Research on retrieval-augmented generation identifies hallucination, outdated knowledge, and limited traceability as important limitations of relying only on parametric knowledge. Retrieval allows a model to use external information and update its working context without retraining the entire model.[1]

That does not mean retrieval automatically makes an answer correct. Retrieval creates another chain of possible failures:

Failure point	Example
Query formulation	The agent searches for the wrong concept or uses an overly broad query.
Retrieval	The relevant documentation is not returned.
Source quality	The result comes from an outdated blog post or an untrusted forum.
Context selection	The correct passage is buried among irrelevant text.
Interpretation	The model misunderstands the retrieved material.
Implementation	The model understands the documentation but applies it incorrectly.
Verification	The system never tests whether the result actually works.

Search is therefore not a magic anti-hallucination button. It is one component in an evidence pipeline.

Search is a capability, not proof of research

Modern agents increasingly expose search tools. OpenCode documents separate tools for web search, page fetching, file reading, editing, shell execution, and repository inspection.[9] Gemini CLI documents a similar workflow: search for current information, fetch the relevant page, and then apply the knowledge to code.[10]

This is useful, but tool availability alone does not guarantee disciplined tool use. An agent may have access to search and still decide to write code immediately. It may search once, read only a snippet, or trust the first result without checking the primary documentation.

The BrowseComp benchmark makes this distinction clear. It was designed around hard-to-find but verifiable questions. The authors describe successful browsing as requiring persistence, depth, creativity in search, and reasoning about the factuality of internet content.[2]

Earlier work on WebGPT reached a similar conclusion. Its model operated in a browser environment, but the system also used imitation learning and human feedback. The model was required to collect references while browsing so that evaluators could assess factual accuracy.[12]

A useful research stage therefore contains more than a search box:

Formulate the query → discover sources → read the actual evidence → compare sources → record provenance.

A search-result snippet should not count as completed research.

The documentation stage is not optional ceremony

One of the strongest pieces of evidence for this architecture comes from research on tool documentation. Hsieh and colleagues studied whether models could use unfamiliar tools from documentation alone. Across several tasks, they reported that documentation-only prompts could enable zero-shot tool use. On a realistic dataset with many APIs, documentation-only prompting outperformed few-shot prompting without documentation.[5]

This suggests that documentation is not merely background reading. It is part of the interface between the model and the tool.

Before using an unfamiliar API, an agent should identify:

• the correct tool and version;

• the required arguments and their types;

• authentication and permission constraints;

• expected output formats;

• failure modes and rate limits; and

• examples that match the current task.

The agent does not need to load an entire documentation website into its context. It needs to retrieve the relevant sections and preserve their source and version information.

Reasoning and action should form a loop

The ReAct paper proposed interleaving reasoning traces with actions. Reasoning helps a model track and update a plan, while actions allow it to obtain information from external sources or interact with an environment.[3]

This is a better model of real work than a single linear prompt. A software task often changes as the agent learns more:

Understand → inspect → observe → update the plan → implement → test

Suppose a user asks for authentication using a particular library. The initial task may appear straightforward. After reading the current documentation, the agent may discover that the version uses a different API. After inspecting the repository, it may find that authentication is already partially implemented. After running tests, it may discover that the first patch breaks an existing route.

A reliable agent must be able to return to research or planning instead of defending its first guess.

Why a small model can still be useful

The argument is not that small models are universally as capable as large models. The more precise argument is that a small model may be sufficient for a constrained role when the surrounding system provides strong tools, narrow responsibilities, and verification.

Toolformer showed that a language model can learn to decide which APIs to call, when to call them, what arguments to provide, and how to incorporate the results. The tools in the study included calculators, question-answering systems, search engines, translation, and calendars.[4]

A 2025 position paper makes a related argument: many agentic workloads involve specialized and repetitive invocations, making smaller models attractive for cost and operational reasons. The paper also argues for heterogeneous systems in which different models handle different responsibilities.[11]

This leads to a practical division of labor:

System component	Responsibility
Small language model	Interpret the request, select tools, formulate queries, plan, and communicate.
Search and retrieval tools	Provide current external evidence.
Documentation reader	Extract API contracts, constraints, and version-specific details.
Local repository tools	Inspect files, symbols, tests, and configuration.
Execution tools	Run commands, tests, builds, and controlled actions.
Policy controller	Enforce permissions, sequencing, and approval requirements.
Verifier	Check factual support, test results, and task completion.

The model does not need to be the entire system. It needs to be a capable coordinator inside the system.

But “always search” is too simple

The research also gives an important correction to my original instinct.

Self-RAG argues that indiscriminately retrieving a fixed number of passages can be harmful when retrieval is unnecessary or the passages are irrelevant. Its approach allows a model to retrieve on demand and reflect on the retrieved passages and its own generation.[7]

This means the correct rule is not:

Search the web before every answer.

The better rule is:

Before acting, make an explicit evidence decision. If the task depends on current, external, uncertain, or specialized information, gather and evaluate evidence before implementation.

For a simple calculation, a calculator may be more appropriate than web search. For a local refactor, repository inspection may be more valuable than internet browsing. For a current API integration, official documentation should be mandatory. For an irreversible action, human approval may matter more than another search.

The policy should be strict about evidence and safety, but flexible about the exact tool.

More context is not the same as better understanding

A common response to knowledge limitations is to put more material into the context window. That does not solve the problem by itself.

Recent research reports that context length alone can hurt language-model performance even when relevant information is perfectly retrieved.[8] The issue is not only whether the right passage is present. The model must also locate, prioritize, and use it correctly.

Therefore, “read the documentation” should not mean “paste every document into the prompt.” A better process is:

1. identify the relevant documentation;

2. retrieve the correct version;

3. select the sections that answer the current question;

4. summarize or compress where necessary;

5. preserve links and provenance; and

6. ask the model to state which evidence supports the implementation decision.

Context management should be treated as a separate engineering problem from context-window size.

Verification must be an independent stage

A search result can be wrong. A citation can be present without supporting the claim. A code patch can look correct without passing tests.

The ALCE benchmark separates answer correctness, fluency, and citation quality. Its authors report that, on the ELI5 dataset, even the strongest evaluated systems lacked complete citation support 50% of the time.[13]

This is why a research-first agent should not merely attach links at the end of an answer. It should connect important claims and implementation decisions to evidence, then verify whether the evidence actually supports them.

For coding tasks, SWE-bench demonstrates the importance of execution-based evaluation. The benchmark gives an AI system a real GitHub issue and asks it to modify a repository. Success is measured using tests that fail before the reference change and pass afterward.[6]

The lesson is straightforward:

Do not evaluate an implementation by how convincing its explanation sounds. Run the tests.

What is new in this proposal?

The individual ingredients are not new. Retrieval-augmented generation, tool use, browsing agents, documentation-aware tool calling, and execution-based verification all have prior research and deployed examples.[1] [3] [4] [5] [6]

The potentially distinctive contribution is the hard-gated orchestration policy.

An agent following this policy would have the following constraints:

Gate	System behavior
Clarification gate	Ambiguous requests pause and ask the user instead of guessing.
Evidence gate	Tasks involving external or changing information require research or an explicit, logged exception.
Documentation gate	Unfamiliar tools require relevant documentation before use.
Write gate	File modification and irreversible actions remain unavailable until the prior gates are satisfied.
Verification gate	The agent cannot claim completion without tests, checks, or another appropriate validation method.

Existing systems expose many of the required capabilities. The open question is whether the workflow is enforced by code or merely suggested to the model. That difference is testable.

How to test the idea

A useful experiment would compare four agents on the same collection of tasks:

Agent	Workflow
Closed-book baseline	Answers or implements from the initial prompt and local context only.
Tool-available baseline	Has search, documentation, and editing tools but chooses its own workflow.
Soft instruction agent	Receives a prompt telling it to research before implementing.
Hard-gated agent	Cannot implement until the evidence stage is completed or an exception is recorded.

The evaluation should measure more than final answer quality. It should include factual correctness, citation support, task success, test pass rate, unnecessary searches, latency, tool cost, clarification quality, and unsupported or unsafe actions.

The central hypothesis would be:

For tasks that depend on external or changing information, a code-enforced evidence-and-verification gate improves correctness and reduces unsupported implementation compared with model-selected tool use, at an acceptable cost in time and tool calls.

That is a specific and falsifiable claim. It is stronger than saying that search is good, and more honest than claiming that search solves hallucination.

Conclusion

The future of reliable AI agents may not be determined only by which model has the largest context window or the most impressive benchmark score.

It may depend more on how the system behaves when the model does not know.

A dependable agent should be able to say, in effect:

“This task depends on information I should not guess. I will clarify the request, find the relevant evidence, read the actual documentation, implement through controlled tools, and verify the result before I claim success.”

That is the idea behind a research-first agent.

Not a model that pretends to know everything.

A system that knows when to learn.

References

[1]: https://arxiv.org/abs/2312.10997 — Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” 2024.

[2]: https://openai.com/index/browsecomp/ — OpenAI, “BrowseComp: a benchmark for browsing agents,” 2025.

[3]: https://arxiv.org/abs/2210.03629 — Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” ICLR 2023.

[4]: https://arxiv.org/abs/2302.04761 — Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools,” 2023.

[5]: https://arxiv.org/abs/2308.00675 — Hsieh et al., “Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models,” 2023.

[6]: https://www.swebench.com/original.html — SWE-bench project, “Can Language Models Resolve Real-world Github Issues?”, benchmark page.

[7]: https://proceedings.iclr.cc/paper\_files/paper/2024/hash/25f7be9694d7b32d5cc670927b8091e1-Abstract-Conference.html — Asai et al., “Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection,” ICLR 2024.

[8]: https://aclanthology.org/2025.findings-emnlp.1264/ — Du et al., “Context Length Alone Hurts LLM Performance Despite Perfect Retrieval,” Findings of EMNLP 2025. [9]: https://opencode.ai/docs/tools/ — OpenCode, “Tools,” official documen (Content truncated due to size limit. Use line ranges to read remaining content)