The Problem: Why “Just Feed It the Whole Repo” Was Never Going to Work
The first instinct almost every engineer has when they hear “AI coding assistant” is some version of: why doesn’t it just read the whole codebase before answering? It sounds like the obvious solution. A model with a large enough context window should, in theory, be able to ingest every file in a repository and reason over all of it at once, the way a new hire might if they had a photographic memory and infinite patience.
That instinct runs headfirst into two separate walls, and understanding both is the entire reason modern coding agents are built the way they are.
The first wall is arithmetic. Even a context window in the range of a million tokens — genuinely enormous by the standards of just a couple of years ago — is dwarfed by real production codebases. A mid-sized company’s monorepo can run into the millions of lines across tens of thousands of files; a large enterprise codebase can be an order of magnitude bigger still. There is no context window on the near horizon large enough to hold “everything,” and even if there were, you wouldn’t want to pay for it: every token an agent loads into context is a token it has to pay to process on every single turn of the conversation, whether or not that token turns out to be relevant to the task at hand.
The second wall is subtler and, in practice, the more important one: even when a huge amount of content technically fits inside a context window, stuffing it full of mostly-irrelevant code doesn’t make a model smarter — it makes it worse. This effect has been observed and measured repeatedly across the industry through 2025 and into 2026, and it has a name in agent-engineering circles: context rot, or context pollution. A model asked to find a single bug in a 40,000-token pile of loosely related files has to do the hard work of figuring out which five of those tokens actually matter, buried inside thirty-nine thousand nine hundred and ninety-five that don’t. Attention, in a very real mechanical sense, gets diluted. Irrelevant code doesn’t just cost money to process — it actively degrades the quality of the answer you get back, because the signal-to-noise ratio inside the context window is doing as much work in determining output quality as the model’s raw capability.
So the actual engineering problem coding agents have to solve isn’t “how do we fit more code into context.” It’s “how do we figure out, for this specific task, the small handful of files and functions that actually matter, and get exactly those into context — nothing more, nothing less — without a human having to manually go select them first.” That’s a search problem, not a memory problem, and it’s worth being precise about that distinction, because the two obvious-sounding approaches to solving it — pre-built vector indexes and live agentic search — turned out to have a genuinely surprising winner when the industry actually measured them on real code.
Building the Mental Model: Two Fundamentally Different Ways to Find Relevant Code
Approach one: index everything up front, retrieve by meaning (RAG)
The dominant pattern borrowed from the broader world of retrieval-augmented generation is intuitive: before the agent ever gets a task, run a background process that chunks every file in the repository, converts each chunk into a vector embedding — a numerical representation of its meaning — and stores those vectors in a searchable database. When a task comes in, embed the task description itself, and retrieve the chunks whose vectors sit closest to it in that meaning-space. This is exactly how most RAG systems built for searching documents, wikis, and support tickets work, and it was the natural first thing the industry tried applying to code as well.
Approach two: don’t pre-build anything — let the agent search live, the way a human would
The alternative approach treats the codebase the way a new engineer treats an unfamiliar repository on their first day: no pre-built map, just a terminal, a handful of sharp tools (grep-style text search, file-globbing, the ability to open and read a file), and the judgment to decide what to look for next based on what the last search turned up. The agent isn’t handed pre-retrieved context before it starts — it drives its own investigation, one tool call at a time, narrowing in on the relevant code exactly the way an engineer would grep for a function name, notice which files it lives in, open the most promising one, and follow the thread from there. This pattern is what the industry now calls agentic search, and the defining property that distinguishes it from RAG is who decides what to look at next: in RAG, a retrieval algorithm decides before the model ever sees the query; in agentic search, the model itself decides, turn by turn, informed by what it has already found.
The analogy that makes the trade-off click
Picture two ways of finding a specific book fact in a library. RAG is like handing a librarian’s assistant a pre-built card catalog keyed by topic — you ask about “coffee brewing,” and the catalog hands you every card filed under that general theme, some of which are exactly what you needed and some of which are tangentially related enough to have been filed there by an imperfect classification process. Agentic search is like sending in a research assistant who walks the stacks themselves: they check the index for the exact title, notice a citation inside the first book that points to a second one, follow that thread, and keep pulling only the specific pages that turn out to matter, adjusting their plan based on what they actually find as they go. The first approach is fast and requires no per-query thinking, but it’s only as good as the pre-built classification and it can miss anything the catalog wasn’t built to anticipate. The second is slower per step but converges on exactly what’s needed, and it can follow leads no pre-built index could have predicted.
This distinction turns out to matter enormously for code specifically, for a reason that becomes clear once you look at what actually happens inside a live search.
Internal Working Deep Dive: What Actually Happens When an Agent Searches Your Codebase
This is the mechanism worth understanding in real detail, because it explains both why modern coding agents are built the way they are, and why the industry’s early bet on vector-embedding retrieval for code specifically didn’t hold up the way it did for other kinds of content.
Why exact-match search wins for code in a way it doesn’t for prose
Vector embeddings excel when the thing you’re searching for is a concept that can be phrased many different ways — “how do I cancel my subscription” should match a support article titled “Ending your membership,” even though the two phrases share almost no words in common. That fuzzy, meaning-based matching is exactly the right tool for prose.
Code is a different kind of artifact. A function is either named createD1HttpClient, or it isn’t — there’s no fuzzy version of that fact. When an engineer needs to find every call site of a specific function, they don’t want “conceptually related” code; they want the exact symbol, every place it appears, with zero false positives and zero false negatives. Embedding-based retrieval, precisely because it matches on meaning rather than exact tokens, tends to surface code that is thematically nearby but textually unrelated — which in a coding context is usually noise rather than signal, and worse, it can just as easily miss the exact match it needed to find if that match happens to sit in an unusual part of embedding space for stylistic reasons (unusual variable naming, terse comments, an unconventional file layout).
This isn’t just an argument from first principles — it’s been measured directly. Anthropic’s own team, in public remarks about Claude Code’s design, has described testing an early version of the product built around a local vector database and finding that agentic, tool-driven search consistently outperformed it for coding tasks. Independent academic research reached the same conclusion from a different angle: a systematic comparison published by Amazon Science in early 2026 found that keyword-based search performed through agentic tool use achieved the large majority of a full RAG pipeline’s retrieval performance without needing a vector database at all — and for codebases with reasonably consistent naming conventions, exact-match search actually outperformed semantic retrieval outright. The broader SWE-bench leaderboard tells a version of the same story at a larger scale: the very first RAG-based baseline on that benchmark — chunk the repo, embed it, retrieve the top matches, generate a patch — scored under 2%. Replacing that retrieval step with an agent driving its own tool calls (open a file, scroll, search, edit) more than sextupled that result immediately, and by 2026 the top-performing systems on the benchmark are agentic, tool-driven architectures rather than vector-retrieval pipelines.
The actual request lifecycle: from a plain-English task to a located answer
When you give Claude Code a task like “find where we validate incoming webhook signatures,” here’s the mechanical sequence that unfolds, expressed as a general pattern rather than an exhaustive account of any single implementation:
Step one: an initial broad probe. Rather than reading files at random, the agent typically starts with a fast, high-recall text search — the equivalent of a grep for terms drawn directly from the task (“webhook,” “signature,” “validate,” and variations). This is intentionally cheap and broad; the goal at this stage isn’t precision, it’s surfacing candidates.
Step two: narrowing by structure, not just text. A raw text search across a large repository often returns far more matches than are useful — the same word can appear in tests, in documentation, in unrelated modules, and in the actual implementation all at once. The agent narrows this using file-pattern filtering (glob-style matching restricted to source directories, excluding test fixtures or generated code, for instance) and by reasoning about which of the surfaced files structurally look like the right place — a file named webhook_handler.py sitting in a payments/ directory is a much stronger candidate than a hit inside a changelog.
Step three: reading, and following the thread. Once the agent has a short list of strong candidates, it reads the actual file contents — not the whole repository, just these specific files — and from there, follows real code relationships: if the signature-validation function it finds calls into a shared crypto utility module, the agent’s next move is to go look at that file too, the same way a human engineer would click through to a function definition. This is the step that a pre-built vector index fundamentally cannot replicate, because the index was built once, in advance, with no idea what specific thread this particular task would need to follow — agentic search, by contrast, generates its next query based on what the current one just revealed, which lets it chase chains of reasoning a static index has no mechanism for.
Step four: convergence and context assembly. The agent repeats this loop — search, narrow, read, follow — until it has enough located, verified context to actually answer the question or make the edit, at which point it stops searching and acts. Critically, only the specific file contents the agent actually chose to read end up occupying space in the context window; everything it didn’t need to open never costs a single token, which is the direct mechanical answer to the “context rot” problem described earlier. Industry write-ups analyzing this pattern have measured context-window savings in the range of roughly 95% compared to naively loading broad swaths of a repository up front, precisely because the agent is only ever paying for what it actually decided, in the moment, that it needed.
Why “no index” is a feature, not a missing feature
A background reason agentic, live search has held up well against pre-built indexing specifically for coding agents is that source code changes constantly, often multiple times an hour during active development. A vector index built at 9 a.m. is already stale by the time a developer has made their third commit of the morning, and keeping it perfectly synchronized in real time is its own nontrivial engineering problem — one that introduces an entire additional class of failure modes: embedding-provider outages, stale collections, rate limits, and the quiet risk that a reviewer sees “relevant chunks were retrieved” and assumes the whole codebase was considered, when in fact the index might have silently excluded generated files, test fixtures, or a private package it was never configured to crawl. Live, tool-driven search sidesteps this entire failure class by construction — there’s no index to go stale, because there’s no index at all. It also has a privacy dimension worth noting: because nothing needs to be pre-processed into embeddings and shipped to a third-party embedding provider, no code has to leave the developer’s machine before a search even happens, which matters a great deal for organizations with strict source-code confidentiality requirements.
Where a persistent index does still earn its place
None of this means vector-based code search is obsolete — it means it solves a different problem than day-to-day agentic coding tasks do. Purpose-built semantic code search, delivered as a pluggable capability an agent can call into (implemented through the Model Context Protocol, for instance, by projects that provide semantic search as an external tool), remains genuinely useful for a specific class of query: broad, conceptual questions across an enormous codebase, like “find functions that handle user authentication” across a codebase so large that even efficient live search would need many exploratory rounds to converge. In that mode, a pre-built semantic index isn’t competing with agentic search — it’s another tool the agent can choose to call, alongside grep and glob and file-reading, and the agent’s own judgment decides which tool fits which kind of question. The lesson underneath both approaches is the same one that runs through all of modern agent design: search should be treated as live infrastructure the agent actively drives, not a static artifact it passively receives.
Persistent project memory as a complement to live search
Live search handles “what does this specific task need right now,” but coding agents also benefit from a second, much smaller and much more durable kind of context: a project-level memory file that captures information a human would otherwise have to re-explain every single session — build commands, coding conventions, architectural decisions, which directories are safe to touch and which aren’t. This kind of persistent file is deliberately tiny compared to the codebase itself, and it complements rather than replaces live search: it answers “what are the standing rules of this project,” while agentic search answers “where, specifically, does the code relevant to this task live.” Treating these as two separate mechanisms, each solving a distinct problem, is part of why modern coding agents avoid the trap of trying to solve everything with one enormous, ever-present context blob.
Engineering Implementation: What This Looks Like as an Actual Tool Loop
Understanding the mechanism is more useful once it’s grounded in the shape of the actual tool-calling loop an agent runs. Below is a simplified but structurally accurate illustration of the pattern — not a reproduction of any proprietary internals, but the same kind of tool-driven search loop publicly described in how modern coding agents operate.
from dataclasses import dataclass, field
from typing import Optional
@dataclass
class SearchState:
"""
Tracks what the agent has learned so far in this task, so each new
tool call can build on prior findings instead of starting blind.
"""
candidate_files: list[str] = field(default_factory=list)
files_read: dict[str, str] = field(default_factory=dict)
ruled_out: set[str] = field(default_factory=set)
def grep_search(pattern: str, exclude_dirs: Optional[list[str]] = None) -> list[str]:
"""
Fast, exact-match text search across the repository. High recall,
intentionally cheap -- this is the agent's first move, not its last.
Excludes noisy directories (tests, generated code, vendored deps)
by default so the first pass isn't drowned in irrelevant hits.
"""
exclude_dirs = exclude_dirs or ["node_modules", "dist", "vendor", "__pycache__"]
# ... invokes an actual grep-equivalent against the working tree ...
return [] # returns matching file paths + line context
def glob_filter(candidates: list[str], pattern: str) -> list[str]:
"""
Narrows a broad grep result set by file path structure -- e.g.
restricting to 'src/**/*.py' to discard incidental matches in
docs, fixtures, or changelogs that happened to contain the search term.
"""
return [f for f in candidates if _matches_glob(f, pattern)]
def read_file(path: str, state: SearchState) -> str:
"""
The only step that actually spends context-window tokens on file
content. Everything upstream (grep, glob) operates on file paths
and small snippets, keeping the exploration phase cheap.
"""
content = _read_from_disk(path)
state.files_read[path] = content
return content
def agentic_search_loop(task_description: str, max_rounds: int = 6) -> SearchState:
"""
The core loop: search, narrow, read, follow references -- repeated
until the agent has enough located, verified context to act, or
a round budget is hit (a safety valve against runaway exploration).
"""
state = SearchState()
query_terms = extract_search_terms(task_description)
for round_num in range(max_rounds):
raw_hits = grep_search(pattern=query_terms[0])
narrowed = glob_filter(raw_hits, pattern="src/**/*")
# A real agent reasons here about which narrowed hits are worth
# opening -- this is the step that is fundamentally *adaptive*:
# the decision depends on what THIS search just returned, not
# on a plan fixed in advance.
promising = rank_candidates(narrowed, task_description, state)
if not promising:
break # nothing new to follow -- converged or exhausted
for path in promising[:2]: # read the strongest candidates only
content = read_file(path, state)
# Following references found inside the file just read is
# what a static, pre-built index structurally cannot do --
# the next query is generated FROM this file's actual content.
referenced = extract_references(content)
query_terms.extend(referenced)
if has_sufficient_context(state, task_description):
break
return state
Why each design choice exists
Grep before read, always. Text search across file paths and short snippets costs a fraction of what reading full file contents costs. Structuring the loop so cheap, broad operations run first and expensive, narrow ones run last is what keeps the overall token bill low without sacrificing precision — you pay full price only for the small number of files that survive multiple rounds of narrowing.
Excluding noisy directories by default. A search that includes node_modules, build output, and vendored dependencies by default will drown genuine signal in thousands of irrelevant hits. This is a small implementation detail with an outsized effect on real-world search quality, and it’s one of the most common mistakes in naive implementations of this pattern.
A round budget as a safety valve. Without an explicit cap, an adaptive search loop that keeps generating new queries from what it just read has no natural stopping point and can spiral into unbounded exploration on a sufficiently confusing codebase. A round limit forces the agent to either converge on an answer with what it has or explicitly report that it needs a narrower task description — both better outcomes than silently burning an unbounded token budget.
Ranking candidates instead of reading everything that matched. Even after narrowing by file path, a grep result can still return more files than are worth opening. Reasoning about which specific candidates are actually worth the cost of a full read — informed by file location, naming, and what’s already been learned — is the step that most directly determines whether the loop converges efficiently or wastes rounds reading dead ends.
Extracting references to drive the next query. This is the single mechanical detail that separates agentic search from a fixed retrieval pipeline. The next search term isn’t decided in advance — it’s generated from the actual content of the file the agent just read, which is exactly how a human engineer follows a function call to its definition, or a class reference to where it’s declared.
Common implementation mistakes
A frequent mistake in naive implementations of this pattern is treating the first grep result as authoritative and reading every match immediately, rather than narrowing first — this burns context-window budget on files that a slightly smarter filtering step would have ruled out for free. Another is failing to exclude generated code, vendored dependencies, and build artifacts from the search surface by default, which floods early rounds with noise and can cause the agent to genuinely miss the actual source file underneath a pile of irrelevant matches. A third, subtler mistake is search that doesn’t terminate — a loop with no round budget and no convergence check can, on a sufficiently large or confusingly structured repository, keep generating new queries indefinitely, burning both time and money without ever actually completing the task.
Real-World Systems: How This Plays Out at Scale
The clearest large-scale validation of the agentic-search approach is the SWE-bench benchmark itself, which measures whether an AI system can resolve real, previously filed GitHub issues against real production codebases — not synthetic toy problems, but the exact kind of “find the relevant code, understand it, fix it correctly” task this whole search problem exists to solve. The benchmark’s own history is a live case study in the RAG-versus-agentic-search question: the original RAG-based baseline, publicly released in 2023, scored under 2%. Replacing pre-built retrieval with a tool-driven agent that opens, scrolls, and searches files on its own initiative pushed that number past 12% almost immediately, and by 2026 the leaderboard’s top-performing systems — spanning multiple vendors and architectures — are dominated by agentic, tool-driven designs rather than vector-retrieval pipelines, with the best systems clearing the majority of the benchmark’s real-world issues.
At the level of individual products, the industry’s convergence on this pattern is broad rather than limited to a single vendor. Public write-ups tracking the space through 2026 describe the same tool-driven, no-pre-index architecture showing up across multiple major coding agents from different companies — evidence that this isn’t one vendor’s idiosyncratic choice, but a pattern the industry converged on independently after testing the alternative and finding it wanting for this specific kind of problem. Enterprise engineering teams working with genuinely large monorepos — codebases spanning hundreds of thousands of lines and thousands of files — have reported, in public technical write-ups, hitting real friction when a codebase’s sheer size pushes past what pure live search can efficiently converge on within a reasonable number of rounds, which is precisely the gap that purpose-built semantic search tools, offered to agents as an additional callable capability via the Model Context Protocol, are designed to fill as a complement rather than a wholesale replacement.
AI Era Relevance: Why This Is a Preview of a Much Bigger Pattern
The RAG-versus-agentic-search question inside coding tools is, in miniature, a preview of a debate now playing out across nearly every serious application of large language models to real, large, constantly-changing bodies of information.
Agentic retrieval is displacing static RAG well beyond code. The same core argument that favored live, tool-driven search for source code — precision over fuzzy recall, no stale index to maintain, adaptive follow-the-thread exploration a fixed retrieval step can’t replicate — is increasingly being applied to enterprise knowledge bases, customer support systems, and internal documentation search, anywhere the underlying corpus changes often enough that a pre-built index is fighting a losing battle against staleness.
Context engineering has become its own discipline. As agents gain access to more tools, more files, and longer-running tasks, deciding what not to put in context has become as important an engineering skill as prompt writing itself. The context-rot problem described earlier — more tokens degrading output quality rather than improving it — means that an agent’s search strategy is now inseparable from its actual reasoning quality; a system that finds the right five files performs measurably better than one that finds the right five files buried inside five hundred irrelevant ones, even holding the underlying model constant.
Multi-agent and sub-agent architectures extend the same pattern to bigger problems. As coding agents take on longer-running, more complex tasks, a common architecture spins up focused sub-agents to investigate a specific thread — one sub-agent traces how authentication works across a service, another investigates a specific failing test — and reports back a compact, already-distilled summary rather than raw file contents, which keeps the orchestrating agent’s own context small even as the total amount of code actually explored across the whole task grows substantially.
This pattern is a leading indicator for how AI systems will handle any large, live, rapidly-changing corpus going forward — not just code, but live databases, constantly-updated product catalogs, and operational systems where a stale snapshot is actively dangerous rather than merely inconvenient. Understanding why agentic search won out for code is a genuinely transferable piece of engineering judgment for anyone building retrieval into an AI system in any domain where “changes constantly” and “needs to be exactly right” both apply at once.
Advantages, Limitations, and Trade-offs
Advantage: precision without staleness. Live, tool-driven search finds exact matches with no fuzzy false positives, and because there’s no index to fall behind, it’s never wrong about the current state of the code the way a stale vector index quietly can be. This matters most in exactly the situation coding agents spend most of their time in: an actively edited repository where the ground truth changes multiple times an hour.
Limitation: convergence isn’t guaranteed, and it costs rounds. Because the agent is discovering the right files through iterative search rather than retrieving them in one shot, a confusingly organized codebase, inconsistent naming conventions, or a task description that doesn’t share vocabulary with the relevant code can cause the search loop to take many rounds to converge — or, in the worst case, to time out against a round budget without finding what it needed. A pre-built semantic index, when it’s well-maintained and the query happens to match well, can sometimes get there in a single retrieval step where live search needs several.
Trade-off: no-index simplicity versus scale ceiling. Avoiding a pre-built index sidesteps an entire class of maintenance and staleness problems, but it also means every session starts from zero — there’s no accumulated map of the codebase carried over from the last time the agent worked in it, only whatever a persistent project-memory file was deliberately set up to capture. On sufficiently enormous codebases, this becomes a genuine scaling limitation, which is exactly why purpose-built semantic search tools exist as an optional, callable complement rather than being treated as unnecessary.
Trade-off: token efficiency versus completeness. Reading only what the search loop specifically decided was relevant keeps costs low and avoids context rot, but it also means the agent is, by construction, operating on a partial view of the codebase at any given moment — it’s a calculated bet that the specific files it chose to read are actually the ones that matter, and an imperfect search strategy can miss a relevant file elsewhere that never surfaced in any of the executed searches.
Career Impact and What to Learn Next
Understanding how coding agents actually locate relevant code — rather than treating the whole thing as an opaque black box — has become a genuinely practical skill for software engineers working alongside these tools daily, not just for the small number of people building agent infrastructure itself. Engineers who understand why an agent’s search sometimes misses a relevant file learn to write more search-friendly task descriptions, structure repositories with clearer naming and directory conventions that make both human and agentic search easier, and set up project-memory files that fill the specific gaps live search structurally can’t close on its own. For engineers building agentic systems directly — an increasingly in-demand specialization spanning AI infrastructure, developer-tools engineering, and platform engineering — fluency in the RAG-versus-agentic-search trade-off, context engineering, and tool-loop design is becoming as foundational a skill as understanding REST API design was a decade ago.
If this topic is new to you, the most productive next steps are studying how the SWE-bench benchmark evolved from its original RAG baseline to today’s agentic leaderboard, since that history is itself the clearest evidence for why this architecture won out; reading published comparisons of retrieval strategies for code specifically, including the academic literature comparing keyword-based agentic search against vector retrieval; and, most usefully, paying close attention the next time you use an AI coding assistant to which tool calls it actually makes before it answers — noticing the sequence of searches, narrows, and reads is the fastest way to build real intuition for how this mechanism behaves on your own codebase’s particular quirks.
The instinct to solve “how does an AI understand my codebase” by simply making the context window bigger turned out to be solving the wrong problem entirely. The real problem was never capacity — it was judgment: figuring out, for this specific task, which small slice of a much larger codebase actually matters, and doing so in a way that keeps pace with code that never stops changing. The industry’s answer, arrived at not through theory but through head-to-head testing against the obvious alternative, was to stop trying to pre-digest the entire codebase into a static index and instead give the agent the same tools and the same iterative judgment a good engineer already uses on their first day in an unfamiliar repository: search broadly, narrow quickly, read only what earns it, and follow the thread wherever the code itself leads. That’s not a workaround for the limits of context windows — it’s a better model of how understanding a large, living system actually works in the first place, for a human or a machine, and it’s a design principle that’s only going to matter more as agents are asked to reason over bigger and faster-changing bodies of information than source code alone.
Further Reading / External Links
- Claude Code — Product Overview
- Vadim’s Blog — Claude Code Doesn’t Index Your Codebase. Here’s What It Does Instead.
- Zilliz — Claude Context: Code Search MCP for Claude Code
- Developers Digest — Claude Context Is Code Search For Agents
