How Cursor Actually Reads 100,000 Files
Phase 1: The Problem — You Cannot Put a Codebase in a Prompt
Here’s the number that breaks the naive mental model of how tools like Cursor work. A mid-sized company codebase — say, 100,000 files — at a conservative average of 200 lines per file and roughly 8 tokens per line, comes out to somewhere around 160 million tokens. Even the most generous context windows available in 2026 top out in the low millions of tokens, and models that accept that much context get slower, more expensive, and measurably worse at using the middle of that context correctly — a well-documented effect sometimes called “lost in the middle.” So the honest starting point is this: no context window, current or near-future, makes it viable to just paste the whole codebase into the model and ask a question.
This isn’t a minor inconvenience — it’s the entire design constraint that shapes everything else. When Cursor answers “where is the rate limiter configured?” or “why does this function throw on null input?”, it is fundamentally impossible for the underlying model to have read all 100,000 files to figure that out. It read some tiny fraction of them, chosen by a system working extremely hard, in the milliseconds before your question was answered, to guess correctly which handful of files actually matter.
The naive alternative — just grep for keywords and hand the top hits to the model — sounds reasonable but fails constantly in real codebases. A search for “authentication” misses a file that implements login logic but calls it verifyUser. A search for “database connection” misses the pool configuration buried in a file named bootstrap.ts that never uses either word. Human engineers navigate real codebases by understanding intent, not by keyword matching — when you ask a senior engineer where auth happens, they don’t grep, they remember what the code does. Any tool that wants to feel like it “understands your codebase” the way a teammate does has to solve this same problem: turning meaning into something searchable, at a scale where you genuinely cannot read everything first.
This is the problem Cursor — and every serious AI coding tool — has to solve before a single line of generated code ever happens: how do you find the right handful of files, out of tens or hundreds of thousands, fast enough and accurately enough that the model’s answer is actually grounded in your real code rather than a plausible-sounding guess?
Phase 2: The Mental Model — A Codebase Is a Library, Not a Book
The cleanest way to build intuition here is to stop thinking of a codebase as one enormous document and start thinking of it as a library with a very good card catalog.
If someone hands you a single 50-page document and asks a question, the right move is obviously to read the whole thing. That’s what happens when a model’s context window comfortably covers what you’ve given it. But nobody expects a librarian to have personally read all four million books in a national library before answering a reference question. The librarian’s actual skill is knowing where to look — which section, which shelf, sometimes which exact book — based on a structured index built long before your question ever arrived.
That’s precisely the architectural shift: instead of treating “understanding the codebase” as a reading problem, treat it as a retrieval problem. Build an index once (or incrementally, as code changes), and at question time, use that index to narrow 100,000 files down to the 10 or 20 that actually matter, then hand only those to the model. The model never sees the library. It sees the handful of books the librarian pulled off the shelf.
The second mental model worth building is what actually goes into that “card catalog” entry for a piece of code. It isn’t just the filename or the raw text — it’s a numeric fingerprint, called an embedding, that captures the code’s meaning in a way that lets similar meanings sit close together mathematically, even when the exact words differ. This is precisely why a query like “where do we handle failed payments” can surface a function named handleDeclinedTransaction that shares almost no literal words with the query — the embedding captures the conceptual neighborhood, not the vocabulary.
The third piece of intuition, and the one most tutorials skip, is that code isn’t prose, and treating it like prose during indexing throws away the most useful signal available: structure. A function’s meaning isn’t just its text — it’s also which functions call it, which files import it, where it sits in the folder hierarchy, and how it fits into the compiler’s own understanding of the codebase (its abstract syntax tree). A serious code-indexing system leans on this structure heavily, because it’s free, precise information that a general-purpose text embedding would otherwise have to guess at from prose alone.
With those three ideas in place — retrieval instead of reading, meaning-based fingerprints instead of keyword matching, and structural signal instead of pure text — the internal architecture in Phase 3 will make a lot more sense than it would cold.
Phase 3: Internal Working Deep Dive — From Keystroke to Answer
This is the core of how a tool like Cursor actually operates across a massive codebase, broken into the two lifecycles that matter: building the index, and answering a query against it.
3.1 Building the Index — What Happens Before You Ever Ask a Question
Step one: chunking, not file-by-file embedding. A naive approach would embed each file as one unit. This fails immediately in real codebases, where a single file can be 2,000 lines covering a dozen unrelated concerns. Instead, files get broken into smaller, semantically coherent chunks — typically aligned to logical boundaries like functions, classes, or methods, rather than arbitrary line counts. This is where the abstract syntax tree (AST) — the compiler’s structural map of the code — becomes essential: it lets the chunker split a file at a function boundary instead of, say, halfway through an if statement, which would produce a chunk that’s syntactically meaningless on its own and therefore embeds poorly.
Step two: enriching each chunk with context before embedding it. A raw function body, stripped of everything around it, often loses critical meaning. A function called validate(input) means almost nothing on its own — but the same function annotated with its file path (src/auth/session-validator.ts), its surrounding class name, and its imports suddenly carries much more signal. Production-grade indexers prepend this kind of lightweight metadata to each chunk before embedding, because the embedding model can only capture meaning that’s actually present in the text it’s given.
Step three: generating embeddings at scale. Each enriched chunk is passed through an embedding model, producing a vector — typically a few hundred to a couple thousand dimensions — that represents its meaning as a point in high-dimensional space. Doing this for 100,000 files, each broken into perhaps 10–30 chunks, means generating on the order of a million to several million embeddings. This is done in batched, parallelized calls rather than one chunk at a time, because the fixed overhead per API call would otherwise dominate the total indexing time.
Step four: storing vectors in a structure built for fast similarity search. A million-plus vectors can’t be searched by brute-force comparison at interactive speed — comparing a query vector against every stored vector one by one simply doesn’t scale to sub-second response times. Instead, these systems use approximate nearest-neighbor (ANN) data structures — commonly graph-based indexes like HNSW (Hierarchical Navigable Small World) — which trade a small amount of accuracy for a massive speed advantage, turning a search that would otherwise scan millions of vectors into one that touches a tiny, well-chosen fraction of them.
Step five: keeping the index fresh without re-indexing everything. Code changes constantly — a full re-index on every keystroke would be prohibitively expensive and slow. Instead, indexing systems watch the filesystem for changes and re-embed only the chunks belonging to modified files, often using a content hash to detect whether a file’s meaningful content actually changed versus, say, just its formatting or comments. This incremental approach is what makes it feasible for the index to stay usefully current across a workday of active editing rather than only being accurate right after a full rebuild.
3.2 Answering a Query — What Happens When You Ask a Question
Step one: the query itself gets embedded, using the same embedding model used for the codebase, so the question and the code chunks live in the same meaning-space and can be meaningfully compared.
Step two: approximate nearest-neighbor search retrieves the top candidates — typically a few dozen chunks whose embeddings sit closest to the query’s embedding. This step is fast specifically because of the ANN index structure built during indexing; without it, this step alone could take seconds instead of milliseconds on a large codebase.
Step three: hybrid retrieval fills the gaps that pure embeddings leave. Semantic search is excellent at conceptual matches but genuinely weak at exact-token matches — a query containing a specific variable name, error code, or config key is often better served by traditional keyword or full-text search than by an embedding comparison. Serious systems run both in parallel and merge the results, because relying on embeddings alone reliably fails on exactly the queries that mention a specific identifier.
Step four: re-ranking narrows dozens of candidates down to the handful that will actually be sent to the model. The first-pass retrieval (steps two and three) is deliberately tuned for speed and recall across the entire codebase, which means it’s imprecise by design — it’s meant to cast a reasonably wide net cheaply. A second, more computationally expensive re-ranking pass — often a smaller, more focused model — then re-scores just those dozens of candidates for genuine relevance to the specific query, because that level of scrutiny is only affordable once the search space has already been narrowed from millions of chunks to dozens.
Step five: the assembled context — the surviving few chunks, plus relevant structural context like the file’s imports or the function’s call sites — gets handed to the model alongside the user’s actual question. This is the moment where the earlier work pays off: the model is now reasoning over a small, highly relevant slice of a 100,000-file codebase, not the codebase itself, and — critically — it usually has no idea how much it didn’t see. Which is precisely why retrieval quality, not model intelligence, is the dominant factor in whether the final answer is actually correct.
Phase 4: Engineering Implementation — A Realistic Retrieval Pipeline
The following pseudocode reflects the real shape of steps described above, with the reasoning behind each design decision made explicit.
def index_codebase(files):
for file in files:
# Chunk at AST boundaries, not arbitrary line counts —
# a chunk that cuts through a function mid-body embeds
# into meaningless, noisy vector space.
ast = parse_to_ast(file)
chunks = split_at_function_and_class_boundaries(ast)
for chunk in chunks:
# Enrich with structural context before embedding.
# A bare function body loses the meaning carried by
# its file path, class name, and imports.
enriched = attach_context(
code=chunk.text,
file_path=file.path,
imports=file.imports,
enclosing_class=chunk.enclosing_class,
)
vector = embed(enriched)
# Store both the vector and enough metadata to
# reconstruct exact source location later —
# the vector alone can't tell you where the code lives.
vector_store.upsert(
id=chunk.id,
vector=vector,
metadata={"path": file.path, "lines": chunk.line_range},
content_hash=hash(chunk.text), # for incremental re-indexing
)
def answer_query(query, vector_store, keyword_index, model):
query_vector = embed(query)
# Hybrid retrieval: semantic search alone misses exact
# identifier matches; keyword search alone misses paraphrase.
semantic_hits = vector_store.approximate_search(query_vector, top_k=40)
keyword_hits = keyword_index.search(query, top_k=40)
candidates = deduplicate(semantic_hits + keyword_hits)
# Re-ranking is only affordable now that we've narrowed
# from millions of chunks down to ~60-80 candidates.
ranked = rerank(query, candidates)
top_chunks = ranked[:12]
# Pull in one hop of structural context — e.g. the immediate
# callers or imports of the top chunks — since a function's
# meaning is often incomplete without its immediate neighbors.
expanded_context = expand_with_call_graph(top_chunks, hop_depth=1)
return model.generate(query=query, context=expanded_context)
The most common real-world mistake isn’t in this pipeline’s shape — it’s skipping the enrichment step and the hybrid search step because they’re not strictly necessary to get a working demo. A pure vector-search-over-raw-code-chunks system will answer conceptual questions reasonably well and then fail, confusingly and unpredictably, on any question involving a specific identifier, exact error message, or config key — because nothing in a bare code embedding was ever built to capture exact-token precision in the first place.
Phase 5: Real-World Systems — How This Plays Out at Scale
GitHub Copilot and similar tools initially leaned much more heavily on the currently-open file and a handful of recently-viewed files rather than full-repository retrieval — a reasonable starting point given the difficulty of the indexing problem, but one that visibly struggled on questions about code far from the cursor’s current location. The industry-wide move toward full-repository semantic indexing, of the kind described in Phase 3, is a direct response to that limitation: repository-wide questions need repository-wide retrieval, not just local context.
Sourcegraph, which built code search as its core product well before the LLM era, is a useful case study in why structural signal matters so much: its search product was built around precise code intelligence (go-to-definition, find-references, symbol graphs) long before embeddings entered the picture, and modern AI-assisted search products borrow heavily from that same structural approach — because a call graph built from exact compiler analysis is strictly more reliable than an embedding’s approximate guess at the same relationship.
Large monorepo companies — Google and Meta are the most-cited examples — operate at a scale (billions of lines of code across a single repository) where even the retrieval architecture described here needs additional layers: sharding the index across machines, aggressive caching of frequent queries, and incremental indexing pipelines that process file changes as a continuous stream rather than periodic batch jobs, because a monorepo of that size can have thousands of commits landing per day. The core retrieval logic doesn’t fundamentally change at that scale — but every piece of the pipeline in Phase 3 needs to be re-engineered for throughput rather than assumed to just scale linearly.
The consistent lesson across all of these: the hard, differentiating engineering work is almost entirely in retrieval quality and infrastructure — not in the final step of handing context to the model, which is comparatively simple once the right chunks have been found.
Phase 6: AI-Era Relevance — Codebase Retrieval as Applied RAG
Strip away the code-specific vocabulary, and everything in Phase 3 is a specialized instance of retrieval-augmented generation — the same architectural pattern used to ground chatbots in company documents, or research assistants in academic papers. The reason it’s worth understanding codebase indexing specifically, even if your interest is AI engineering broadly, is that code retrieval is one of the hardest and most instructive versions of the RAG problem, for a specific reason: code has far more usable structure than prose.
A paragraph in a PDF has essentially no formal structure an indexer can lean on beyond its position in the document. A function in a codebase has an exact, compiler-verified structure — its call graph, its type signature, its import dependencies — that can be extracted with zero ambiguity, unlike the fuzzy, embedding-only relationships you’re stuck inferring in unstructured text. This is why hybrid retrieval (combining exact structural signals with approximate semantic search) is far more mature and effective in code-focused RAG systems than in general-purpose document RAG — the ceiling on retrieval quality is simply higher when structure is available for free.
This also connects directly to agentic coding systems, which are becoming the dominant interface for AI-assisted software engineering. An agent that’s asked to “fix the bug causing intermittent login failures” doesn’t just need one good retrieval — it needs to retrieve, reason, decide it needs more context, retrieve again based on what it just learned (say, discovering a call to an external session store it didn’t know about), and repeat that loop several times before it has enough grounded understanding to propose a real fix. Every one of those retrieval steps depends on exactly the indexing and search architecture described in Phase 3 — which means codebase retrieval isn’t a solved, static component sitting underneath agentic coding tools. It’s a live, repeatedly-invoked capability that the entire quality of the agent depends on, turn after turn.
Phase 7: Advantages, Limitations, and Honest Trade-offs
Advantage — it scales sublinearly with codebase size. Because retrieval narrows the search space before the model ever gets involved, query latency and cost stay roughly stable whether the codebase has 10,000 files or 500,000, as long as the underlying ANN index is well-built. This is precisely what makes the approach viable at all — a linear-scaling strategy (reading more of the codebase as it grows) would become unusably slow and expensive well before reaching real enterprise scale.
Limitation — retrieval is probabilistic, and wrong retrieval produces confidently wrong answers. If the right chunk simply never makes it into the top-ranked candidates — because it’s phrased unusually, or the enrichment step missed important context — the model has no way of knowing it’s missing something. It will typically answer fluently and plausibly based on whatever it did receive, with no visible signal to the user that the answer might be incomplete. This is arguably the most consequential limitation of the entire approach: failures are silent, not loud.
Trade-off — freshness versus cost. Fully re-indexing on every single file save would keep the index perfectly current but would be prohibitively expensive at scale; batching or debouncing index updates saves enormous cost but introduces some lag, meaning very recent changes might not yet be reflected in retrieval results. Every real system makes an explicit choice about how much staleness is acceptable, and that choice becomes very visible to users the moment they ask about code they just wrote seconds ago.
Limitation — structural signal isn’t available for every language or codebase equally. Languages with mature, widely-supported AST tooling get much higher-quality chunking and structural enrichment than obscure or highly dynamic languages, where accurate static analysis is genuinely harder. This means retrieval quality isn’t uniform across a company’s stack — a well-typed backend service often gets meaningfully better AI assistance than a loosely-structured configuration or scripting layer sitting right next to it.
Phase 8: Career Impact — Why This Matters Beyond Coding Tools
Understanding this architecture is directly useful career signal for a few overlapping roles that are in high demand in 2026: engineers building internal AI tooling on top of a company’s own codebase or document set, engineers working on developer-tools products themselves, and AI engineers more broadly, since codebase retrieval is simply the sharpest, most structurally rich version of the RAG problem you’ll encounter — mastering it transfers cleanly to less structured domains.
In interviews, this shows up less as “explain what RAG is” and more as scenario-based system design: “design a system that lets an AI assistant answer questions about a 200,000-file codebase,” which is really asking whether you understand the chunking, hybrid retrieval, re-ranking, and incremental-indexing trade-offs covered in Phase 3 — not whether you can name the concept of embeddings. Candidates who can explain why pure semantic search fails on exact-identifier queries, and why re-ranking only becomes affordable after a first-pass narrowing step, consistently stand out over candidates who can only describe the pipeline’s happy path.
What to learn next, if this area is genuinely interesting: go hands-on with an actual vector database and ANN index (rather than only reading about HNSW conceptually), and separately, spend time with a language’s AST tooling directly — parsing and chunking real source files yourself. The combination of those two hands-on skills, semantic retrieval infrastructure and structural code analysis, is precisely the intersection this entire category of tooling lives in.
Phase 9: Final Thoughts — The Illusion of Reading, Built From Not Reading At All
The experience of using a tool like Cursor on a large codebase feels, from the outside, like talking to something that has read your entire project. That feeling is an illusion carefully engineered by everything described in this article — and it’s worth sitting with why that illusion is actually the right design goal, rather than a shortcut being papered over. No system, human or model, actually needs to read every file to answer a specific question well. What it needs is a genuinely excellent index and a genuinely disciplined retrieval process, the same way a great research librarian doesn’t need to have memorized every book to point you, in seconds, at exactly the right page.
The deeper lesson generalizes well past coding tools: as context windows keep growing and it becomes tempting to believe “just throw everything at the model” is a viable long-term strategy, the retrieval-first architecture described here is a reminder that scale isn’t solved by bigger context windows alone — it’s solved by knowing, with precision, what not to show the model in the first place. That discipline — deciding what matters before generation ever happens — is the real engineering achievement behind tools that seem to “understand” a hundred thousand files, and it’s a discipline that will keep mattering long after today’s specific context-window limits are a distant memory.
