Skip to content
RepoKnit

From a repository URL to an answer you can check.

RepoKnit reads a repository, splits it into meaningful pieces of code, and finds the pieces that answer your question. Since an answer can only be as good as the evidence found, the finding gets measured too.

This page describes the intended system. Only the frontend exists today, and each component below says where it stands.

The pipeline

  1. Connect

    Fetch a public GitHub repository and record the exact commit, so every citation points at one version of the code.

    repo + commit SHA

  2. Parse

    Find files, languages, functions, classes and imports. Skip binaries, vendored code and anything that looks like a secret.

    symbol-aligned chunks

  3. Embed and index

    Embed each chunk and store it in Turbopuffer with its path, symbol and line range.

    searchable index

  4. Retrieve

    Find the chunks relevant to a question, then answer only from them. Citations are checked after the answer is written.

    answer + citations

  5. Evaluate

    Run retrieval setups against hand-labelled questions. Metrics appear only where labels exist to compute them.

    benchmark runs

Why retrieval matters most

A language model can only reason about the code it's shown. If the function that answers your question ranks just outside the results it receives, the model never sees it. It either says it doesn't know or writes something plausible.

So RepoKnit shows retrieved chunks next to every answer, and Search Lab measures retrieval on its own, separately from how convincing the answer reads.

top_k = 5
  1. 1docs/routing.md
  2. 2tests/test_routes.py
  3. 3app/views.py
  4. 4README.md
  5. 5app/config.py
  6. the model sees nothing below this line
    6app/router.py Router.matchthe answer
An illustration, not a measurement. Recall@5 here is 0, and the fix is better retrieval rather than a better prompt.

System architecture

Requests flow left to right. Indexing runs in the background and never inside an HTTP request. Select a component to see what it does.

In progress Planned Optional

  • Browser

    In progress

    Client

    Where engineers ask questions, read source and compare retrieval runs.

    Calls or writes to

    • Next.js appHTTPS

  • Next.js app

    In progress

    Frontend

    UI, routing and streaming display. This is the only part built so far.

    Calls or writes to

    • FastAPItyped API

    Used by

    • BrowserHTTPS

  • FastAPI

    Planned

    API

    The authority for validation, access control, quotas and retrieval.

    Calls or writes to

    • PostgreSQLstate
    • Turbopufferretrieve
    • LangGraphmulti-step
    • Job queueenqueue

    Used by

    • Next.js apptyped API

  • PostgreSQL

    Planned

    Application state

    Relational source of truth for everything that is not a vector.

    Used by

    • FastAPIstate

  • Turbopuffer

    Planned

    Vector search

    Stores embedded code chunks with metadata and serves retrieval queries. Not an application database.

    Used by

    • FastAPIretrieve
    • LangGraphretrieve
    • Indexing workerupsert

  • LangGraph

    Planned

    Orchestration

    Bounded multi-step investigations. Simple questions skip it entirely.

    Calls or writes to

    • Turbopufferretrieve
    • LLM provideranswer
    • LangSmithtrace

    Used by

    • FastAPImulti-step

  • LLM provider

    Planned

    Answer synthesis

    Writes answers from supplied evidence only, behind a service interface.

    Used by

    • LangGraphanswer

  • LangSmith

    Optional

    Tracing & evals

    Traces tool calls and evaluation runs, only when explicitly configured.

    Used by

    • LangGraphtrace

  • Job queue

    Planned

    Durable queue

    Decouples indexing from HTTP requests.

    Calls or writes to

    • Indexing workerconsume

    Used by

    • FastAPIenqueue

  • Indexing worker

    Planned

    Background worker

    Fetches, parses, chunks and embeds a repository at a pinned commit.

    Calls or writes to

    • GitHubfetch @ SHA
    • Embedding modelembed
    • Parser / chunkerparse
    • Turbopufferupsert

    Used by

    • Job queueconsume

  • GitHub

    Planned

    Source

    Public repositories only, fetched at a recorded commit SHA.

    Used by

    • Indexing workerfetch @ SHA

  • Parser / chunker

    Planned

    Tree-sitter

    Turns files into symbol-aligned chunks with line ranges.

    Used by

    • Indexing workerparse

  • Embedding model

    Planned

    Provider

    Embeds chunks at index time and queries at search time.

    Used by

    • Indexing workerembed

Two stores with different jobs

PropertyPostgreSQLTurbopuffer
HoldsRepositories, commits, jobs, usage, benchmark labels and runsEmbedded code chunks with path, symbol and line metadata
Queried byExact lookups and joinsSimilarity search plus metadata filters
Source of truth forApplication stateNothing. It's a retrieval index and can be rebuilt

Rules the backend will follow

None of these are enforced yet, because the backend doesn't exist yet.

Repository code is never executed.
Files are read as text at a pinned commit.
Citations are checked, not trusted.
Paths and line ranges are validated against stored chunk metadata after the answer is written.
Missing metrics stay missing.
A metric that wasn't measured is shown as unavailable, never as zero.
The server enforces limits.
Quotas, rate limits and a global kill switch live in the API, not the browser.
Public repositories only.
Nothing is sent to tracing tools without explicit configuration.