TestIQ is a CLI tool that reads a codebase, understands it through retrieval-augmented generation, and uses a fully local LLM (via Ollama) to write contextually-aware unit tests and explain why existing tests are failing, all without sending a single line of code to the cloud.

I built it primarily to learn RAG from the inside out, not to read about it, but to actually wire up the indexing, retrieval, and generation pipeline myself and see where it breaks. It's an archived project now (I built it, used it, and moved on), but the last time I ran it, the full pipeline, from indexing a project to generating and validating tests, still worked end to end.

The Problem

I wanted to understand retrieval-augmented generation well enough to work around its actual failure modes, not just its happy path. RAG demos online almost always show the easy case: a clean question, a clean document, a clean answer. I wanted to throw something messier at it, an entire codebase, with functions that call each other, shared state, and edge cases, and see what it actually takes to get a local model to generate a test that's both relevant and correct.

Unit test generation turned out to be the perfect torture test for that. It's not enough to retrieve "related" code. The tool has to understand what a function actually does, find the right surrounding context (imports, sibling functions, class structure), and then produce something that isn't just plausible-looking Python, but code that actually runs and actually tests the right thing.

Key Features

Codebase indexing. testiq index <path> parses a project with Tree-sitter and stores the resulting structure in a local ChromaDB vector database, so retrieval works on the actual shape of the code, not just raw text.
Context-aware test generation. testiq generate can target a whole directory, a single file, or one specific function, pulling in only the relevant context before handing it to the LLM.
Scenario prediction. testiq assume analyzes a file and predicts happy-path, failure, and critical edge cases before a single test is written.
Plain-English failure explanations. testiq explain takes a failing test, explains what went wrong in plain English, and proposes a patch.
Risk-ranked coverage scanning. testiq scan walks a codebase and surfaces untested functions, ranked by how risky they'd be to leave uncovered.
Zero cloud dependency. Every model call, embeddings and generation alike, runs locally through Ollama. No API keys, no usage limits, no code ever leaving the machine.

Under the Hood

The pipeline is a straight line from raw source to validated tests, with a feedback loop built in at the end:

Tree-sitter parses the codebase into an AST, so the tool understands functions, classes, and structure instead of treating files as plain text.
ChromaDB stores those parsed chunks as embeddings (via mxbai-embed-large), acting as the local vector database for retrieval.
When generating tests, a retriever pulls the most relevant chunks of context for the target function, and that context gets handed to a local Ollama model (gemma4:e2b) to draft the actual test.
Every generated test is run straight through Pytest, and failures feed back into a self-correction loop that tries to fix the test before handing it back.
Everything is configured through a single testiq.config.toml, so swapping models or endpoints doesn't touch any code.

Challenges Worth Mentioning

Getting all the pieces to actually work together. RAG sounds simple as a diagram: parse, embed, retrieve, generate. In practice, every stage has its own failure modes, and they compound. A slightly-off chunking strategy from Tree-sitter throws off retrieval, bad retrieval gives the model the wrong context, and a model with the wrong context confidently writes a test for the wrong function. Most of the real work was in the seams between these stages, not any one of them in isolation.

Validating whether the output was actually correct. A generated test that runs isn't the same as a generated test that's right. A test can pass while asserting something trivial, or fail for a reason that has nothing to do with the code it's testing. That's what pushed me to build the self-correction loop around Pytest in the first place: instead of trusting the model's first output, the tool actually executes what it generated and uses the real result to judge and improve it, rather than taking the LLM's word for it.

What I Took Away From It

Building TestIQ made RAG's failure modes concrete in a way tutorials never did: retrieval quality quietly caps generation quality no matter how good the model is, and "the LLM produced an output" is a much lower bar than "the LLM produced a correct output." Building the self-correction loop, where generated tests get executed and judged by their actual results, was the part that taught me the most, and it's a pattern I'd reach for again in any project that hands an LLM's output real responsibility.