qikly

Functional tests for backend Python

Independent test generation for AI-written code

The coder never sees the answer key. Its tests come from criteria it was never shown, so they can disagree with it. Two code panels kept apart: calc.py written by the coding agent, and test_calc.py written by the test agent, which asserts a tax rate limit the coding agent was never given. The suite passes 9 of 9.

When one AI writes both the code and its tests, a green suite proves little: the tests were written to pass. qikly splits the work between two agents. A test agent reads the acceptance criteria and writes the tests. A coding agent writes the code and never sees those criteria, so the tests can disagree with it.

In the run shown above, only the test agent was told a rate may not exceed 100. The coding agent figured that bound out from a failing test and added or rate > 100 itself, so the 9 of 9 is code passing an acceptance bar it never saw.

$ pip install --upgrade qikly $ qikly --explain CALC_TAX

That prints a bundled task twice, once as each agent receives it, and the difference between them. No API key, no model call, about a second. Then qikly --demo runs a task end to end with a Gemini, OpenAI or Anthropic key for well under a cent, and five steps take your own module to a first run.

Source on GitHub How it works, with one repair end to end

What it is for

The behaviour of backend logic, one module at a time

It tests

What a Python module does with its inputs: data transforms and ETL steps, calculations, validation rules, merges and business logic. Integration, system and unit tests, in pytest, for small to mid-sized repositories.

It does not test

User interfaces, performance, load, security or any other non-functional property, and it does not run end-to-end tests across services.

It fits most naturally where verification already has to be independent of the people who build the thing: automotive, medical devices, fintech and defence.

How it works

The coding agent never sees the acceptance criteria

One task file holds the requirements, the interface and the acceptance criteria. The test agent reads all three and writes the integration and system tests before any code exists. The coding agent gets the same file with the criteria cut out, and works from failing tests until the suite passes.

How the task file is split, and how the loop turns One task file feeds two agents. The test generation agent sees all three sections. The coding agent sees only the first two, never the acceptance criteria. Both meet at pytest. A failing run returns to the coding agent with the failing test and its error, never the criteria. Once the integration and system stages pass, the test generation agent writes the unit stage, and that is the only point at which it is allowed to read the implementation. one task file 1 requirements 2 interface (signatures) 3 acceptance_criteria test generation agent sees 1, 2 and 3, not the code coding agent sees 1 and 2, never 3 the suite integration, system, then unit the code pytest pass converged code + suite fail: the failing test, never the criteria integration and system pass: now write the unit stage, code visible
The main flow consists of two loops. The dashed purple one is the repair cycle: a failing run returns the failing test and its error, never the criteria, so the coding agent can only learn the bar one failed test at a time. The dashed teal one is the staged expansion: integration and system tests are written before any code exists, and only once they pass is the unit stage written, which is the single point where test generation may read the implementation.

Two obvious objections

"Just use one model for the code and another for the tests." That helps a little. It does not fix the underlying issue: both models still read the same criteria, so the code is still built to the standard it is about to be judged by. Changing who types it does not change what they were shown. qikly removes the criteria from the coding agent's prompt, qikly --explain shows the cut, and a test fails the build if any call site lets a criterion through.

"Doesn't a failing test give the criteria away?" It shows the coding agent that one test, its source and its error, and the test often states the rule it checks. It cannot change a test already written: the integration and system suites exist before any code does, so what the agent learns from a failure can shape the code, never the bar.

See it run

One command, about thirty seconds, nothing written outside its folder

A qikly --demo run: the acceptance criteria are withheld from the coding agent, three test suites are generated, a test fails, a patch is applied, and the suite goes green.
One qikly --demo run, unedited. The criteria are withheld, the suites are generated, a test fails, a patch lands, and it goes green. 38 seconds on gemini-3.5-flash-lite, not sped up.

Start

Pick the command that fits what you already have

Every command, by what you already have, and the design write-up: the case, how well it works, with the measured results and how it is built.

The separation this tool enforces is ordinary practice in safety-critical engineering, where verification is required to be independent of implementation as part of a V&V (Verification & Validation) methodology for testing. This library's author worked in that setting on General Motors' autonomous vehicle program before building this toolset.