Addressing deficiencies 1 and 2 at the smallest unit
Which unit tests touch this change. Proof they ran.
Mark's first deficiency: no system for selecting tests for a particular release. Second deficiency: no system for checking whether they ran. CODITECT addresses both at unit level: only the tests that touch the change run, and every result is recorded against the change ID.
"We and every other software company in the world are outstripping our ability to test what we're building."
Why now: the velocity of agentic coding has decoupled from the velocity of testing, auditing, and validation - the knowledge and proof that AI agents did what they were tasked to perform, i.e. testing, in this case. An AI agent can produce more code in a day than a team used to write in a sprint. The test, audit, and compliance layers did not get faster at the same rate. The gap is structural and widens with every model release.
Three deficiencies - in every company today - that no software addresses:
- determining which tests need to run for a particular release
- checking whether they ran
- recording the outcome
Mark Walker, nue.io - meeting transcript [00:46:36]
Unit tests are the smallest piece CODITECT can select, run, and record per change. The first of the three deficiencies is addressed automatically here.
Absence 1 at unit level - which tests touch this change
On every commit, CODITECT inspects the diff and resolves it to specific functions, classes, or modules. From the dependency graph, it identifies the unit tests that exercise those code paths. Only those tests are queued for the commit - not the whole suite.
The selection plan is a JSON artifact: which tests, why each was chosen, what each covers. The plan is attached to the commit as evidence of decision.
Absence 2 at unit level - that they ran
Selected tests run via the project's own test framework (pytest, vitest, jest, go test, cargo test, others). Each test reports start time, end time, status, assertion message on failure. A test that fails to start counts as a failure, not as 'skipped'.
Test results are normalised into a standard JSON shape regardless of framework, then recorded against the commit ID.
Absence 3 at unit level - recording the outcome
Per-test result rows are written to the audit database with file, line, assertion, timing, and the change that triggered the run. The full console output is captured. Coverage gaps are flagged automatically.
When a code path lacks any unit test, CODITECT generates one with rationale and proposes it as a pull request. The operator reviews, approves, and the recorded outcome of the next run includes the new test.