The field worker's app showed four appointments. The dispatcher's screen showed no tour for any of them. Both views looked plausible in isolation. Together they described two different systems.1
That contradiction surfaced while the DATAMIMIC team was building a field-service scheduling prototype with coding agents: Flutter clients for three user roles, a FastAPI backend, PostgreSQL, and asynchronous workers. The individual functions were easy. The failures lived between persistence, process transitions, background execution, and role-specific projections. Local tests passed, endpoints responded, and the connected business process was still inconsistent.1
The team's answer was not a longer prompt.1 It was to stop letting the agent define the reality its own code gets judged against. DATAMIMIC, which hit the Hacker News front page this week under the title "don't let your coding agent invent its own test world,"2 is a concrete version of that idea, and the write-up of how they used it is unusually candid about what it can and cannot do.
Why coding agents make this worse
A human team has built-in friction. One person interprets the requirement, another verifies it later, and fixtures may predate the code. An agent compresses that into one session: it interprets, implements, and verifies in a single context, so one wrong assumption about the domain propagates through the code, the fixtures, and the assertions together. Everything is internally consistent, green, and wrong.1
Unit tests and mocks still have a job. They answer focused questions about functions, contracts, and error handling. They do not prove that a paid appointment enters planning exactly once, reaches the assigned employee, and survives retries without duplication. Those properties cross persistence, APIs, asynchronous execution, and user-role projections, and no fixture written beside the feature under test is going to challenge them.1
What DATAMIMIC is
DATAMIMIC is a model-driven, deterministic-first test data platform. The Community Edition in the repo is MIT-licensed, Python-native, and MCP-ready, and it is what the field report above ran on (4.1.0). Install with pip install datamimic-ce.3
Its core promise is a determinism contract: same engine version, same model, same seed means byte-identical output on every run and machine, held at several layers and gated in CI. Every facade result carries a determinism_proof.content_hash, so the exact data an agent saw can be re-executed and verified later.3 The generators are domain-aware rather than random: finance, healthcare, and demographics models keep relationships and business logic consistent, which is the thing the README demonstrates Faker-style tools get wrong (a generated "25-year-old with Alzheimer's").3
Supported targets run from PostgreSQL, MySQL, Oracle, MS SQL, SQLite, and MongoDB to CSV, JSON, XML, XLSX, and fixed-width files. There is also pseudonymization for staging exports in two modes: seeded, where the same source record maps to the same masked value every run, and non-seeded, with no reversible mapping at field level, for one-time deliveries that want maximum privacy.3 CE is the open-source core of a larger Enterprise Platform that adds governed workflows, a probability-scored PII scanner, multi-system execution, and a Rust fastpath designed for billion-record workloads.3
The agent-facing workflow
The part that matters for this blog's beat is how the tool is shaped for agents to author data without being able to fudge it.
New models start as model.dm.json, a declarative statement of intent. The agent does not hand-write XML; the authoring service deterministically compiles the JSON into the XML descriptor the engine executes.4 The minimal model is a few lines:
{
"version": "1",
"seed": 7,
"products": [
{
"kind": "generated",
"name": "cities",
"count": 3,
"fields": [
{"kind": "values", "name": "region", "values": ["north", "south"]}
]
}
]
}{
"version": "1",
"seed": 7,
"products": [
{
"kind": "generated",
"name": "cities",
"count": 3,
"fields": [
{"kind": "values", "name": "region", "values": ["north", "south"]}
]
}
]
}Authoring runs as one transaction per attempt. datamimic scaffold model.dm.json --format json compiles, lints, performs one bounded run, and evaluates acceptance against that same capture; failures come back as structured issues with a path, allowed fields, and sometimes a typed repair. The agent declares an expectation for every stated requirement, and verified=true certifies only what was declared: a requirement nobody encoded passes verification silently, which is why declaring expectations is mandatory. Stop on verified=true, then run the generated descriptor for real.4
For MCP environments there is an optional adapter (pip install "datamimic-ce[mcp]") exposing four tools: datamimic_reference, datamimic_scaffold, datamimic_check, and datamimic_run, mirroring the CLI's reference, scaffold, lint, and dry-run commands. It deliberately exposes only those four operations; domain generation stays on the Python/CLI side rather than becoming a parallel MCP authoring path.3
What it caught, and what it didn't
The field report earns its keep by showing both sides. The clearest win was a cross-role invariant: the same confirmed appointment must look the same to the customer, the dispatcher, and the employee. The seeded, versioned state made the dispatcher/employee contradiction visible, which local endpoint tests never would. To be fair to the evidence, the report is explicit that the case shows the need for such a cross-role regression test; it is not proof that an automated DATAMIMIC test already detected the defect.1
The most instructive miss was the F1 planning comparison. The declared scenario had 300 objects, 25 employees, and 1,320 contractual services under a fixed seed. Both planning strategies scheduled all 1,320 services and left none unplanned. Approximated drive time fell from 2,110,175 seconds to 214,337 seconds, a reduction of 89.84%, and two runs of the full pipeline produced the same aggregate hash across all 27 application tables (3e9741523fd08e0f5ddd44505f5081230e38e65e91e6e519fbc03726698faa82).1
The test passed even though the workload-spread indicator jumped from 790.11 to 2,823.44 service minutes. Why? The acceptance gate defined a threshold for drive time (an 8% improvement) and none for workload distribution. The scenario was not broken; the acceptance specification was incomplete, and the deterministic state made that omission measurable instead of invisible.1
That is the actual value. Determinism makes failures attributable: change one implementation detail, rerun the same difficult state, and any difference in the result belongs to the implementation, not to fixture drift.1
The lines the report draws
Two caveats are worth keeping in mind.
Determinism is not coverage. A perfectly repeatable scenario can still be shallow, and the report pairs named deterministic scenarios with controlled variation in cardinalities, boundaries, and lifecycle states. Random data, on its own, can be diverse and irrelevant at the same time.1
The agent is not the final authority. In this setup it may author the model and run the gates, but acceptance properties live outside the implementation and the running system provides the verdict: contract validation, state-machine contract tests, integration tests, process tests, and UI tests on a simulator all stay in the loop.1
Their decision rule for when a declarative model beats local fixtures is the part worth stealing:1
Use code when setup is a local test detail. Use a declarative model when the state itself is part of the behaviour under review.
For stateful, relational applications where correctness crosses role views and async boundaries, that describes most of the interesting cases. DATAMIMIC is one concrete way to build that world, and to keep the agent from inventing the world its own code is graded against.
- Alexander Kell, "Deterministic Test Data Specifications for AI Coding Agents" (August 10, 2026): https://datamimic.io/blog/deterministic-test-data-ai-coding-agents/
- "Datamimic - don't let your coding agent invent its own test world" on Hacker News: https://news.ycombinator.com/item?id=49722276
- DATAMIMIC Community Edition README: https://github.com/rapiddweller/datamimic
- AGENTS.md, DATAMIMIC repository: https://github.com/rapiddweller/datamimic/blob/development/AGENTS.md