Engineering an agent’s eyes and hands

The problem

A solo build structured as a director-and-agent division of labour. The codebase is largely agent-written; the engineering documented here is the workflow that made that possible and kept it correct.

Mulk (Arabic: dominion) is a discrete-tick world simulation. Civilizations arise on a real Earth — loaded from a GEBCO elevation heightmap — or on procedural worlds, then spread, trade, war, convert, invent, fragment and collapse on their own across seven ages, while the player reaches in with about 37 god-powers. It runs two modes, a free-form Sandbox and a scored Shepherd challenge, over a 512×256 world grid, exported to the browser.

The development challenge, and the actual subject of this write-up, is that an AI coding agent has two blind spots: it cannot see the screen, and it cannot play. For ordinary software that is survivable. For an emergent simulation whose behaviour only appears over thousands of ticks, it is fatal — you cannot tell whether mega-nations fragment correctly, or whether the climate model depopulates the subarctic, by reading code or glancing at a single frame.

The entire workflow is an apparatus engineered to close those two gaps: to give the agent eyes (visual and numeric) and hands (drive the editor and the running game).

The hands: three control channels

The project drives Godot through three distinct channels, configured in .mcp.json and project.godot:

The two editor MCPs let the agent write the game; the runtime bridge lets it observe and puppeteer the game while it runs, clicking buttons and sampling a property every N frames into a ring buffer.

All three are security-hardened — shared-secret auth with lockout, constant-time compare, value whitelists, read-only allowlists — because they expose a live engine to an automated client. The tooling co-evolved with the game: when the enhanced server’s auth key rotated on every reload and trapped the client in a connect → auth-fail loop, the fix was to patch the server to reuse a stable key. The agent maintaining its own instruments mid-project.

The eyes: the agent builds its own instruments

This is the distinctive part. A god-game’s behaviour is emergent over thousands of ticks, so instead of eyeballing, the workflow builds bespoke headless diagnostic harnesses that print quantitative summaries, runs them, reads the numbers, tunes constants, and re-runs. Empirical, measurement-driven tuning via CLI flags:

Reproducible seeding is the key. It lets the agent A/B a constant change against a fixed world and trust that a metric delta came from its edit, not from RNG. For the things judged by eye, a separate deterministic screenshot pipeline (about 35 --shot<name> recipes) seeds a state, parks the camera, optionally fast-forwards, and saves a PNG — so the agent generates and curates its own visual evidence.

Cross-session memory and the inner loop

The build spanned many sessions over nine days, so Claude Code’s file-based memory carries hard-won operational knowledge forward: nine cross-linked memory files encoding the non-obvious — the port-to-plugin map and how to diagnose “not connected”, the auth-key rotation fix, a six-place checklist for adding a god-power, and the design rationale for each major system. This is what kept a multi-day, 15k-line build coherent.

A typical feature or balance cycle: plan against the design docs, edit the simulation via MCP, validate headlessly (errors = 0), measure with the relevant probe and tune until metrics land in range, look by regenerating screenshots or driving the live game over the runtime bridge, commit with a structured body stating root cause and verification, and persist any new gotcha into memory.

A recurring discipline: substantial changes are staged in the working tree and confirmed with the director before committing, rather than self-committed.

Numbers

Real figures from the codebase and history: measured, not estimated.

Measured tuning outcomes Measured

The empirical loop, working. Each delta traced to a deliberate constant change against a seeded world:

The point of these is not the game balance. It is that an agent that could not watch the simulation still produced quantified, verifiable results, because the measurement apparatus existed.

What I’d do differently, and known limitations

Stated plainly: these are real.

• No conventional test suite. The diagnostic harnesses are the test suite. Defensible for a solo simulation build, but a true assertion-based regression suite would catch silent breakage the probes do not.
• Web build size. The browser export’s index.wasm is large (~38 MB), a downstream consequence of targeting web export from this engine and pipeline. Worth a size-reduction pass.
• Documentation drift. The design docs ran ahead of, then behind, the implementation as the game evolved emergently from its original concept. Trivial to reconcile, and worth doing.

Competencies

What this build demonstrates, in hiring terms: