Can an agent get better without changing its weights?

“Does memory help agents?” is one of those claims that gets asserted far more often than it gets tested. Reflection loops, lesson banks, and self-improvement pipelines are everywhere, but most demos can’t tell you whether the agent actually got better — or whether its memory quietly filled up with confident nonsense. ZenArena exists to turn the vague question into something falsifiable.

Turning a vibe into a delta

The same LLM agent plays chess under three conditions: no memory at all, an append-only memory that keeps everything it ever writes down, and a governed memory that is actively curated. Chess is the domain for one reason: Stockfish provides a clean, external truth signal. Every move can be scored in centipawns against the engine’s evaluation, so “the agent improved” stops being a vibe and becomes a measurable delta.

Memory with an immune system

The interesting arm is the governed one. Self-reflecting agents have a documented failure mode — sometimes called misevolution — where the lessons an agent teaches itself gradually degrade its behaviour. ZenArena’s memory module targets that directly with four anti-degradation mechanisms:

• a replay validation gate — a lesson must survive re-testing before it is kept;
• Jaccard-similarity deduplication that merges near-duplicate lessons;
• confidence × decay weighting, so stale lessons lose influence over time;
• full provenance — every lesson records the game IDs, verifier deltas, and board positions that produced it.

If governed memory wins, then curation — not volume — is the load-bearing part of agent self-improvement.

No cheating allowed

The easiest way to fake a result here would be to leak the engine into the agent, so verifier isolation is enforced structurally in code: Stockfish is never reachable at move-selection time, and dedicated isolation tests guarantee it. Any measured gain is attributable to the memory pathway, nothing else.

Around the experiment sits a proper research platform rather than a one-off script: a single pluggable LLM abstraction spanning seven backends (from mock to Anthropic, OpenAI, Gemini, Ollama, LM Studio, and OpenAI-compatible gateways), per-match telemetry for cost, latency, tokens, and fallbacks, and a dark-glass React dashboard with learning curves, A/B/C analytics, provenance drill-down, and live model-vs-model matches — all over a database-free FastAPI service. Engineering hygiene is research-grade: 73 pytest functions across 13 modules, CI running tests, lint, and the frontend build, and capture labelling that keeps smoke runs out of the analytics.

Beyond chess

Chess is scaffolding, not the point. The governance pattern — validate before keeping, deduplicate, decay, record provenance — is domain-agnostic, and it is exactly what any long-running agent with a memory will eventually need, whether it writes code or answers support tickets.

The experimental method is just as reusable as the mechanism: pick a domain with an external truth signal, isolate the verifier from the agent, and let your claim be wrong. If governed memory beats append-only against the engine, that’s evidence curation matters. If it doesn’t, that’s worth knowing too. Either way, the platform makes the answer measurable instead of anecdotal.