Testing

I have a project that four AI agents work on while I’m not at my computer. One drives the application end to end and files bugs. One reads a bug and proposes how to fix it. I approve or push back. One implements the approved plan and opens a pull request. A fourth runs a long-horizon soak overnight, looking for drift that no short session can see. There is no orchestrator. No message queue, no coordinator agent, no shared database, no framework. The agents never talk to each other and none of them holds state between runs. ...

Most AI-assisted QA workflows assume you have access to everything: Playwright MCP configured in VS Code, Copilot Vision enabled, the embedded browser panel working. In an enterprise environment, those assumptions often don’t hold. Security policies restrict which tools can connect to which services. Features get disabled. The standard setup isn’t available. This post documents a different approach that factors in certain constraints. The combination: playwright-cli for browser interaction, GitHub Copilot CLI for the agent loop, and a plain natural language prompt describing what to test. No MCP. No generated test files. No vision model. Just a coding agent running shell commands against a real browser. ...

A working RAG pipeline is easy. Knowing whether it will keep working after you change something is harder, and most projects skip that part entirely. Here, the focus is on designing an eval harness that catches real problems, using the Anthropic docs RAG agent as the example. What an eval harness does An eval harness is a script that runs a fixed set of test cases against your pipeline and produces a pass/fail score. Run it before and after a change — if the score drops, the change broke something. If it improves, the change helped. ...