I Gave My Coding Agent Eyes Three Different Ways. Here's the Honest Scorecard.

Your coding agent just finished a change. It says "done ✅". You open the browser and the button does nothing.

This happens because the agent never actually saw the running app. It edited code, reasoned about what should happen, and reported success based on the diff. Andrej Karpathy described coding agents as "effectively programming with a blindfold on", and that line stuck with me because it is exactly the failure mode. The model writes plausible code and has no way to check whether the page it just changed behaves the way it claimed.

There are three real ways to take the blindfold off right now, and all three speak the Model Context Protocol so any MCP agent (Claude Code, Cursor, OpenCode) can use them:

I built Iris. Below is what happened when I put all three through the same harness. I am going to lead with the cases where Iris loses, because if you are reading this on r/programming you have already smelled the vendor pitch coming and you deserve the caveats first.

A screenshot is the actual rendered frame. That gives Playwright and DevTools a category of bug that an in-app tool genuinely cannot see.

Here is the one I measured. I dropped a stray CSS filter: hue-rotate(90deg) saturate(1.6) onto the html element. That re-tints the entire rendered page. It changed 21,393 pixels, about 2.3% of the screen. Every computed-style property Iris reads (color, backgroundColor, opacity, geometry) stayed byte-for-byte identical, because the filter happens at paint time, after the values Iris can see.

This is the screenshot's home turf and Playwright wins it cleanly. Iris only matched the catch when I explicitly drove it through its opt-in iris_visual_diff (which uses CDP under the hood). The always-on, no-install SDK reads computed style, so anything that only shows up in actual pixels (font-load failures, paint order, GPU and compositing bugs) is a real blind spot.

The other places the browser drivers win outright:

• →Any site, zero cooperation. Playwright and DevTools test a URL you have never touched. Iris has to embed @syrin/iris-browser, so it physically cannot test a third-party site you do not ship code into.
• →Trusted native input. Playwright drives real CDP input: native keyboard, mouse, file pickers, drag-and-drop, all with isTrusted: true. Iris defaults to synthetic dispatch and real input is opt-in only.
• →Cross-browser. Playwright runs WebKit, Firefox, and Chromium. Iris runs on whatever single engine your app runs on.
• →Browser-level scope. Multi-tab, popups, cross-origin, downloads, auth dialogs, and network mock/intercept all live at the browser level. Iris is scoped to one page runtime, so it observes network but mocking stays the app's job.
• →Protocol-level debugging. Chrome DevTools MCP speaks raw CDP, so for low-level network and performance traces on any site it is the right tool. Iris only sees app-level network, so the wire protocol is out of its reach.

So if you are testing someone else's site, running a cross-browser release matrix, or chasing a pixel-level visual regression, stop reading and use Playwright or DevTools. That is genuinely their job. Iris is built for a narrower situation: an agent building an app you own, verifying its own work on every edit.

A stray CSS filter re-tinted 2.3% of the page. The screenshot diff caught it; Iris's always-on computed-style read missed it.

Now the other direction.

When a browser-driver agent wants to know what is on the page, the common move is to dump the full accessibility tree (or a screenshot) and let the model read it. That is a lot of tokens for a question as small as "did the modal open".

Iris asks narrow questions instead. A query plus an observe plus an assert is a few small structured calls.

Across a 20-step flow that is roughly 2,000 tokens with Iris against roughly 146,000 with full-tree snapshots.

Here is the honest asterisk, because this number gets misread. If you force Iris to also dump the whole accessibility tree, the gap shrinks to about 1.8×. The big multiple comes from Iris usually not needing the whole tree to answer the question, so most of the saving is architectural rather than a magic compression trick. I would rather you hear that from me than catch it yourself.

A single verification is interesting. The thing a test suite actually does is run the same verification over and over, on every commit, forever. That repetition is where the gap stops being a percentage and starts being orders of magnitude.

A screenshot or DOM agent has to re-drive the whole flow with the LLM on every single run. The model clicks through, reads the page, and judges the result again, paying full token cost each time. Iris records the flow once and replays it with no model in the loop: it re-resolves the anchors, advances the clock, and checks the declared consequence.

Two things matter here. First, the per-run cost is two orders of magnitude apart and it compounds with every commit. Second, and I think this is the bigger deal, the flake rate. I replayed the same flow eight times and got one status and one verdict, identical across all eight runs, because there is no LLM sampling temperature or tool ordering to vary. A browser-driver agent re-drives with a live model every run, so its verdict is sampled rather than fixed. Flakiness is the number one tax on any regression suite, and a model-free replay pays zero of it by construction.

The suite-scale number (47 tokens whether you have 2 flows or 4) comes from consolidating into one verdict: passing flows are counted, only failures get detailed. That is the only place a within-field 100× multiple is physically real. You cannot catch 100× more bugs than exist, but you can re-verify a suite 2,574× cheaper.

Re-verifying one known flow costs about 175 tokens with Iris's model-free replay against roughly 30,000 when a browser-driver agent re-drives with an LLM. 128 to 184 times cheaper per run, 2,574 times across a four-flow suite, at 0% flake.

The DOM is a lossy projection of your program. Plenty of real bugs live in the gap between what the program did and what the page shows. A tool standing outside the browser, looking at pixels or markup, has no way to see them. Iris sits in the runtime, so it can.

I built a small suite of bugs that look completely fine on screen and measured who caught what:

• →UI lies about the store. The displayed count says one thing, the store holds another. A screenshot and a DOM read both report the visible value and have no source of truth to contradict it. Iris caught 2 of 2; the browser drivers caught 0 of 2.
• →Dead handler, green button. A "Ship" button is present, clicks fine, and changes nothing in the store. Iris declares the success condition as deployments.0.status == live, so the replay fails when the store never moved, with no element drift to hide behind.
• →Double-submit. An action that fires its network request twice passes any presence check. A declared net { count: 1 } consequence catches the second call. A count: 0 rule catches a forbidden call, like a reverted migration endpoint or a stray privacy beacon that should never fire.
• →Silent console error. The action logs a console.error and the page still renders. A console { absent: true } consequence fails on it, read after the action settles so it cannot pass before the error fires.
• →Wasted-render storm. A React component thrashing at 108 commits per second while producing identical output. There is no DOM mutation, so an outside tool sees an idle page. Iris reads the commit rate off the devtools hook (108 vs a healthy 36) for about 50 tokens.
• →Blast radius. An action corrupts an unrelated piece of store state for a view that is not even on screen. Nothing visible changes anywhere. This is the deepest one, and no out-of-page tool can see it because the damage lives entirely in program state.

I want to be precise about the honest boundary here, because "Iris sees state" can sound like more than it is. Raw network counting and raw console capture are parity: Playwright has route and page.on('console'), DevTools reads both too. The advantage is not that Iris can observe these. It is that the count or the clean console is a declared, deterministic consequence of a replayed flow, checked after settle, that cannot be faked by a locator that healed onto the wrong element. The genuinely Iris-only catches are the ones that require reading the store: the UI-vs-store desync, the dead-handler oracle, the blast radius, and the render storm.

There is also a time dimension. Iris controls the app's setTimeout and Date, so a flow gated on a 2.6-second transition (a deploy going from "building" to "live") verifies in about 202 milliseconds by freezing and advancing the clock. The browser drivers are perfectly capable of testing this, but they have to sleep through real time and guess the duration. Under-wait and the test goes flaky. That gap scales with the timer: a 5-minute timeout makes it roughly 1000×.

Marketing pages love to imply total dominance. The honest reality is that a large slice of web bugs are things a human can see on screen, and any tool with a JavaScript evaluate can reach those.

Every visually-observable bug I tested (computed style, geometry, occlusion, color, theme, six bugs in total) came back 6 of 6 for all three tools. Iris is more ergonomic there because it is one native call instead of authoring a JS snippet (roughly 117 to 259 fewer tokens per check), but it is not more capable. A tie is a tie.

In a live gpt-4o tool-use loop over five broken-app scenarios, using authoritative model usage tokens rather than a proxy:

Iris caught the most, but it spent about 1.7× the tokens to do it on this particular run. That is the trade in a single fresh detection loop. The token advantage I keep talking about only shows up once you start re-running the same flow. One model, one turn budget, five scenarios is a small sample, so treat the accuracy ordering as directional rather than gospel.

Pick by what you are actually doing. Ignore which vendor wrote the blog. Plenty of teams run both: Iris for the inner build-verify loop, Playwright for the cross-browser release gate.

The rule of thumb I keep landing on: if you own the app and an agent is building it, Iris is the cheap, deterministic, state-aware inner loop, and the regression suite that does not go flaky. If you are driving someone else's site, many engines, real input, or true pixels, that is Playwright and DevTools territory.

One table, wins and ties and losses stated plainly. Every row comes from a committed harness in the repo, and you can reproduce it with pnpm bench.

The full scorecard across Iris, Playwright MCP, and Chrome DevTools MCP: Iris wins the agent loop, regression cost, store-state desync, and render storms; ties on visual bugs; and loses on pixel regressions, no-install reach, and cross-browser.

The One Honest Sentence

If you want the whole thing compressed: Iris ties on anything a person can see in the DOM (with a real and growing cost advantage on re-runs), and it wins where the bug requires seeing the program itself, its state and the same flow replayed deterministically. Playwright and DevTools win on true pixels, any-site reach, real input, and cross-browser.

It is not that one of these is better and the rest are obsolete. They are aimed at different moments. A browser driver is the right tool when you are standing outside an app you do not control. An in-app layer is the right tool when an agent is building an app you own and needs to check its own work cheaply on every edit, then re-check it forever without flaking.

I built Iris because that second moment, the agent's inner loop, did not have a tool that read program truth instead of guessing from pixels. The benchmark above is my attempt to show exactly where that bet pays off and exactly where it does not. Every harness behind these numbers is open source at github.com/syrin-labs/iris, so if you find a case where they do not hold, clone it and show me. If the project looks useful, a star on GitHub helps other people building with agents find it.

Continue reading

Growth Tools

OpenOutreach Setup Guide 2026: Free AI LinkedIn Outreach

June 3, 2026Read

AI Experiments

Your AI Agent Has Been in Production for Weeks. Do You Even Know What's Working?

May 19, 2026Read

Research

Agent Drift is Now Quantifiable: Here's the Math Behind It

April 28, 2026Read