AI-assisted visual QA: Playwright captures at phone and desktop, light and dark, before you say done

What you will build

A visual QA loop you run after every front-end change, whether you or an AI agent made it. A Playwright script opens your pages at phone and desktop sizes in light and dark mode, saves screenshots, counts the elements that must be there, exercises a few real interactions, and records every page error. A short result note and a review prompt then hand that evidence to an AI agent, which has to judge the screenshots and numbers instead of saying "looks good". This is how every change to this site was checked before it was called done.

Before you start

• A web project you can run locally, and Node.js.
• Playwright as a library: npm i -D playwright, then npx playwright install chromium. To use your installed Google Chrome instead, launch with channel: "chrome"; Playwright supports the branded Chrome and Edge channels.
• Claude Code or Codex, if you want the agent review in Step 8.
• About an hour to set up; a few minutes per run after that.

How it works

An AI agent that edits CSS cannot see the page unless you give it a way to. Build success and a passing lint only prove the code compiles. The loop below produces three kinds of evidence: images a person or model can look at, numbers a script can compare, and error logs that are empty or not.

1. Start the site the way you want to test it, and confirm it responds.
2. For each theme and each viewport, open a fresh browser context with that color scheme and size.
3. Save screenshots of full pages and of key sections.
4. Count the elements that must exist, and read the data attributes your components set.
5. Click, filter and type like a visitor, and count again.
6. Record page errors, console errors and failed requests.
7. Sample animations at fixed times, and check reduced motion and pause.
8. Write the numbers and file names into a short result note.
9. Give the note and the images to an agent with acceptance lines to check.

Step 1: Start the site and confirm it responds

For quick iterations, run the dev server. Before a release, check a production build (npm run build then npm run start), because that is what visitors get. Put the base URL in an environment variable so the same script can later check the live site.

npm run dev &
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:4310/

A 200 means go. Anything else means stop: fix the server first, or every screenshot is of an error page.

Check

• The curl line prints 200.
• You know which build you are testing: dev, local production or live.

Step 2: Build the capture matrix

Use a new browser context per combination. colorScheme makes the page see prefers-color-scheme as light or dark, and viewport sets the size. 390 by 844 stands in for a phone and 1440 by 900 for a laptop; those are the sizes this site is checked at.

import { chromium } from "playwright";
const B = process.env.BASE || "http://127.0.0.1:4310";
const b = await chromium.launch({ channel: "chrome" });
for (const theme of ["light", "dark"]) for (const [w, h, vw] of [[1440, 900, "1440"], [390, 844, "390"]]) {
  const ctx = await b.newContext({ viewport: { width: w, height: h }, colorScheme: theme });
  const p = await ctx.newPage();
  const errs = [];
  p.on("pageerror", (e) => errs.push(e.message.slice(0, 140)));
  await p.goto(B + "/library", { waitUntil: "networkidle" });
  await p.waitForTimeout(800);
  await p.screenshot({ path: `library-${theme}-${vw}.png`, fullPage: vw === "1440" });
  // counts and interactions go here (Steps 3 to 5)
  console.log(theme, vw, errs.length ? errs.join(";") : "no errors");
  await ctx.close();
}
await b.close();

Full-page screenshots on desktop show the whole layout in one file. On the phone size, a viewport screenshot shows what a visitor sees first. For a single section, screenshot a locator instead: await p.locator("#about").screenshot({ path: "about.png" }), after scrollIntoViewIfNeeded() so lazy content has rendered.

Check

• Four files per page: light and dark, 390 and 1440.
• Dark screenshots are actually dark. If not, your site may use a theme toggle instead of the system setting.

Step 3: Count what must be there

Screenshots show how things look. Counts show that nothing went missing. Pick selectors for the parts that matter and print how many exist. This is the core of the check used for this site's library:

const biz = await p.locator(".lib-card").count();
await p.locator(".lib-tab").nth(1).click();
const build = await p.locator(".lib-card").count();
await p.locator(".lib-tab").nth(0).click();
await p.locator(".lib-chip", { hasText: "Safety" }).click();
const safety = await p.locator(".lib-card").count();
await p.fill(".lib-search input", "three.js");
const search = await p.locator(".lib-card").count();
console.log(theme, vw, "| business", biz, "| build", build, "| safety", safety, "| search three.js", search);

That run printed 14 business cards, 8 builder cards, 3 cards under Safety and 4 results for "three.js" in all four combinations, which matched the content files. A wrong number points straight at a filter or data bug that a screenshot might hide.

Components can also report their own state. The 3D hero sets data-wordmark3d="on" when WebGL is running, so the script reads it with p.evaluate(() => document.querySelector(".personal-hero")?.dataset.wordmark3d). Counting canvas elements gives a second signal.

Playwright's docs note that for asserting a count in a test suite, toHaveCount is better than reading count(), because it waits and retries. For a capture script that prints numbers for review, count() is fine.

Check

• Every count has an expected value written down before you run.
• The same counts appear in light and dark and at both sizes.

Step 4: Record errors, and filter the noise

Listen before you navigate. pageerror fires for uncaught exceptions, console for console messages (keep type error), and requestfailed when a request fails.

p.on("pageerror", (e) => errs.push("pageerror " + e.message.slice(0, 160)));
p.on("console", (m) => m.type() === "error" && errs.push("console " + m.text().slice(0, 160)));
p.on("requestfailed", (r) => errs.push("failed " + r.url().slice(0, 120)));

Read failures before you act on them. On the live check of this site, some "failed" entries were Next.js prefetches cancelled when the script navigated away. Left to load, the page showed none, and the prefetched URL returned 200 on its own.

Check

• Every combination prints "no errors", or each error is explained in the result note.
• A deliberate error in a test page shows up in the output, so you know the listener works.

Step 5: Exercise behaviour, not just looks

Click the controls a visitor would use and check the result. On this site the header has a pause control for decorative motion, which sets data-motion on a root element:

await p.locator(".header-tool[title*='Pause']").click();
const motion = await p.evaluate(() => document.querySelector(".motion-root")?.dataset.motion);
const floating = await p.locator(".motion-control").count();

After moving the controls into the header, this check confirmed the old floating controls were gone (count 0) and that pausing set data-motion to off. Also test routes with special rules. This site keeps the Academy pages in light mode, so the live check opens /academy/assess with a dark color scheme and confirms the page still reports light.

Check

• Every interactive control you changed has one scripted click and one printed result.
• Rules that should hold across routes have their own line in the output.

Step 6: Sample animations and check reduced motion

For animation, screenshots at fixed times show each stage. Compute each wait from one start time, so small delays do not add up:

const t0 = Date.now();
for (const [name, at] of [["solved", 1.2], ["scrambled", 7.0], ["shape", 12.2], ["solving", 18.2]]) {
  const wait = at * 1000 - (Date.now() - t0);
  if (wait > 0) await p.waitForTimeout(wait);
  await p.locator(".about-cube").screenshot({ path: `cube-${name}.png` });
}

Then check the other side. Playwright's newContext accepts reducedMotion: "reduce", which makes the page match prefers-reduced-motion: reduce. The cube should show one still, solved frame, and the hero should keep its plain text heading.

For WebGL pages captured on a Mac, the hero capture launched Chrome with --use-angle=metal and --enable-gpu to get rendered frames.

Check

• Each stage image shows a different stage.
• The reduced-motion capture shows no animation stage, only the still state.

Step 7: Write the result note

Keep one short note per change, next to the images. The site's notes follow the same shape every time:

# [Change] result (YYYY-MM-DD)
## Change
- What changed, in which files.
## Checks (raw)
- lint: 0 errors. build: exit 0.
- capture.mjs light/dark x 1440/390: business 14, build 8, safety 3, search 4, no page errors.
## Remaining
- What was not verified (for example: physical phone smoothness, live site).

Raw means copied output, not a paraphrase. The note is what you, a teammate or an agent reviews later.

Check

• Every number in the note appears in a command's output.
• The "Remaining" section is never empty unless everything truly was checked.

Step 8: Hand the evidence to an AI agent

Ask for a review against acceptance lines, with the files as the only source of truth. A fresh agent session works best, because it has not seen the reasoning behind the change.

Review this UI change using only the files in evidence/library-v2/. Do not trust my summary.
Acceptance:
1. Library shows 14 business and 8 builder cards in light and dark at 390 and 1440.
2. No page errors in any combination.
3. Dark screenshots have readable text and no light boxes left over from light mode.
4. At 390 wide nothing is cut off and there is no horizontal scroll.
For each line: PASS or FAIL, the output line or image that proves it, and what you saw.
List anything you could not verify. Ignore style preferences that do not break a line.

Claude Code's Read tool returns PNG and JPG files as images Claude can see, and codex exec can attach images with -i. The Claude Code docs note that large screenshots are downscaled first, so for fine detail, capture a section with a locator screenshot rather than a full page. The answer should cite files and lines. If it only restates your note, ask it to open the images.

Check

• Each acceptance line has a verdict and a cited file or output line.
• Failures come back to you before the change is called done.

Step 9: Check the live site after deploy

A local pass is not a live pass. After a deploy, rerun the same script with BASE set to the live URL and read-only steps only. Do not submit forms or write data on the live site without the owner's say-so. On this site, the live check confirmed the 3D wordmark mounted, the news and library pages rendered their items, and no page or console errors appeared, while a real newsletter sign-up was deliberately left unverified.

Check

• The live run uses the same counts as the local run.
• The note records what live checks were skipped and why.

Gotchas we hit

• The dev server had died. Once, nothing was listening on the port, so every capture would have been an error page. That is why Step 1 checks for a 200 first.
• Prefetch cancellations looked like failures. Filter requestfailed noise before you report it.
• Old controls lingered. After moving theme and pause buttons into the header, a count of the old floating controls proved they were gone.
• Waiting strategies. Our scripts use networkidle plus a short wait before screenshots. Playwright's docs discourage networkidle and fixed timeouts for tests and recommend web assertions, so use those when a check must pass or fail on its own.

Take it further

• Wrap the script in a skill so one command runs it: see Write a Claude Code skill for a job you repeat.
• Add the checks to your project instructions: see Set up a project for Claude Code.
• Move stable counts into Playwright Test with toHaveCount for a suite that fails on its own.
• Compare screenshots with the previous run to spot unintended changes.

Quick checklist

• The server responds before capturing.
• Light and dark at 390 and 1440 for every changed page.
• Counts with expected values for the elements that matter.
• Page errors, console errors and failed requests recorded and explained.
• Every changed control clicked once in the script.
• Animation stages sampled; reduced motion checked.
• A result note with raw output and a "Remaining" list.
• An agent review against acceptance lines, citing files.
• The live site checked read-only after deploy.