Visual regression testing for WordPress: how to catch a broken layout before your client does
Visual regression testing means taking a screenshot of a page before a change and again after it, then comparing the two for differences nobody intended. On WordPress it is the fastest way to find out that an update or an edit broke a page you were not looking at. The hard part is not taking the screenshots. It is stopping the false alarms.
How it works
- Take a full page screenshot of each important page, on desktop and on a phone. This is the baseline.
- Make the change: an update, a content edit, a theme tweak.
- Take the same screenshots again, the same way.
- Compare them pixel by pixel. Anything above a small tolerance is flagged.
- A person, or a second check, decides whether each difference was intended.
A sensible tolerance is around 2% of the page. Lower than that and font rendering alone will trip it.
The false alarms, and how to fix each one
These come from running this on real client sites. Each one produced a "regression" that was not one.
| What you see | What is really happening | The fix |
|---|---|---|
| An image is a blank box in one shot and fine in the other | WordPress marks images to decode in the background. In a full page shot the browser can paint a fully loaded image as empty, a different one each time | Force images to decode before the shot, and wait for them, not only for the network to go quiet |
| The lower half of the page is missing images | Lazy loading. Images below the fold have not been asked for yet | Scroll the whole page first, or switch lazy images to load straight away |
| A carousel shows a different slide | It moved between shots | Freeze animations and transitions before capturing |
| A banner covers part of the page in one shot | A cookie notice, a staging notice or a chat bubble appeared | Hide those elements on both shots. Do not paint a coloured block over them: that changes the page height and shifts everything below |
| Numbers differ | Stock counts, dates, "3 people viewing this" | Hide or ignore those regions, or accept them as explained |
| Everything is a few pixels off | A web font loaded late | Wait for fonts before capturing |
The first row cost us a real rollback. A change that altered nothing was reversed because two image tiles painted blank in the "after" shot. On the same page, with no fix, a blank tile turned up in four runs out of six. With images forced to decode first, none in six.
The rule that follows from all of this: take the baseline and the comparison shot through exactly the same code, with the same waits and the same hidden elements. Most false alarms are two shots taken slightly differently.
Staging against live
You can compare a page with itself before and after a change on staging. You should also compare the live page before and after publishing, because live has caches, a CDN and real data that staging does not. A change that is clean on staging can still fail on live.
Where screenshots are not enough
A screenshot cannot tell you that a form no longer sends its email, that add to cart fails, or that a payment method vanished at checkout. For those you need a test that does the thing: submit the form and confirm the email arrives, add a product and reach checkout. Use screenshots for layout and a real test for anything a visitor has to complete.
Ways to run it
- A plugin on the site that compares pages on a schedule or after updates. Simple to start, limited control over the false alarms above.
- A hosted service that crawls your pages and compares them. More control, a monthly cost per site or per page.
- Your own script with an open source tool such as BackstopJS or Playwright. Free and fully controllable, and you maintain it.
- Your host. Some managed WordPress hosts include visual checks with their automatic plugin updates.
- A managed loop. SquareJuice runs the comparison on every client request, on staging before you approve and on live after publishing, and reverses the change if the live check fails.
Questions
What is visual regression testing in WordPress?
Comparing screenshots of a page from before and after a change to find layout differences nobody intended.
Why does my visual regression test fail when nothing changed?
Usually images that had not finished decoding or loading, a moving carousel, a late web font, or a banner that appeared in one shot. Capture both shots the same way, wait for images and fonts, and freeze animations.
How much difference should count as a failure?
About 2% of the page is a workable starting point. Then look at where the difference is. A small difference in the header matters more than a large one in a rotating carousel.
Can visual regression testing check a WooCommerce checkout?
It can show that the checkout page looks the same. It cannot show that an order goes through. That needs a test that adds a product and reaches checkout.
Scan your sites free
Leave your details and get one of the first invites.