# snapdiff

Visual regression testing for web pages, with no SaaS. snapdiff screenshots your pages at several viewports and colour schemes using headless Chrome (driven over a hand-written DevTools Protocol client), compares them to approved baselines with an anti-aliasing-aware perceptual diff (a numpy reimplementation of the pixelmatch algorithm), and writes a self-contained HTML report you can review and attach to CI.

![report](docs/images/report.png)

## Features

- YAML config: base URL or a static directory (served automatically), pages, viewports with `device_scale_factor`/mobile emulation, light/dark scheme, full-page / viewport / element screenshots, per-page thresholds and retries.
- Wait strategies (selector, network-idle-ish, fixed delay) and actions (`click`, `hover`, `type`, `scroll`, `wait`) before capture.
- Determinism helpers: animations, transitions and the caret are disabled; `hide` selectors are made invisible without moving layout; `mask` selectors are painted over with solid boxes (dates, ads, random content).
- Own CDP client over WebSockets; parallel capture in N tabs; `google-chrome --screenshot` fallback for simple pages.
- Diff engine: YIQ delta with threshold, anti-aliasing detection, red/yellow diff image, mismatch percentage, bounding boxes of changed regions, padding for size changes.
- Workflow: `capture`, `compare` (exit 1 on regressions), `approve`, `report`, `run`.
- CI: JUnit XML, Markdown summary, stable `page@viewport@scheme.png` file names.
- HTML report: side by side, overlay slider, onion skin, diff-only view with region boxes, filters for changed / new / removed / passed. Images are embedded, so it is one file.

## Install

Requires Python 3.12+ and Google Chrome (`/usr/bin/google-chrome`, or set `chrome_path` / `SNAPDIFF_CHROME`).

```bash
uv venv .venv && uv pip install -p .venv/bin/python -e '.[dev]'
```

## Quickstart

Run the bundled example: approve `examples/site` (v1) as the baseline, then check `examples/site-v2`, which contains deliberate visual changes.

```bash
cd examples
../.venv/bin/snapdiff --port 8172 capture            # screenshots of site v1
../.venv/bin/snapdiff approve --all                  # v1 becomes the baseline
../.venv/bin/snapdiff --static-dir site-v2 --port 8172 run \
    --junit .snapdiff/junit.xml --markdown .snapdiff/summary.md
open .snapdiff/report.html                           # or xdg-open
```

Real output of the last command (exit code 1):

```
Capturing 7 screenshot(s) from http://127.0.0.1:8172
Captured 7/7 into examples/.snapdiff/current
  passed   about@desktop@light                      0.000%
  changed  home@desktop@dark                        0.915%
  changed  home@desktop@light                       0.915%
  changed  home@mobile@dark                         2.348%
  changed  home@mobile@light                        2.348%
  changed  pricing-plan@desktop@light               9.393%
  changed  pricing@desktop@light                    5.296%
6 changed, 0 new, 0 removed, 1 passed
Report written to examples/.snapdiff/report.html
```

What v2 changed, and what snapdiff made of it:

| Change in v2 | Result |
|---|---|
| Button colour blue to green, headline text change | `home` flagged (2 regions) |
| Promo banner pushes pricing content down (layout shift) | `pricing` flagged, 21 regions |
| Avatar circle edge shifted by 0.4 px (anti-aliasing only) | `about` **passes**: 196 anti-aliased pixels ignored |
| `#today` timestamp and `#ad` box (dynamic content) | masked / hidden, never flagged |

The pricing diff image (red = changed pixels, yellow = anti-aliasing, faded = unchanged):

![pricing diff](docs/images/diff-pricing.png)

The Markdown summary written by `--markdown`:

```
**6 changed**, 0 new, 0 removed, 1 passed

| Status | Screenshot | Mismatch | Regions |
|---|---|---|---|
| changed | `home@desktop@dark` | 0.915% | 2 |
| changed | `pricing@desktop@light` | 5.296% | 21 |
| passed | `about@desktop@light` | 0.000% | - |
```

## Usage

```
snapdiff [-c snapdiff.yaml] [--static-dir DIR] [--base-url URL] [--port N] COMMAND
```

| Command | What it does |
|---|---|
| `capture [-o PAGE]... [-j N]` | Screenshots into `.snapdiff/current` (clears it first unless `--only` is used). |
| `compare [-o PAGE]... [--strict] [--junit F] [--markdown F]` | Diffs current vs baseline, writes `.snapdiff/diff/` and `results.json`. Exit 1 if any screenshot is `changed`; `--strict` also fails on `new`/`removed`. |
| `approve [NAME... \| --all]` | Copies current to baseline. `NAME` is a page name or full id (`home@mobile@dark`). Removed screenshots delete their baseline. |
| `report [-o FILE]` | Builds `.snapdiff/report.html`. |
| `run [...]` | `capture` + `compare` + `report`. Exit 1 on a regression or a failed capture. |

Statuses: `passed`, `changed` (mismatch above the page's `max_diff_percent`, or the image size changed), `new` (no baseline), `removed` (no current). Exit code 2 means a config error.

`--static-dir` and `--base-url` override the config's site source, which is how the same config checks a different build. All paths in the config are relative to the config file.

## Configuration

`examples/snapdiff.yaml`:

```yaml
static_dir: site          # or base_url: https://staging.example.com
output_dir: .snapdiff
concurrency: 3            # parallel tabs
# serve_port: 0           # port of the built-in server (0 = free port)
# chrome_path: /usr/bin/google-chrome
# fallback: true          # use `chrome --screenshot` if CDP fails (simple pages only)

viewports:                # default: desktop 1280x800, tablet, mobile
  desktop: {width: 1280, height: 800}
  mobile: {width: 390, height: 844, device_scale_factor: 2, mobile: true}

defaults:                 # any page key except name/path/selector
  threshold: 0.1
  wait: {delay_ms: 100}

pages:
  - name: home
    path: /index.html
    schemes: [light, dark]
    viewports: [desktop, mobile]
    wait: {selector: "#cta", network_idle_ms: 100}
  - name: pricing-plan
    path: /pricing.html
    mode: element              # full_page (default) | viewport | element
    selector: ".plans"
    actions:
      - hover: ".plan .btn"
  - name: about
    path: /about.html
    mask: ["#today"]           # painted over with a solid box
    hide: ["#ad"]              # visibility:hidden, layout preserved
```

Page keys: `name`, `path`, `mode`, `selector`, `schemes`, `viewports`, `wait`, `actions`, `hide`, `mask`, `threshold`, `max_diff_percent`, `retries`.

- `threshold` (0 to 1, default 0.1): per-pixel perceptual tolerance, as in pixelmatch.
- `max_diff_percent` (default 0): share of changed pixels a page may have and still pass.
- `wait`: `selector` (+ `timeout_ms`), `network_idle_ms` (no new resource requests for that long), `delay_ms`. A bare number or string is shorthand for a delay or a selector.
- `actions`: `click: sel`, `hover: sel`, `type: {selector, text}`, `scroll: sel | y | {selector|y}`, `wait: ms`. Click and hover use real mouse events.
- Names may not contain `@` or `/`; viewport names may not contain `@`.

Invalid configs fail fast with a message naming the offending key.

## Architecture

| Module | Role |
|---|---|
| `snapdiff/config.py` | YAML parsing and validation into dataclasses. |
| `snapdiff/cdp.py` | Chrome launch, minimal CDP client (`Connection`, `Tab`, `Browser`). |
| `snapdiff/capture.py` | Per-page capture pipeline, parallel jobs, retries, CLI fallback. |
| `snapdiff/server.py` | Static file server for `static_dir`. |
| `snapdiff/diff.py` | Pixelmatch reimplementation, connected components, size handling. |
| `snapdiff/workflow.py` | compare / approve over the `.snapdiff/` directories. |
| `snapdiff/ci.py` | JUnit XML and Markdown summary. |
| `snapdiff/report.py` | Self-contained HTML report. |
| `snapdiff/cli.py` | Click commands. |

Layout of the output directory: `baseline/`, `current/`, `diff/` (PNG named `page@viewport@scheme.png`), `results.json`, `report.html`. Commit `baseline/` to git, ignore the rest.

Deeper reading: [docs/diff-algorithm.md](docs/diff-algorithm.md), [docs/cdp-notes.md](docs/cdp-notes.md), [docs/stable-visual-tests.md](docs/stable-visual-tests.md).

## Tests

```bash
.venv/bin/python -m pytest -q
```

53 tests: diff engine, config validation, workflow and reports, plus integration tests that drive real headless Chrome (skipped when Chrome is missing). The Chrome tests use ports 8173 to 8175 and take about two minutes in total on a machine where Chrome's first navigation is slow.

## Limitations

- Baselines are only valid for the machine/Chrome/font setup that produced them; generate them in the same environment (ideally CI) that compares them. See the stable-tests doc.
- The `google-chrome --screenshot` fallback captures the viewport only and cannot do actions, masks, element mode or dark mode.
- Full-page captures are capped at 16000 CSS px tall.
- `network_idle_ms` approximates network idle by watching the Performance resource timeline; it does not see WebSockets or fetches that never complete.
- No retry-until-stable heuristics: flaky pages should be fixed with masks and waits rather than averaged away.