GitHub Action Planned
First-class GitHub Action that runs your scenarios against the PR preview deployment and posts the report as a comment.
Testing Suite · Coming soon
Every change to your site, checked before customers feel it.
Your site changes every week. Testing Suite is designed to re-run your customer journeys after every change, in the same real browsers, and block the ones that break or slow down. When checkout breaks on the way out, the release will halt instead of your revenue. When LCP regresses past your budget on staging, promotion to production will block.
Testing Suite is on the roadmap. We'll write to you when there's a usable build. Written for the engineering team? Yes. Forward it.
What it does
Build once. Reuse on every push.
Performance testing tells you whether your site stays fast under load. That's once a release, maybe weekly. The other 95% of the week, you ship code without testing what users actually see. Testing Suite is planned for that 95%.
After every deploy, it will run the scenarios you already built (homepage to checkout, search and add to cart, login and dashboard) against the deployed environment. Same real browser. Same isolation. Same Web Vitals capture. Smaller scale, faster runtime than a full performance test.
When a step in the scenario breaks, the pipeline will be able to halt so you can roll back. When LCP regresses past your budget on staging, promotion to production will block. When a console error appears on the checkout page that wasn't there yesterday, you'll catch it before the next deploy reaches customers. The aim is to catch what unit tests miss.
And because the scenarios are the same ones you performance test with, there will be one place to maintain them. No duplicated test code between your QA and performance pipelines.
What you get back
A PR check that links to the report.
A green check will mean the post-deploy scenarios passed and no Web Vital budget was busted. A red check will mean verification failed, so your pipeline can halt, roll back, or block the next promotion. Either way, the check will link straight to the report for the run.
The report will be the same one you get from a full performance test. Open any session. Watch the video. Read the logs. The bug that broke the build is one click away.
Roadmap
What's planned.
Here's the rough shape of what Testing Suite will ship with. We'll write to the waitlist as each piece lands.
GitLab CI Planned
A GitLab CI template that drops into your pipeline. Same scenarios. Same report. Run after every deploy to staging or production.
Threshold rules Planned
Set per-test budgets. LCP under 2.5s. INP under 200ms. No console errors. The check fails when budgets break, and your pipeline can halt or roll back.
Webhook API Planned
Trigger smoke tests from anything. Custom CI runners. Deployment hooks. Internal tools.
Diff reports Planned
Side-by-side comparison of the current run versus the previous green deploy. See what got faster and what got slower.
Slack notifications Planned
Failure alerts in a channel. Includes the broken scenario, the failed step, and a link to the session video.
What teams use it for
Six smoke-test patterns we hear on demo calls.
Different reasons to put the scenarios into your CI pipeline. Same forensic detail in every report.
Pull-request gates
Every PR with a preview deployment will run the critical-path scenarios against it. The PR check will fail the same way a unit-test check fails, except this one is designed to catch Web Vitals regressions in a real browser.
Pre-release smoke
Tag a release candidate and the smoke suite will run against the staging environment. Ship only if it stays green.
Web Vitals budgets
Per-page LCP and INP budgets set at the URL level. The post-deploy check will fail the moment a page busts its budget. Not later. Now.
Pre-merge diffs
Compare the candidate's preview-deployment run against the last green main-branch run. See exactly what got slower and what got faster, before the PR merges.
Regression hunting
When something regresses next week, look back at the smoke runs. The regression is in one of the deploys. The forensic detail tells you which.
Recurring nightly
Run the full smoke suite nightly against production. Catch slow regressions that ship-day smoke tests miss.
Questions
FAQ
What is the Evaluat Testing Suite?
Testing Suite is a planned product that re-runs the real-browser scenarios you already built as post-deploy smoke checks. After every change, the same customer journeys run again in real browsers, at smaller scale and faster runtime than a full performance test, and block the deploys that break or slow them.
Will Testing Suite work with GitHub Actions or GitLab CI?
Both are on the roadmap: a GitHub Action that runs your scenarios against the PR preview deployment and posts the report as a comment, and a GitLab CI template that drops into your pipeline. A webhook API is planned for custom runners and deployment hooks.
What will a red check mean?
Verification failed: a scenario step broke, or a Web Vitals budget (for example LCP under 2.5s) was busted. Your pipeline can halt, roll back, or block the next promotion, and the check will link straight to the report for the run, with session video and logs.
What can I use today?
Performance Testing is available through assisted private access. The scenarios you build for it are the same ones Testing Suite is designed to re-run, so nothing is thrown away. Testing Suite is planned. Join the design-partner waitlist.