// work / software qa & testing

Test automation and release QA for a SaaS platform

A QA audit, a Playwright and API automation suite running on every pull request, and a release process that cut escaped bugs by two thirds for a B2B SaaS team with no dedicated tester.

Client
B2B subscription software company
Industry
SaaS
Year
2025
Role
QA lead and automation engineer
Duration
6 weeks, then monthly retainer
Laptop screen showing lines of programming code
Photo: @clark_fransa on Unsplash

Context

A B2B SaaS company with eight developers and no QA function was shipping weekly. Every release went out after a quick manual click-through by whoever had time, and every second release came back with a customer-reported regression. The team had tried automation twice; both suites had rotted into a folder of skipped tests within a few months.

The problem

Catch regressions before customers do, without slowing the weekly release down, and build something the developers would keep running after I left. The previous suites failed on the last point: they were flaky, slow and nobody owned them.

Approach

Audit first. A week walking the product with support and engineering, reading the last six months of production incidents and mapping them to features. Twelve user journeys accounted for almost all of the customer-facing bugs: sign-up and billing, the import pipeline, permissions, and the reporting screens. That became the priority list.

Test plan with deliberate gaps. A risk-based plan that said what would be automated end to end, what would be covered at the API level, what stayed manual, and what was consciously not tested. Writing the gaps down was as useful as the coverage; it ended the “why didn’t tests catch this?” conversation.

The suite. Playwright in TypeScript, structured around user journeys rather than pages. 42 end-to-end tests for the critical paths, each creating its own isolated test data through the API so tests never depend on each other or on a shared database state. API tests in Postman and code cover the contract and the negative paths the UI cannot reach. A k6 load test gives the reporting endpoints a baseline the team can compare against after each change.

Flake as a metric. Every test runs three times in isolation before it is allowed into the main suite; anything that is not deterministic is fixed or rejected. Flake rate is reported monthly, alongside pass rate and duration. It has stayed under 1%.

CI. The suite runs on every pull request in about eleven minutes, with Allure reports and a screenshot and trace attached to every failure. Nightly runs cover the full matrix on BrowserStack. A failing test blocks the merge, which was the cultural change that mattered most.

Release process. A one-page checklist, an exploratory pass on each release by me, and a short note on what was covered. Bugs go to Jira with steps, environment, severity and evidence attached, so the developer can start on the fix rather than on reproduction.

Outcome

Customer-reported regressions fell by about two thirds over the first quarter, measured against the previous six months of support tickets. The release cadence stayed weekly. Six months on, the developers are adding tests to the suite themselves as part of feature work, which is the outcome the two earlier attempts never reached. The engagement continues as a monthly retainer for release passes and suite maintenance.

Need something like this?

This is the kind of work I do under software qa & testing. Tell me about your version of the problem.