Automation that counts towards the verdict
Automated results belong in the same place as manual ones, with the same honesty: every attempt recorded, flaky tests shown for what they are, and passes weighed by how much they prove.
Runner
Start TestPlum's runner on a laptop, CI box or cloud machine. It checks out a repository, runs its Playwright tests on demand, on a schedule or for each pull request, and reports results with screenshots, videos and traces. Each job gets its own short-lived token.
Playwright reporter
Records every attempt of every test, including retries, with its evidence, so flaky tests show up for what they are. Tests TestPlum has not seen yet are created with their steps.
JUnit from any CI
Upload JUnit XML from any pipeline. Uploads are idempotent with a key, and results are matched to tests by automation ID.
REST API and webhooks
A versioned REST API described in OpenAPI 3.1, with organization API keys, and webhooks signed with HMAC-SHA256 that are retried and logged.
MCP server for coding assistants
Claude Code, Cursor, VS Code and other MCP clients can read requirements, write and link tests, start runs, sort failures and ask whether a release is ready, within the API key's permissions.
Git export
Turn manual tests into Playwright test files with one step per test step: download a patch for git apply, or have a runner push a new branch for a pull request. Existing files are never overwritten.
From failure to fix
When an automated test breaks
Sorted
The failure is grouped with others of the same cause and sorted as a broken test, with the reasons.
Reproduced
A runner reproduces it in a fresh checkout. A test that passes again gets no fix.
Proven
AI proposes a change; guardrails refuse anything that hides a failure, and the fix must pass every time.
Approved
A person reviews the diff and screenshots and applies the patch. TestPlum never writes to your repository.
Do I need to change my Playwright tests?
No. Add the TestPlum reporter to your Playwright configuration, or let a runner run the repository as it is. Tests TestPlum has not seen yet are created with their steps, and later runs find them by automation ID.
Where do the runners run?
On machines you control: a laptop, a CI box or a cloud machine. A runner claims jobs for its organization, checks out the repository, runs the tests and reports back with a token that only that job can use.
Which CI systems work?
Any that can run Playwright or produce JUnit XML. Uploads carry an idempotency key, so a retried pipeline step never records results twice.
Can a coding assistant change my data?
Only within the permissions of the API key you give it. The MCP server talks to the same REST API with an organization API key, and fixes it proposes to tests still need a person's approval.
Connect your first runner
Create a workspace, register a runner under Integrations, then add a test setup under Runs, Automation that points at a repository with Playwright tests.