Frequently asked questions

TestPlum is a test management platform for QA teams. It brings requirements, test cases, test runs and defects together, records manual and automated results in one history, and turns those results into release readiness, risk and flaky-test reports.

TestPlum is for software teams that test before they release: QA leads who plan and report on testing, testers who execute cases and file defects, developers who read failures and feed CI results in, and admins who manage members, roles and fields.

Yes. Testers execute manual cases step by step in test runs, and CI pipelines upload JUnit XML results to the same runs, so manual and automated results appear in the same reports.

TestPlum checks recorded results against thresholds you set: runs in scope, execution completion, pass rate, critical cases failing or blocked, open critical issues and passing requirements. The release is ready only when every check passes, and each check shows the numbers behind it.

Yes. Organizations have owner, admin and member roles, and each project has configurable roles built from 14 permissions. A tester role can be limited to only the run cases assigned to them.

Yes. Every query is scoped to your organization and project, resources from other organizations return 'not found', and automated tests check isolation across read and write endpoints and API keys.

Yes. You can send an issue from TestPlum to GitHub or Jira and keep the link, and signed webhooks let you connect other tools to TestPlum events.

AI is optional. When an administrator enables it, it can draft test cases from a requirement and summarise run failures, with cost controls. Every core feature, including coverage-gap rules, works without it.

Yes. TestPlum has light and dark themes, works down to 375-pixel-wide screens and is built to WCAG 2.1 AA intent: keyboard operable, labelled controls, managed focus in dialogs and automated accessibility checks in both themes.

Yes. TestPlum imports CSV files of up to 1,000 rows per upload. The import is validated first and applied all-or-nothing, and rows with an external ID update the matching case instead of creating a duplicate.

Yes. Every save creates a new version, and you can diff any two versions. Test runs snapshot the case version they were created with, so past results always refer to the steps that were actually executed.

Yes. Admins define custom fields and group them into field sets, so each type of case shows the fields that apply to it. Custom fields are included in CSV import and export.

Nothing changes in the run. Runs snapshot the case versions they were created with. TestPlum flags that the case changed, and you can refresh the snapshot explicitly when you want the new steps.

Yes. Project roles are configurable, and a tester role without the permission to view all cases only sees and executes the run cases assigned to them.

Completion is the share of run cases whose current status is final. Pass rate is passed cases divided by cases with a final status. The burndown counts cases without a first final result on each day.

A requirements traceability matrix is a table that maps each requirement to the tests that verify it and to their results and defects. It shows which requirements are covered, which are failing and which have no tests.

Requirement coverage is the number of live requirements with at least one live linked test case, divided by the number of live requirements.

A release is ready only when every check passes: at least one run is in scope, completion and pass rate meet your thresholds, no critical-priority cases are failing or blocked, open critical issues are within your limit and enough requirements are passing with none failing.

For cases with at least three decisive results that include both passes and failures, the flaky score is the number of pass-to-fail or fail-to-pass flips divided by the number of decisive results minus one.

The risk score adds points for high priority (up to 40), a failed or blocked last result (35), edits since last execution (25), never executed (20), a failing linked requirement (15) and a flaky score of at least 0.3 (10). Higher scores are tested first.

Yes. An issue can be sent to GitHub or Jira from its page, and the link to the created issue is saved. Access tokens are encrypted with AES-256-GCM and are never returned by the API.

Yes. Teams that do not use an external tracker can manage defects in TestPlum with statuses, priorities, assignees, due dates, comments and history.

Any CI system that can run curl. TestPlum ingests JUnit XML, the report format produced by most test runners, including those for Java, JavaScript, Python, Go and .NET.

Yes. The API is versioned under /api/v1 and described by an OpenAPI 3.1 document served by the application. An automated test checks that every documented endpoint exists.

Each delivery carries an X-TestPlum-Signature header with a timestamp and an HMAC-SHA256 of the timestamp and body, computed with your webhook secret. Recompute it and compare before trusting the payload.