Xray alternative
An Xray alternative for Jira teams
Keep Jira for bugs and stories. Move your tests to a place built around one question: is the release ready?
TestPlum is an Xray alternative for Jira teams that want their tests outside the issue queue and a Ready or Not ready verdict for each release. It imports Xray tests as JSON or CSV, including Jira's CSV export, and each test keeps its Jira key. Bugs still go to Jira and stay in sync both ways, and you pay per tester rather than for every user on the Jira site.
Who should switch, and who should stay
Xray describes itself as test management built inside Jira. That is a real strength if your team wants tests to be Jira issues. TestPlum takes a different approach: tests live in TestPlum, and Jira keeps the bugs and stories.
- Switch if most people on your Jira site never touch tests, and you would rather pay only for the people who do.
- Switch if you want automated and manual results to add up to one release answer, with flaky passes and retries weighed honestly.
- Stay if your team wants tests, test plans and test sets to be Jira issues, searchable with JQL next to everything else.
- Stay if you work mainly in Cucumber. TestPlum keeps a Cucumber scenario as text in the test's description; it does not turn Gherkin into steps.
- Stay if you need step-by-step results history. Each test's result in an execution comes across, but the status of each step stays in Xray.
TestPlum and Xray side by side
For Xray we list only what Xray, its Atlassian Marketplace listing and Atlassian's own documentation say. Where they don't say, check their docs.
| Capability | TestPlum | Xray |
|---|---|---|
| Where tests live | In TestPlum, linked to Jira | Inside Jira |
| Pricing model | Per tester; viewers and Jira users not counted | Marketplace app; on Jira Cloud the license tier matches the Jira site's user tier |
| Free option | Free plan: up to 3 testers, 1 project, no time limit | Free trial on the Marketplace |
| Editions | Free, Team, Business, Enterprise | Standard and Advanced |
| Import from Xray | JSON, GraphQL results, Test Case Importer CSV or Jira CSV; Jira keys kept | Not applicable |
| CI results | JUnit XML, CTRF and Allure by one curl step | Connects to JUnit and CI tools such as Jenkins, GitHub, GitLab and Azure DevOps |
| BDD | Cucumber scenarios kept as text | Cucumber tests in Jira |
| Release verdict | Ready or Not ready from agreed checks, with plain reasons | See their docs |
| Pass weighed by evidence | Trusted pass rate: retries, flaky passes and passes with no assertions count partly | See their docs |
| Failure grouping and sorting | Grouped by cause; sorted as bug, broken test, environment or flaky | See their docs |
| Flaky-test quarantine | Yes, with reason, owner and expiry | See their docs |
| Jira bugs and stories | Bugs sent to Jira with two-way status sync; stories imported by JQL as requirements | Native to Jira |
What the Xray importer brings over
Upload an export under Tests, Import, Xray. TestPlum recognises Xray's bulk JSON, the JSON its GraphQL API returns, a CSV laid out for the Test Case Importer, or Jira's CSV export of the test issues. You see what will happen before anything is saved. The Xray migration guide has the full field map.
- Test repository folders become folders, in the same nesting.
- Steps keep their action, test data and expected result together.
- Labels become tags, Jira priorities map to TestPlum priorities, and requirement links become references.
- Each test keeps its Jira key, such as SHOP-12, so a newer export updates the same tests.
- Attachments and images on the test issue can be copied with a Jira API token you give once and that is never stored.
- Executions come across as past results: each becomes a signed-off run dated as it happened. Test plans and test sets stay in Xray.
Send CI results with one curl step
If your pipeline already produces a JUnit report for Xray, the same file works for TestPlum.
TestPlum takes JUnit XML, CTRF JSON and Allure results at one address and works out the format from the content. CTRF covers reporters for Cypress, Jest, Vitest, pytest and more. Retries in CTRF and Allure are kept as separate attempts, so a test that passed only on its second try shows up as flaky instead of green.
Open Runs, Connect CI, pick your test tool (Playwright, Jest, Vitest, pytest or any JUnit XML) and your CI (GitHub Actions, GitLab CI or another), and create a CI key. The key is shown once and can only send results and read the answer for that project. The page waits for your first run and shows it as soon as it lands. Nothing to install: the step is plain curl.
curl -f -sS -X POST \
-H "Authorization: Bearer $TESTPLUM_CI_KEY" \
-H "Content-Type: application/xml" \
--data-binary @junit.xml -o /dev/null \
"https://app.testplum.com/api/v1/projects/<project-id>/automation/report?release=%40next&run_name=CI%20%23<build>"A release verdict your whole Jira site can see
Viewers are free, so developers and product managers can open the verdict without a paid seat. Read how it works on the release verdict page.
- Ready or Not ready, from checks you agree in advance: runs in scope, completion, pass rate, trusted pass rate, critical tests failing or blocked, open critical bugs and requirements verified.
- Each failing check reads as one plain sentence, such as “Only 50% of tests pass. Needs 95%.”, with one button to the page that fixes it.
- A pass counts by what it proves. A pass on retry, from a flaky test or with no assertions counts only partly, so a green suite of weak tests cannot make a release look ready.
- Your build can print the answer in its log, or stop when the release is Not ready.
Failures sorted by cause, bugs filed in Jira
When automated tests fail, TestPlum sorts out why before anyone opens a ticket. More on failure triage, and on the Jira connection.
- Failures that share a cause are grouped, so ten tests failing for one reason are one problem.
- Each group is sorted as a product bug, a broken test, an environment problem or a flaky test, by visible rules, with the reasons and a confidence.
- File one bug per cause and send it to Jira or GitHub. Its status stays in sync both ways.
- Flaky tests can be quarantined with a reason, an owner and an expiry. They stay out of the verdict counts until a fix is confirmed, except that a quarantined critical test keeps the verdict at Not ready.
How the pricing models differ
Xray is sold through the Atlassian Marketplace in Standard and Advanced editions, with monthly or annual billing and a free trial. Atlassian's documentation says a cloud app's license tier matches the number of users licensed for Jira on the site. In practice that means the app is priced for everyone on the Jira site, not only the people who test.
TestPlum is priced per tester. Free covers up to 3 testers, unlimited viewers and one project with no time limit. Team is $19 and Business $35 per tester per month, billed yearly. Viewers, such as developers, product managers and managers, are free on every plan, and people in connected tools such as Jira or GitHub are never counted. See pricing.
For example, a Jira site of 200 people with 10 testers pays TestPlum for 10 testers: $190 a month on Team, billed yearly. For Xray's price at your Jira tier, use the pricing tab on its Marketplace listing. Paid TestPlum plans are set up with the TestPlum team for now.
How to move from Xray
- Create a free workspace. Sign up and create a project. No Jira or Xray credentials are needed to import.
- Export your tests. Use Xray's JSON, its GraphQL API, the Test Case Importer CSV or Jira's CSV export of the test issues.
- Check and import. Upload it under Tests, Import, Xray. Fix anything the check lists, then import.
- Bring executions. Import Xray's JSON results as past results, matched by Jira key.
- Connect Jira. Connect your Jira project so bugs go there and stay in sync, and import stories by JQL as requirements.
- Connect CI. Add the curl step from Runs, Connect CI, using the JUnit report you already have.
Frequently asked questions
Can I import Xray tests?
Yes. Upload Xray's bulk JSON, its GraphQL results, a Test Case Importer CSV or Jira's CSV export under Tests, Import, Xray. Folders, steps with test data and labels come across, and each test keeps its Jira key.
Does TestPlum work with Jira?
Yes, but it is not a Jira app. Bugs are sent to a connected Jira project and their status stays in sync both ways, and Jira stories can be imported by JQL as requirements.
Do I pay for every Jira user?
No. TestPlum is priced per tester only. Viewers and people in connected tools such as Jira are never counted.
Is there a free Xray alternative?
TestPlum has a free plan for up to 3 testers with unlimited viewers and one project, with no time limit. The Xray import is included.
Are Xray executions imported?
Yes. Import Xray's JSON results after the tests. Each execution becomes a signed-off run dated as it happened, matched by Jira key. Test plans, test sets and step-level statuses stay in Xray.
What happens to Cucumber tests?
The scenario is kept as text in the test's description, and the import check lists which tests this applies to. TestPlum does not turn Gherkin into steps.
Sources
- Xray home page, checked November 19, 2026
- Xray on the Atlassian Marketplace: pricing, checked November 19, 2026
- Atlassian: apps and user license, checked November 19, 2026
Xray and Jira are trademarks of their owners. TestPlum is not affiliated with or endorsed by them; the names are used only to compare and to describe what TestPlum can import.