Zephyr Scale alternative

A Zephyr Scale alternative with a release verdict

Bring your Zephyr Scale test cases across with their folders, steps and keys, and get one plain answer about each release.

By the TestPlum teamChecked

Short answer

TestPlum is a Zephyr Scale alternative for Jira teams that want tests outside Jira and a Ready or Not ready verdict for every release. It imports Zephyr Scale XML or CSV exports with folders, steps, test data and custom fields, and each test keeps its key, such as SHOP-T12. Bugs still go to Jira, and you pay per tester rather than per Jira user.

A note on the name

SmartBear has brought Zephyr Squad and Zephyr Scale together into one product family called Zephyr. Its Atlassian Marketplace listing is now named Zephyr, Test Management and Automation for Jira. Many teams still call it Zephyr Scale, and so does the export TestPlum reads, so this page uses that name.

Who should switch, and who should stay

  • Switch if your Jira site is much larger than your test team, and you want to pay only for testers.
  • Switch if you want CI results, manual runs and open bugs to add up to one release verdict with plain reasons.
  • Stay if you want tests to live inside Jira, next to the issues they cover.
  • Stay, or plan for it, if you need your Zephyr execution history. TestPlum imports Zephyr Scale test cases, but not their past results or attachments yet.
  • Stay if you use Zephyr Squad. TestPlum's importer reads Zephyr Scale exports only.

TestPlum and Zephyr side by side

For Zephyr we list only what SmartBear, its Marketplace listing and Atlassian's own documentation say. Where they don't say, check their docs.

TestPlum compared with Zephyr
CapabilityTestPlumZephyr (formerly Zephyr Scale)
Where tests liveIn TestPlum, linked to JiraInside Jira
Pricing modelPer tester; viewers and Jira users not countedMarketplace app; on Jira Cloud the license tier matches the Jira site's user tier
Free optionFree plan: up to 3 testers, 1 project, no time limitFree trial on the Marketplace
EditionsFree, Team, Business, EnterpriseStandard and Advanced on the Marketplace
Import from Zephyr ScaleXML or CSV test cases, keys keptNot applicable
CI resultsJUnit XML, CTRF and Allure by one curl stepSee their docs
BDDGherkin not run or parsedBDD/Gherkin listed on the Marketplace
Release verdictReady or Not ready from agreed checks, with plain reasonsSee their docs
Pass weighed by evidenceTrusted pass rate: retries, flaky passes and passes with no assertions count partlySee their docs
Failure grouping and sortingGrouped by cause; sorted as bug, broken test, environment or flakySee their docs
Flaky-test quarantineYes, with reason, owner and expirySee their docs
Jira bugs and storiesBugs sent to Jira with two-way status sync; stories imported by JQL as requirementsNative to Jira

What the Zephyr Scale importer brings over

In Zephyr Scale, open Tests, select the test cases or a folder, choose Export, then XML or CSV. Upload the file in TestPlum under Tests, Import, Zephyr Scale. You see what will happen before anything is saved. By default the tests go into a folder called Zephyr Scale, with your folders inside it, or you can name it.

  • Folders become folders.
  • Steps keep their test data and expected results.
  • Objective, precondition and priority come across, and rich text is turned into plain text.
  • Labels become tags, and covered issues become references.
  • Custom fields fill matching fields, or an administrator can have them created, typed from their values and shown in the check first.
  • Each test keeps its key, such as SHOP-T12, so a newer export updates the same tests.
  • Not yet: past executions, attachments and Zephyr Squad exports. See all sources on the migration page.

Send CI results with one curl step

Automated results from your pipeline land in the same runs as manual testing, and count toward the release you are working on.

TestPlum takes JUnit XML, CTRF JSON and Allure results at one address and works out the format from the content. CTRF covers reporters for Cypress, Jest, Vitest, pytest and more. Retries in CTRF and Allure are kept as separate attempts, so a test that passed only on its second try shows up as flaky instead of green.

Open Runs, Connect CI, pick your test tool (Playwright, Jest, Vitest, pytest or any JUnit XML) and your CI (GitHub Actions, GitLab CI or another), and create a CI key. The key is shown once and can only send results and read the answer for that project. The page waits for your first run and shows it as soon as it lands. Nothing to install: the step is plain curl.

One CI step: send a JUnit XML report (a CTRF or Allure report goes to the same address)
curl -f -sS -X POST \
  -H "Authorization: Bearer $TESTPLUM_CI_KEY" \
  -H "Content-Type: application/xml" \
  --data-binary @junit.xml -o /dev/null \
  "https://app.testplum.com/api/v1/projects/<project-id>/automation/report?release=%40next&run_name=CI%20%23<build>"

One answer: can we ship?

Each project opens on a green or red light for the release due next. Read how it works on the release verdict page.

  • Ready or Not ready, from checks you agree in advance: runs in scope, completion, pass rate, trusted pass rate, critical tests failing or blocked, open critical bugs and requirements verified.
  • Each failing check reads as one plain sentence, such as “Only 50% of tests pass. Needs 95%.”, with one button to the page that fixes it.
  • A pass counts by what it proves. A pass on retry, from a flaky test or with no assertions counts only partly, so a green suite of weak tests cannot make a release look ready.
  • Your build can print the answer in its log, or stop when the release is Not ready.

Failures sorted by cause

Instead of a list of red tests, you get a short list of causes. More on failure triage.

  • Failures that share a cause are grouped, so ten tests failing for one reason are one problem.
  • Each group is sorted as a product bug, a broken test, an environment problem or a flaky test, by visible rules, with the reasons and a confidence.
  • File one bug per cause and send it to Jira or GitHub. Its status stays in sync both ways.
  • Flaky tests can be quarantined with a reason, an owner and an expiry. They stay out of the verdict counts until a fix is confirmed, except that a quarantined critical test keeps the verdict at Not ready.

How the pricing models differ

Zephyr is sold through the Atlassian Marketplace in Standard and Advanced editions, with monthly or annual billing and a free trial. SmartBear's own pricing page compares its Zephyr editions without listing prices. Atlassian's documentation says a cloud app's license tier matches the number of users licensed for Jira on the site.

TestPlum is priced per tester. Free covers up to 3 testers, unlimited viewers and one project with no time limit. Team is $19 and Business $35 per tester per month, billed yearly. Viewers, such as developers, product managers and managers, are free on every plan, and people in connected tools such as Jira or GitHub are never counted. See pricing.

For Zephyr's price at your Jira tier, use the pricing tab on its Marketplace listing. Paid TestPlum plans are set up with the TestPlum team for now.

How to move from Zephyr Scale

  1. Create a free workspace. Sign up and create a project. No Jira credentials are needed to import.
  2. Export test cases. In Zephyr Scale, select a folder or test cases, choose Export, then XML. XML keeps steps, test data and labels.
  3. Check and import. Upload it under Tests, Import, Zephyr Scale, let TestPlum create any missing custom fields, then import.
  4. Connect Jira. Connect your Jira project so bugs go there and stay in sync, and import stories by JQL as requirements.
  5. Connect CI. Add the curl step from Runs, Connect CI so automated results count toward the release.
  6. Import again while you switch. Tests are matched by key, so a newer export updates them instead of adding copies.

Frequently asked questions

Can I import Zephyr Scale test cases?

Yes. Upload a Zephyr Scale XML or CSV export under Tests, Import, Zephyr Scale. Folders, steps with test data, objective, precondition, priority, labels, covered issues and custom fields come across.

Do my Zephyr test keys change?

No. Each test keeps its key, such as SHOP-T12, so importing a newer export updates the same tests instead of adding copies.

Are Zephyr executions and attachments imported?

Not yet. The importer brings test cases. Past results can be imported from TestRail and Xray today, but not from Zephyr.

Does it work with Zephyr Squad?

No. The importer reads Zephyr Scale exports. For other tools, a CSV with one row per step works.

Does TestPlum work with Jira?

Yes. Bugs are sent to a connected Jira project and their status stays in sync both ways, and Jira stories can be imported by JQL as requirements.

Is there a free Zephyr Scale alternative?

TestPlum has a free plan for up to 3 testers with unlimited viewers and one project, with no time limit. The Zephyr Scale import is included.

Sources

Zephyr, Zephyr Scale and Jira are trademarks of their owners. TestPlum is not affiliated with or endorsed by them; the names are used only to compare and to describe what TestPlum can import.

Find out whether your next release is ready

Create a workspace, import your tests or write a few, and record a run. Everything on this site is in the product today.