JUnit XML integration

A JUnit XML test results dashboard: upload JUnit XML from any CI

Almost every test tool can write JUnit XML. Upload that one file to TestPlum and it becomes a run with history, failure groups and a release verdict, whatever language your tests are in.

By the TestPlum teamChecked

Short answer

Have your test tool write a JUnit XML file, then POST it to TestPlum with curl and a project CI key from Runs → Connect CI. Each test case is matched by its class name and name, so results build a history, and the project answers Ready or Not ready with plain reasons.

Step 1: write a JUnit XML file

These are the commands the Connect CI page shows for each tool. Playwright and Vitest have a JUnit reporter built in, pytest has a flag for it, and Jest uses the jest-junit package. Any other tool that writes JUnit XML works the same way.

Pick the lines for your tool
# Playwright: add the JUnit reporter in playwright.config (keep your other reporters)
reporter: [["list"], ["junit", { outputFile: "results.xml" }]],

# Jest: install jest-junit, then run Jest with it
npm install --save-dev jest-junit
npx jest --ci --reporters=default --reporters=jest-junit

# Vitest: run it with its JUnit reporter
npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xml

# pytest: write a JUnit report
pytest --junitxml=junit.xml

Step 2: upload JUnit XML with curl

In TestPlum, open Runs → Connect CI and press Create CI key. Save it as a secret named TESTPLUM_CI_KEY where your tests run. Add these two commands after your tests, with the path to your file in place of junit.xml (Playwright's line above writes results.xml).

The upload goes to an endpoint that also reads CTRF and Allure. It tells the format from the content, so you do not need to name it.

Upload and print the answer
# After your tests, with TESTPLUM_CI_KEY set to the key:
curl -f -sS -X POST \
  -H "Authorization: Bearer $TESTPLUM_CI_KEY" \
  -H "Content-Type: application/xml" \
  --data-binary @junit.xml -o /dev/null \
  "https://app.testplum.com/api/v1/projects/<project-id>/automation/report?release=%40next&run_name=CI%20%23$BUILD_NUMBER"
curl -f -sS -H "Authorization: Bearer $TESTPLUM_CI_KEY" "https://app.testplum.com/api/v1/projects/<project-id>/status?format=text" | tee testplum-answer.txt

How JUnit XML maps to TestPlum

A test case TestPlum has not seen is created in an Automated suite. The next file with the same automation ID adds a result to that same test, so you can open any test and see every run it was in.

How each part of a JUnit XML file is read
In the JUnit fileIn TestPlum
testsuite nameThe section the test sits in
testcase classname and nameThe automation ID, joined with a dot
failure or errorFailed
skippedSkipped
timeThe result's duration
Failure messageKept on the result as its comment

What the dashboard gives you

  • Run history. Each upload becomes a run named after the build, such as CI #412, next to your manual runs.
  • Flaky detection and quarantine. Tests whose results flip between pass and fail are suggested for quarantine. A quarantined test is left out of the verdict counts until it passes enough times in a row, or the quarantine expires.
  • Failure grouping by cause. Failures with the same normalized error become one group, sorted as product bug, broken test, environment or flaky, with the reasons shown.
  • Trust-weighted pass rate. Each pass counts by how much it proves. A pass on retry, a flaky test or a test edited after the run counts for less, so a 100% pass rate can still fail the trusted pass rate check.
  • A release verdict in plain words. The project answers Ready or Not ready, with each reason in one line and a link to where to fix it.
  • Answer history. TestPlum remembers when the answer turned and what changed: up to five changes, newest first, with the reasons that were new or fixed.

Send results to the release you are working on

The upload URL ends with release=@next. That means the open release due soonest, the same one the project home answers for. You never edit the step between releases. With no open release, the run is tied to none.

You can also name a release, as release=2.4.0 for example. An open release with that name is used, a missing one is created, and a shipped or cancelled release takes no new runs.

Security: a CI key locked to one project

The key you paste into CI is a CI key, made under Runs → Connect CI. It belongs to one project and can do three things there: send results, run the release gate and read the status. Any other request with it is refused with 403, and a revoked key gets 401.

A CI key acts as the person who created it, so it can never do more than they can. Only a signed-in person who can run and edit tests can create one. It lasts a year, a project can have at most 20 live keys, and creating or revoking a key is written to the audit log.

Store it as a masked secret in your CI, never in a file you commit. If it leaks, revoke it on the same Connect CI page and make a new one.

Troubleshooting

  • 422: the report has no tests. The file is empty or has no testcase elements. Check the path and that the tests ran.
  • 422: could not tell the report format. The body is not JUnit XML, CTRF or Allure. Add format=junit to the URL if your file is unusual, and check that curl sends the file with --data-binary @file.
  • Nothing arrives when tests fail. Make the step run after failures: if: always() on GitHub Actions, after_script on GitLab CI.
  • The same build uploaded twice. Add an Idempotency-Key header that is unique per build. TestPlum honours it for seven days, so a retried upload is recorded once.
  • Retries are missing. JUnit XML gives one outcome per test case. Flaky tests still show across runs. To record each retry, send CTRF.

Frequently asked questions

Which tools can upload JUnit XML to TestPlum?

Any tool that writes JUnit XML. The Connect CI page has ready commands for Playwright, Jest, Vitest and pytest, and an option for anything else.

Do I have to name the format in the upload?

No. TestPlum detects JUnit XML, CTRF and Allure from the content. Add format=junit only if detection fails.

How does TestPlum know two uploads are the same test?

By the automation ID, made from the test case's class name and name. Keep those stable and the history stays on one test.

Can I upload several JUnit files from one build?

Each upload makes its own run. To add a second file to an existing run, pass that run's ID as run_id in the upload URL.

Does uploading need a full API key?

No. A CI key is enough, and safer. It can only send results, run the gate and read the status of its own project.

What if a report is very large?

Uploads are limited by the server's request size. If a file is refused as too large, split the suite and send each part to the same run by passing its run_id.

Sources

Playwright, Jest, Vitest and pytest are trademarks of their owners. TestPlum is not affiliated with or endorsed by them.

Find out whether your next release is ready

Create a workspace, import your tests or write a few, and record a run. Everything on this site is in the product today.