Test cases as code: keeping manual tests as Markdown in Git

Keep test cases as Markdown files in Git: file layout, IDs and front matter, pull request review, syncing results back, conflicts and limits.

By the TestPlum teamUpdated 8 min read

Short answer

Test cases as code means storing each manual test case as a Markdown file in the product's Git repository, with front matter for fields such as ID and priority and sections for preconditions and steps. Changes go through pull requests, and a sync step keeps the test management tool up to date, where results and history stay.

What “test cases as code” means

Test cases as code means keeping your manual test cases, the written steps and expected results, as plain text files in the same Git repository as the product. They change through the same branches and pull requests as the code they describe, and a sync step keeps a test management tool up to date so results, history and reports still live in one place.

This is about the test descriptions, not about automation. Automated tests are already code. The question here is where the human-readable cases live: in a tool’s database that only testers open, or in the repository where developers work every day.

Why keep test cases in Git

Benefit What it looks like in practice
Review A change to a test is a diff in a pull request, read by the developer who changed the feature
Same change, same place A pull request that changes checkout can update the checkout tests in the same commit
History and blame Git shows who changed a step, when, and in which pull request
Branches Tests for an unreleased feature live on the feature branch until it merges
Tooling Search, bulk edits, linting and AI coding assistants work on text files
No lock-in Plain Markdown is readable without any tool

The trade-off is that a file in Git cannot record results. Who ran the test, on which build, and whether it passed belong in a system built for that. So the workable model is definitions in Git, results in the test management tool, with a sync in between.

When it is a good fit, and when it is not

It suits teams where developers write or review a fair share of the tests, where features ship through pull requests, and where people already think of the repository as the source of truth.

It suits teams less well when testers do not use Git at all and have no wish to, when many cases are edited daily in a spreadsheet-like way, or when tests are mostly owned outside engineering. You can still mix: keep some folders as code and edit the rest in the tool. What you must avoid is two people editing the same test in two places without a rule about which one wins. More on that under conflicts.

A file layout that scales

Use one file per test case, and let folders mean something:

tests/cases/
  checkout/
    payment/
      pay-with-a-saved-card.md
      pay-with-a-new-card.md
    discount-codes.md
  account/
    sign-up.md
    reset-password.md

A few rules keep this tidy:

  • Folders are the structure. The path is the test’s folder in the tool. Moving a file moves the test.
  • File names are slugs of the title. Lowercase, hyphens, no spaces. They are for humans; the ID does the matching.
  • One root folder. Keep all cases under one directory, such as tests/cases/, so the sync never touches anything else.
  • Small files. One test per file means a pull request diff shows exactly which tests changed.

IDs and front matter

Each file starts with front matter, a small block of fields between --- lines, followed by the test itself as Markdown sections. This is the format TestPlum reads and writes:

---
id: SHOP-C12
title: Pay with a saved card
priority: high            # critical, high, medium or low
type: Functional          # a test type set up in TestPlum
tags: [payments, smoke]
refs: [SHOP-7]
automation_id: "checkout.spec.ts > pays"
estimate: 5               # minutes
fields:                   # custom fields by key
  browser: Chrome
---

## Preconditions
A card is saved.

## Steps
1. Open checkout
   Expected: The saved card is preselected
2. Click **Pay**
   Expected: The order is confirmed

## Notes
Anything else, in Markdown.

What each part does:

Part Purpose
id The test’s key. A new file has none; TestPlum assigns one on the first push
title, priority, type, tags The fields you filter and report on
refs Links to requirements or tickets
automation_id Matches this case to an automated test’s results from CI
estimate, fields Expected minutes and any custom fields your project defines
## Preconditions What must be true before you start
## Steps A numbered list; an Expected: line under a step is its expected result
## Notes, ## Postconditions Anything else, and how to clean up

Why the ID matters. Titles change, and files get renamed. A stable ID is how the tool knows that a renamed file is the same test, so its history carries on instead of starting from zero. When a new file has no ID yet, TestPlum finds it again by its path, and the push writes the new ID into the file for you to commit. It skips this in CI, where nobody commits the checkout.

Why a strict format matters. Free-form Markdown is easy to write and hard to sync. A fixed set of fields and section names means the tool can tell you exactly what is wrong. TestPlum reports an unknown field or a malformed step with the file and line number.

Reviewing tests through pull requests

The real payoff is review. A test change shows up as a diff, next to the code change it belongs to:

 2. Click **Pay**
-   Expected: The order is confirmed
+   Expected: The order is confirmed and a receipt email is sent

Make review dependable with three habits:

  1. Ask for tests in the pull request template. “Did you add or update test cases under tests/cases/?” is one checkbox.
  2. Give the folder owners. A CODEOWNERS file makes GitHub request a review from the owners automatically when a pull request touches their files (GitHub: about code owners). Making QA a code owner of tests/cases/ means no test changes without them seeing it.
  3. Validate in CI. A check that fails on an invalid file or a conflict catches problems before merge, not after.

A GitHub Actions workflow can run that check only when test files change, using the paths filter on pull_request (workflow syntax: paths):

on:
  pull_request:
    paths:
      - "tests/cases/**"

One catch: GitHub’s reference notes that when a workflow is skipped by a path filter, its checks stay “Pending”. If you make the check required, pull requests that do not touch tests/cases/ can be blocked from merging, so read that note before you turn it on.

Syncing both ways

There are two directions, and each has a natural moment:

Command When What it does
pull Once to start, then whenever tests changed in the tool Writes every test as a file under tests/cases/ (or a folder you choose)
check On every pull request Fails on an invalid file or a conflict, without writing anything
push After merge to the main branch New files become tests, edited files update them, moved files move them

With the TestPlum CLI that looks like this. The CLI is not yet published as a public package; it is available to workspaces on request, and the same pull, check and push are available through the REST API.

node testplum.mjs tests pull     # writes every test under tests/cases; commit them
node testplum.mjs tests check    # for pull requests: exits 1 on an invalid file or a conflict
node testplum.mjs tests push     # after merging: new files become tests, edited files update them

Pushing only after merge matters. If every feature branch pushed, the tool would show half-finished tests from branches that may never merge.

Deleted files. By default a push leaves alone a test whose file is gone, because a missing file is easy to cause by accident. When you push the whole folder, you can ask for those tests to be listed, and with an extra flag moved to the recycle bin: tests push --complete --delete-missing.

Results stay where they are. Editing a test in Git creates a new version in the tool. Past results keep pointing at the version that was actually run, so history stays honest. See test management in TestPlum for versions and coverage, and test runs for how results are recorded.

Handling conflicts

A conflict happens when the same test changed in both places since the last sync. The safe approach is to remember what both sides agreed at the last sync and compare each side with that:

Repository since last sync Tool since last sync Outcome
Unchanged Unchanged Nothing to do
Changed Unchanged Push updates the test
Unchanged Changed Push leaves it and says to pull
Changed Changed, differently Conflict: nothing is written for that test

TestPlum works this way. It keeps a fingerprint of each test’s Markdown from the last sync, and that fingerprint ignores formatting, so re-wrapping a line or adding spaces inside a tag list is not a change. When there is a real conflict, pull overwrites the file with the tool’s version, and you merge with Git as you would any other conflict: look at git diff, combine the two edits, and push again.

Two rules make conflicts rare:

  • Decide who edits where. For folders kept as code, edit in Git. If someone fixes a typo in the tool, pull it into the repository soon.
  • All or nothing. If any file in a push is invalid, TestPlum writes nothing at all, so a push never leaves the tool half updated.

Limits and honest trade-offs

Test cases as code is not free. Plan for these:

  • Not everyone uses Git. A tester who works only in the tool will see their edits reported as “newer in the tool” until someone pulls them.
  • Attachments and screenshots do not fit well in front matter. Keep evidence with results, not in the case file.
  • Rich formatting is limited to what Markdown can express. Tables inside steps, for example, are awkward.
  • Large suites mean many files. TestPlum handles up to 5,000 tests in one sync, which covers most projects, but very large suites may want to sync folder by folder.
  • Paths are fenced. For safety, the sync refuses paths that try to leave the tests folder, and hidden files.
  • The sync needs a key that can edit tests. A CI key that can only send results is not enough to push changes.

Getting started in a week

  1. Day 1: Pull every test into tests/cases/ on a branch, open a pull request and let the team read the files.
  2. Day 2: Merge it. Add the check step to pull requests that touch tests/cases/.
  3. Day 3: Add the push step on merge to main, using a key stored as a CI secret.
  4. Day 4: Add CODEOWNERS and a pull request template line about tests.
  5. Day 5: Agree which folders are edited in Git and which in the tool, and write it in the repository’s README.

Once tests live next to code, link each case to its automated counterpart with automation_id, and send CI results back with a single step; see JUnit XML and GitHub Actions. The release decision then rests on both manual and automated results, as described in the release readiness checklist.

Frequently asked questions

Why store test cases in Git?

So test changes are reviewed in pull requests next to the code they describe, with full history and blame, on the same branches. Plain Markdown also stays readable without any tool.

Where are test results stored if test cases live in Git?

In the test management tool. A file can describe a test but cannot record who ran it, on which build and with what outcome, so the usual model is definitions in Git and results in the tool, kept in step by a sync.

What happens if a test is edited in Git and in the tool at the same time?

A safe sync compares both sides with what they agreed at the last sync. If only one side changed, that change wins; if both changed differently, it is a conflict and nothing is overwritten until a person merges the two edits.

Does TestPlum support test cases as Markdown?

Yes. The TestPlum CLI pulls every test as a Markdown file with front matter, checks files in pull requests and pushes changes after merge. A test changed in both places is reported as a conflict instead of being overwritten.

Do testers need to learn Git?

Not necessarily. Teams often keep some folders as code and edit the rest in the tool. What matters is agreeing where each folder is edited, so the same test is not changed in two places at once.

Find out whether your next release is ready

Create a workspace, import your tests or write a few, and record a run. Everything on this site is in the product today.