---
title: Pytest Integration with Mergify
description: Report your test results from pytest to Mergify
---

<IntegrationLogo src={pytestLogo} alt="Pytest logo" />

This guide explains how to integrate Pytest with Test Insights using the
`pytest-mergify` plugin. Once installed, test results are automatically
uploaded to Test Insights without any extra workflow changes.

## Installation

You need to install the
[`pytest-mergify`](https://pypi.python.org/pypi/pytest-mergify) plugin to
automatically upload your test results to **Test Insights**. This can be done in
different ways depending on your Python dependency setup. Below are a few
examples:

### Pip / Requirements File

```bash
pip install pytest-mergify
```

Or add `pytest-mergify` to your `requirements.txt`.

### `setup.py`

```python
setup(
    name="your-package",
    ...
    install_requires=[
        ...
    ],
    extras_require={
        "dev": ["pytest-mergify"]
    },
    ...
)
```

Make sure those dependencies are installed when running your tests.

### `setup.cfg`

```ini
[options.extras_require]
dev =
    pytest-mergify
```

Make sure those dependencies are installed when running your tests.

### Poetry

```bash
poetry add --group dev pytest-mergify
```

## Update Your CI Workflow

<CIInsightsSetupNote />

Your workflow should run your tests as usual while exporting the secret
`MERGIFY_TOKEN` as an environment variable.

### GitHub Actions

Add the following to the GitHub Actions step running your tests:

```yaml
env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
```

For example:

```yaml
- name: Run Tests 🧪
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: pytest
```

### Buildkite

Set `MERGIFY_TOKEN` in the environment of the agents running your tests.
The step itself then needs no Mergify-specific configuration:

```yaml
steps:
  - label: "Run Tests 🧪"
    command: pytest
```

<BuildkiteTokenNote plugin={false} />

The plugin collects your test results and sends them to Test Insights.

## Quarantine and Crashed Runs

The plugin applies [quarantine](/test-insights/quarantine) inside the pytest
session. It marks each quarantined test as a non-strict `xfail`, so a
quarantined test that fails is reported as `xfailed` and does not count against
pytest's exit code. The exit code of your test step already accounts for
quarantine, so the step needs nothing more:

- Do not add `continue-on-error: true`. The recipes that upload a JUnit report
  with the `mergifyio/gha-mergify-ci` action need it because the action decides
  the job's result after the tests. Here nothing does, so it would let every
  real failure through.

- There is no step `id` to set and no `test_step_outcome` to pass: both belong
  to that action, which this setup does not use.

A crash cannot pass for a green run either. If pytest dies mid-run or fails
outside a test, such as a collection error, it exits non-zero and the step
fails. The plugin uploads results when the session ends, so a process killed
outright before then, such as by the out-of-memory killer, sends nothing to
Test Insights. A run that stops early without the process being killed, such as
an interrupted session or one cut short by `--maxfail`, still uploads the
results collected up to that point.

If the plugin cannot fetch the quarantine list, it quarantines nothing for that
run, and a quarantined test that fails makes the step fail as usual.

## Using with Tox

If you’re using [Tox](https://tox.wiki/) to manage test environments, you can
still use `pytest-mergify` by passing the `MERGIFY_TOKEN` and the rest of the
CI environment variables into the test environment. On Buildkite the token comes
from the agent's environment, as [above](#buildkite), so only the GitHub Actions
step names it.

In your CI workflow:

```yaml
# GitHub Actions
- name: Run Tox Tests
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: tox
```

```yaml
# Buildkite
steps:
  - label: "Run Tox Tests"
    command: tox
```

In your `tox.ini`, make sure the plugin is included in your testenv dependencies:

```ini
[testenv]
# You need to pass the MERGIFY_*, CI, GITHUB_*, etc variables
passenv = *
deps =
    pytest
    pytest-mergify
commands = pytest
```

If you’re using multiple environments (e.g. `py38`, `py39`, etc.), the plugin
will work for all of them as long as the token is set correctly.

If you’re running multiple Tox environments (e.g., py38, py39, etc.), we
recommend setting the `MERGIFY_TEST_JOB_NAME` environment variable to identify each
environment’s report in Test Insights:

In your CI workflow:

```yaml
# GitHub Actions
- name: Run Tox Tests
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
    MERGIFY_TEST_JOB_NAME: tox-${{ matrix.python-version }}
  run: tox
```

```yaml
# Buildkite
steps:
  - label: "Run Tox Tests ({{matrix}})"
    command: tox
    matrix:
      - "3.10"
      - "3.11"
      - "3.12"
    env:
      MERGIFY_TEST_JOB_NAME: "tox-{{matrix}}"
```

:::tip
  Use `MERGIFY_TEST_JOB_NAME` to make reports clearer in Test Insights,
  especially when running multiple Tox environments or using a matrix.
:::

## Verify and Review in Test Insights

After pushing these changes, your next CI run reports its pytest results
automatically.

<ReviewInTestInsights />