Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions public/_redirects
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,7 @@
/integrations/teamcity/ /integrations/ci-status-checks 301
/integrations/gitlab /integrations/ci-status-checks 301
/integrations/gitlab/ /integrations/ci-status-checks 301
/ci-insights/test-frameworks/* /test-insights/test-frameworks/:splat 301
/ci-insights/quarantine /test-insights/quarantine 301
/ci-insights/quarantine/ /test-insights/quarantine 301

116 changes: 4 additions & 112 deletions src/content/docs/ci-insights.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -86,115 +86,7 @@ Select your CI system below to enable CI Insights and configure flaky test detec

## Test Framework Configuration

CI Insights supports many test frameworks and captures test-level data from them.
Each framework requires a quick setup to ensure test results are parsed and uploaded properly.

Refer to the dedicated page for your specific framework to configure and verify your test output.

<DocsetGrid>
<Docset
title="Cypress"
path="/ci-insights/test-frameworks/cypress"
icon="simple-icons:cypress"
>
Integrate Cypress with CI Insights.
</Docset>
<Docset
title="Go"
path="/ci-insights/test-frameworks/golang"
icon="simple-icons:go"
>
Integrate Go with CI Insights.
</Docset>
<Docset
title="Jest"
path="/ci-insights/test-frameworks/jest"
icon="simple-icons:jest"
>
Integrate Jest with CI Insights.
</Docset>
<Docset
title="JUnit"
path="/ci-insights/test-frameworks/junit"
icon="simple-icons:junit5"
>
Integrate JUnit with CI Insights.
</Docset>
<Docset
title="Minitest"
path="/ci-insights/test-frameworks/minitest"
icon="simple-icons:ruby"
>
Integrate Minitest with CI Insights.
</Docset>
<Docset
title="MSTest"
path="/ci-insights/test-frameworks/mstest"
icon="simple-icons:dotnet"
>
Integrate MSTest with CI Insights.
</Docset>
<Docset
title="NUnit"
path="/ci-insights/test-frameworks/nunit"
icon="simple-icons:dotnet"
>
Integrate NUnit with CI Insights.
</Docset>
<Docset
title="Pest"
path="/ci-insights/test-frameworks/pest"
icon="simple-icons:php"
>
Integrate Pest with CI Insights.
</Docset>
<Docset
title="PHPUnit"
path="/ci-insights/test-frameworks/phpunit"
icon="simple-icons:php"
>
Integrate PHPUnit with CI Insights.
</Docset>
<Docset
title="Playwright"
path="/ci-insights/test-frameworks/playwright"
icon="simple-icons:playwright"
>
Integrate Playwright with CI Insights.
</Docset>
<Docset
title="Pytest"
path="/ci-insights/test-frameworks/pytest"
icon="simple-icons:pytest"
>
Integrate Pytest with CI Insights.
</Docset>
<Docset
title="RSpec"
path="/ci-insights/test-frameworks/rspec"
icon="simple-icons:ruby"
>
Integrate RSpec with CI Insights.
</Docset>
<Docset
title="Rust"
path="/ci-insights/test-frameworks/rust"
icon="simple-icons:rust"
>
Integrate Rust with CI Insights.
</Docset>
<Docset
title="TestNG"
path="/ci-insights/test-frameworks/testng"
icon="simple-icons:testinglibrary"
>
Integrate TestNG with CI Insights.
</Docset>
<Docset
title="Vitest"
path="/ci-insights/test-frameworks/vitest"
icon="simple-icons:vitest"
>
Integrate Vitest with CI Insights.
</Docset>
</DocsetGrid>
To capture test-level data from your CI pipeline, configure your test
framework to upload results to Mergify. See the
[Test Insights test framework guides](/test-insights#test-framework-configuration)
for framework-specific setup instructions.
4 changes: 2 additions & 2 deletions src/content/docs/ci-insights/setup/buildkite.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,6 @@ every 12 hours on weekdays.

## Test Framework Configuration

See the [test framework guides](/ci-insights#test-framework-configuration) for
See the [test framework guides](/test-insights#test-framework-configuration) for
framework-specific instructions on generating JUnit XML reports and uploading
them to CI Insights with both GitHub Actions and Buildkite.
them with both GitHub Actions and Buildkite.
122 changes: 121 additions & 1 deletion src/content/docs/test-insights.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ description: Monitor, detect, and manage unreliable tests across your repositori
hubAccent: true
---

import Docset from '../../components/DocsetGrid/Docset.astro';
import DocsetGrid from '../../components/DocsetGrid/DocsetGrid.astro';

Test Insights helps you manage test reliability across the full lifecycle.
It catches flaky tests on pull requests before they merge, surfaces unhealthy
tests across your repositories, and lets you quarantine problematic tests so
Expand Down Expand Up @@ -53,4 +56,121 @@ framework:

- [GitHub Actions setup](/ci-insights/setup/github-actions)
- [Jenkins setup](/ci-insights/setup/jenkins)
- [Test framework configuration](/ci-insights#test-framework-configuration)
- [Test framework configuration](#test-framework-configuration)

## Test Framework Configuration

Test Insights supports many test frameworks and captures test-level data from
them. Each framework requires a quick setup to ensure test results are parsed
and uploaded properly.

Refer to the dedicated page for your specific framework to configure and
verify your test output.

<DocsetGrid>
<Docset
title="Cypress"
path="/test-insights/test-frameworks/cypress"
icon="simple-icons:cypress"
>
Integrate Cypress with Test Insights.
</Docset>
<Docset
title="Go"
path="/test-insights/test-frameworks/golang"
icon="simple-icons:go"
>
Integrate Go with Test Insights.
</Docset>
<Docset
title="Jest"
path="/test-insights/test-frameworks/jest"
icon="simple-icons:jest"
>
Integrate Jest with Test Insights.
</Docset>
<Docset
title="JUnit"
path="/test-insights/test-frameworks/junit"
icon="simple-icons:junit5"
>
Integrate JUnit with Test Insights.
</Docset>
<Docset
title="Minitest"
path="/test-insights/test-frameworks/minitest"
icon="simple-icons:ruby"
>
Integrate Minitest with Test Insights.
</Docset>
<Docset
title="MSTest"
path="/test-insights/test-frameworks/mstest"
icon="simple-icons:dotnet"
>
Integrate MSTest with Test Insights.
</Docset>
<Docset
title="NUnit"
path="/test-insights/test-frameworks/nunit"
icon="simple-icons:dotnet"
>
Integrate NUnit with Test Insights.
</Docset>
<Docset
title="Pest"
path="/test-insights/test-frameworks/pest"
icon="simple-icons:php"
>
Integrate Pest with Test Insights.
</Docset>
<Docset
title="PHPUnit"
path="/test-insights/test-frameworks/phpunit"
icon="simple-icons:php"
>
Integrate PHPUnit with Test Insights.
</Docset>
<Docset
title="Playwright"
path="/test-insights/test-frameworks/playwright"
icon="simple-icons:playwright"
>
Integrate Playwright with Test Insights.
</Docset>
<Docset
title="Pytest"
path="/test-insights/test-frameworks/pytest"
icon="simple-icons:pytest"
>
Integrate Pytest with Test Insights.
</Docset>
<Docset
title="RSpec"
path="/test-insights/test-frameworks/rspec"
icon="simple-icons:ruby"
>
Integrate RSpec with Test Insights.
</Docset>
<Docset
title="Rust"
path="/test-insights/test-frameworks/rust"
icon="simple-icons:rust"
>
Integrate Rust with Test Insights.
</Docset>
<Docset
title="TestNG"
path="/test-insights/test-frameworks/testng"
icon="simple-icons:testinglibrary"
>
Integrate TestNG with Test Insights.
</Docset>
<Docset
title="Vitest"
path="/test-insights/test-frameworks/vitest"
icon="simple-icons:vitest"
>
Integrate Vitest with Test Insights.
</Docset>
</DocsetGrid>
6 changes: 3 additions & 3 deletions src/content/docs/test-insights/mitigation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Quarantine works on any branch, not just the default branch.
:::note
Quarantined tests must still be uploaded through one of the supported CI
integrations. See the
[test framework configuration](/ci-insights#test-framework-configuration)
[test framework configuration](/test-insights#test-framework-configuration)
for setup details.
:::

Expand All @@ -40,7 +40,7 @@ through [Detection](/test-insights/detection) and want to stop it from
blocking your team while you investigate.

For technical details on how quarantine integrates with your CI pipeline,
see the [Quarantine documentation](/ci-insights/quarantine).
see the [Quarantine documentation](/test-insights/quarantine).

## Auto-quarantine

Expand Down Expand Up @@ -77,5 +77,5 @@ overhead and keeps your CI pipeline moving.

Mitigation uses the same CI integration as Detection. To ensure quarantine
works correctly, your CI must be configured to check quarantine status. See
the [Quarantine documentation](/ci-insights/quarantine) for technical setup
the [Quarantine documentation](/test-insights/quarantine) for technical setup
details.
2 changes: 1 addition & 1 deletion src/content/docs/test-insights/prevention.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,6 @@ reliability.
Prevention requires test framework plugins that instrument test runs to track
flakiness on pull requests.

See the [test framework configuration](/ci-insights#test-framework-configuration)
See the [test framework configuration](/test-insights#test-framework-configuration)
for setup instructions specific to your framework (pytest-mergify,
rspec-mergify, etc.).
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ When you quarantine a test in the Mergify dashboard, its future executions will
This avoids pipeline disruption while still surfacing issues in the UI and keeping historical data intact.

Quarantined tests must still be uploaded appropriately using one of the supported CI integrations.
To ensure the correct setup, refer to [the documentation for your test framework](/ci-insights/#test-framework-configuration).
To ensure the correct setup, refer to [the documentation for your test framework](/test-insights#test-framework-configuration).

## Adding and removing tests from quarantine

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
When using the Buildkite plugin for CI Insights Quarantine, the plugin automatically
When using the Buildkite plugin for Test Insights Quarantine, the plugin automatically
detects test failures using the step's exit code. No extra configuration is needed.

The `mergifyio/mergify-ci` plugin runs in the `post-command` hook and reads
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
In order to benefit from CI Insights Quarantine, you need to add `continue-on-error: true`
In order to benefit from Test Insights Quarantine, you need to add `continue-on-error: true`
in your GitHub Actions step that executes your tests and generates the JUnit file.
The step running the `gha-mergify-ci` action will determine the success or failure conclusion,
considering quarantined tests.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Cypress Integration with CI Insights
description: Report your test results from Cypress to CI Insights
title: Cypress Integration with Mergify
description: Report your test results from Cypress to Mergify
---

import cypressLogo from "../../../images/ci-insights/cypress/logo.svg"
Expand All @@ -18,7 +18,7 @@ import MergifyCliUploadStep from "../../../../components/MergifyCliUploadStep.as
<IntegrationLogo src={cypressLogo} alt="Cypress logo" />

This guide shows how to configure [Cypress](https://www.cypress.io/) to produce
JUnit reports and upload them to CI Insights using your CI workflow.
JUnit reports and upload them to Test Insights using your CI workflow.

## Generate a JUnit Report with Cypress

Expand Down Expand Up @@ -52,7 +52,7 @@ both have JUnit generated while keeping the standard output.
<CIInsightsSetupNote />

Once you have the JUnit file produced by Cypress, add a step to upload these
results to **CI Insights** via the `mergifyio/gha-mergify-ci` action.
results to **Test Insights** via the `mergifyio/gha-mergify-ci` action.

In the workflow file where Cypress is launched, after running `npm run
test:cypress` (or similar), include a step like:
Expand All @@ -73,16 +73,16 @@ test:cypress` (or similar), include a step like:

<MergifyCliUploadStep reportPath="cypress/results/junit*.xml" testCommand="npm run test:cypress" />

## Verify and Review in CI Insights
## Verify and Review in Test Insights

After pushing these changes:

1. Your GitHub Actions workflow will run tests with Cypress.
2. Cypress generates JUnit files.
3. The Mergify CI action uploads that file to CI Insights.
3. The Mergify CI action uploads that file to Test Insights.

You can then see your test results, including failures and flaky tests,
directly in the CI Insights dashboard.
directly in the Test Insights dashboard.

## Troubleshooting Tips

Expand Down
Loading