Looking through CI logs to find out which of your Cypress tests failed can be time consuming and error-prone, especially when your tests are spread out across several machines. With the Mochawesome test report generator and Github Actions, you can easily generate a beautiful, easy-to-read report of your test run, which merges results from all parallel nodes.
First, add Mochawesome as a dependency of your project:
yarn install --save-dev mochawesome or
npm install --save-dev mochawesome.
Next, tell Cypress to use Mochawesome by adding the following to your
Now, when you run a Cypress test suite, a
.json test report file will be generated for each
.spec file ran, inside the
mochawesome-report directory. Since Knapsack Pro runs tests one at a time, we need
"overwrite": false to ensure that previous test reports are kept during subsequent test runs.
Note that we wrote
"html": false and
"json": true. Since we’re running tests one at a time, generating an HTML report now will be useless, since it will only show the results of one spec file. These options tell Mochawesome to only generate a JSON report. After all the tests are done, we will merge all the JSON reports together, then generate an HTML report, so that we have one report with all of our test results.
Github Actions Workflow
Now we can configure our Github Actions workflow. The first job will run our Cypress tests with Knapsack Pro:
In this case, I am using 4 parallel CI nodes, but you should modify this accordingly if you’re using more or less. We will use the
$KNAPSACK_PRO_CI_NODE_INDEX environment variable later, as this tells us the index of the currently running parallel node. Since I’m using 4 machines, this will be a number from 0 - 3.
Now we can run
knapsack-pro-cypress as usual:
Running the tests will create the
mochawesome-report directory, containing all our individual test reports. To pass these along to the next job, which will take care of merging them, we use Github’s
upload-artifact action. Note that we provide the condition
if: always(), to ensure that test reports will always be uploaded, even if some of our tests failed.
When naming the artifact, we’re using the node index environment variable from earlier. When running on 4 machines for example, this means that 4 artifacts will be uploaded:
The next job, which runs after all the parallel tests finish, will download the test reports, merge all of them, and upload a final HTML report.
if: always() ensures that reports will be generated even if any tests fail. Next, we install the dependencies needed to merge and generate the report.
Now we need to download all our test artifacts, which were uploaded as
test-reports-1, etc. Unfortunately, since Github Actions doesn’t support YAML anchors, this part requires a bit of copy and paste. Just like before, tweak this part depending on how many parallel nodes you’re using:
Now we have a
reports directory, with subdirectories containing reports from each node. Our directory structure will now look like this:
To merge these reports, we want to have them all in one directory. The next step is a short Bash script:
This script finds all the
.json report files and moves them to the
mochawesome-report folder, which will look like this:
Note that a number is appended to the end of each filename. This is only to prevent filename conflicts when we’re moving all the reports into one directory. The name of the file won’t have any affect on the generated report.
Now we can actually generate the final report:
We ran these commands to:
mkdir report: Create the directory that will hold the final report
npx mochawesome-merge > report/report.json: Merge all of our
.jsonfiles from the
mochawesome-reportdirectory into one file,
npx marge --inline report/report.json -o report/: Take the merged report and generate a pretty HTML report from it
Now we upload this report:
Now you will be able to download your report from the build artifacts: