Looking through CI logs to find out which of your Cypress tests failed can be time consuming and error-prone, especially when your tests are spread out across several machines. With the Mochawesome test report generator and Github Actions, you can easily generate a beautiful, easy-to-read report of your test run, which merges results from all parallel nodes.

mocha, mochajs, mocha reports, mochawesome

Cypress Setup

First, add Mochawesome as a dependency of your project:

yarn install --save-dev mochawesome or

npm install --save-dev mochawesome.

Next, tell Cypress to use Mochawesome by adding the following to your cypress.json:

{
  "reporter": {
    "reporterOptions": {
      "overwrite": false,
      "html": false,
      "json": true
    }
  }
}

Now, when you run a Cypress test suite, a .json test report file will be generated for each .spec file ran, inside the mochawesome-report directory. Since Knapsack Pro runs tests one at a time, we need "overwrite": false to ensure that previous test reports are kept during subsequent test runs.

Note that we wrote "html": false and "json": true. Since we’re running tests one at a time, generating an HTML report now will be useless, since it will only show the results of one spec file. These options tell Mochawesome to only generate a JSON report. After all the tests are done, we will merge all the JSON reports together, then generate an HTML report, so that we have one report with all of our test results.

Github Actions Workflow

Now we can configure our Github Actions workflow. The first job will run our Cypress tests with Knapsack Pro:

  test_e2e:
    ...
    strategy:
      fail-fast: false
      matrix:
        node-version: [10.16.2]
        # Knapsack Pro parallelization
        ci_node_total: [4]
        ci_node_index: [0, 1, 2, 3]
    steps:
      ...
      - name: Configure Knapsack Pro
        run: |
          echo ::set-env name=KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS::${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS }}
          echo ::set-env name=KNAPSACK_PRO_TEST_FILE_PATTERN::"cypress/**/*.spec.{js,ts}"
          echo ::set-env name=KNAPSACK_PRO_CI_NODE_TOTAL::${{ matrix.ci_node_total }}
          echo ::set-env name=KNAPSACK_PRO_CI_NODE_INDEX::${{ matrix.ci_node_index }}

In this case, I am using 4 parallel CI nodes, but you should modify this accordingly if you’re using more or less. We will use the $KNAPSACK_PRO_CI_NODE_INDEX environment variable later, as this tells us the index of the currently running parallel node. Since I’m using 4 machines, this will be a number from 0 - 3.

Now we can run @knapsack-pro/cypress as usual:

      - name: Run E2E tests
        run: npx @knapsack-pro/cypress
      - name: Upload E2E test reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-reports-${{ env.KNAPSACK_PRO_CI_NODE_INDEX }}
          path: mochawesome-report

Running the tests will create the mochawesome-report directory, containing all our individual test reports. To pass these along to the next job, which will take care of merging them, we use Github’s upload-artifact action. Note that we provide the condition if: always(), to ensure that test reports will always be uploaded, even if some of our tests failed.

When naming the artifact, we’re using the node index environment variable from earlier. When running on 4 machines for example, this means that 4 artifacts will be uploaded: test-reports-0, test-reports-1, test-reports-2, test-reports-3.

The next job, which runs after all the parallel tests finish, will download the test reports, merge all of them, and upload a final HTML report.

  gen_report:
    name: Generate test report
    needs: test_e2e
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Install dependencies
        run: npm install mochawesome-merge mochawesome-report-generator

The if: always() ensures that reports will be generated even if any tests fail. Next, we install the dependencies needed to merge and generate the report.

Now we need to download all our test artifacts, which were uploaded as test-reports-0, test-reports-1, etc. Unfortunately, since Github Actions doesn’t support YAML anchors, this part requires a bit of copy and paste. Just like before, tweak this part depending on how many parallel nodes you’re using:

      - name: Create reports directory
        run: mkdir reports
      - name: Download test-reports-0
        uses: actions/download-artifact@v4
        with:
          name: test-reports-0
          path: reports/0
      - name: Download test-reports-1
        uses: actions/download-artifact@v4
        with:
          name: test-reports-1
          path: reports/1
      - name: Download test-reports-2
        uses: actions/download-artifact@v4
        with:
          name: test-reports-2
          path: reports/2
      - name: Download test-reports-3
        uses: actions/download-artifact@v4
        with:
          name: test-reports-3
          path: reports/3

Now we have a reports directory, with subdirectories containing reports from each node. Our directory structure will now look like this:

❯ tree reports
reports
├── 0
│   ├── mochawesome_001.json
│   ├── mochawesome_002.json
│   ├── mochawesome_003.json
│   ├── mochawesome_004.json
│   ├── mochawesome_005.json
│   └── mochawesome.json
├── 1
│   ├── mochawesome_001.json
│   ├── mochawesome_002.json
│   ├── mochawesome_003.json
│   └── mochawesome.json
├── 2
│   ├── mochawesome_001.json
│   ├── mochawesome_002.json
│   ├── mochawesome_003.json
│   └── mochawesome.json
└── 3
    ├── mochawesome_001.json
    ├── mochawesome_002.json
    ├── mochawesome_003.json
    └── mochawesome.json

4 directories, 18 files

To merge these reports, we want to have them all in one directory. The next step is a short Bash script:

      - name: Move all reports into one directory
        run: |
          mkdir -p mochawesome-report
          i=0
          for file in $(find reports -type f -name mochawesome\*.json); do
            filename=$(basename "$file" .json)-"$i".json
            mv "$file" mochawesome-report/"$filename"
            ls mochawesome-report
            i=$((i + 1))
          done

This script finds all the .json report files and moves them to the mochawesome-report folder, which will look like this:

❯ tree mochawesome-report 
mochawesome-report
├── mochawesome_001-10.json
├── mochawesome_001-16.json
├── mochawesome_001-2.json
├── mochawesome_001-6.json
├── mochawesome_002-15.json
├── mochawesome_002-1.json
├── mochawesome_002-5.json
├── mochawesome_002-9.json
├── mochawesome_003-0.json
├── mochawesome_003-14.json
├── mochawesome_003-4.json
├── mochawesome_003-8.json
├── mochawesome_004-13.json
├── mochawesome_005-12.json
├── mochawesome-11.json
├── mochawesome-17.json
├── mochawesome-3.json
└── mochawesome-7.json

0 directories, 18 files

Note that a number is appended to the end of each filename. This is only to prevent filename conflicts when we’re moving all the reports into one directory. The name of the file won’t have any affect on the generated report.

Now we can actually generate the final report:

      - name: Merge and generate reports
        run: |
          mkdir report
          npx mochawesome-merge > report/report.json
          npx marge --inline report/report.json -o report/

We ran these commands to:

  • mkdir report: Create the directory that will hold the final report
  • npx mochawesome-merge > report/report.json: Merge all of our .json files from the mochawesome-report directory into one file, report/report.json
  • npx marge --inline report/report.json -o report/: Take the merged report and generate a pretty HTML report from it

Now we upload this report:

      - name: Upload report
        uses: actions/upload-artifact@v4
        with:
          name: E2E Test Reports
          path: report/

Now you will be able to download your report from the build artifacts:

Github build artifacts

If you want to learn more about how to test your Javascript applications faster then check out the blog and website about the Knapsack Pro tool for CI parallelisation.