If you run tests on parallel jobs (using CI parallelisation) you need to merge CodeClimate reports from each parallel job (CI node) into a unified report.

Semaphore CI

An additional problem may happen that some of your parallel jobs (one of your parallel CI node) may not have reported CodeClimate result when tests were never executed on that CI node.

It could happen when you run tests with dynamic tests allocation across parallel jobs using Knapsack Pro Queue Mode. For instance if once of CI node started running tests after other CI nodes already executed the whole test suite distributed for particular CI build then the node has no CodeClimate report.

Here is a list of steps we need to follow to ensure we can correctly prepare a final report for CodeClimate.

Below is the full example for Semaphore CI 2.0 config but it applies to any other CI provider, just use syntax specifically for your CI tool.

version: v1.0
name: My app
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804

blocks:
  - name: Install dependencies
    task:
      agent:
        machine:
          type: e1-standard-4
          os_image: ubuntu1804

      env_vars:
        - name: RAILS_ENV
          value: "test"

      secrets:
        - name: semaphore_secrets

      jobs:
        - name: Build gems and assets
          commands:
            - checkout
            # ...
            - cache restore $SEMAPHORE_PROJECT_NAME-cc-test-reporter
            - if [ -d 'cc-reporter' ]; then
              echo 'Found cc-reporter in cache';
              else
              mkdir cc-reporter;
              curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-reporter/cc-test-reporter;
              chmod +x ./cc-reporter/cc-test-reporter;
              cache store $SEMAPHORE_PROJECT_NAME-cc-test-reporter cc-reporter;
              fi

  - name: Run specs
    task:
      env_vars:
        - name: RAILS_ENV
          value: "test"
        # write knapsack_pro logs to file instead of stdout:
        # log/knapsack_pro_node_0.log
        # log/knapsack_pro_node_1.log
        - name: KNAPSACK_PRO_LOG_DIR
          value: "log"
        # text we expect to find in logs when there was no tests executed for particular parallel job (CI node)
        - name: NO_TEST_EXECUTED_STR
          value: "[knapsack_pro] No test files were executed on this CI node"

      secrets:
        - name: semaphore_secrets

      prologue:
        commands:
          - checkout
          # ...
          - cache restore $SEMAPHORE_PROJECT_NAME-cc-test-reporter
          - mkdir -p coverage/final
          - ./cc-reporter/cc-test-reporter before-build

      # run 2 parallel jobs with Knapsack Pro Queue Mode
      jobs:
        - name: RSpec node 0 - Knapsack Pro
          commands:
            - KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 bundle exec rake knapsack_pro:queue:rspec
            - STR_COUNT=`grep -F "$NO_TEST_EXECUTED_STR" log/knapsack_pro_node_0.log | wc -l | tr -d ' '`
            - if [ $STR_COUNT -eq 0 ]; then ./cc-reporter/cc-test-reporter format-coverage -t simplecov --output "coverage/final/codeclimate.$SEMAPHORE_JOB_ID.json"; fi
            - cache store coverage-0-$SEMAPHORE_WORKFLOW_ID coverage/final

        - name: RSpec node 1 - Knapsack Pro
          commands:
            - KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=1 bundle exec rake knapsack_pro:queue:rspec
            - STR_COUNT=`grep -F "$NO_TEST_EXECUTED_STR" log/knapsack_pro_node_1.log | wc -l | tr -d ' '`
            - if [ $STR_COUNT -eq 0 ]; then ./cc-reporter/cc-test-reporter format-coverage -t simplecov --output "coverage/final/codeclimate.$SEMAPHORE_JOB_ID.json"; fi
            - cache store coverage-1-$SEMAPHORE_WORKFLOW_ID coverage/final

  - name: Post-test tasks
    task:
      secrets:
        - name: semaphore_secrets

      jobs:
        - name: CodeClimate
          commands:
            - cache restore $SEMAPHORE_PROJECT_NAME-cc-test-reporter
            # we have 2 parallel jobs so 0..1 range
            - for i in {0..1}; do cache restore coverage-$i-$SEMAPHORE_WORKFLOW_ID; done
            - FILES_COUNT=`ls -1 coverage/final | wc -l | tr -d ' '`
            - ./cc-reporter/cc-test-reporter sum-coverage --output - --parts $FILES_COUNT coverage/final/codeclimate.*.json | ./cc-reporter/cc-test-reporter upload-coverage --input -
            - for i in {0..1}; do cache delete coverage-$i-$SEMAPHORE_WORKFLOW_ID; done

Summary and Queue Mode for dynamic test suite split

CI builds can be much faster thanks to leveraging parallel jobs on Semaphore CI 2.0 and CI parallelisation on any CI provider (see more parallelisation examples for your CI providers). You can check Knapsack Pro tool for CI parallelisation and learn more about Queue Mode and what problems it solves in below video.