Skip to main content

Overview

Knapsack Pro wraps your current test runner(s) and works with your existing CI infrastructure to parallelize tests optimally:

  • Dynamically splits your tests based on up-to-date test execution data
  • Is designed from the ground up for CI and supports all of them
  • Tracks your CI builds to detect bottlenecks
  • Does not have access to your source code and collects minimal test data (with opt-in encryption)
  • Enables you to export historical metrics about your CI builds
  • Supports out-of-the-box any Ruby test runners, Cypress, Jest, Vitest (and provides both SDK and API to integrate with any other language)
  • Replaces local dependencies like Redis with an API and runs your tests regardless of network problems

Components​

Knapsack Pro API​

The Knapsack Pro API splits test files between parallel CI nodes based on the test execution time of your previous CI runs. See Test Orchestration Modes below for more information.

Knapsack Pro Client​

The Knapsack Pro Client is a library to install in your project that wraps your current test runners/frameworks. It connects with the Knapsack Pro API to:

  • Retrieve the list of tests to run for the current parallel CI node
  • Record the test execution time, including additional information from the CI environment variables:
    • Commit hash
    • Branch name
    • Number of parallel CI nodes
    • CI node index

Dashboard​

Knapsack Pro comes with a web UI that displays graphs and stats about your test runs:

Tests Orchestration Modes​

Normal Mode​

Knapsack Pro splits your tests dynamically between parallel CI nodes.

Each Knapsack Pro Client (parallel CI node):

  1. Retrieves from the Knapsack Pro API a batch of tests to execute
  2. Runs those tests
  3. Repeats 1. and 2. as long as there are more tests in the API queue
  4. Sends the test execution times to the API for the subsequent CI runs

Knapsack Pro distributes tests so that each CI node finishes at the same time. Thanks to that, your CI build time is as fast as possible.

Knapsack Pro allows distributing the tests in a single batch per node too, but it’s rarely a good solution:

  • Tests themselves, especially end to end, have varying execution times
  • Failing tests execute faster than green ones
  • CI nodes may differ in hardware/performance
  • CI nodes boot at different times
  • CI nodes are more or less busy when the environment is shared/virtualized
  • CI nodes may load their caches at dissimilar times
  • Test execution starts at different times on different CI nodes
  • Flaky tests are.. Well, flaky!

Fallback Mode​

In case of network issues, Knapsack Pro logs a warning and automatically switches to Fallback Mode to run your tests with a split based on test file names.

Knapsack Pro will retry to connect with the API multiple times before switching to Fallback Mode.

Split by test examples​

Knapsack Pro can parallelize tests across CI nodes by example. This is useful when you have slow test files but don't want to manually split test examples into smaller test files.

Spot CI nodes​

When you use spot CI nodes like Google Cloud Preemptible VMs or Amazon EC2 Spot Instances, Knapsack Pro gracefully handles preemptions: when you retry a node that was shut down the same tests are executed before continuing consuming the queue.

Retry-only-Failures​

When you retry one (or all the) nodes, Knapsack Pro executes only the tests that failed last time on that node (see the docs for Ruby, Jest, Cypress, Vitest, Playwright).

SAML-based single sign-on (SSO)​

Besides traditional authentication (magic links, email/password), Knapsack Pro supports your organization's Identity Provider (IdP) like Okta, OneLogin, Google Workspace, or any other SAML SSO provider.

Customer Support​

If you have questions, feel free to contact us directly and we will be happy to help.