Maelstrom packages your tests into hermetic micro-containers, then distributes them to be run on an arbitrarily large cluster of test-runners, or locally on your machine. You might use Maelstrom to run your tests because:

• It’s easy. Maelstrom functions as a drop-in replacement for cargo-test and pytest. In most cases, it just works with your existing tests with minimal configuration.
• It’s reliable. Maelstrom runs every test hermetically in its own lightweight container and runs each test independently, eliminating confusing errors caused by inter-test or implicit test-environment dependencies.
• It’s scalable. You can run Maelstrom as a cluster — add more worker machines to linearly increase test throughput.
• It’s clean. Maelstrom has a from-scratch, rootless container implementation (not relying on docker or RunC) written in Rust, optimized to be low-overhead and start quickly.
• It’s fast. In most cases, Maelstrom is faster than cargo test, even without adding clustering. Maelstrom’s test-per-process model is inherently slower than pytest’s shared-process model, but Maelstrom provides test isolation at a low performance cost.

Maelstrom is currently available for Rust and Pytest on Linux. C++, Typescript, and Java are coming soon.

While our focus thus far has been on running tests, Maelstrom’s underlying job execution system is general-purpose. We provide a command line utility to run arbitrary commands, as well a gRPC-based API and Rust bindings for programmatic access and control.

See the book for more information:

• Maelstrom Book

Installing Pre-Built Binaries

To run your tests using Maelstrom, you need a test-runner binary. The easiest way to get it is using cargo-binstall.

For Rust tests:

cargo binstall cargo-maelstrom

For Python tests:

cargo binstall maelstrom-pytest

This will install a pre-built binary from the github releases page. If you don’t have cargo-binstall, you can download the binary manually.

Check out the book for more ways to install Maelstrom.

Running cargo-maelstrom

To run your Rust tests, use cargo-maelstrom:

cargo maelstrom

This runs in “standalone” mode, meaning all tests are run locally. Each test is run in its own container, configured with a few common dependencies. It may work for your project without any further configuration.

If some tests fail, however, it likely means those tests have dependencies on their execution environment that aren’t packaged in their containers. You can remedy this by adding directives to the cargo-maelstrom.toml file. To do this, run:

cargo maelstrom --init

Then edit the created cargo-maelstrom.toml file as described in the book

Running maelstrom-pytest

Choosing a Python Image

maelstrom-pytest --init

Then update the file to include a python image

[[directives]]
image = "docker://python:3.11-slim"
image.use = ["layers", "environment"]

This example installs an image from Docker

Including Your Project Python Files

added_layers = [ { glob = "**.py" } ]

This example just adds all files with a .py extension. You may also need to include .pyi files or other files.

Including pip Packages

test-requirements.txt.

pytest==8.1.1

Now we are ready to try to run tests. Just invoke maelstrom-pytest:

maelstrom-pytest

This runs in “standalone” mode, meaning all tests are run locally. Each test is run in its own container.

Setting Up a Cluster

To get even more out of Maelstrom, you can set up a cluster to run your tests on. You will need to run one copy of the broker (maelstrom-broker) somewhere, and one copy of the worker (maelstrom-worker) on each node of the cluster.

You can install these using multiple methods, including cargo-binstall:

cargo binstall maelstrom-worker maelstrom-broker

Then you can start the broker:

maelstrom-broker --port=1234

Then a few workers:

maelstrom-worker --broker=broker-host:1234

Then run cargo-maelstrom against the cluster:

cargo maelstrom --broker=broker-host:1234

See the book for more information:

• Maelstrom Book