Why Maelstrom?

2024-10-16

In my 25 years of writing software professionally, I’ve witnessed a major shift in our industry’s attitude towards testing. When I started, we developers rarely tested our software. We would run a few manual, ad hoc tests, and then call it good. We would then throw our code over the figurative wall to a dedicated testing team. If we were lucky, they’d get to our changes in a week or two.

Today, we strive to have as many automated tests as possible, and to run those tests early and often. We continuously integrate our code, running those automated tests when we do. We’ve all experienced, sometimes with great pain, the fact that the earlier a defect is caught, the cheaper and easier it is to fix.

Even though things have gotten better, we still don’t run our tests nearly early nor often enough. Our testing tools have failed to keep up with our testing needs. This post will take a look at why that is, and will introduce Maelstrom, a toolset that we’ve build to address the problem.

Implementing a Container Runtime Part 1: Spawning Processes on Linux

2024-11-13

Spawning child processes using your programming language’s provided APIs can be very straightforward in a modern language. For example Rust’s std::process:Command provides an easy interface:

use anyhow::Result;

fn std_command_spawn_wait() -> Result<()> {
    let mut child = std::process::Command::new("/bin/echo")
        .stdout(std::process::Stdio::null())
        .spawn()?;

    let status = child.wait()?;
    assert!(status.success());

    Ok(())
}

These APIs make it really easy to get things right. They are the first thing you should reach for. Sometimes, though, you may find yourself needing to do things that just aren’t supported by these simple APIs. We found ourselves in this position working on Maelstrom, since we run each test in its own set of Linux namespaces. Maybe you too want to do something with namespaces, or maybe you want to use a pidfd. If that’s the case, then you might need to dig deeper and discover what the underlying APIs are capable of.

Of course, once you dig deeper, you might quickly find yourself confused. On Linux there are several APIs that all spawn a child process. There is fork, vfork, posix_spawn and clone. So which one do you pick? How are they different?

This article tries to answer these questions. Also, at the end I provide a simple flow chart that I hope makes it easy for you to decide which approach to take.

Maelstrom 0.12.0 Release

2024-09-16

We’re excited to announce Maelstrom 0.12.0. In this release, we made significant usability improvements to the Maelstrom test runners.

Maelstrom 0.11.0 Release

2024-07-30

We’re excited to announce Maelstrom 0.11.0. There are two main changes this release. We have overhauled the terminal UI for all of our test runners, and we have added a new test runner for the Go programming language.