Testing in Rust: Execution Control and Test Organization

This chapter covers advanced testing workflows in Rust. It explains how to customize test execution using command-line arguments, filter and ignore specific tests, and structure your codebase using unit tests and integration tests.

1. The Mechanics of cargo test and Binary Arguments

When you run cargo test, Cargo compiles your code in test mode and builds a test binary. It then executes this binary to run all functions annotated with the #[test] attribute.

By default, the test binary:

• Runs all tests in parallel using separate threads.
• Captures all standard output (stdout) generated during passing tests, preventing it from printing to the screen.

To customize this behavior, you can pass command-line options. There are two distinct sets of command-line arguments:

1. Cargo Test Arguments: Options that configure how Cargo manages the build and test process.
2. Test Binary Arguments: Options that configure the behavior of the resulting test binary.

These two sets of arguments are separated by a double dash (--):

cargo test [cargo_options] -- [binary_options]

Command-Line Help Commands

To inspect the available options for both Cargo and the test binary, use the following help commands:

2. Controlling Concurrency with Test Threads

By default, Rust runs tests in parallel, allocating a separate thread to each test. This speeds up test suite execution. However, parallel execution can cause issues if your tests share state or access shared resources.

The Shared Resource Problem

If multiple tests read and write to the same file on disk or modify a shared database, they can interfere with one another, leading to intermittent, non-deterministic test failures (race conditions).

The Solution: Serial Execution

To prevent tests from running in parallel, you can limit the number of execution threads to 1 using the --test-threads binary option:

cargo test -- --test-threads=1

Note: Running tests serially will increase the overall execution time of your test suite. Use this option only when your tests rely on shared, non-isolated environments, or refactor your tests so that each test operates on its own isolated resource (e.g., writing to unique, randomly named temporary files).

3. Capturing and Displaying Standard Output

By default, if a test passes, Rust's test runner captures all standard output (such as println! macros) and hides it from the console. Output is only displayed for failing tests to help you debug.

Consider the following code containing a function that prints a message and returns a value:

fn prints_and_returns_10(a: i32) -> i32 {
    println!("I got the value {}", a);
    10
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn this_test_passes() {
        let value = prints_and_returns_10(8);
        assert_eq!(value, 10);
    }

    #[test]
    fn this_test_fails() {
        let value = prints_and_returns_10(4);
        assert_eq!(value, 5); // This assertion will fail
    }
}

Default Output Behavior

Running cargo test produces the following output behavior:

• this_test_fails: Fails, and the captured output (I got the value 4) is printed to the console.
• this_test_passes: Passes, and its output (I got the value 8) is hidden.

Showing Output for Passing Tests

To force the test runner to display printed output for both passing and failing tests, pass the --show-output flag to the test binary:

cargo test -- --show-output

4. Filtering and Running Subsets of Tests

If your test suite is large, you may want to run only a specific test or a subset of tests. You can filter tests by passing a search string directly to cargo test.

Consider a test suite with the following test functions:

• tests::add_two_and_two
• tests::add_three_and_five
• tests::one_hundred

Running a Single Test

To run a single test, pass its exact name as an argument:

cargo test one_hundred

Running Multiple Tests via Partial Name Matching

If you pass a string that matches the beginning or a portion of multiple test names, Cargo will run all tests that match that filter.

cargo test add

This command matches and runs both tests::add_two_and_two and tests::add_three_and_five.

Filtering by Module Namespace

Because module names form part of the full path of a test name (e.g., tests::one_hundred), you can run all tests within a specific module by filtering on the module's name:

cargo test tests::

5. Ignoring Slow or Expensive Tests

Some tests, such as those performing heavy computations, network requests, or disk I/O, can take a long time to run. You can exclude these tests from normal test runs using the #[ignore] attribute.

#[test]
fn it_works() {
    assert_eq!(2 + 2, 4);
}

#[test]
#[ignore]
fn expensive_test() {
    // Code that takes a long time to run (e.g., up to an hour)
    assert_eq!(2 + 2, 4);
}

Execution Workflows for Ignored Tests

• Normal Run: Running cargo test runs it_works and marks expensive_test as ignored without executing it.
• Run Only Ignored Tests: To run only the tests marked with the #[ignore] attribute, use the --ignored binary flag:

cargo test -- --ignored

6. Unit Testing in Rust (In-File Convention)

The Rust community divides testing into two main categories: unit tests and integration tests.

The Unit Testing Pattern

In Rust, unit tests are written directly inside the source files containing the code they test. By convention, you create a module named tests at the bottom of the file to hold these tests.

pub fn add_two(a: i32) -> i32 {
    inner_adder(a, 2)
}

// Private helper function
fn inner_adder(a: i32, b: i32) -> i32 {
    a + b
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_internal_adder() {
        // Testing a private function is permitted in Rust
        assert_eq!(inner_adder(2, 2), 4);
    }
}

Key Components of the Unit Test Pattern

1. #[cfg(test)] Attribute: This annotation tells Rust to compile and run this module only when executing cargo test. It ensures that the test code is excluded from normal production builds (such as cargo build or cargo run), keeping the compiled binary small.
2. use super::*;: Because tests is an inner child module, it must bring the items from its parent module into scope. Using use super::*; imports all public and private items from the parent module.
3. Testing Private Functions: In Rust, child modules can access private items in their parent modules. This allows you to write unit tests for private helper functions (like inner_adder) if needed.

7. Integration Testing (External Interface)

Integration tests are completely external to your library. They use your library in the same way any external client crate would, meaning they can only access and test your library's public API.

Directory Structure

Integration tests must reside in a directory named tests at the root of your project (at the same level as your src directory):

Writing an Integration Test

Each file inside the tests/ directory is compiled by Cargo as a separate, individual crate. Because of this, you must explicitly import your library into scope at the top of each integration test file.

// tests/integration_test.rs

// Bring the library crate into scope
use adder; 

#[test]
fn test_add_two_integration() {
    // Can only call public APIs (e.g., add_two)
    assert_eq!(adder::add_two(2), 4);
}

Note: You do not need to annotate integration test files or modules with #[cfg(test)]. Cargo automatically treats all files in the tests/ directory as test suites and compiles them only during cargo test.

Running Integration Tests

When you run cargo test, Cargo runs unit tests first, followed by integration tests, and finally documentation tests:

cargo test

To run only a specific integration test file, use the --test flag followed by the name of the file (excluding the .rs extension):

cargo test --test integration_test

8. Sharing Code in Integration Tests (The Common Module Pitfall)

As your integration test suite grows, you may want to share helper code (such as setup or teardown functions) across multiple integration test files.

The Naive (Incorrect) Approach

If you create a helper file directly under the tests/ directory, Cargo will treat it as an integration test crate:

Because common.rs is in the root of tests/, Cargo compiles it as a separate test crate. This results in empty test suites and unnecessary output in your console:

running 0 tests
...

The Correct Approach: Using Subdirectories

To prevent Cargo from compiling shared helper code as an integration test crate, place the shared code inside a subdirectory of tests/. Files inside subdirectories of tests/ are not compiled as separate test crates.

Use the old-style module layout (mod.rs) to define the shared module:

1. Define the Shared Helper Module

Inside tests/common/mod.rs, define your shared functions:

// tests/common/mod.rs

pub fn setup() {
    // Setup code specific to your library tests
    println!("Setting up test environment...");
}

2. Import and Use the Shared Module in Integration Tests

You can now declare the common module inside your integration test files:

// tests/integration_test.rs

use adder;

// Declare the common module
mod common;

#[test]
fn test_add_two_integration() {
    common::setup();
    assert_eq!(adder::add_two(2), 4);
}

9. Testing Binary Crates (src/main.rs Limitations)

If a project contains only a binary crate (src/main.rs) and no library crate (src/lib.rs), you cannot write integration tests for it in the tests/ directory.

Why This Limitation Exists

Integration tests must import the target crate using use crate_name;. However, binary crates are designed to run as standalone executables and do not expose their functionality as an importable library.

tests/integration_test.rs
   |
   | Cannot import!
   v
src/main.rs (Binary Crate)

The "Thin Wrapper" Architecture

To make binary projects testable, the standard Rust design pattern is to split your project into two parts:

1. src/lib.rs: Contains all the core logic, functions, and types. This is a library crate and can be imported.
2. src/main.rs: A thin wrapper that imports the library crate and calls its entry point (e.g., a run function).

By structuring your project this way, you can write comprehensive integration tests against src/lib.rs to verify your application's core logic.

Summary and Key Takeaways

• Command Separation: Use a double dash (--) to separate arguments meant for Cargo from arguments meant for the compiled test binary (e.g., cargo test -- [binary_options]).
• Serial Execution: Run tests serially using cargo test -- --test-threads=1 when tests share state or access the same resources.
• Output Capture: By default, Rust captures and hides standard output for passing tests. Use cargo test -- --show-output to display output for all tests.
• Test Filtering: Run a subset of tests by passing a partial name match to Cargo (e.g., cargo test add).
• Ignoring Tests: Mark slow or resource-intensive tests with the #[ignore] attribute, and run them using cargo test -- --ignored.
• Unit Test Location: Place unit tests in the same file as your production code inside a module named tests annotated with #[cfg(test)].
• Private Testing: Unit tests can test private functions because child modules in Rust have access to all items in their parent modules.
• Integration Test Location: Place integration tests in a tests/ directory at the root of your project. These tests can only access your library's public API.
• Shared Test Code: To share helper code across integration tests without creating empty test suites, place the code in a subdirectory (e.g., tests/common/mod.rs).
• Binary Testing: You cannot write integration tests for a binary-only crate (src/main.rs). To work around this, extract your core logic into a library crate (src/lib.rs) and keep your binary crate as a thin wrapper.