Crust of Rust: Build Scripts and Foreign-Function Interfaces (FFI)

This chapter explores how Rust programs interface with native C-compatible libraries and how Cargo's build script mechanism (build.rs) automates the compilation, linking, and code-generation steps required for cross-language integration. Through a practical walkthrough of binding to the libsodium cryptography library, we will demystify the mechanics of Foreign-Function Interfaces (FFI), opaque pointers, memory management across language boundaries, and the trade-offs of automated binding generation.

1. Why Do We Need Build Scripts?

In the Rust ecosystem, Cargo handles dependency resolution and compilation for pure Rust code seamlessly. However, real-world software often relies on legacy codebases, system libraries, or platform-specific APIs written in C, C++, or assembly.

To bridge this gap, Cargo provides a hook called a Build Script. A build script is an auxiliary Rust program that runs before your crate is compiled. It is a Cargo-specific construct rather than a feature of the Rust language itself.

Common Use Cases for Build Scripts

• Building or linking native C/C++ libraries: Compiling C source files bundled with the crate or locating pre-installed libraries on the host system.
• Code generation: Generating Rust source code dynamically (e.g., parsing a protocol buffer definition, compiling a database schema, or generating FFI bindings) and writing it to a dedicated output directory.
• Platform-specific configuration: Probing the compiler or host environment to conditionally enable features based on the target architecture, operating system, or compiler version.

2. The Build Script Mental Model & Execution Flow

A build script is simply a Rust file—conventionally named build.rs—placed in the root directory of your Cargo package (at the same level as Cargo.toml).

The Execution Workflow

When you run cargo build or cargo run, Cargo executes the compilation process in two distinct phases:

Key Architectural Constraints

1. Host vs. Target: The build script is compiled and run on the host machine (the machine doing the compilation). If you are cross-compiling (e.g., compiling on x86_64 Linux for an ARM64 Android target), build.rs runs as an x86_64 binary. It must not assume that the host and target share the same architecture, pointer size, or library paths.
2. Sandboxing and Side Effects: Currently, build scripts are un-sandboxed. They can access the network, read and write arbitrary files, or execute system commands. This makes them highly powerful but introduces security risks and potential issues for reproducible build environments.
3. No Modification of src/: A build script must never modify files inside the src/ directory. Doing so interferes with Cargo's change-detection algorithms, causing the project to rebuild continuously on every invocation. All generated files must be written to the designated output directory (OUT_DIR).

3. Communication Channels: Environment Variables & OUT_DIR

Because the build script and the compiler run as separate processes, they communicate primarily through environment variables and standard output.

The OUT_DIR Environment Variable

During execution, Cargo provides the build script with an environment variable named OUT_DIR. This points to a unique, writable directory nested deep inside Cargo's target/ folder.

To write generated code, the build script retrieves this path, creates the necessary files, and writes the generated Rust code into them.

// build.rs
use std::env;
use std::fs;
use std::path::Path;

fn main() {
    // Retrieve the output directory provided by Cargo
    let out_dir = env::var("OUT_DIR").unwrap();
    let dest_path = Path::new(&out_dir).join("generated_constants.rs");
    
    // Write a simple generated Rust module
    fs::write(
        &dest_path,
        "pub const GENERATED_VERSION: &str = \"1.0.18\";"
    ).unwrap();
}

Consuming Generated Code in lib.rs

To use this generated code in your main library or binary, use the include! macro combined with the concat! and env! macros.

// src/lib.rs

// The env! macro reads the environment variable set *at compile time*
include!(concat!(env!("OUT_DIR"), "/generated_constants.rs"));

pub fn print_version() {
    println!("Version: {}", GENERATED_VERSION);
}

Note on env! vs. std::env::var:

• std::env::var("OUT_DIR") is a runtime lookup. It queries the environment of the machine running the final compiled binary. At runtime, Cargo is no longer present, and OUT_DIR will not exist.
• env!("OUT_DIR") is a compiler directive. It reads the environment variable during compilation and inlines the string literal directly into the AST.

4. Instructing Cargo: Standard Output Commands

A build script communicates instructions back to Cargo by printing specially formatted strings to standard output. These lines must begin with the prefix cargo:.

If a build script completes successfully, Cargo swallows its standard output. If the build script fails (exits with a non-zero status or panics), Cargo prints the standard output and standard error to the terminal to assist in debugging.

Change Detection and Rebuild Behavior

By default, if a crate contains a build script, Cargo will rerun it on every single build to ensure correctness. This can severely degrade compile times.

To opt out of this behavior, the build script must emit cargo:rerun-if-changed or cargo:rerun-if-env-changed instructions. Once emitted, Cargo will cache the build script's output and only rerun it if the specified files or environment variables change.

// build.rs
fn main() {
    // Re-run this build script only if wrapper.h changes
    println!("cargo:rerun-if-changed=wrapper.h");
    
    // Re-run if the system-level library path environment variable changes
    println!("cargo:rerun-if-env-changed=SODIUM_LIB_DIR");
}

5. The links Attribute and the -sys Crate Pattern

When linking against native libraries, we must prevent multiple crates in a dependency graph from linking against the same native library. Doing so can lead to duplicate symbol errors, linker collisions, or undefined behavior.

The links Attribute

To prevent this, Cargo provides the links attribute in Cargo.toml.

# Cargo.toml
[package]
name = "libsodium-sys"
version = "0.1.0"
links = "sodium" # Tells Cargo this crate links against libsodium

If two different crates in a single dependency graph declare links = "sodium", Cargo will abort the build with an error before compilation begins.

The -sys Crate Pattern

To maintain a clean ecosystem, the Rust community adheres to the -sys crate pattern:

1. The -sys Crate: A minimal, low-level crate whose sole responsibility is to locate, compile, link, and expose raw, unsafe C bindings. It declares the links attribute.
2. The High-Level Crate: A safe, idiomatic Rust wrapper built on top of the -sys crate. It exposes clean, safe Rust APIs (using types like Result, slices, and lifetimes) and does not declare the links attribute.

6. Locating and Linking Libraries: System Probe vs. Vendoring

A -sys crate's build script typically uses one of two strategies to locate the target C library: System Probing or Vendoring.

Strategy A: System Probing (via pkg-config)

On Unix-like systems, libraries install .pc files that describe their location, version, and required linker flags. The pkg-config crate wraps this utility.

// build.rs
fn main() {
    // Probe the system for a library named "libsodium" that is at least version 1.0.18
    match pkg_config::Config::new().atleast_version("1.0.18").probe("libsodium") {
        Ok(library) => {
            // pkg_config automatically emits the correct cargo:rustc-link-lib and
            // cargo:rustc-link-search instructions based on the .pc file.
            println!("cargo:warning=Found system libsodium version: {}", library.version);
        }
        Err(e) => {
            panic!("Failed to find system libsodium: {}", e);
        }
    }
}

Strategy B: Vendoring (Building from Source)

If the library is not installed on the host system, the crate can bundle the C source code (often as a Git submodule) and compile it on the fly using the cc crate.

// build.rs (Simplified Vendoring)
fn main() {
    println!("cargo:rerun-if-changed=c_src/helper.c");
    
    // Compile the bundled C code using the host's C compiler
    cc::Build::new()
        .file("c_src/helper.c")
        .include("c_src")
        .compile("helper"); // Outputs static library libhelper.a
}

Comparison of Linking Strategies

7. Writing FFI Bindings: Manual Declarations vs. bindgen

Once a library is linked, we must declare its functions and types in Rust so the compiler can generate the correct calls.

Manual FFI Declarations

For small, stable APIs, you can write the declarations by hand using an extern "C" block.

// src/lib.rs
#[repr(C)]
pub struct GitRevSpec {
    pub from: *mut GitObject,
    pub to: *mut GitObject,
    pub flags: u32,
}

extern "C" {
    // Declares a function defined in the linked C library
    pub fn git_revparse(
        revspec: *mut GitRevSpec,
        repo: *mut GitRepository,
        spec: *const libc::c_char,
    ) -> libc::c_int;
}

Automated Bindings with bindgen

For large libraries, manual translation is tedious and error-prone. The bindgen tool parses C header files (.h) using libclang and automatically generates the equivalent Rust declarations.

1. Create a C Header Wrapper

Create a wrapper.h file containing the headers you want to bind:

// wrapper.h
#include <sodium.h>

2. Configure bindgen in build.rs

Add bindgen as a build dependency in Cargo.toml and invoke it in build.rs:

// build.rs
use std::env;
use std::path::PathBuf;

fn main() {
    println!("cargo:rerun-if-changed=wrapper.h");

    // Generate bindings
    let bindings = bindgen::Builder::default()
        .header("wrapper.h")
        // Restrict generation to specific APIs to keep bindings clean and stable
        .allowlist_function("sodium_init")
        .allowlist_function("crypto_generichash")
        .allowlist_var("crypto_generichash_.*")
        .generate()
        .expect("Unable to generate bindings");

    // Write the bindings to the $OUT_DIR/bindings.rs file.
    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("bindings.rs"))
        .expect("Couldn't write bindings!");
}

8. Memory Layout & Opaque Types in FFI

When passing data between Rust and C, we must ensure that both languages agree on the memory layout of structs and pointers.

The Importance of #[repr(C)]

By default, the Rust compiler does not guarantee a stable field ordering or alignment for structs. To force Rust to lay out a struct exactly how a C compiler would, you must annotate it with #[repr(C)].

// Without #[repr(C)], Rust might reorder these fields to minimize padding.
#[repr(C)]
pub struct CCompatibleStruct {
    pub byte: u8,      // 1 byte
    // 3 bytes of padding inserted here to align the next field
    pub integer: u32,  // 4 bytes
}

Representing Opaque C Structs

C libraries often use "opaque pointers" to hide implementation details. A header might declare a pointer to a struct without defining its fields:

// C Header
typedef struct git_commit git_commit;
void git_commit_free(git_commit *commit);

In Rust, we must represent this type in a way that prevents users from constructing it directly, while still allowing us to pass pointers to it.

The Empty Enum Pattern

The idiomatic way to represent an opaque type in Rust is an empty enum:

// An enum with no variants cannot be instantiated in Rust.
pub enum GitCommit {}

extern "C" {
    // We can safely pass raw pointers to this uninhabited type.
    pub fn git_commit_free(commit: *mut GitCommit);
}

Why not use an empty struct? If we declared pub struct GitCommit;, a user could write let commit = GitCommit; and instantiate it on the stack. An empty enum has no valid states, making instantiation impossible and guaranteeing compile-time safety.

9. Step-by-Step Walkthrough: Binding to libsodium

Let's build a safe, idiomatic Rust wrapper around libsodium's initialization and generic hashing APIs.

Step 1: Project Setup

Create a new library crate:

cargo new libsodium-sys --lib

Update Cargo.toml to include the build dependencies and metadata:

[package]
name = "libsodium-sys"
version = "0.1.0"
edition = "2021"
links = "sodium"

[dependencies]
libc = "0.2"

[build-dependencies]
pkg-config = "0.3"
bindgen = "0.69"

Step 2: Write the Build Script

Create build.rs in the root directory:

// build.rs
use std::env;
use std::path::PathBuf;

fn main() {
    println!("cargo:rerun-if-changed=wrapper.h");

    // 1. Locate libsodium using pkg-config
    let library = pkg_config::Config::new()
        .atleast_version("1.0.18")
        .probe("libsodium")
        .expect("libsodium must be installed on the system");

    // 2. Configure and run bindgen
    let mut builder = bindgen::Builder::default()
        .header("wrapper.h")
        .allowlist_function("sodium_init")
        .allowlist_function("crypto_generichash")
        .allowlist_var("crypto_generichash_BYTES.*");

    // Pass include paths from pkg-config to clang
    for path in library.include_paths {
        builder = builder.clang_arg(format!("-I{}", path.display()));
    }

    let bindings = builder.generate().expect("Failed to generate bindings");

    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("bindings.rs"))
        .expect("Failed to write bindings.rs");
}

Create wrapper.h in the root directory:

// wrapper.h
#include <sodium.h>

Step 3: Expose Low-Level Bindings in lib.rs

Include the generated bindings inside a private module, then selectively expose them or build a safe wrapper.

// src/lib.rs
#![allow(non_upper_case_globals)]
#![allow(non_camel_case_types)]
#![allow(non_snake_case)]

mod ffi {
    include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
}

Step 4: Design the Safe, Idiomatic Wrapper

libsodium requires calling sodium_init() before invoking any other cryptographic functions. We can enforce this invariant at compile time using a token struct.

// src/lib.rs
use std::mem::MaybeUninit;

/// A token proving that libsodium has been successfully initialized.
#[derive(Clone, Copy, Debug)]
#[non_exhaustive] // Prevents external construction
pub struct Sodium;

impl Sodium {
    /// Initializes libsodium. Safe to call multiple times.
    pub fn new() -> Result<Self, ()> {
        // Safety: sodium_init is safe to call concurrently.
        let res = unsafe { ffi::sodium_init() };
        if res < 0 {
            Err(())
        } else {
            Ok(Sodium)
        }
    }

    /// Hashes a message using the Blake2b-based generic hash API.
    pub fn generic_hash<'a>(
        self,
        input: &[u8],
        output: &'a mut [MaybeUninit<u8>],
        key: Option<&[u8]>,
    ) -> Result<&'a mut [u8], ()> {
        let out_len = output.len();

        // Enforce libsodium's size constraints at runtime
        let min_bytes = ffi::crypto_generichash_BYTES_MIN as usize;
        let max_bytes = ffi::crypto_generichash_BYTES_MAX as usize;
        if out_len < min_bytes || out_len > max_bytes {
            return Err(());
        }

        let (key_ptr, key_len) = match key {
            Some(k) => {
                let k_len = k.len();
                let min_key = ffi::crypto_generichash_KEYBYTES_MIN as usize;
                let max_key = ffi::crypto_generichash_KEYBYTES_MAX as usize;
                if k_len < min_key || k_len > max_key {
                    return Err(());
                }
                (k.as_ptr(), k_len)
            }
            None => (std::ptr::null(), 0),
        };

        // Safety:
        // 1. We verified that `output` length is within valid bounds.
        // 2. We verified that the optional `key` length is within valid bounds.
        // 3. `sodium_init` has run (guaranteed by the possession of `self`).
        let res = unsafe {
            ffi::crypto_generichash(
                output.as_mut_ptr() as *mut u8,
                out_len,
                input.as_ptr(),
                input.len() as u64,
                key_ptr,
                key_len,
            )
        };

        if res == 0 {
            // Safety: crypto_generichash guarantees that it has fully initialized
            // the output buffer up to out_len bytes upon returning 0.
            unsafe {
                Ok(std::slice::from_raw_parts_mut(
                    output.as_mut_ptr() as *mut u8,
                    out_len,
                ))
            }
        } else {
            Err(())
        }
    }
}

Step 5: Write an Integration Test

Verify that our safe wrapper correctly computes a hash and matches standard test vectors.

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_sodium_hashing() {
        let sodium = Sodium::new().expect("Failed to initialize");
        
        let message = b"Hello, world!";
        let mut out_buffer = [MaybeUninit::uninit(); 32];
        
        let hash = sodium
            .generic_hash(message, &mut out_buffer, None)
            .expect("Hashing failed");

        assert_eq!(hash.len(), 32);
        
        // Verify against known Blake2b 256-bit hash of "Hello, world!"
        let expected_hex = "dffd6021bb2bd5b0af6762d8ca0d1a3209105824750a7074a8d6cba2265953c9";
        assert_eq!(hex::encode(hash), expected_hex);
    }
}

10. Reverse FFI: Calling Rust from C & C++ Integration

Sometimes, you need to compile Rust code into a static or dynamic library and expose it to a C or C++ application.

Exposing Rust Functions to C

To make a Rust function callable from C, you must:

1. Change its calling convention to C using extern "C".
2. Disable Rust's name-mangling using #[no_mangle].
3. Use C-compatible types for all arguments and return values.

// src/lib.rs

/// Computes the length of a C-style null-terminated string.
/// 
/// # Safety
/// The caller must ensure that `s` points to a valid, null-terminated string.
#[no_mangle]
pub unsafe extern "C" fn rust_strlen(s: *const libc::c_char) -> libc::size_t {
    if s.is_null() {
        return 0;
    }
    let mut len = 0;
    while *s.add(len) != 0 {
        len += 1;
    }
    len
}

Generating C Headers with cbindgen

Just as bindgen generates Rust code from C headers, cbindgen parses your Rust library and automatically generates a C/C++ header file (.h) containing the corresponding function declarations and struct definitions.

Safe C++ Integration with cxx

If you are specifically bridging Rust and C++, standard FFI is highly limiting because C++ features like templates, classes, and exceptions do not map cleanly to the C ABI.

The cxx crate provides a safe, declarative bridge between Rust and C++. It enforces safety invariants across the boundary, handles string and vector conversions automatically, and avoids the need for raw pointers in many scenarios.

// Example cxx bridge declaration
#[cxx::bridge]
mod ffi {
    unsafe extern "C++" {
        include!("path/to/cpp_header.h");
        type CoolCppClass;
        fn do_something(self: &CoolCppClass, val: i32) -> UniquePtr<CoolCppClass>;
    }

    extern "Rust" {
        fn call_back_into_rust(data: &str) -> bool;
    }
}

11. Compiler Feature Detection & autocfg

Build scripts are also used to detect capabilities of the compiler currently building the crate. This allows library authors to adopt modern language features (like const generics or new standard library APIs) while maintaining backwards compatibility with older compiler versions.

The autocfg crate simplifies this process by attempting to compile small test snippets inside the build script.

// build.rs
fn main() {
    let ac = autocfg::AutoCfg::new().unwrap();
    
    // Check if the compiler supports const generics
    if ac.probe_rustc_version(1, 51).unwrap() {
        autocfg::emit("has_const_generics");
    }
}

In your library code, you can then use conditional compilation:

// src/lib.rs
#[cfg(has_const_generics)]
pub fn process_array<const N: usize>(arr: [u8; N]) {
    // Modern implementation
}

#[cfg(not(has_const_generics))]
pub fn process_array(arr: &[u8]) {
    // Fallback implementation
}

12. Summary and Key Takeaways

• Build scripts (build.rs) are compiled and executed on the host machine before the target crate is compiled. They are used for linking native libraries, generating code, and probing compiler features.
• OUT_DIR is the only directory a build script should write files to. Crate code accesses this directory at compile time using the env!("OUT_DIR") macro.
• Standard output commands prefixed with cargo: allow build scripts to pass linker search paths (rustc-link-search), link libraries (rustc-link-lib), and define compilation flags (rustc-cfg).
• The links attribute in Cargo.toml ensures that only one crate in the dependency graph links to a specific native library, preventing duplicate symbol errors.
• The -sys crate pattern separates raw, unsafe FFI bindings (which declare the links attribute) from safe, idiomatic Rust wrappers.
• Opaque C pointers are best represented in Rust using empty enums (e.g., pub enum OpaqueType {}), which prevents accidental stack instantiation while allowing pointer manipulation.
• MaybeUninit<T> is highly useful when passing uninitialized buffers to C functions that write data, satisfying Rust's safety guarantees without incurring the performance cost of zero-initializing memory.
• bindgen and cbindgen automate the bidirectional generation of bindings between Rust and C, while cxx provides a highly specialized, safe bridge for C++ integration.