Introduction

Fuzz testing is a software testing technique used to find security and stability issues by providing pseudo-random data as input to the software.

Rust is a high performance, safe, general purpose programming language.

This book demonstrates how to perform fuzz testing for software written in Rust.

There are two tools for fuzzing Rust code documented in this book: afl.rs and cargo-fuzz.

The source of this book is available on GitHub at https://github.com/rust-fuzz/book.

Fuzzing with cargo-fuzz

cargo-fuzz is the recommended tool for fuzz testing Rust code.

cargo-fuzz is itself not a fuzzer, but a tool to invoke a fuzzer. Currently, the only fuzzer it supports is libFuzzer (through the libfuzzer-sys crate), but it could be extended to support other fuzzers in the future.

Setup

Requirements

libFuzzer needs LLVM sanitizer support, so this only works on x86-64 Linux, x86-64 macOS and Apple-Silicon (aarch64) macOS for now. Requires a C++ compiler with C++11 support. Rust provides multiple compilers. This project requires the nightly compiler since it uses the -Z compiler flag to provide address sanitization. Assuming you used rustup to install Rust, you can check your default compiler with:

$ rustup default
stable-x86_64-unknown-linux-gnu (default) # Not the compiler we want.

To change to the nightly compiler:

$ rustup install nightly
$ rustup default nightly
nightly-x86_64-unknown-linux-gnu (default) # The correct compiler.

Installing

cargo install cargo-fuzz

Upgrading

cargo install --force cargo-fuzz

Tutorial

For this tutorial, we're going to be fuzzing the URL parsing crate rust-url. Our goal here is to find some input generated by the fuzzer such that, when passed to Url::parse, it causes some sort of panic or crash to happen.

To start, clone the rust-url repository and change directories into it:

git clone https://github.com/servo/rust-url.git
cd rust-url

Although we could fuzz the latest commit on master, we're going to checkout a specific revision that is known to have a parsing bug:

git checkout bfa167b4e0253642b6766a7aa74a99df60a94048

Initialize cargo-fuzz:

cargo fuzz init

This will create a directory called fuzz_targets which will contain a collection of fuzzing targets. It is generally a good idea to check in the files generated by init. Each fuzz target is a Rust program that is given random data and tests a crate (in this case, rust-url). cargo fuzz init automatically generates an initial fuzz target for us. Use cargo fuzz list to view the list of all existing fuzz targets:

cargo fuzz list

The source code for this fuzz target by default lives in fuzz/fuzz_targets/<fuzz target name>.rs. Open that file and edit it to look like this:

#![no_main]
#[macro_use] extern crate libfuzzer_sys;
extern crate url;

fuzz_target!(|data: &[u8]| {
    if let Ok(s) = std::str::from_utf8(data) {
        let _ = url::Url::parse(s);
    }
});

libFuzzer is going to repeatedly call the body of fuzz_target!() with a slice of pseudo-random bytes, until your program hits an error condition (segfault, panic, etc). Write your fuzz_target!() body to hit the entry point you need.

Since the generated data is a byte slice, we'll need to convert it to a UTF-8 &str since rust-url expects that when parsing.

To begin fuzzing, run:

cargo fuzz run <fuzz target name>

Congratulations, you're fuzzing! The output you're seeing is generated by the fuzzer libFuzzer. To learn more about what the output means see the 'output' section in the libFuzzer documentation.

If you leave it going for long enough you'll eventually discover a crash. The output would look something like this:

...
#56232	NEW    cov: 2066 corp: 110/4713b exec/s: 11246 rss: 170Mb L: 42 MS: 1 EraseBytes-
#58397	NEW    cov: 2069 corp: 111/4755b exec/s: 11679 rss: 176Mb L: 42 MS: 1 EraseBytes-
#59235	NEW    cov: 2072 corp: 112/4843b exec/s: 11847 rss: 178Mb L: 88 MS: 4 InsertByte-ChangeBit-CopyPart-CopyPart-
#60882	NEW    cov: 2075 corp: 113/4953b exec/s: 12176 rss: 183Mb L: 110 MS: 1 InsertRepeatedBytes-
thread '<unnamed>' panicked at 'index out of bounds: the len is 1 but the index is 1', src/host.rs:105
note: Run with `RUST_BACKTRACE=1` for a backtrace.
==70997== ERROR: libFuzzer: deadly signal
    #0 0x1097c5500 in __sanitizer_print_stack_trace (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x62500)
    #1 0x108383d1b in fuzzer::Fuzzer::CrashCallback() (fuzzer_script_1:x86_64+0x10002fd1b)
    #2 0x108383ccd in fuzzer::Fuzzer::StaticCrashSignalCallback() (fuzzer_script_1:x86_64+0x10002fccd)
    #3 0x1083d19c7 in fuzzer::CrashHandler(int, __siginfo*, void*) (fuzzer_script_1:x86_64+0x10007d9c7)
    ...
    #33 0x10838b393 in fuzzer::Fuzzer::Loop() (fuzzer_script_1:x86_64+0x100037393)
    #34 0x1083650ec in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (fuzzer_script_1:x86_64+0x1000110ec)
    #35 0x108396c3f in main (fuzzer_script_1:x86_64+0x100042c3f)
    #36 0x7fff91552234 in start (libdyld.dylib:x86_64+0x5234)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 2 InsertByte-EraseBytes-; base unit: 3c4fc9770beb5a732d1b78f38cc8b62b20cb997c
0x68,0x74,0x74,0x70,0x3a,0x2f,0x2f,0x5b,0x3a,0x5d,0x3a,0x78,0xc5,0xa4,0x1,0x3a,0x7f,0x1,0x59,0xc5,0xa4,0xd,0x78,0x78,0x3a,0x78,0x69,0x3a,0x0,0x69,0x3a,0x5c,0xd,0x7e,0x78,0x40,0x0,0x25,0xa,0x0,0x29,0x20,
http://[:]:x\xc5\xa4\x01:\x7f\x01Y\xc5\xa4\x0dxx:xi:\x00i:\\\x0d~x@\x00%\x0a\x00)
artifact_prefix='/private/tmp/rust-url/fuzz/artifacts/fuzzer_script_1/'; Test unit written to /home/user/rust-url/fuzz/artifacts/fuzzer_script_1/crash-e9b1b5183e46a288c25a2a073262cdf35408f697
Base64: aHR0cDovL1s6XTp4xaQBOn8BWcWkDXh4OnhpOgBpOlwNfnhAACUKACkg

The line in the output that starts with http is the input that causes a panic in rust-url.

Guide

All available commands available for cargo-fuzz:

cargo fuzz --help

Run a target:

cargo fuzz run <fuzz target name>

Cargo features

It is possible to fuzz crates with different configurations of Cargo features by using the command line options --features, --no-default-features and --all-features. Note that these options control the fuzz_targets crate; you will need to forward them to the crate being fuzzed by e.g. adding the following to fuzz_targets/Cargo.toml:

[features]
unsafe = ["project/unsafe"]

#[cfg(fuzzing)]

Every crate instrumented for fuzzing -- the fuzz_targets crate, the project crate, and their entire dependency tree -- is compiled with the --cfg fuzzing rustc option. This makes it possible to disable code paths that prevent fuzzing from working, e.g. verification of cryptographic signatures, with a simple #[cfg(not(fuzzing))], and without the need for an externally visible Cargo feature that must be maintained throughout every dependency.

#[cfg(fuzzing_repro)]

When you run cargo fuzz run <fuzz target name> <crash file>, every crate is compiled with the --cfg fuzzing_repro rustc option. This allows you to leave debugging statements in your fuzz targets behind a #[cfg(fuzzing_repro)]:

#![allow(unused)]
fn main() {
#[cfg(fuzzing_repro)]
eprintln!("Input data: {}", expensive_pretty_print(&data));
}

libFuzzer configuration options

See all the libFuzzer options:

cargo fuzz run <fuzz target name> -- -help=1

For example, to generate only ASCII inputs, run:

cargo fuzz run <fuzz target name> -- -only_ascii=1

Structure-Aware Fuzzing

Not every fuzz target wants to take a buffer of raw bytes as input. We might want to only feed it well-formed instances of some structured data. Luckily, the libfuzzer-sys crate enables us to define fuzz targets that take any kind of type, as long as it implements the Arbitrary trait.

libfuzzer_sys::fuzz_target!(|input: AnyTypeThatImplementsArbitrary| {
    // Use `input` here...
})

The arbitrary crate implements Arbitrary for nearly all the types in std, including collections like Vec and HashMap as well as things like String and PathBuf.

For convenience, the libfuzzer-sys crate re-exports the arbitrary crate as libfuzzer_sys::arbitrary. You can also enable #[derive(Arbitrary)] either by

• enabling the arbitary crate's "derive" feature, or
• (equivalently) enabling the libfuzzer-sys crate's "arbitrary-derive" feature.

See the arbitrary crate's documentation for more details.

This section concludes with two examples of structure-aware fuzzing:

1. Example 1: Fuzzing Color Conversions

2. Example 2: Fuzzing Allocation API Calls

Example 1: Fuzzing Color Conversions

Let's say we are working on a color conversion library that can turn RGB colors into HSL and back again.

Enable Deriving Arbitrary

We are lazy, and don't want to implement Arbitrary by hand, so we want to enable the arbitrary crate's "derive" cargo feature. This lets us get automatic Arbitrary implementations with #[derive(Arbitrary)].

Because the Rgb type we will be deriving Arbitrary for is in our main color conversion crate, we add this to our main Cargo.toml.

# Cargo.toml

[dependencies]
arbitrary = { version = "1", optional = true, features = ["derive"] }

Derive Arbitrary for our Rgb Type

In our main crate, when the "arbitrary" cargo feature is enabled, we derive the Arbitrary trait:

// src/lib.rs

#[derive(Clone, Debug)]
#[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
pub struct Rgb {
    pub r: u8,
    pub g: u8,
    pub b: u8,
}

Enable the Main Project's "arbitrary" Cargo Feature for the Fuzz Targets

Because we made arbitrary an optional dependency in our main color conversion crate, we need to enable that feature for our fuzz targets to use it.

# fuzz/Cargo.toml

[dependencies]
my_color_conversion_library = { path = "..", features = ["arbitrary"] }

Add the Fuzz Target

We need to add a new fuzz target to our project:

$ cargo fuzz add rgb_to_hsl_and_back

Implement the Fuzz Target

Finally, we can implement our fuzz target that takes arbitrary RGB colors, converts them to HSL, and then converts them back to RGB and asserts that we get the same color as the original! Because we implement Arbitrary for our Rgb type, our fuzz target can take instances of Rgb directly:

// fuzz/fuzz_targets/rgb_to_hsl_and_back.rs

libfuzzer_sys::fuzz_target!(|color: Rgb| {
    let hsl = color.to_hsl();
    let rgb = hsl.to_rgb();

    // This should be true for all RGB -> HSL -> RGB conversions!
    assert_eq!(color, rgb);
});

Example 2: Fuzzing Allocator API Calls

Imagine, for example, that we are fuzzing our own malloc and free implementation. We want to make a sequence of valid allocation and deallocation API calls. Additionally, we want that sequence to be guided by the fuzzer, so it can use its insight into code coverage to maximize the amount of code we exercise during fuzzing.

Add the Fuzz Target

First, we add a new fuzz target to our project:

$ cargo fuzz add fuzz_malloc_free

Enable Deriving Arbitrary

Like the color conversion example above, we don't want to write our Arbitrary implementation by hand, we want to derive it.

# fuzz/Cargo.toml

[dependencies]
libfuzzer-sys = { version = "0.4.0", features = ["arbitrary-derive"] }

Define an AllocatorMethod Type and Derive Arbitrary

Next, we define an enum that represents either a malloc, a realloc, or a free:

// fuzz_targets/fuzz_malloc_free.rs

use libfuzzer_sys::arbitrary::Arbitrary;

#[derive(Arbitrary, Debug)]
enum AllocatorMethod {
    Malloc {
        // The size of allocation to make.
        size: usize,
    },
    Free {
        // Free the index^th allocation we've made.
        index: usize
    },
    Realloc {
        // We will realloc the index^th allocation we've made.
        index: usize,
        // The new size of the allocation.
        new_size: usize,
    },
}

Write a Fuzz Target That Takes a Sequence of AllocatorMethods

Finally, we write a fuzz target that takes a vector of AllocatorMethods and interprets them by making the corresponding malloc, realloc, and free calls. This works because Vec<T> implements Arbitrary when T implements Arbitrary.

// fuzz/fuzz_targets/fuzz_malloc_free.rs

libfuzzer_sys::fuzz_target!(|methods: Vec<AllocatorMethod>| {
    let mut allocs = vec![];

    // Interpret the fuzzer-provided methods and make the
    // corresponding allocator API calls.
    for method in methods {
        match method {
            AllocatorMethod::Malloc { size } => {
                let ptr = my_allocator::malloc(size);
                allocs.push(ptr);
            }
            AllocatorMethod::Free { index } => {
                match allocs.get(index) {
                    Some(ptr) if !ptr.is_null() => {
                        my_allocator::free(ptr);
                        allocs[index] = std::ptr::null();
                    }
                    _ => {}
                }
            }
            AllocatorMethod::Realloc { index, size } => {
                match allocs.get(index) {
                    Some(ptr) if !ptr.is_null() => {
                        let new_ptr = my_allocator::realloc(ptr, size);
                        allocs[index] = new_ptr;
                    }
                    _ => {}
                }
            }
        }
    }

    // Free any remaining allocations.
    for ptr in allocs {
        if !ptr.is_null() => {
            my_allocator::free(ptr);
        }
    }
});

Code Coverage

Visualizing code coverage helps you understand which code paths are being fuzzed and — more importantly — which aren't. To help the fuzzer exercise new code paths, you can look at what it is failing to reach and then either add new seed inputs to the corpus, or tweak the fuzz target. This chapter describes how to generate coverage reports for your fuzz target and its current corpus.

Prerequisites

First, install the LLVM-coverage tools as described in the rustc book.

If you are using a non-nightly toolchain as your default toolchain, remember to install the rustup components for the nightly toolchain instead of the default (rustup component add --toolchain nightly llvm-tools-preview ...).

You must also have cargo fuzz version 0.10.0 or newer to use the cargo fuzz coverage subcommand.

Generate Code-Coverage Data

After you fuzzed your program, use the coverage command to generate precise source-based code coverage information:

$ cargo fuzz coverage <target> [corpus dirs] [-- <args>]

This command

• compiles your project using the -Cinstrument-coverage Rust compiler flag,

• runs the program without fuzzing on the provided corpus (if no corpus directory is provided it uses fuzz/corpus/<target> by default),

• for each input file in the corpus, generates raw coverage data in the fuzz/coverage/<target>/raw subdirectory, and

• merges the raw files into a coverage.profdata file located in the fuzz/coverage/<target> subdirectory.

Afterwards, you can use the generated coverage.profdata file to generate coverage reports and visualize code-coverage information as described in the rustc book.

Example

Suppose we have a my_compiler fuzz target for which we want to visualize code coverage.

1. Run the fuzzer on the my_compiler target:

   $ cargo fuzz run my_compiler
   

2. Produce code-coverage information:

   $ cargo fuzz coverage my_compiler
   

3. Visualize the coverage data in HTML:

   $ cargo cov -- show fuzz/target/<target triple>/release/my_compiler \
       --format=html \
       -instr-profile=fuzz/coverage/my_compiler/coverage.profdata \
       > index.html
   

   There are many visualization and coverage-report options available (see llvm-cov show --help).

Targets

A collection of community maintained cargo-fuzz compatible fuzz targets can be found here.

Fuzzing with afl.rs

American fuzzy lop (AFL) is a popular, effective, and modern fuzz testing tool. afl.rs allows one to run AFL on code written in the Rust programming language.

Setup

Requirements

Tools

• C compiler (e.g. gcc or clang)
• make

Platform

afl.rs works on x86-64 Linux, x86-64 macOS, and ARM64 macOS.

cargo install cargo-afl

Upgrading

cargo install --force cargo-afl

Tutorial

For this tutorial, we are going to fuzz the URL parser rust-url. Our goal here is to find some input generated by the fuzzer such that, when passed to Url::parse, it causes some sort of panic or crash to happen.

Create a fuzz target

The first thing we’ll do is create a fuzz target in the form of a Rust binary crate. AFL will call the resulting binary, supplying generated bytes to standard input that we’ll pass to Url::parse.

cargo new --bin url-fuzz-target
cd url-fuzz-target

We’ll need two dependencies in this crate:

• url: the crate we’re fuzzing
• afl: not required, but includes a couple utility functions to assist in creating fuzz targets

So add these to the Cargo.toml file:

[dependencies]
afl = "*"
url = "*"

Now we’ll need to write the source for the fuzz target in src/main.rs:

#[macro_use]
extern crate afl;
extern crate url;

fn main() {
    fuzz!(|data: &[u8]| {
        if let Ok(s) = std::str::from_utf8(data) {
            let _ = url::Url::parse(&s);
        }
    });
}

fuzz! is a utility macro provided by the afl crate that reads bytes from standard input and passes the bytes to the provided closure.

In the body of the closure, we call Url::parse with the bytes that AFL generated. If all goes well, url::Url::parse will return an Ok containing a valid Url, or an Err indicating a Url could not be constructed from the String. If Url::parse panics while parsing the String, AFL will treat it as a crash and the AFL UI will indicate as such.

One important detail about the fuzz! macro: if a panic occurs within the body of the closure, the panic will be caught and process::abort will be subsequently called. Without the call to process::abort, AFL would not consider the unwinding panic to be a crash.

Build the fuzz target

Normally, one uses cargo build to compile a Cargo-based Rust project. To get AFL to work with Rust, a few extra compiler flags need to be passed to rustc during the build process. To make this easier, there is an AFL cargo subcommand (provided by the afl crate) that automatically passes these rustc flags for us. To use it, you’ll do something like:

cargo afl <cargo command>

Since we want to build this crate, we’ll run:

cargo afl build

Provide starting inputs

AFL doesn't strictly require starting inputs, but providing some can make AFL’s job easier since it won’t need to ‘learn’ what a valid URL looks like. To do this, we'll create a directory called in with a few files (filenames don’t matter) containing valid URLs:

mkdir in
echo "tcp://example.com/" > in/url
echo "ssh://192.168.1.1" > in/url2
echo "http://www.example.com:80/foo?hi=bar" > in/url3

Start fuzzing

To begin fuzzing, we’ll run:

cargo afl fuzz -i in -o out target/debug/url-fuzz-target

The fuzz subcommand of cargo-afl is the primary interface for fuzzing Rust code with AFL. For those already familiar with AFL, the fuzz subcommand of cargo-afl is identical to running afl-fuzz.

The -i flag specifies a directory full of input files AFL will use as seeds.

The -o flag specifies a directory AFL will write all its state and results to.

The last argument target/debug/url-fuzz-target specifies the fuzz target binary AFL will call, supplying random bytes to standard input.

As soon as you run this command, you should see AFL’s interface start up:

For more information about this UI and what each of the sections mean, see this resource hosted on the AFL website.

AFL will run indefinitely, so if you want to quit, press CTRL-C.

Reproducing

Once you have a few crashes collected from running your fuzzer, you can reproduce them by passing them in manually to your test case. This is typically done via stdin. E.g. for url-fuzz-target the command would be:

cargo afl run url-fuzz-target < out/default/crashes/crash_file

where out is the -o parameter from your fuzz command and crash_file is an arbitrary file in the crashes directory.

Trophy Case

A collection of bugs found in Rust code through fuzz testing can be found in the trophy-case repository.