moodmosaic

Fuzzing meets property testing

This article is part of a four-part series on fuzzing in memory-safe languages.

The line between fuzzing and property-based testing has blurred. Modern tools combine coverage-guided exploration with semantic correctness checks, creating hybrid approaches that leverage the best of both worlds.

Convergence in Practice

Traditional distinctions are dissolving:

This convergence addresses each approach’s limitations: fuzzing’s semantic blindness and property testing’s shallow exploration.

Rust: propfuzz and bolero

Facebook’s propfuzz (now archived) demonstrated the potential by adapting proptest to work with coverage-guided fuzzers:

use propfuzz::prelude::*;
use proptest::prelude::*;
use serde_json::Value as JsonValue;

propfuzz! {
    #[test]
    fn prop_json_roundtrip(value in any::<JsonValue>()) {
        let serialized = serde_json::to_string(&value).unwrap();
        let deserialized: JsonValue = serde_json::from_str(&serialized).unwrap();
        prop_assert_eq!(value, deserialized);
    }
}

bolero represents the current state-of-the-art, supporting both property-based and fuzzing approaches:

use bolero::check;

#[test]
fn fuzz_sort() {
    check!().for_each(|input: &[u8]| {
        if let Ok(mut numbers) = parse_numbers(input) {
            let original = numbers.clone();
            numbers.sort();

            // Properties to validate.
            assert!(is_sorted(&numbers));
            assert_eq!(numbers.len(), original.len());
            assert!(numbers.iter().all(|n| original.contains(n)));
        }
    });
}

The same property could run as a quick test or long-running fuzzer, using identical logic with different exploration strategies.

The bolero framework can run the same test with different engines:

use bolero::check;

#[test]
fn test_parser() {
    check!()
        .with_type::<String>()
        .for_each(|input| {
            match parse(input) {
                Ok(ast) => {
                    // Property: pretty-printing should roundtrip.
                    assert_eq!(parse(&ast.to_string()), Ok(ast));
                }
                Err(_) => {
                    // Property: invalid input should consistently fail.
                    assert!(parse(input).is_err());
                }
            }
        });
}

# Run with libFuzzer.
bolero test --engine=libfuzzer fuzz_sort

# Run with AFL.
bolero test --engine=afl fuzz_sort

This runs with QuickCheck-style generation by default, but can switch to AFL/libFuzzer for extended campaigns.

Go: Native Fuzzing as Hybrid Approach

Go’s built-in fuzzing exemplifies convergence. Developers write property-like functions that the runtime explores with coverage guidance:

import (
    "encoding/json"
    "reflect"
    "testing"
)

func FuzzJSONRoundtrip(f *testing.F) {
    f.Add(`{"name": "test", "value": 42}`)

    f.Fuzz(func(t *testing.T, jsonStr string) {
        var data interface{}

        // Parse the input.
        if err := json.Unmarshal([]byte(jsonStr), &data); err != nil {
            return // Skip invalid JSON.
        }

        // Property: roundtrip should preserve data.
        encoded, err := json.Marshal(data)
        if err != nil {
            t.Fatalf("Marshal failed: %v", err)
        }

        var roundtrip interface{}
        if err := json.Unmarshal(encoded, &roundtrip); err != nil {
            t.Fatalf("Roundtrip unmarshal failed: %v", err)
        }

        if !reflect.DeepEqual(data, roundtrip) {
            t.Errorf("Roundtrip failed: %v != %v", data, roundtrip)
        }
    })
}

Note: reflect.DeepEqual can occasionally flag differences due to map key ordering in JSON, but it serves well as a simple roundtrip property.

This combines fuzzing’s exploration power with property testing’s semantic focus, running coverage-guided search while checking logical correctness.

Structured Generation Meets Coverage Guidance

Modern approaches use structured generators (from property testing) with coverage feedback (from fuzzing):

// Pseudo-code: assumes to_http_string() and parse_http() are implemented.
use arbitrary::Arbitrary;

#[derive(Arbitrary, Debug)]
struct HttpRequest {
    method: Method,
    path: String,
    headers: Vec<(String, String)>,
    body: Vec<u8>,
}

fuzz_target!(|req: HttpRequest| {
    let serialized = req.to_http_string();
    match parse_http(&serialized) {
        Ok(parsed) => {
            // Property: parsing should preserve semantics.
            assert_eq!(parsed.method, req.method);
            assert_eq!(parsed.path, req.path);
        }
        Err(_) => {
            // Should only fail on truly invalid input.
            panic!("Valid request failed to parse: {:?}", req);
        }
    }
});

This reaches deeper program states than raw byte fuzzing while maintaining systematic exploration.

Development Workflow Integration

Hybrid approaches fit naturally into development cycles:

  1. Quick feedback: Run property tests on each commit (seconds).
  2. Nightly campaigns: Extended fuzzing with same properties (hours).
  3. Corpus management: Save interesting inputs for regression testing.
  4. Continuous fuzzing: Run long-lived fuzzing jobs in dedicated infrastructure (e.g. OSS-Fuzz or internal fuzzing clusters) to keep discovering edge cases over time.

Future Directions

The convergence continues with emerging patterns:

Conclusion

Memory-safe languages haven’t eliminated the need for systematic testing – they’ve shifted the focus from memory corruption to logical correctness.

The future belongs to tools that combine fuzzing’s relentless exploration with property testing’s semantic precision. By writing tests that work as both quick checks and deep fuzzers, developers can confidently tackle the complexity of modern software.

Whether hunting for panics in Rust or race conditions in Go, the hybrid approach provides comprehensive coverage that neither technique achieves alone.

This concludes the series on fuzzing in memory-safe languages. The techniques explored here apply beyond Rust and Go to any system where correctness matters more than just avoiding crashes.