A benchmark definition. Can be a single measurement, a comparison of multiple implementations, a scaling analysis, or a named group.

Benchmark a single function.

benchmark "List.map" <|
    \_ -> List.map ((+) 1) (List.range 1 100)

Compare two implementations. Automatically verifies they produce the same result before benchmarking.

compare "reversing"
    ( "List.reverse", \_ -> List.reverse data )
    ( "List.foldl (::)", \_ -> List.foldl (::) [] data )

Compare multiple implementations with a shared runner function. Automatically verifies that all implementations produce the same result.

rank "mapToList"
    (\mapToList -> mapToList negate ints1To100)
    [ ( "with foldr", Array.Extra.MapToList.withFoldr )
    , ( "with toIndexedList", Array.Extra.MapToList.withListMap )
    ]

Organize benchmarks into a named group.

describe "Array.Extra"
    [ rank "mapToList" ...
    , rank "filterMap" ...
    ]

Run multiple implementations at different input sizes. Useful for understanding algorithmic complexity. Automatically verifies correctness at each size.

scale "sorting"
    [ 10, 100, 1000 ]
    (\n -> List.range 1 n |> List.reverse)
    [ ( "List.sort", List.sort )
    , ( "mergeSort", mergeSort )
    ]

A simple series of named benchmarks. Does not verify equality (use rank if you want correctness checks).

series "approaches"
    [ ( "List.sort", \_ -> List.sort data )
    , ( "List.sortBy identity", \_ -> List.sortBy identity data )
    ]

Like compare, but generates input from a fuzzer. The fuzzer is also used for the correctness check (fuzz test).

compareFuzz "reversing"
    (Fuzz.list Fuzz.int)
    ( "List.reverse", List.reverse )
    ( "List.foldl (::)", List.foldl (::) [] )

Like rank, but generates input from a fuzzer.

rankFuzz "mapToList"
    (Fuzz.array Fuzz.int)
    (\mapToList array -> mapToList negate array)
    [ ( "with foldr", Array.Extra.MapToList.withFoldr )
    , ( "with toIndexedList", Array.Extra.MapToList.withListMap )
    ]

Like scale, but generates input from a size-dependent fuzzer.

scaleFuzz "sorting"
    [ 10, 100, 1000 ]
    (\n -> Fuzz.listOfLength n Fuzz.int)
    [ ( "List.sort", List.sort )
    , ( "mergeSort", mergeSort )
    ]

Sample a fuzzer with a specific seed. Use this in elm repl to reproduce the exact input that was used during a benchmark run.

When you see unexpected results from a fuzz-based benchmark, copy the seed from the benchmark output and run in elm repl:

> import Bench
> import Fuzz
> Bench.sampleFuzzer 1234567 (Fuzz.list Fuzz.int)
[3, -7, 42, ...] : List Int

This gives you the exact value the fuzzer produced during that run.

(We cannot print the value in the benchmark run, since unlike elm-test, elm-bench runs in --optimize mode and so doesn't have access to Debug.log.)

Disable the automatic equality check for a benchmark. Use this when implementations intentionally produce different results (e.g. different orderings that are both valid).

rank "set operations"
    (\toList -> toList mySet)
    [ ( "Set.toList", Set.toList )
    , ( "custom", customToList ) -- different order, both valid
    ]
    |> skipEqualityCheck

Filter benchmarks by name (case-insensitive substring match). Groups are kept if any child matches. Returns Nothing if nothing matches.

Convert to elm-explorations/benchmark's Benchmark type for execution. The seed is used to generate deterministic fuzzer-based input.

Extract the correctness test from a benchmark tree. The CLI runner executes this before benchmarking to catch broken implementations early.

Encode the benchmark tree structure as JSON. Used by the CLI to understand the tree shape for filtering and display.