benchmark()

Creates a benchmark item that will be run when the benchmark target is run. The behavior of this function is identical to test() except for:

  • benchmark() has no is_parallel keyword because benchmarks are not run in parallel
  • benchmark() does not automatically add the MALLOC_PERTURB_ environment variable

Defined benchmarks can be run in a backend-agnostic way by calling meson test --benchmark inside the build dir, or by using backend-specific commands, such as ninja benchmark or msbuild RUN_TESTS.vcxproj.

Signature

# Creates a benchmark item that will be run when the benchmark target is
void benchmark(
  str                                                           name,           # The *unique* test id
  exe | jar | external_program | file | custom_tgt | custom_idx executable,     # The program to execute

  # Keyword arguments:
  args        : array[str | file | tgt | external_program]  # Arguments to pass to the executable
  depends     : array[build_tgt | custom_tgt]               # specifies that this test depends on the specified
  env         : env | array[str] | dict[str]                # environment variables to set, such as `['NAME1=value1',
  priority    : int                                         # specifies the priority of a test
  protocol    : str                                         # specifies how the test results are parsed and can
  should_fail : bool                                        # when true the test is considered passed if the
  suite       : str | array[str]                            # `'label'` (or array of labels `['label1', 'label2']`)
  timeout     : int                                         # the amount of seconds the test is allowed to run, a test
  verbose     : bool                                        # if true, forces the test results to be logged as if `--verbose` was passed
  workdir     : str                                         # absolute path that will be used as the working directory
)

Arguments

The function benchmark() accepts the following positional arguments:

Name Type Description Tags
name str

The unique test id
executable exe | jar | external_program | file | custom_tgt | custom_idx

The program to execute. (Since 1.4.0) A CustomTarget is also accepted.

Finally, benchmark() accepts the following keyword arguments:

Name Type Description Tags
args array[str | file | tgt | external_program]

Arguments to pass to the executable
depends array[build_tgt | custom_tgt]

specifies that this test depends on the specified target(s), even though it does not take any of them as a command line argument. This is meant for cases where test finds those targets internally, e.g. plugins or globbing. Those targets are built before test is executed even if they have build_by_default : false.

(since 0.46.0)
env env | array[str] | dict[str]

environment variables to set, such as ['NAME1=value1', 'NAME2=value2'], or an env object which allows more sophisticated environment juggling. (Since 0.52.0) A dictionary is also accepted.
priority int

specifies the priority of a test. Tests with a higher priority are started before tests with a lower priority. The starting order of tests with identical priorities is implementation-defined. The default priority is 0, negative numbers are permitted.

(since 0.52.0)

default = 0
protocol str

specifies how the test results are parsed and can be one of exitcode, tap, or gtest. For more information about test harness protocol read Unit Tests. The following values are accepted:

  • exitcode: the executable's exit code is used by the test harness to record the outcome of the test).

  • tap: Test Anything Protocol.

  • gtest (since 0.55.0): for Google Tests.

  • rust (since 0.56.0): for native rust tests

(since 0.50.0)

default = 'exitcode'
should_fail bool

when true the test is considered passed if the executable returns a non-zero return value (i.e. reports an error)

default = false
suite str | array[str]

'label' (or array of labels ['label1', 'label2']) attached to this test. The suite name is qualified by a (sub)project name resulting in (sub)project_name:label. In the case of an array of strings, the suite names will be (sub)project_name:label1, (sub)project_name:label2, etc.
timeout int

the amount of seconds the test is allowed to run, a test that exceeds its time limit is always considered failed, defaults to 30 seconds. Since 0.57 if timeout is <= 0 the test has infinite duration, in previous versions of Meson the test would fail with a timeout immediately.

default = 30
verbose bool

if true, forces the test results to be logged as if --verbose was passed to meson test.

(since 0.62.0)

default = false
workdir str

absolute path that will be used as the working directory for the test

The results of the search are