← groundrun.io > Groundrun docs

Groundrun YAML scenario format cheat sheet

groundrun test scenarios are written in YAML, across two file types that work together. A rig file describes one physical test setup: which boards, phones and power switches are wired up, and how. A scenario file describes one test: which firmware to flash, what steps to run in order, and what counts as a pass. A scenario file references a rig file by name, and many scenario files can share one rig file.

Steps are listed under steps: (a flat, ordered list) or phases: (the same steps grouped into named story beats) — a scenario picks exactly one of the two forms. Every field below is required unless marked optional; a default value, when one exists, is stated in the Description column.

Examples use placeholder names throughout: my-rig for a rig, board-1/ board-2/board-3 for board roles, phone for a phone role, backend for a cloud/service role, and com.example.myapp for an app package. Replace them with your own. For the groundrun command-line tool that pushes, validates and runs these files, see cli-cheat-sheet.md.

Contents

Rig file

Field/Key Description Example
rig.name Required. Unique identifier for this rig, referenced by scenario files' own scenario.rig field. rig: {name: my-rig}
rig.description Optional. Human-readable summary. rig: {description: "Three-board mesh bench"}
rig.capabilities Optional list. Shared hardware the whole rig offers, not tied to one actor — today only wifi_capture (a USB Wi-Fi adapter that can capture 802.11 traffic near the rig). rig: {capabilities: [wifi_capture]}
actors.<role-name> Required, at least one. A map of role names (used by scenario files) to actor definitions. Each actor has a type (board, phone, power_switch, or cloud) plus type-specific fields — see Actor types below. actors: {board-1: {type: board, ...}}
power.type / power.ip / power.outlets Optional. A rig-wide power switch used by power_cycle steps: type is shelly or kasa, ip is the switch's network address, outlets maps a human-readable label to an outlet number. power: {type: shelly, ip: 192.0.2.10, outlets: {usb-hub: 1}}
supervisor.host Optional. user@ip of a supervisory Pi that can hard-reset this rig's main controller. Enables the coordinator's supervisory power-cycle path. supervisor: {host: "user@192.0.2.11"}
supervisor.power_switch.type / .ip / .outlets Required when supervisor is present. The switch controlling the main controller's own power; same shape as the top-level power fields, with the outlet mapping keyed testing-pi. power_switch: {type: shelly, ip: 192.0.2.12, outlets: {testing-pi: 1}}
supervisor.power_cycle.off_duration / .boot_timeout Optional, under supervisor. Seconds to hold power off (default 10) and seconds to wait for the main controller to come back up over SSH (default 120). power_cycle: {off_duration: 10, boot_timeout: 120}
redaction Optional list. Operator-supplied patterns for masking secrets your firmware prints on UART, applied to every captured log. Each entry has a prefix (regex context to keep, e.g. "secret=") and a pattern (regex for the value to mask), plus an optional label for the redaction record. These add to, not replace, Groundrun's own built-in defaults. redaction: [{label: api-key, prefix: "key=", pattern: "[0-9a-f]+"}]

Actor types

Every actor under actors: declares a type. Four types exist.

Field/Key Description Example
board.type Required. Always "board" — an embedded development board connected over USB. type: board
board.chip Required. A chip family name (e.g. efr32mg24, nrf52840, esp32) — informational, and also used by the coordinator to match a rig's capabilities against what a scenario needs. chip: nrf52840
board.serial Required. The board's USB serial number, used to find it among connected devices. Quote a numeric serial so YAML does not parse it as an integer. serial: "0123456789"
board.flash_tool Required. Which flashing tool to use: commander (Silicon Labs), nrfutil (Nordic), mcumgr (Nordic, over an existing running app), or esptool (Espressif). flash_tool: commander
board.connection Required. The board's UART connection settings — see Connection types → uart. connection: {type: uart, baud: 115200}
board.capabilities Optional list. What this board's own hardware can do beyond flashing and UART. Today only packet_capture (onboard radio packet trace, board-specific hardware only). capabilities: [packet_capture]
phone.type Required. Always "phone" — a phone connected over USB. type: phone
phone.platform Required. "android" or "ios". platform: android
phone.adb_serial Optional, Android only. The device's ADB serial number. Omit when only one Android device is attached. adb_serial: "R58N123ABCD"
phone.udid Optional, iOS only. The device's UDID. Omit when only one iOS device is attached. udid: "00001234-ABCDEF012345"
phone.app_package Optional, Android only. Default app package name for this phone; a scenario's own app_launch step overrides it for the rest of the run. app_package: com.example.myapp
phone.bundle_id Optional, iOS only. Default app bundle ID, same role as app_package. bundle_id: com.example.myapp
phone.connection Optional. Defaults to a USB connection. See Connection types → adb (Android) or ios. connection: {type: adb, transport: usb}
power_switch.type Required. Always "power_switch" — a network-controllable smart plug or strip, for per-outlet control beyond the rig-level power section. type: power_switch
power_switch.model Required. "shelly" or "kasa". model: shelly
power_switch.ip Required. The switch's address on the local network. ip: 192.0.2.10
power_switch.outlets Required. Maps a human-readable label to an outlet number. outlets: {aux-power: 2}
cloud.type Required. Always "cloud" — an HTTP-reachable backend or API server. type: cloud
cloud.base_url Required. Base URL including port. base_url: "http://192.0.2.20:8080"
cloud.auth Optional. "none" (default), "bearer", or "basic". Credentials are supplied through environment variables at run time, never written into the YAML. auth: bearer

Connection types

A board or phone actor's connection block declares its own type.

Field/Key Description Example
uart.type Required. Always "uart" — a serial connection to a board over USB. type: uart
uart.baud Optional. Baud rate. Default 115200. baud: 115200
uart.flow_control Optional. "none" (default) or "hardware" (RTS/CTS) — must match what your firmware actually uses. flow_control: hardware
uart.protocol Optional. "cli" (default, line-based text) or "hdlc" (binary-framed, e.g. Spinel). protocol: cli
uart.vcom_sync Optional, flash_tool: commander only. When true, a board reset also re-applies baud/flow_control to the debug adapter's own virtual COM port first. Default false. vcom_sync: true
adb.type Required. Always "adb" — an Android Debug Bridge connection to a phone. type: adb
adb.transport Required. "usb" (phone cabled to this rig) or "tcp" (phone reached over the network). transport: usb
adb.address Optional. The phone's host:port for a tcp connection. Normally omitted — the phone's address is discovered automatically over the cable; only set it for a phone cabled to a different rig. address: "192.0.2.30:5555"
adb.remote_rig Optional. Names another rig whose own controller physically holds this phone's USB cable, when driving that phone over the network is not an option (for example, a step is about to move the phone off its Wi-Fi network). Requires adb_serial to be set explicitly. Mutually exclusive with transport: tcp. remote_rig: my-other-rig
ios.type Required. Always "ios" — an iOS device connected over USB. No further properties; the device is found by its UDID. type: ios
http Used implicitly by cloud and power_switch actors — no separate connection block to write. (not written directly)

Scenario file

Field/Key Description Example
scenario.name Required. Unique identifier; also names the generated test file. Do not include an owner/directory prefix here — that is supplied separately when pushing the scenario. scenario: {name: my-scenario}
scenario.owner Optional. Qualifies the scenario's registry name to owner/name, for a scenario stored as a file rather than pushed with an explicit name. owner: my-team
scenario.description Optional. One-line summary of what the scenario tests. description: "Three boards form a mesh"
scenario.rig Required. The rig this scenario runs on: either a rig file's filename (for a scenario stored as a file) or a rig's rig.name identity (for a scenario submitted over the API). rig: rig-my-rig.yaml
scenario.manual_equivalent_hours Optional. Hours of manual testing this scenario replaces, for value reporting. Must be greater than 0. manual_equivalent_hours: 2.5
scenario.failure_severity Optional. critical, high, medium, or low — how bad it is if this scenario's failure goes undetected. failure_severity: high
scenario.budget_seconds Optional. This scenario's own wall-clock time limit, 30 to 14400 seconds, overriding the coordinator's shared default. A whole number; out-of-range values are refused rather than clamped. budget_seconds: 1800
scenario.requires_lease Optional list. Extra rig names this scenario depends on without directly touching their hardware (for example, reading a code displayed on another rig's screen). Never includes this scenario's own rig. requires_lease: [my-other-rig]
scenario.measure_power Optional. When true, captures per-board current draw and phone battery current for the whole run as a separate artifact. measure_power: true
scenario.power_capture_datarate Optional, requires measure_power: true. Sample rate in Hz for the whole-run power capture. Default 100. power_capture_datarate: 100
scenario.power_capture_stagger Optional, requires measure_power: true. Seconds to wait between starting each board's power capture. Default 0 (all at once). power_capture_stagger: 0.5
firmware.<role-name> Optional map. A board role's default firmware image path. Must start with builds/ and be a single non-empty string (not a list — use a per-step override for a second image). firmware: {board-1: builds/firmware.bin}
files.<slot-name> Optional map. Declares a non-firmware payload file (a script, a certificate) this scenario needs, mapping a slot name to a destination path. The caller supplies the actual file content per run; every declared slot must be supplied on every run. files: {ca-cert: certs/ca.pem}
tool_overrides.<tool-name> Optional map. Substitutes a different build of a named tool (nrfutil, mcumgr, commander, silink, pti_jar, otbr_agent) for the run, staged onto the rig beforehand. tool_overrides: {nrfutil: /tmp/groundrun-push/nrfutil-2.9.0}
steps One of steps or phases is required (never both, never neither). An ordered list of step maps, each with an action, usually a target, and step-specific params — see the step-type sections below. steps: [{action: flash, target: board-1}]
phases The alternative to steps. A list of named story beats, each a story label plus its own steps list (same step grammar). Flattening every phase's steps in order gives the same sequence a flat steps list would; phases add narrative grouping, no new behavior. phases: [{story: "Flash firmware", steps: [...]}]
pass_criteria Required, at least one entry. Evaluated once every step has finished — see Pass criteria below. pass_criteria: [{criterion: device_state, actor: board-1, expected: Ready, timeout: 30}]

Pass criteria

Evaluated once every step has finished; if any step fails, no criteria are evaluated at all (the run's outcome is decided by the failed step instead).

Field/Key Description Example
device_state Checks whether a regex matched anywhere in a board's UART output collected over the whole run so far. Fields: actor, expected (regex, or the literal usb_present to check the board has a device path), timeout (accepted for schema compatibility but does not bound a live wait — this criterion checks already-collected output). {criterion: device_state, actor: board-1, expected: Ready, timeout: 30}
discovery_count Checks that a phone's app reported discovering at least a given number of distinct devices. Fields: actor (a phone), expected (integer), timeout (optional, milliseconds — restricts which collected events count). {criterion: discovery_count, actor: phone, expected: 2}
connection_count Same mechanism as discovery_count, counting distinct devices the app reported connecting to. {criterion: connection_count, actor: phone, expected: 1}
read_count Same mechanism as discovery_count, counting distinct devices the app reported reading from. {criterion: read_count, actor: phone, expected: 1}
max_time Checks the run's total elapsed time against a limit. Fields: metric (only total is implemented today), expected (milliseconds). Must not exceed the scenario's own budget_seconds when one is declared. {criterion: max_time, metric: total, expected: 120000}

Board and rig steps

Field/Key Description Example
power_cycle (action) Turns the rig's power off and back on, for a clean hardware state. No target. {action: power_cycle}
power_cycle.off_duration Optional. Seconds to keep power off. Default 5. params: {off_duration: 5}
power_cycle.boot_wait Optional. Seconds to wait after power-on for boards to enumerate and boot. Default 15. params: {boot_wait: 15}
flash (action) Writes firmware onto one or more boards. {action: flash, target: board-1}
flash.target Required. One board role, or a list of board roles. target: [board-1, board-2]
flash.firmware Optional. A firmware path for this step only, overriding the role's firmware: entry — for flashing a second image (e.g. a bootloader, then the application) onto one board. params: {firmware: builds/bootloader.bin}
configure (action) Sends a list of commands to a board over UART, before or during a scenario. {action: configure, target: board-1}
configure.target Required. One board role, or a list. target: board-1
configure.commands Required list. Command strings, sent one at a time. params: {commands: ["status"]}
configure.expect_response Optional. When true, fails the step immediately if the board produced no output at all — instead of letting a later wait_for time out. Refused inside a try_each_as block or on a non-board target. Default off. params: {expect_response: true}
wait_for (action) Blocks until a regex pattern appears in a board's UART output, or the timeout expires. {action: wait_for, target: board-1}
wait_for.target Required. One board role, or a list — a list means "any one of these". target: [board-1, board-2]
wait_for.pattern Required. Regex to match against UART lines. params: {pattern: "Ready"}
wait_for.timeout Required. Seconds to wait before failing. params: {timeout: 30}
wait_for.fresh Optional. When true, only matches output printed after this wait started (or after the last write to this board); required explicitly inside a repeat block, to avoid re-matching one old line on every iteration. Default false (scans everything collected so far). params: {fresh: true}
wait_for.match Optional. "text" (default, line-based) or "bytes" (byte-exact, for a binary protocol). params: {match: bytes}
device_reset (action) Resets one board via its debug probe, without cutting power to the whole rig. {action: device_reset, target: board-1}
device_reset.target Required. One board role. target: board-1
device_reset.wait Optional. Seconds to pause after reset. No wait by default. params: {wait: 2}
device_recover (action) Clears a board's non-volatile memory — needed before flashing an image incompatible with what is currently on the board. {action: device_recover, target: board-1}
device_recover.target Required. One board role. No other params. target: board-1
rig_check (action) Checks that one or more boards are reachable, before the scenario runs. {action: rig_check, target: board-1}
rig_check.target Required. One board role, or a list. target: [board-1, board-2]
rig_check.check Required. "reachable" (the actor exists), "uart_responsive" (a write gets any reply within 5s), or "usb_present" (a USB device path is set). params: {check: uart_responsive}

Phone steps

Field/Key Description Example
app_install (action) Installs an app onto the phone. {action: app_install, target: phone}
app_install.target Required. Role name of the phone. target: phone
app_install.path Required. Path to the app binary (APK for Android, IPA for iOS). Names the base APK when split_paths is also given. params: {path: builds/myapp.apk}
app_install.split_paths Optional list, Android only. Additional split APKs to install alongside the base APK, for an Android App Bundle split set. All splits must share the base APK's package, version and signature. params: {split_paths: [builds/split-arm64.apk]}
app_launch (action) Launches an app on the phone. {action: app_launch, target: phone}
app_launch.target Required. Role name of the phone. target: phone
app_launch.package Required. Android package name, or iOS bundle ID. params: {package: com.example.myapp}
app_launch.activity Required on Android, ignored on iOS. Activity class to launch. params: {activity: .MainActivity}
app_launch.extras Optional map, Android only. Intent extras passed at launch. params: {extras: {mode: test}}
app_launch.grant_permissions Optional list, Android only. Fully-qualified runtime permissions to grant before launching — useful on a shared phone where a previous session's grants may have been reset. params: {grant_permissions: [android.permission.ACCESS_FINE_LOCATION]}
app_launch.stop_packages Optional list, Android only. Package names to force-stop before launching, so a leftover app from an earlier session cannot hold the foreground. params: {stop_packages: [com.example.otherapp]}
app_action (action) Sends a programmatic command to the running app (Android: a broadcast intent; iOS: a URL scheme) and optionally collects log output. Fastest option when the app under test has a broadcast-receiver test harness. {action: app_action, target: phone}
app_action.target Required. Role name of the phone. target: phone
app_action.action Required. Action identifier — an intent action suffix on Android, a URL path on iOS. params: {action: SCAN_START}
app_action.extras Optional map. Key-value parameters passed along with the action. params: {extras: {timeout: "10"}}
app_action.collect_duration Optional. Milliseconds of log output to collect after sending the command. params: {collect_duration: 2000}
app_ui (action) Drives the phone's screen through touch gestures — for an app with no test harness, or to test the real UI flow itself. {action: app_ui, target: phone}
app_ui.target Required. Role name of the phone. target: phone
app_ui.gesture Required. One of tap, type, swipe, drag, wait, assert_text. params: {gesture: tap}
app_ui.selector Required for tap/assert_text unless at is given (for tap). Exactly one of text, resource_id, or content_desc naming the element. params: {selector: {text: "Add Device"}}
app_ui.at For tap/drag endpoints. A fractional screen coordinate {x, y} (each 0 to 1), for a surface with no selectable element (a map, a chart). Mutually exclusive with selector. params: {at: {x: 0.5, y: 0.42}}
app_ui.text Required for type. The text to enter into a focused field. params: {gesture: type, text: "hello"}
app_ui.direction Required for swipe. up, down, left, or right. params: {direction: up}
app_ui.start / app_ui.end Required for drag. Each is a selector or a fractional coordinate, marking the drag's start and end points. params: {start: {x: 0.5, y: 0.5}, end: {x: 0.3, y: 0.6}}
app_ui.duration_ms Optional, for drag. Press-move-release duration in milliseconds. Default 800. params: {duration_ms: 800}
app_ui.seconds Required for wait. Seconds to pause. params: {gesture: wait, seconds: 3}
app_ui.expect Optional, for assert_text. "present" (default) or "absent". params: {expect: absent}
app_ui.timeout Optional, for tap/assert_text. Seconds to poll for the element before failing. Default 10. params: {timeout: 10}
app_ui.optional Optional, tap only. When true, a missing element does not fail the step — for a screen that only sometimes appears. Default false. params: {optional: true}
phone_shell (action) Runs raw shell commands on the phone — for port forwarding, permission grants, screen wake, or anything with no dedicated step type. {action: phone_shell, target: phone}
phone_shell.target Required. Role name of the phone. target: phone
phone_shell.commands Required list. Shell command strings, run in sequence. params: {commands: ["input keyevent KEYCODE_WAKEUP"]}
phone_charging (action) Turns the phone's USB charging off or on, so a power measurement reflects real battery draw rather than charge current. Restored automatically at the end of every run. {action: phone_charging, target: phone}
phone_charging.target Required. Role name of the phone. target: phone
phone_charging.state Required. "off" or "on" (always quote it — unquoted off/on parse as YAML booleans). params: {state: "off"}
phone_wifi (action) Turns the phone's Wi-Fi radio on/off, or joins/forgets a network by name. Refused on a phone reached over the network (since disabling Wi-Fi would cut the connection driving the step). {action: phone_wifi, target: phone}
phone_wifi.target Required. Role name of the phone. target: phone
phone_wifi.state Required. "off", "on", "join", or "forget" (always quote it). params: {state: "join"}
phone_wifi.ssid Required for join/forget. The network name — open networks only, never a password. params: {ssid: my-network}
phone_wifi.via Optional, join only. "shell" (default, fast) or "settings-ui" (drives the real Wi-Fi settings screen, needed when a scenario depends on Android's own captive-portal detection). params: {via: "settings-ui"}
phone_wifi.await_window_pattern Optional, join + via: settings-ui only. A pattern to poll for in the phone's window state right after the join tap (for example, a captive-portal login screen appearing). params: {await_window_pattern: "CaptivePortalLoginActivity"}
phone_wifi.await_window_count / .await_window_interval Optional, with await_window_pattern. Poll attempts (default 6) and seconds between them (default 0.3). params: {await_window_count: 6, await_window_interval: 0.3}
phone_wifi.form_fill Optional, join + await_window_pattern only. Fills a login form's SSID/password fields and taps Connect, in one pass: {ssid_field_at, password_field_at, connect_at} (each {x, y}) plus ssid_text/password_text. params: {form_fill: {ssid_field_at: {x: 0.5, y: 0.3}, password_field_at: {x: 0.5, y: 0.4}, connect_at: {x: 0.5, y: 0.8}, ssid_text: "${ssid}", password_text: "${password}"}}

Host and service steps

Steps that run on the test host itself, rather than on a board or phone.

Field/Key Description Example
manual_gate (action) Pauses the run and shows a message to the operator, resuming on confirmation or timeout. No target. {action: manual_gate}
manual_gate.message Required. Message shown to the operator. params: {message: "Confirm cable seated"}
manual_gate.timeout Optional. Seconds to wait for confirmation. Default 300. params: {timeout: 300}
build (action) Runs a build command on the host to produce firmware or other artifacts. No target. {action: build}
build.command Required. Shell command. params: {command: "make firmware"}
build.env Optional map. Extra environment variables. params: {env: {TARGET: release}}
build.working_dir Optional. Working directory for the command. params: {working_dir: "src/firmware"}
run_script (action) Runs an external script or tool for anything not covered by a dedicated step type. No target. {action: run_script}
run_script.command Required. Shell command. Its own bash variables must be written $var, never ${var} — the latter is parsed as a template reference. params: {command: "python3 tools/provision.py"}
run_script.env Optional map. Extra environment variables. params: {env: {CA_CERT: "/opt/certs/ca.pem"}}
run_script.timeout Optional. Seconds before the command is killed. Default 300. params: {timeout: 300}
run_script.exclusive_boards Optional list of board roles. Boards whose serial ports are released for the duration of this command (for a tool that needs sole access to the raw device, like a DFU transfer) and reopened afterward. params: {exclusive_boards: [board-1]}
run_script.capture Optional. Reads a regex capture group out of the command's own stdout into a scenario variable — pattern (with a named group (?P<value>...)) and variable. params: {capture: {pattern: "id=(?P<value>[0-9a-f]+)", variable: session_id}}
service_start (action) Launches a background process — a mock backend, a capture tool — for the rest of the scenario to use. {action: service_start}
service_start.target Required unless name is given. Role name of a cloud actor this service represents. target: backend
service_start.name Required unless target is given. A label for a host-side tool with no actor of its own. params: {name: pti-capture}
service_start.command Required. Shell command that launches the service. params: {command: "python3 tools/backend.py"}
service_start.health_check Optional. An HTTP URL polled until it returns 200 OK before the step completes. params: {health_check: "http://localhost:8080/health"}
service_start.startup_timeout Optional. Seconds to wait for the health check. Default 30. params: {startup_timeout: 30}
service_start.exclusive_uart Optional list of board roles. Boards whose real device nodes are handed to this service for its whole lifetime — for a tool that must own the device directly rather than a shared copy of its output (for example, one enforcing hardware flow control). Released automatically by the matching service_stop. params: {exclusive_uart: [board-1]}
service_stop (action) Stops a service started by service_start. {action: service_stop}
service_stop.target / .name Required, matching whichever service_start used. params: {name: pti-capture}

Both run_script and service_start can read a board's live UART stream without opening its device directly, by referencing ${uart.<role>} (read-only) or ${uart_rw.<role>} (read and write) inside command. Both resolve at run time to a path carrying that board's output; the literal device path itself can never be written in a command. With exclusive_uart, ${uart_dev.<role>} resolves to the board's real device node.

Capture and measurement steps

Field/Key Description Example
verify (action) Asserts a condition on data already collected by earlier steps (unlike wait_for, which blocks live). {action: verify, target: board-1}
verify.target Required. Role name of the actor to check — a board, phone, or cloud actor, matching the condition used. target: board-1
verify.condition Required. "uart_pattern" (board UART matched), "uart_absent" (never matched), "logcat_event" (phone logcat matched), or "http_status" (cloud response code). params: {condition: uart_pattern}
verify.expected Required. A regex pattern (for the UART/logcat conditions) or an HTTP status code. params: {expected: "Ready"}
verify.settle Optional, uart_absent only. Seconds to wait before scanning, so the assertion covers a real interval rather than a moment too early to have seen anything. params: {settle: 5}
verify.timeout Optional, uart_pattern only. Seconds to wait for the pattern. Default 10. params: {timeout: 10}
packet_capture_start / packet_capture_stop (action) Captures a board's own radio traffic (via its onboard packet-trace hardware) between the two steps, as a .pcapng artifact. Requires the board to declare the packet_capture capability. {action: packet_capture_start, target: board-1}
packet_capture_start.target Required. Role name of the board being captured. target: board-1
packet_capture_start.output Optional. Output filename, must end .pcapng. Defaults to a name derived from the role. params: {output: air-trace.pcapng}
wifi_capture_start / wifi_capture_stop (action) Captures nearby 802.11 traffic through the rig's own monitor-mode Wi-Fi adapter, as a .pcapng artifact. Requires the rig to declare the wifi_capture capability. Target is a board or phone actor. {action: wifi_capture_start, target: phone}
wifi_capture_start.target Required. Role name of the board or phone actor to capture near. target: phone
wifi_capture_start.output Optional. Output filename, must end .pcapng. Defaults to a name derived from the role. params: {output: wifi-trace.pcapng}
capture (action) Extracts a value from a board's UART output and stores it as a scenario variable, referenced later as ${variable_name}. Fails the scenario if the pattern never appears. {action: capture, target: board-1}
capture.target Required. One board role — not a list. target: board-1
capture.pattern Required. Regex with a named group (?P<value>...). params: {pattern: "id: (?P<value>[0-9a-f]+)"}
capture.variable Required. Variable name, unique across the scenario. params: {variable: device_id}
capture.timeout Required. Seconds to wait for the pattern. params: {timeout: 30}
record_value (action) Reads a value off a board's UART and puts it on the run's record — without failing the scenario if it is not found. Use this instead of capture when the reading is informational rather than needed by a later step. {action: record_value, target: board-1}
record_value.target Required. One board role, or a list — each board's reading is recorded independently. target: [board-1, board-2]
record_value.pattern Required. Regex with a named group (?P<value>...). params: {pattern: "state=(?P<value>[0-9]+)"}
record_value.name Required. The key the reading is recorded under. Not required to be unique — recording the same name before and after a disruptive step is a normal pattern. params: {name: join_state}
record_value.timeout Required. Seconds to wait for the pattern. params: {timeout: 30}
record_value.last_match Optional. When true, waits the full timeout and records the last match seen rather than the first — for a value that changes over time. Default false. params: {last_match: true}
power_measure (action) Measures a board's average current draw over a window and fails the scenario if it falls outside declared bounds. Requires a board using the commander flash tool. {action: power_measure, target: board-1}
power_measure.target Required. Role name of the board to measure. target: board-1
power_measure.window_ms Optional. Measurement window in milliseconds. Default 2000. params: {window_ms: 10000}
power_measure.expect_max_ma / .expect_min_ma At least one required, unless baseline is used instead. Upper/lower bounds on average current in mA. params: {expect_max_ma: 5, expect_min_ma: 0.5}
power_measure.baseline Alternative to the absolute bounds above. Names a recorded baseline reading to compare against instead of a hand-written number. params: {baseline: setup-advertising}
power_measure.expect_max_delta_ma Required with baseline. Allowed drift in mA above the baseline's recorded average. params: {expect_max_delta_ma: 0.3}

Control-flow steps

Field/Key Description Example
repeat (action) Runs a block of inner steps a fixed number of times — the only loop construct, used for a soak test. No target; inner steps carry their own. {action: repeat, params: {count: 60}, steps: [...]}
repeat.count Required. Iterations, 1 to 100000. params: {count: 60}
try_each_as (action) Retries a block of inner steps with a different board playing a named role each attempt, stopping at the first attempt that does not fail — for a scenario where which physical board should play a role is not known in advance. {action: try_each_as, params: {...}, steps: [...]}
try_each_as.role Optional. A descriptive label for the rotating role; documentation only. params: {role: gateway}
try_each_as.candidates Required list, at least 2 board roles to try. params: {candidates: [board-1, board-2, board-3]}
try_each_as.role_firmware Required. Firmware path for whichever board is currently playing the role. params: {role_firmware: builds/role.bin}
try_each_as.others_firmware Required. Firmware path for the remaining candidates on each attempt. params: {others_firmware: builds/other.bin}
try_each_as.timeout_per_attempt Required. Seconds allowed per attempt; relies on the inner steps' own timeouts to actually enforce it. params: {timeout_per_attempt: 180}
retry_block (action) Retries a fixed block of inner steps against the same target(s), on any failure, up to a set number of attempts — for absorbing a known-flaky sequence with no external signal to check. No target. {action: retry_block, params: {count: 3}, steps: [...]}
retry_block.count Required. Attempts, 1 to 100000. params: {count: 3}
retry_once_if (action) Retries a fixed block of inner steps exactly once, but only after an external probe command confirms a specific known fault caused the failure, then waits for a readiness command before retrying. No target. {action: retry_once_if, params: {...}, steps: [...]}
retry_once_if.probe_command Required. A shell command; exit code 0 means the known fault is confirmed. params: {probe_command: "python3 tools/fault_probe.py"}
retry_once_if.readiness_command Required. A shell command polled until it exits 0, checked before the retry runs — must check the actual condition the retried steps depend on, not just that a process is alive. params: {readiness_command: "curl -sf http://localhost:8081/ready"}
retry_once_if.readiness_timeout Optional. Seconds to poll the readiness command before giving up. Default 90. params: {readiness_timeout: 120}
retry_once_if.readiness_poll_interval Optional. Seconds between readiness polls. Default 5. params: {readiness_poll_interval: 5}
retry_once_if.probe_timeout Optional. Seconds the probe command itself is allowed to run. Default 30. params: {probe_timeout: 30}

Inside any of these blocks, string parameters may reference an earlier capture step's variable as ${variable_name}.