Hardware-in-the-loop test framework for the Armbian device lab.
It discovers the boards on the lab network, keeps the hardware inventory in NetBox in sync, and runs per-board test pipelines — flashing a clean image where the hardware allows it, otherwise testing upgrade / reboot / performance — then publishes results as a time series so regressions are visible over time.
The previous generation of this tool lives on the
masterbranch and is being deprecated. Thismainbranch is a clean, modular redesign.
┌───────────────────────────────────────────────┐
│ NetBox (netbox.armbian.com) │
│ Device Types · Devices · status · capabilities│
└───────────────▲───────────────┬───────────────┘
discover + reconcile (write) │ │ read capabilities
┌───────────────┴──────┐ │
two lab subnets ─────▶│ scan-daemon │ │
(nmap / arp / ssh) │ lib/discovery.py │ │
│ daemon/reconcile.py │ │
└──────────────────────┘ │
│
per-board run: orchestrator/run-board ──reads caps──┘
│
├─ has SD_Swap? → modules/flash/flash-sdwire ┐
├─ has maskrom? → modules/flash/flash-rockchip │ clean image
│ → modules/provision/firstlogin │ + full suite
│ → modules/test/* ┘
└─ neither? → modules/test/{upgrade,reboot,hw-performance}
│
└─ results ─────────▶ results/publish ─▶ time-series store (armbian.github.io)
power for any step: modules/power/power ──dispatch──▶ infra/{apc7920,tplink-poe,dut-relay}
See docs/architecture.md for the full design,
docs/netbox-model.md for the data model + the
scanner's reconcile rules, and docs/capabilities.md
for how a board declares what it can do.
| Path | What it is |
|---|---|
daemon/ |
network-scan → NetBox reconcile loop (the inventory/status daemon) |
lib/ |
shared library: NetBox client, discovery, capability resolution, bash helpers |
modules/power/ |
power abstraction → dispatches to the infra power backends |
modules/flash/ |
clean-image flashing: flash-sdwire, flash-rockchip, shared helpers |
modules/provision/ |
firstlogin, maintainer-keys, sync-keys — first-boot bring-up + SSH key sync |
modules/test/ |
individual tests: upgrade, reboot, hw-performance, … |
orchestrator/ |
run-board — the per-board pipeline that ties modules together |
results/ |
results schema + publisher (time-series) |
config/ |
example config (real config is gitignored — holds secrets) |
docs/ |
architecture and operations docs |
Every module is a standalone script with a small, documented contract
(inputs by env/flags, one job, exit code + a JSON result line on stdout) so each
can be developed and tested in isolation. See
docs/module-contract.md.
armbian/infra— owns the lab primitives this framework calls: power (apc7920,tplink-sg2428p-poe,dut-relay-pd) and NetBox conventions. autotests orchestrates them; it does not re-implement power control.armbian/armbian.github.io— publishing target for the time-series results (samedata-branch pattern the build farm already uses), so the website can chart them.- NetBox — the single source of truth for what hardware exists and what each board can do.
cp config/autotests.conf.example config/autotests.conf
chmod 600 config/autotests.conf # holds the NetBox token
$EDITOR config/autotests.conf # set NETBOX_TOKEN, SUBNETS, infra tool paths
# one-time: create the NetBox custom fields the model uses (idempotent)
./tools/netbox-bootstrap-fields.py --dry-run
./tools/netbox-bootstrap-fields.py
# one-shot inventory reconcile (dry-run first)
./daemon/reconcile.py --once --dry-run
./daemon/reconcile.py --once
# run the scanner as a daemon (or install daemon/autotests-scan.service)
./daemon/scan-daemon.py
# run the test pipeline for one board (by NetBox name or IP)
./orchestrator/run-board orangepi5-01
