Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions aimdb-bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,18 @@ harness = false
name = "b0_alloc_linkable"
harness = false

[[bench]]
name = "b0_alloc_remote_ir"
harness = false

[[bench]]
name = "b1_b2_remote_ir"
harness = false

[[bench]]
name = "b1_b2_remote_envelope"
harness = false

[features]
default = ["std"]
# Gates `profiles`/`reports`/`harness` and their criterion/serde_json/
Expand All @@ -55,6 +67,7 @@ std = [
"dep:tokio",
"dep:serde",
"dep:serde_json",
"dep:ciborium",
"dep:criterion",
"dep:futures",
"dep:aimdb-data-contracts",
Expand Down Expand Up @@ -88,6 +101,9 @@ tokio = { workspace = true, optional = true }
# JSON output for baseline snapshots
serde = { workspace = true, optional = true }
serde_json = { workspace = true, optional = true }
# Issue #156 prototype dependency. This stays isolated in the unpublished
# benchmark crate while we measure whether a production CBOR path is justified.
ciborium = { version = "0.2.2", default-features = false, optional = true }

# migration_chain! for the b0_alloc_migration bench — std-gated like
# criterion/serde_json above, since that's the only bench that needs it.
Expand Down
38 changes: 38 additions & 0 deletions aimdb-bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ Plus two informational benches that exercise the full runner-driven pipeline.
- **Tokio** — `b0_alloc_tokio`, `b1_b2_tokio` (host).
- **postcard `Linkable` codec** — `b0_alloc_linkable` (host). This isolates the
issue #177 `encode_into` seam; it is not a whole-connector allocation claim.
- **Type-erased remote representation** — `b0_alloc_remote_ir` and
`b1_b2_remote_ir` compare JSON tree/direct, CBOR-to-JSON, and native CBOR
paths for issue #156. `b1_b2_remote_envelope` conditionally compares the
production AimX JSON envelope with an isolated native-CBOR consumer.
- **Embassy** — `b0_alloc_embassy`, `b1_b2_embassy`
(host). These drive the real [`EmbassyBuffer`] backend via
`futures::executor::block_on` over embassy-sync's poll methods — no
Expand Down Expand Up @@ -61,6 +65,11 @@ cargo bench -p aimdb-bench --bench b0_alloc_linkable
# B1 + B2 — latency (time/iter) and throughput (msgs/sec), one Criterion suite
cargo bench -p aimdb-bench --bench b1_b2_tokio

# Issue #156 — allocation, representation latency, and envelope prototype
cargo bench -p aimdb-bench --bench b0_alloc_remote_ir
cargo bench -p aimdb-bench --bench b1_b2_remote_ir
cargo bench -p aimdb-bench --bench b1_b2_remote_envelope

# Embassy buffer backend (host) — same classes
cargo bench -p aimdb-bench --bench b0_alloc_embassy
cargo bench -p aimdb-bench --bench b1_b2_embassy
Expand Down Expand Up @@ -91,6 +100,35 @@ Criterion writes HTML reports to `target/criterion/`.

---

## JSON/CBOR remote representation

The shared fixtures cover numeric telemetry, nested state, and a byte-heavy
record. Every representation is measured in both directions:

| Path | Purpose |
|---|---|
| `json_tree` | current `T -> serde_json::Value -> bytes` model |
| `json_direct` | control that removes only the dynamic JSON tree |
| `cbor_json_transcode` | CBOR dynamic tree forced back through a JSON edge |
| `cbor_tree_native` | dynamic native-CBOR consumer |
| `cbor_direct` | control that isolates the dynamic CBOR tree cost |

The envelope target runs request, write, and subscription-event flows. Its JSON
paths use the production `AimxCodec`; the CBOR protocol is benchmark-only and
does not modify `EnvelopeCodec` or a connector. It carries the record's CBOR
bytes opaquely inside a CBOR byte string and decodes them to `ciborium::Value`
at the consumer.

The measured decision is documented in
[`docs/design/046-cbor-self-describing-remote-access.md`](../docs/design/046-cbor-self-describing-remote-access.md):
keep JSON for current consumers, avoid CBOR-to-JSON transcoding, and pursue a
native binary edge only for a concrete byte-heavy workload with negotiation
and bounded decoding. The isolated Cortex-M code-size comparison and exact
commands live in
[`fixtures/remote-representation-footprint`](fixtures/remote-representation-footprint/README.md).

---

## B0 — allocation gate

`b0_alloc_tokio` does not use Criterion. It runs a fixed warmup + batch cycle and writes JSON results to `aimdb-bench/target/bench-results/b0_alloc_tokio.json` (the path is anchored to the crate dir, so it is the same regardless of the directory you run from).
Expand Down
148 changes: 148 additions & 0 deletions aimdb-bench/benches/b0_alloc_remote_ir.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
//! B0-Remote-IR — isolated allocation comparison for issue #156.
//!
//! This measures representation work only. It does not include AimX framing,
//! connector copies, scheduler wakeups, transport I/O, or backpressure.

use std::hint::black_box;

use aimdb_bench::alloc::{reset, snapshot};
use aimdb_bench::remote_ir::{
verify_round_trips, ByteHeavySample, NestedState, NumericTelemetry, RemoteIrFixture,
RemoteIrPath,
};

const WARMUP_ITERS: usize = 200;
const MEASURE_ITERS: usize = 1_000;
const ALLOCATOR_PROBE_BYTES: usize = 64;

#[global_allocator]
static GLOBAL: aimdb_bench::alloc::CountingAllocator<std::alloc::System> =
aimdb_bench::alloc::CountingAllocator(std::alloc::System);

#[derive(Debug)]
struct AllocationRow {
shape: &'static str,
operation: &'static str,
path: &'static str,
payload_bytes: usize,
allocations: u64,
allocated_bytes: u64,
}

fn warm_encode<T>(path: RemoteIrPath, value: &T, iterations: usize)
where
T: RemoteIrFixture,
{
for _ in 0..iterations {
let encoded = path
.encode(black_box(value))
.expect("fixture encode should succeed");
black_box(encoded.as_slice());
}
}

fn warm_decode<T>(path: RemoteIrPath, encoded: &[u8], iterations: usize)
where
T: RemoteIrFixture,
{
for _ in 0..iterations {
let decoded: T = path
.decode(black_box(encoded))
.expect("fixture decode should succeed");
black_box(decoded);
}
}

fn measure_fixture<T>(rows: &mut Vec<AllocationRow>)
where
T: RemoteIrFixture,
{
verify_round_trips::<T>().expect("all representation paths must round-trip");
let value = T::sample();

for path in RemoteIrPath::ALL {
let encoded = path.encode(&value).expect("fixture encode should succeed");
let payload_bytes = encoded.len();

warm_encode(path, &value, WARMUP_ITERS);
reset();
warm_encode(path, &value, MEASURE_ITERS);
let (allocations, allocated_bytes) = snapshot();
rows.push(AllocationRow {
shape: T::NAME,
operation: "encode",
path: path.name(),
payload_bytes,
allocations,
allocated_bytes,
});

warm_decode::<T>(path, &encoded, WARMUP_ITERS);
reset();
warm_decode::<T>(path, &encoded, MEASURE_ITERS);
let (allocations, allocated_bytes) = snapshot();
rows.push(AllocationRow {
shape: T::NAME,
operation: "decode",
path: path.name(),
payload_bytes,
allocations,
allocated_bytes,
});
}
}

/// Prove this bench binary's global allocator reaches the shared counters.
fn allocation_counter_positive_control() -> (u64, u64) {
reset();

let mut probe = Vec::<u8>::with_capacity(black_box(ALLOCATOR_PROBE_BYTES));
probe.push(black_box(0xA5));
black_box(probe.as_slice());

let measured = snapshot();
assert!(
measured.0 >= 1,
"allocation counter missed the deliberate Vec allocation"
);
assert!(
measured.1 >= ALLOCATOR_PROBE_BYTES as u64,
"byte counter missed the deliberate Vec capacity"
);

drop(probe);
measured
}

fn main() {
let mut rows = Vec::with_capacity(3 * RemoteIrPath::ALL.len() * 2);
measure_fixture::<NumericTelemetry>(&mut rows);
measure_fixture::<NestedState>(&mut rows);
measure_fixture::<ByteHeavySample>(&mut rows);

// Run after every measurement tuple is captured so the deliberate
// allocation cannot contaminate any representation window.
let (probe_allocations, probe_bytes) = allocation_counter_positive_control();

println!("=== B0 Remote IR allocation comparison ===");
println!("Measured iterations per row: {MEASURE_ITERS}");
println!("Allocator probe: {probe_allocations} calls / {probe_bytes} bytes");
println!(
"{:<20} {:<7} {:<22} {:>9} {:>12} {:>15} {:>12} {:>14}",
"shape", "op", "path", "payload", "allocations", "allocated_bytes", "allocs/op", "bytes/op"
);

for row in rows {
println!(
"{:<20} {:<7} {:<22} {:>9} {:>12} {:>15} {:>12.3} {:>14.1}",
row.shape,
row.operation,
row.path,
row.payload_bytes,
row.allocations,
row.allocated_bytes,
row.allocations as f64 / MEASURE_ITERS as f64,
row.allocated_bytes as f64 / MEASURE_ITERS as f64,
);
}
}
73 changes: 73 additions & 0 deletions aimdb-bench/benches/b1_b2_remote_envelope.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
//! Conditional Phase-4 B1/B2 envelope prototype for issue #156.
//!
//! Runs production AimX JSON framing against an isolated native-CBOR consumer
//! for request, write, and subscription-event flows. Transport I/O is excluded.

use std::hint::black_box;
use std::time::Duration;

use aimdb_bench::remote_envelope::{
encoded_frame_sizes, round_trip, verify_envelope_round_trips, EnvelopeOperation,
EnvelopeProtocol,
};
use aimdb_bench::remote_ir::{ByteHeavySample, NestedState, NumericTelemetry, RemoteIrFixture};
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};

fn benchmark_fixture<T>(criterion: &mut Criterion)
where
T: RemoteIrFixture,
{
verify_envelope_round_trips::<T>().expect("envelope fixtures must round-trip");
println!("\n=== remote envelope sizes: {} ===", T::NAME);
for (operation, protocol, bytes) in
encoded_frame_sizes::<T>().expect("envelope sizes must encode")
{
println!(
"{:<8} {:<24} {:>6} bytes",
operation.name(),
protocol.name(),
bytes
);
}

let sample = T::sample();
let mut group = criterion.benchmark_group(format!("remote_envelope/{}", T::NAME));
group.throughput(Throughput::Elements(1));

for operation in EnvelopeOperation::ALL {
for protocol in EnvelopeProtocol::ALL {
group.bench_with_input(
BenchmarkId::new(operation.name(), protocol.name()),
&(protocol, operation),
|bencher, &(protocol, operation)| {
bencher.iter(|| {
let observation = round_trip(protocol, operation, black_box(&sample))
.expect("trusted envelope fixture");
black_box(observation)
});
},
);
}
}
group.finish();
}

fn remote_envelope(criterion: &mut Criterion) {
benchmark_fixture::<NumericTelemetry>(criterion);
benchmark_fixture::<NestedState>(criterion);
benchmark_fixture::<ByteHeavySample>(criterion);
}

fn configure() -> Criterion {
Criterion::default()
.sample_size(30)
.warm_up_time(Duration::from_secs(1))
.measurement_time(Duration::from_secs(2))
}

criterion_group! {
name = benches;
config = configure();
targets = remote_envelope
}
criterion_main!(benches);
Loading