spike: TXE runs world state + AVM fully in-process (Slice C)#24642
Draft
charlielye wants to merge 12 commits into
Draft
spike: TXE runs world state + AVM fully in-process (Slice C)#24642charlielye wants to merge 12 commits into
charlielye wants to merge 12 commits into
Conversation
…+ services Adds `ipc_codegen/in_process.hpp` (`dispatch_sync`) to the C++ template dir the codegen copies into every generated service. It drives a service's generated `make_<prefix>_handler(ctx)` synchronously across a direct function-call boundary (NAPI/FFI) instead of a socket — the in-process counterpart to the `<prefix>_ipc_server.hpp` socket serve loop. The concrete entry point (a NAPI method or named FFI symbol) is instantiated by the host per service, deliberately not a single global `ipc_ffi_entry`, so multiple services can be co-hosted in one process without symbol collisions. First building block of the TXE in-process spike (run the AVM + world state in the TXE's own process, no bb-avm-sim/aztec-wsdb child processes).
Carves the AVM simulator's dispatch + command handlers out of the bb-avm-sim executable into a transport-agnostic `avm_sim_ffi` library, and adds a plain-C ABI (`avm_ffi.h`: avm_create_ipc / avm_call / avm_destroy) that runs a simulation in-process via the codegen `dispatch_sync` primitive — no socket. The bb-avm-sim executable now just links the library and wraps its dispatch in the socket serve loop (out-of-process path unchanged). Any in-process host — a small NAPI wrapper generated up the stack, pointed at this library — is the other consumer. Barretenberg itself stays free of any Node/NAPI knowledge; the ABI is plain C. avm_execute.hpp gains explicit-specialization declarations for the AvmRequest handlers so both the executable and the library link against them rather than implicitly instantiating the definition-less primary template. Builds green (AVM=ON): libavm_sim_ffi.a exports the three C symbols and bb-avm-sim relinks onto it.
A ~90-line node-addon-api wrapper (`InProcessAvm`) over barretenberg's avm_sim_ffi C ABI. `new InProcessAvm(wsdbPath, cdbPath)` → `avm_create_ipc`; `.call(requestFrame)` → `avm_call` (dispatch_sync, in-process, no subprocess); `.destroy()` → `avm_destroy`. Generic byte-in/byte-out `call`; the only AVM-specific bit is the constructor's dep wiring. Builds standalone with barretenberg's zig toolchain (ABI match) and links its static archives into the .node — no shared lib, no PIC issues (bb is built -fPIC). Barretenberg stays Node/NAPI-free; this wrapper is the in-process consumer of its C ABI, mirroring bb-avm-sim as the out-of-process consumer. Verified: builds, loads in Node, exposes InProcessAvm. This proves the whole native chain (bb C ABI → linked .node → loadable) — the spike's riskiest part. The generalized form (codegen emits this given a lib path) is the follow-up; TS backend + InProcessAvmSimulator + TXE wiring come next.
Wires the in-process AVM through to a working round-trip, behind the existing AvmSimulator interface so nothing downstream changes. - NAPI wrapper `call` now runs avm_call on a worker thread and returns a Promise (was synchronous). Mandatory even for Slice A: the CDB server runs on the JS event loop, so a synchronous AVM call on that thread would block the loop and deadlock against its own CDB requests. Off-thread frees the loop to serve CDB. - `NapiAvmSimulator implements AvmSimulator` — sibling of AvmSimulatorPool: owns the same CDB reverse channel (CdbIpcServer + per-simulate fork registration) and drives the generated AvmService via a NapiBackend (IpcClientAsync over the addon), so the in-process path runs behind the codegen'd API, not bespoke. Only the transport differs: an in-process worker thread vs a child process. - CdbIpcServer.ready() so the addon's synchronous connect waits for the socket to bind. - TXESynchronizer + PublicProcessorTestEnv select the in-process backend behind TXE_IN_PROCESS / AVM_INPROCESS_NODE; the pool remains the default. Verified: the public-processor deployments test (deploy + public calls + CDB callbacks + block-cache revert) passes with AVM_INPROCESS_NODE set — identical results to the subprocess pool, no bb-avm-sim child process. Slice A done.
Adds the generic host_call(target, bytes) reverse channel so an in-process AVM can reach contract data via a direct host callback instead of a CDB socket: - avm_ffi.h gains `avm_host_call_fn` (the native twin of the wasm host_call import — same (target, req, resp) contract) and `avm_create_hostcall`. - HostCallContractDB implements ContractDBInterface over that proxy, speaking the exact same CDB wire protocol/framing as CdbIpcContractDB — only the transport differs (host callback vs socket). - AvmRequest is decoupled from the concrete CDB: it now holds a ContractDBInterface& plus a set_fork_id closure (fork routing is a transport concern, not part of the DB interface), so the socket and host_call CDBs both slot in. bb-avm-sim's out-of-process path is updated to match and still links. Builds green (AVM=ON). NAPI TSFN delivery of host_call + the TS host router (→ CdbIpcServer dispatch) come next; wasm reuses the same (target, bytes) shape.
Completes the generic host_call(target, bytes) reverse channel so the in-process AVM reaches contract data by calling straight into the host, dropping the CDB socket: - host_call callback gains a `void* ctx` (standard C-callback pattern) so the NAPI wrapper can route it to the owning instance's ThreadSafeFunction. - NAPI wrapper: `new InProcessAvm(wsdbPath, onHostCall)` builds a TSFN over the JS onHostCall; the AVM's worker-thread host_call blocks on a std::promise the TSFN callback fulfils — handling onHostCall returning a Promise (the CDB read is async). This is the native twin of the wasm host_call import; same (target, bytes) contract, so the future wasm build reuses the router. - CdbIpcServer can run socketless (listenSocket=false) and exposes handle(bytes) = its generated dispatch; NapiAvmSimulator routes host_call target CDB→handle. registerFork/unregisterFork unchanged. Verified: the public-processor deployments test (deploy + public calls + CDB lookups) passes with the AVM in-process AND no CDB socket. Only the wsdb socket remains between the in-process AVM and the host (Slice C).
…ice C) The generated C++ IPC client was hardwired to a socket (make_client by path suffix). Add a `Transport` (byte request→response function) and a constructor that takes one, routing the per-command `send<>` through it. The path constructor is unchanged — it now just installs a socket-backed Transport — so every existing consumer (CdbIpcContractDB, WsdbIpcMerkleDB, bb-avm-sim) is unaffected. This is the enabler for an in-process AVM to reach a co-hosted world-state server directly (C++→C++) instead of over a socket: WsdbIpcMerkleDB can wrap a WsdbIpcClient built over an in-process transport that drives the wsdb dispatch of a shared WorldState — the last step to zero child processes for the TXE (Slice C). Mirrors the zig/rust clients, which are already backend-parameterized. Builds green (AVM=ON): cdb + wsdb clients regenerate with the injectable transport and all consumers relink; the socket path is untouched.
WsdbScheduler took an `ipc::IpcServer&` but used it for exactly one thing: the `has_pending_request()` check that gates the inline fast path. Replace it with an injected `std::function<bool()>` predicate, so the scheduler (read-batch / write-barrier per-fork ordering + thread pool) is transport-agnostic — unchanged behaviour, but no longer tied to the socket server. This lets the wsdb dispatch run behind a plain C ABI: the socket server passes its `has_pending_request()`, and an in-process (FFI/NAPI) host passes its own predicate. First step toward running wsdb in-process (shared WorldState with the in-process AVM) via the same generalized-backend model as the AVM, rather than reverting to a pre-wire binding. aztec-wsdb relinks; the socket path is unchanged.
Mirror avm_sim_ffi: move the WorldState, per-fork scheduler, and generated dispatch behind a plain-C ABI (wsdb_create / wsdb_call / wsdb_destroy) in a new wsdb_ffi static library. wsdb_call is async — it decodes and schedules synchronously but delivers the response through a callback, so the socket server's concurrency is preserved. aztec-wsdb becomes a thin socket adapter that feeds wire frames into wsdb_call; an in-process host (NAPI wrapper, or a co-linked AVM sharing the WorldState) drives the same ABI without a socket.
The Slice C AVM shape: no sockets. World state is reached through a synchronous wsdb byte transport (avm_wsdb_call_fn — the host drives a co-hosted WorldState's dispatch, so leaf reads stay C++<->C++ and never bounce through JS), and contract data through the existing host_call proxy. The AVM sees a plain WsdbIpcClient built via its Transport ctor and does not know world state is in-process. avm_sim_ffi stays free of any wsdb_ffi / world_state dependency, so bb-avm-sim is unaffected.
Add an InProcessWsdb ObjectWrap over the wsdb_ffi C ABI: it owns a WorldState in-process and exposes call()/destroy() so a generated WsdbService (AsyncApi) drives it exactly like the spawned aztec-wsdb backend, no child process. Add a third InProcessAvm constructor — (inProcessWsdb, onHostCall) — that co-hosts the AVM against the SAME WorldState via avm_create_inprocess: world-state reads go C++<->C++ through a synchronous wsdb trampoline (never bouncing to JS), contract data via host_call. The scheduler's inline fast path is disabled (always-pending predicate) since in-process requests arrive on multiple threads. Together: a TXE session with zero child processes.
…ce C) Wire the TS side so TXE_IN_PROCESS runs both the world state and the AVM in one process sharing a single WorldState, with zero child processes: - world-state: loosen IpcWorldState to drive any AsyncApi (a spawned WsdbService or an in-process NAPI backend); getIpcPath guards on absence. Add NativeWorldStateService.fromWsdbBackend(IpcClientAsync) and buildInProcessWsdbOptions (the in-process twin of getWsdbExtraArgs). - simulator: createInProcessWsdb + NapiWsdbBackend (drive the wsdb AsyncApi over the addon's InProcessWsdb), and NapiAvmSimulator.spawnCoHosted, which hands that same InProcessWsdb to the AVM so both share one WorldState. - txe: TXESynchronizer.createInProcess() creates the shared InProcessWsdb, builds the world state over a NapiWsdbBackend, and co-hosts the AVM. Session teardown already disposes the AVM before closing the world state. TXE_IN_PROCESS now means full in-process (was AVM-only).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spike: TXE runs world state + AVM fully in-process (zero child processes)
Draft / spike — proves feasibility and surfaces the edges; productionization is a follow-up.
The AVM cutover made public execution an out-of-process
bb-avm-simservice and world state anaztec-wsdbservice. Right for production nodes; heavy for the TXE, where each of hundreds of Noir tests spins up its own(aztec-wsdb + bb-avm-sim + cdb)process-triple. Pre-cutover the TXE ran both in-process.This spike makes where a service runs a backend choice expressed through the codegen service model, and has the TXE choose to run both the AVM and world state in one process, sharing a single
WorldState— recovering the pre-cutover topology while production keeps the out-of-process backends unchanged. SetTXE_IN_PROCESS=1.How it fits the service model
wsdb_ffi,avm_sim_ffi). The out-of-process executables (aztec-wsdb,bb-avm-sim) and a small NAPI wrapper are both thin consumers of the same dispatch — the socket server just feeds wire frames into the FFI entry.create/call/destroyis the symmetric backend lifecycle (process for IPC, thread/instance for NAPI).host_call(target, bytes)is the generic reverse channel (same contract as the wasmonHostCall): for NAPI it reaches TS-resident services (the CDB). World state stays C++↔C++ (a synchronous byte transport), so leaf reads never bounce through JS.Shape
wsdb_ffi:wsdb_create/wsdb_call(async) /wsdb_destroy;aztec-wsdbbecomes a thin socket adapter over it.avm_create_inprocess: the AVM reaches world state via a sync wsdb byte-callback and contract data viahost_call.avm_sim_ffistays free ofworld_state, sobb-avm-simis unaffected.avm_inprocessaddon co-hostsInProcessWsdb+ a co-hostingInProcessAvm(inProcessWsdb, onHostCall)— one sharedWorldState.NativeWorldStateService.fromWsdbBackend+NapiWsdbBackend;NapiAvmSimulator.spawnCoHosted;TXESynchronizer.createInProcess().Verification
aztec-wsdb/bb-avm-simstill build and run.public_immutableread + init,nft transfer_in_publicwrite) with noaztec-wsdband nobb-avm-simchild processes.Non-goals (spike)
Production node in-process;
MemoryMerkleDB/ no-disk world state; folding the hand-written NAPI wrapper into the codegen; browser/wasm backend.