Agent-ergonomics fixes from CLI transcript mining: py3.12 sandbox, list_instances, usage-on-error, JSON query-raw errors

[qu0b]

Mined every Claude Code session that drove the panda CLI (June 12 – July 15: 777 invocations, 43 hard failures) and fixed the failure classes that repeatedly burned real devnet investigations. Each fix is verified against the exact command that failed in the transcripts.

sandbox: Python 3.11 → 3.12 (PEP 701)

LLM-generated code routinely nests same-quote strings and backslashes inside f-string expressions — a SyntaxError on 3.11, legal on 3.12. Base image bumped to pinned python:3.12-slim; the hash lock was recompiled with the same pinned uv (0.11.17) and resolution is unchanged (header-only diff). Verified: image builds, ethpandaops/pandas import, transcript-failing f-strings parse on 3.12.13.

dora + ethnode: get_clients / list_instances

Models guess ethnode.list_instances(network) after list_networks() and hit an ImportError; instance labels were documented as non-enumerable. They are enumerable: Dora's /api/v1/clients/consensus client_name is exactly the ethnode instance label (verified against live hoodi). New dora.get_clients server op + Python wrapper, and ethnode.list_instances(network) returns sorted {name, status} — status distinguishes online/offline nodes for free. Covered by two new server tests.

cli: stop making agents guess syntax

SilenceUsage left flag typos as a bare Error: unknown flag: --network with no correction path — transcripts show 6+ guessed-flag failures (--network, --like, --database, --datasource, -c, arg-count misses). Now:

• unknown-flag errors append the command's usage line + --help pointer (SetFlagErrorFunc)
• arg-count errors carry the usage line too (Args validators wrapped tree-wide)
• panda execute -c works as shorthand for --code

cli: machine output mode — JSON in, JSON out

The single most frequent wasted-iteration pattern (~15 occurrences): panda clickhouse query-raw ... 2>&1 | python3 json.load fails, the JSONDecodeError masks the real ClickHouse error, and the agent retries blind. The general condition is stdout is a JSON document and the consumer merges stderr into the pipe — which the transcripts also do with -o json on other commands.

So the rule is per-run, not per-command: when output is JSON (-o json/--json, or an always-JSON command like query-raw), the CLI owns error rendering — stdout carries exactly one JSON document, stderr stays empty, and the exit code conveys failure. Update notices are suppressed in that mode too (they broke even successful pipelines). Human text mode is untouched: cobra still prints Error: ... to stderr.

This also fixes two pre-existing machine-honesty bugs that the mode change makes safe to correct: panda execute --json and panda build --json exited 0 when the execution or build had actually failed, unlike their text paths.

server: JSON-RPC rejections are not gateway failures

A JSON-RPC error object rides back on a healthy node connection — the node processed and rejected the request. Returning 502 (with the CLI's "temporarily unreachable — retry" hint) made deterministic failures look transient: transcripts show an agent retrying debug_getBadBlocks (-32601) four hours after first learning the client doesn't support it, and revert reasons ("stack underflow", "invalid opcode") arriving triple-wrapped as gateway errors during EIP testing. JSON-RPC errors now return 400 with the error's data payload appended (it often carries the answer, e.g. a revert reason), and the CLI adds a do-not-retry hint keyed on the message — so it also improves output against older servers still wrapping these in 502.

embedding + search: outages are bounded and honest

On 2026-07-08 the embedding upstream (OpenRouter) hung and three consecutive panda search calls each blocked for the full 2-minute batch timeout during a live devnet investigation. Root cause: a single remoteEmbedTimeout shared by 500-document index batches and single-query embeds, with server and proxy timeouts identical so the server always timed out first with a generic error.

• single-item embeds (the interactive search shape) get a 15s deadline; batches get 130s, deliberately outlasting the proxy's 2-minute upstream timeout so its specific error surfaces
• search failures from an unreachable embedding upstream, a warming index, or an index that never activated return 503 instead of 400 — retryable service conditions, not query errors. Filter-value rejections ("unknown category") stay 400
• CLI adds retry-shortly hints for both conditions

Test plan

• go test -race ./pkg/... ./modules/... clean, make lint 0 issues
• make docker-sandbox builds; PEP 701 smoke test + new sandbox functions import in the image
• End-to-end against a running server: flag/arg errors show usage, -c executes, query-raw success and failure both parse through 2>&1 | json.load
• New unit tests: dora.get_clients (bare list + non-JSON body), ethnode execution_rpc JSON-RPC error → 400 with data, searchErrorStatus classification, serverErrorHint for JSON-RPC/-32601 across old and new server statuses

[github-actions]

🐼 Smoke eval — c0129bc: ✅ 8/8 pass

📊 Interactive report — tokens p50 17,174 · tokens/solve 19,913.

Reference points: qu0b/cli-agent-ergonomics@862f602 100% · qu0b/cli-agent-ergonomics@189cd99 100% · qu0b/cli-agent-ergonomics@361a627 100%.

question	result	tokens	tools
forky_node_coverage	✅	16,491	7
tracoor_node_coverage	✅	16,661	6
mainnet_block_arrival_p50	✅	15,725	7
list_datasources	✅	13,139	2
block_count_24h	✅	20,410	11
missed_slots_24h	✅	25,047	20
chartkit_default_arrival_distribution	✅	34,143	19
storage_upload_session_scoped	✅	17,687	11

🔭 Langfuse traces (8 runs; ⚠️ = failed)

• forky_node_coverage
• tracoor_node_coverage
• mainnet_block_arrival_p50
• list_datasources
• block_count_24h
• missed_slots_24h
• chartkit_default_arrival_distribution
• storage_upload_session_scoped

_{The report walks this branch's commits against the master baseline and the most recent release. A self-contained copy is in the run's eval-smoke-* artifact.}