An "ability to learn" ML challenge — two-script submissions, locked data, challenge-owned scoring.
Overview · Miners · Validators · Architecture · Scoring · Security
PRISM is a BASE subnet that measures a model's ability to learn from
scratch. Miners submit a two-script bundle — architecture.py (build_model(ctx)) and
training.py (train(ctx)) — and the challenge owns everything else: a locked FineWeb-Edu
dataset (read-only, no network) and the scoring. The miner owns the model and the training loop;
the challenge owns the data and the score.
Every scored run is re-executed by the challenge under a forced random init, so the score is a prequential (online) compression metric in bits-per-byte — the area under the from-scratch loss curve, normalized by bytes consumed. Weights are derived from those scores through the LLM gateway-gated pipeline and published dry-run. Faster learning ⇒ better compression ⇒ better score.
flowchart LR
M[Miner two-script bundle] --> G{Static sandbox + param cap}
G -- reject --> X[[rejected]]
G --> L{LLM gateway hard gate}
L -- reject --> X
L --> A[Master assigns 1 GPU work unit]
A --> V[Validator re-executes<br/>forced random init]
V --> S[Prequential bpb + held-out delta]
S --> W[Two-tier dry-run weights]
- Submit — a signed
architecture.py+training.pybundle (a single combined module is rejected). - Static gates — AST sandbox, 150M parameter cap, single-node multi-GPU contract; any failure is terminal before GPU.
- LLM hard gate — a strong model reviews both scripts via the master gateway; a
rejectis terminal. - Forced-init re-execution — one validator re-runs the loop on the locked FineWeb-Edu train split and captures the online loss itself (miner-reported numbers are ignored).
- Scoring — the master computes prequential bits-per-byte plus a secret held-out delta tie-breaker.
- Weights — emission splits two-tier (best architecture
0.60/ best training variant0.40), published dry-run viaget_weights(never on-chain here).
Common cheats are inert, not merely detected:
- No pretrained weights — forced random init makes smuggled weights inert; an anomalous step-0 loss zeroes the score; the container runs
network=none. - No metric gaming — the challenge recomputes the metric from the loss it captured; miner-reported numbers and manifests are ignored.
- No memorization — the secret
val/testsplits never leave the master; an excessive train-vs-held-out gap is penalized. - Deterministic — fixed seeds and a challenge-controlled data order reproduce the same score within tolerance.
PRISM can move GPU re-execution onto miner-funded workers (deployed on Lium/Targon via the BASE
base worker CLI). Validators then run verify-only plausibility checks plus probabilistic audits,
and each result carries an ExecutionProof (manifest hash + worker sr25519 signature, with optional
image-digest and attestation tiers). Gated behind worker_plane (default off). See the
worker deployment guide.
| Guide | Contents |
|---|---|
| Overview | The subnet in one page |
| Miner guide | Build and submit a two-script bundle |
| Validator guide | Run evaluation on your own broker |
| Architecture | Service design and forced-init re-execution |
| Submission format | The two-script contract and PrismContext |
| Scoring & rewards | Prequential bits-per-byte and tie-breakers |
| Scaling | Single-node multi-GPU contract |
| Security model | Sandbox, LLM gate, anti-cheat |
| API | Internal and public routes |
| Operators | Deploy and run |
uv run ruff check .
uv run mypy
uv run pytest --cov=prism_challenge --cov-fail-under=80GPU re-execution, HuggingFace publication, and LLM provider calls are mocked in tests; the real GPU, HuggingFace token, and provider keys are wired only at deploy.
Apache-2.0