feat(vae): support running VAEs on CPU via cpu_only setting

[Pfannkuchensack]

Summary

Extends the cpu_only mechanism from #8777 (text encoders) to VAE decode. Adds a cpu_only field to all standalone VAE configs; the loader already forces standalone configs with cpu_only=True onto the CPU. The 7 decode invocations now move latents to the VAE's effective device instead of hard-coding CUDA, and the SD/SDXL path falls back to fp32 on CPU (fp16 conv is unsupported there). Adds a "Run on CPU" toggle to the VAE model settings panel and regenerates the API schema.

Decode-only for now; encode and main-model VAE submodels are unchanged.

Backend

• cpu_only field added to all standalone VAE configs (SD1/SD2/SDXL/FLUX checkpoint, SD1/SDXL diffusers, FLUX.2/Qwen-Image/Anima, FLUX.2 diffusers).
• No loader change needed — _get_execution_device already returns cpu for any standalone config with cpu_only=True.
• All 7 latents-to-image invocations (l2i, FLUX, FLUX.2, SD3, CogView4, Qwen-Image, Z-Image, Anima) now send latents to get_effective_device(vae) rather than TorchDevice.choose_torch_device().
• SD/SDXL decode is forced to fp32 when the VAE is on CPU (fp16 conv is not implemented on CPU).
• Cache invalidation on toggle is already covered by the existing _LOAD_AFFECTING_SETTINGS (cpu_only is in it).

Frontend

• New VAEModelSettings panel + useVAEModelSettings hook (mirrors the encoder panel, reuses the shared form-data type).
• ModelView renders the panel for type === 'vae'.

Related Issues / Discussions

Closes #7276 (VAE part — the CLIP/text-encoder part was delivered in #8777)

QA Instructions

1. In Model Manager, select a standalone VAE model → the settings panel now shows a Run on CPU toggle. Enable it and save.
2. Run a generation that decodes with that VAE (e.g. SDXL → Latents to Image). Confirm:
   • The image decodes correctly (no device-mismatch / "Half not implemented on CPU" errors).
   • During decode, VRAM usage stays flat for the VAE (it loads into system RAM, not VRAM) — visible as reduced peak VRAM on low-memory GPUs.
3. Toggle the setting off, save, regenerate → decode runs on GPU again (no restart needed; the cached entry is evicted on the settings change).
4. Repeat the smoke test across architectures: SD1/SDXL (l2i), FLUX, FLUX.2, SD3, CogView4, Qwen-Image, Z-Image, Anima.
5. Expected trade-off: CPU decode is noticeably slower (esp. SDXL at high resolution, now fp32) — this is intentional (VRAM savings vs. speed).

Tested on SD1,SDXL,Flux1,Flux2klein9b,Anima,Zimage. Open are SD3,CogView4, Qwen-Image,Flux2klein4b(should work). Anima is really slow on CPU, anything else is ok.

Automated:

• pytest tests/backend/model_manager/load/test_load_default_cpu_only.py tests/app/routers/test_update_model_record_cache_invalidation.py
• Frontend: pnpm lint + pnpm test:no-watch (all green).

Merge Plan

Standard merge. No DB schema or redux migration.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Review

Overview

Extends the #8777 text-encoder cpu_only mechanism to standalone VAE decode. I verified the load-bearing claims against the branch: _get_execution_device (load_default.py:120) already handles any config with a cpu_only attr, so no loader change is genuinely needed; cache invalidation is covered by the explicit cpu_only comparison at model_manager.py:471; the new INFO log does mirror an existing per-lock INFO line (model_cache.py:554); and the new test passes (4/4 locally). The design is sound and consistent with the established precedent.

Issues

1. i18n copy is text-encoder-specific — wrong on the VAE panel (should fix)
VAEModelSettings.tsx reuses modelManager.runOnCpu, which reads "Run text encoder model on CPU only", and the cpuOnly popover says "only the text encoder component will run on CPU… The conditioning outputs are automatically moved to GPU for the denoiser." Both render on the VAE settings panel and are incorrect/confusing there. Suggest a VAE-specific label (e.g. runVaeOnCpu) and popover feature, or generalizing the shared strings.

2. lock() INFO log fires for all models on GPU-less installs, mislabeled (cpu_only) (should fix)
The early-return at model_cache.py:443 triggers whenever compute_device.type == "cpu" — which is true for every model on a CPU-only install (no GPU), not just models configured cpu_only. Previously that branch logged at debug; now every lock of every model on such systems emits "Loaded model ... onto cpu device (cpu_only); skipping VRAM load" at INFO — noisy and factually wrong about the cause. Suggest only using the (cpu_only) wording (and INFO level) when the model's compute device differs from the cache's default execution device, keeping the plain-CPU-install case at debug.

3. The l2i comment's rationale is stale (minor)
"fp16 VAE ops are not supported on CPU" is false on the pinned torch — conv2d and conv3d run fine in fp16 on CPU with torch 2.10 (verified empirically). The fp32 force is still the right call (SD/SDXL VAE has known fp16 overflow → black images, and CPU fp16 conv is much slower than fp32), but the comment should give the real reason. Related asymmetry worth a look: standalone Qwen and Anima VAEs load as fp16 on CUDA-default systems (vae.py uses self._torch_dtype) and their decode paths do fp16 conv3d on CPU with no dtype guard. It works, but fp16 CPU conv3d is very slow — likely a big part of the "Anima is really slow on CPU" observation. Casting to fp32/bf16 on the CPU path there would probably help substantially.

4. Frontend duplication (minor)
useVAEModelSettings is byte-for-byte useEncoderModelSettings with a narrower type, and VAEModelSettings.tsx duplicates EncoderModelSettings.tsx including the whole DefaultCpuOnly controller. Widening the existing hook/component's accepted config union to include VAEModelConfig would eliminate ~140 duplicated lines and keep the two panels from drifting. If they stay separate, the shared EncoderModelSettingsFormData type import from the encoder folder is an awkward cross-dependency.

5. SD3/CogView4 changes are no-ops (informational)
There are no standalone SD3 or CogView4 VAE configs, so those graphs always use the main model's VAE submodel, where cpu_only never applies — get_effective_device returns the GPU device exactly as before. The changes are harmless consistency, but the QA instruction to smoke-test cpu_only on SD3/CogView4 can't actually be exercised since the toggle can't exist for those architectures.

What's good

• Zero-loader-change design: the config field alone activates the existing _get_execution_device → ModelCache.put(execution_device=...) path, and toggle-time eviction was already wired up via _load_settings_changed.
• get_effective_device is the right helper — it handles partially-loaded models and matches all seven existing text-encoder call sites from #8777.
• The new test file is focused and passes; testing the helper via ModelLoader.__new__ is a pragmatic way to avoid heavy fixtures, and the SimpleNamespace-without-attrs case guards the hasattr fallback.
• Schema/openapi regen is consistent with the pattern the encoder configs established.

Future Enhancement

After the multi-GPU PR goes in, a good follow-on PR would be to execute the VAE on an idle GPU in a multi-GPU environment, following the same pattern as is used for the text encoder.

Verdict

Approve with minor changes — items 1 and 2 are the ones I'd want fixed before merge (both small); 3–5 are comment/cleanup suggestions. No security concerns, no migration needed, and the untested-architecture risk turns out to be perf-only, not correctness.

[lstein]

Minor changes requested. Please see earlier review comment.

[lstein]

Everything looks good!