mAi: #10 - multi-model backend expansion (workflow templates + compare harness)

Path 1 architecture: one comfyui adapter, workflows as data.

- workflow_template.go: embed.FS + token substitution with type-preserving
  whole-value placeholders. ${prompt} → string, ${seed} → int64,
  ${cfg} → float64 — no JSON round-tripping. Partial matches ignored.
- comfyui.go: refactored to load workflow from embedded FS or filesystem
  path. Back-compat preserved: workflow: defaults to flux1-schnell.
- workflows/{flux1-schnell,flux2-klein,sd35-medium}.json — bundled
  templates. flux1-schnell migrated from hardcoded with identical node IDs.
- compare.go: new `imagen compare` subcommand. Sequential N-backend run
  (one GPU on mRock — parallel would OOM), per-backend PNG, sidecar JSON
  with per-model metadata + errors, composite contact sheet via Go image
  package (no ImageMagick dep).
- Sample config gains flux2-klein-local + sd35-medium-local instances.
- docs/backends.md: architecture rationale + per-model HF download paths
  + how to add a new bundled workflow + compare-harness reference.

Live smoke verified: compare mock + flux-schnell-local at 768×768 →
both PNGs written, sidecar JSON has workflow="flux1-schnell" + full
metadata, contact sheet renders. Worker contract (Request → Generate)
unchanged, so flexsiebels /imagine UI API surface preserved.

Tests: 11 existing comfyui + 6 new workflow_template + 5 new compare
tests, all green.

Adding a new model is now yaml + JSON, never Go.
This commit is contained in:
mAi
2026-05-11 17:29:57 +02:00
parent 623dd290c5
commit 8435817ce1
15 changed files with 1638 additions and 122 deletions

View File

@@ -144,15 +144,54 @@ output:
cloud_sync: auto
backends:
# FLUX.1-schnell on the local ComfyUI server. The "workflow" key picks the
# bundled template under internal/backend/workflows/; omit it for back-compat
# (defaults to flux1-schnell). See docs/backends.md for the per-model setup.
flux-schnell-local:
type: comfyui
base_url: http://mrock:8188
workflow: flux1-schnell
# Filename of the unet checkpoint inside the ComfyUI server's
# models/unet/ directory. See docs/setup-comfyui-mrock.md.
# models/unet/ directory.
model: flux1-schnell.safetensors
vae: ae.safetensors
clip_l: clip_l.safetensors
clip_t5: t5xxl_fp8_e4m3fn.safetensors
dtype: fp8_e4m3fn
default_steps: 4
default_sampler: euler
default_scheduler: simple
default_cfg: 1.0
# FLUX.2 [klein] 4B distilled — sub-second on RTX 4070 Ti SUPER.
# Weights: BFL non-commercial; flux-2-klein-base-4b-fp8 in models/unet/,
# qwen_3_4b in models/text_encoders/, flux2-vae in models/vae/.
flux2-klein-local:
type: comfyui
base_url: http://mrock:8188
workflow: flux2-klein
model: flux-2-klein-base-4b-fp8.safetensors
vae: flux2-vae.safetensors
clip: qwen_3_4b.safetensors
dtype: fp8_e4m3fn
default_steps: 4
default_sampler: euler
default_scheduler: simple
default_cfg: 1.0
guidance: 4.0
# SD3.5 medium — single-checkpoint variant that bundles the three text
# encoders inside the .safetensors. Drop into models/checkpoints/.
sd35-medium-local:
type: comfyui
base_url: http://mrock:8188
workflow: sd35-medium
model: sd3.5_medium_incl_clips_t5xxlfp8scaled.safetensors
default_steps: 28
default_sampler: dpmpp_2m
default_scheduler: sgm_uniform
default_cfg: 4.5
shift: 3.0
mock:
type: mock