mAi: #8 - imagen.jobs queue + worker subcommand (flexsiebels write path)
Async write path for the flexsiebels owner-mode UI: flexsiebels INSERTs into imagen.jobs, the worker on mRiver claims pending rows via LISTEN/NOTIFY + 5s safety poll, runs the same generate pipeline imagen generate uses, and writes the result through internal/cloud into imagen.images. - Schema migration imagen_jobs_init: table + status CHECK + two indexes + owner-scoped RLS + grants + AFTER INSERT trigger publishing on the imagen_jobs channel via pg_notify. - internal/worker: DB-agnostic loop over a Queue interface. Drains the whole pending backlog on each wake. Job-scoped contexts are derived from Background so SIGTERM lets the in-flight generation finish (no half-state). ResetStaleRunning at startup unsticks rows left over from a previous crash. Eight unit tests cover the done / failed / missing-id / drain / NOTIFY-wake / shutdown / transient-error paths against a fake queue (no real Postgres in CI). - cmd/imagen/worker.go: pgx-backed Queue (one dedicated conn for LISTEN + UPDATE), plus the workerPipeline that reuses buildBackend + attachUsageSink + prompt.Apply + buildWriter + maybeCloudSync. The per-job owner_user_id overrides the env-level fallback so each row in imagen.images is attributed correctly. - maybeCloudSync now returns (*cloud.SyncResult, error) so the worker can link imagen.jobs.image_id to the inserted imagen.images row. The CLI generate path keeps printing its stderr summary unchanged. - scripts/imagen-worker.service + .env.example for the systemd --user unit on mRiver. EnvironmentFile lives in ~/.dotfiles and is never committed. - docs/setup-worker-mriver.md walks through installation + the spec's SQL-INSERT smoke; docs/architecture.md grows an "async write path" section. - worker_integration_test.go (env-guarded by IMAGEN_WORKER_INTEGRATION=1) drives one real job through the full pipeline against msupabase using the mock backend, then verifies imagen.images + Storage object landed and the row flipped to done with image_id linked. Verified end-to-end: pickup latency ~7ms, total 74ms, failure path captures error text.
This commit is contained in:
22
scripts/imagen-worker.env.example
Normal file
22
scripts/imagen-worker.env.example
Normal file
@@ -0,0 +1,22 @@
|
||||
# Environment for the imagen-worker.service systemd unit.
|
||||
# Copy to ~/.dotfiles/.env.imagen-worker and fill in real values.
|
||||
# Never commit the populated file — it carries the Supabase service-role key.
|
||||
|
||||
# Direct Postgres DSN for LISTEN/NOTIFY + imagen.jobs UPDATE statements.
|
||||
# PostgREST cannot LISTEN, so the worker connects to Postgres directly.
|
||||
# Host + port + password come from the msupabase compose env on mlake.
|
||||
IMAGEN_WORKER_DATABASE_URL=postgres://postgres:CHANGE_ME@100.99.98.201:6789/postgres?sslmode=disable
|
||||
|
||||
# PostgREST endpoint for the imagen.images cloud-sync writer (same as
|
||||
# `imagen generate`'s cloud-sync code path).
|
||||
SUPABASE_URL=https://supa.flexsiebels.de
|
||||
SUPABASE_SERVICE_KEY=CHANGE_ME
|
||||
|
||||
# Default owner_user_id. Per-job owner from the imagen.jobs row overrides
|
||||
# this, so it's only used as a fallback when a job arrives with a NULL
|
||||
# owner_user_id — which the schema disallows. Keep it set for safety.
|
||||
IMAGEN_OWNER_USER_ID=ac6c9501-3757-4a6d-8b97-2cff4288382b
|
||||
|
||||
# Optional: REPLICATE_API_TOKEN if any imagen.jobs.backend may resolve to
|
||||
# a Replicate adapter instance.
|
||||
# REPLICATE_API_TOKEN=CHANGE_ME
|
||||
Reference in New Issue
Block a user