m/mGPUmanager
1
0
Fork 0
Code Issues 13 Pull Requests Actions Packages Projects Releases Wiki Activity

comfyui's vram_budget_mib is 11000, measured usage is 4346 — the broker refuses image jobs that would fit comfortably #11

New Issue
Open
opened 2026-07-28 23:46:53 +00:00 by mAi · 3 comments
mAi commented 2026-07-28 23:46:53 +00:00
Copy Link

Measured, not estimated

consumers.yaml:  comfyui.vram_resident_mib: 11000
nvidia-smi during a real flux-schnell generation:  4346 MiB

The declared budget is 2.5x the actual requirement. flux-schnell is a much smaller model than flux-dev; the 11000 figure looks sized for the latter.

What it costs

The broker gates on the declared number, so it refuses jobs that would run fine. Observed 2026-07-29 00:19-00:23, six consecutive failures:

broker: insufficient VRAM to load the model even after eviction
(untracked GPU usage, e.g. a game?): cannot free enough VRAM for comfyui
even after evicting every evictable consumer

Free VRAM at the time was ~9300 MiB — far more than the 4346 actually needed, and less than the 11000 demanded. The job was refused for want of memory it did not need.

A generation run after the machine reconnected succeeded with 8967 MiB free, i.e. still below the declared budget, and settled at 11049/16376 total with whisper, ollama, Brave and an RDP session all resident. It fits comfortably.

The error message actively misleads

"untracked GPU usage, e.g. a game?" sent the investigation at desktop processes. Brave (403 MiB) and a stale xfreerdp3 session (420 MiB) were real and worth cleaning up, but they were never the cause — 823 MiB against a 4346 MiB requirement on a 16 GB card. I spent an hour doing increasingly precise arithmetic on a number that was simply wrong, and m's "ComfyUI braucht keine 11gb... das ging sonst auch" was what broke the loop.

When a budget check fails, the message should state required vs available vs freeable, so a reader can see at once whether the requirement is plausible. A message naming a probable external cause invites confirmation bias toward that cause.

Fix

  1. Correct comfyui.vram_resident_mib to the measured value plus a margin — 4346 measured, so something like 6000 leaves comfortable headroom and still guards the card.
  2. If both flux-schnell and flux-dev may run, the budget cannot be a single constant on the consumer. Either declare per-model budgets, or set it to the largest model actually installed and document which one.
  3. Improve the refusal message: required=X available=Y freeable=Z after evicting [names]. The current wording names a hypothesis, not a measurement.

Second, separate finding: whisper-server cannot actually be evicted

whisper-server:
  vram_resident_mib: 2050
  # No HTTP unload; mGPUmanager evicts via systemd restart (Schritt 5).

The systemd-restart eviction path appears unimplemented, so whisper's ~2 GB is pinned in practice while the broker's accounting may treat it as reclaimable. That did not cause this incident, but it means the eviction budget is smaller than the config implies, and it will matter for genuinely large models.

Third: gpu_resident_mib reads 0 for every consumer

/v1/status reports gpu_resident_mib: 0 for comfyui, mvoice, ollama and whisper-server, while nvidia-smi shows 4346 / 242 / 2012 / 2082 respectively. So the broker's per-consumer residency accounting is not populated, and any decision derived from it is working off zeros. Worth checking whether the scheduler relies on this field.

Acceptance

  • A flux-schnell job is admitted when free VRAM exceeds its real requirement.
  • The refusal message states required, available and freeable as numbers.
  • gpu_resident_mib reflects measured residency, or the field is removed if nothing populates it.
## Measured, not estimated ``` consumers.yaml: comfyui.vram_resident_mib: 11000 nvidia-smi during a real flux-schnell generation: 4346 MiB ``` The declared budget is **2.5x the actual requirement**. `flux-schnell` is a much smaller model than `flux-dev`; the 11000 figure looks sized for the latter. ## What it costs The broker gates on the declared number, so it refuses jobs that would run fine. Observed 2026-07-29 00:19-00:23, six consecutive failures: ``` broker: insufficient VRAM to load the model even after eviction (untracked GPU usage, e.g. a game?): cannot free enough VRAM for comfyui even after evicting every evictable consumer ``` Free VRAM at the time was ~9300 MiB — far more than the 4346 actually needed, and less than the 11000 demanded. The job was refused for want of memory it did not need. A generation run after the machine reconnected succeeded with **8967 MiB free**, i.e. still below the declared budget, and settled at 11049/16376 total with whisper, ollama, Brave and an RDP session all resident. It fits comfortably. ## The error message actively misleads "untracked GPU usage, e.g. a game?" sent the investigation at desktop processes. Brave (403 MiB) and a stale `xfreerdp3` session (420 MiB) were real and worth cleaning up, but they were never the cause — 823 MiB against a 4346 MiB requirement on a 16 GB card. I spent an hour doing increasingly precise arithmetic on a number that was simply wrong, and m's "ComfyUI braucht keine 11gb... das ging sonst auch" was what broke the loop. When a budget check fails, the message should state **required vs available vs freeable**, so a reader can see at once whether the requirement is plausible. A message naming a probable external cause invites confirmation bias toward that cause. ## Fix 1. **Correct `comfyui.vram_resident_mib` to the measured value** plus a margin — 4346 measured, so something like 6000 leaves comfortable headroom and still guards the card. 2. **If both flux-schnell and flux-dev may run**, the budget cannot be a single constant on the consumer. Either declare per-model budgets, or set it to the largest model actually installed and document which one. 3. **Improve the refusal message**: `required=X available=Y freeable=Z after evicting [names]`. The current wording names a hypothesis, not a measurement. ## Second, separate finding: whisper-server cannot actually be evicted ```yaml whisper-server: vram_resident_mib: 2050 # No HTTP unload; mGPUmanager evicts via systemd restart (Schritt 5). ``` The systemd-restart eviction path appears unimplemented, so whisper's ~2 GB is pinned in practice while the broker's accounting may treat it as reclaimable. That did not cause this incident, but it means the eviction budget is smaller than the config implies, and it will matter for genuinely large models. ## Third: `gpu_resident_mib` reads 0 for every consumer `/v1/status` reports `gpu_resident_mib: 0` for comfyui, mvoice, ollama and whisper-server, while `nvidia-smi` shows 4346 / 242 / 2012 / 2082 respectively. So the broker's per-consumer residency accounting is not populated, and any decision derived from it is working off zeros. Worth checking whether the scheduler relies on this field. ## Acceptance - A flux-schnell job is admitted when free VRAM exceeds its real requirement. - The refusal message states required, available and freeable as numbers. - `gpu_resident_mib` reflects measured residency, or the field is removed if nothing populates it.
mAi commented 2026-07-29 00:00:30 +00:00
Author
Copy Link

Root cause found, and it is not the budget number — the broker does not credit memory the requesting consumer already holds

The wrong 11000 was real and worth fixing, but it was not the cause. Correcting my own report.

The measurement

After one successful generation, with vram_resident_mib already corrected to 6000:

nvidia-smi:  comfyui process holds 4346 MiB   (model loaded)
broker:      free=4863  reserved=1024  →  available 3839
             comfyui  loaded=True  budget=6000  gpu_resident_mib=0
             → refuses: short by 2161

ComfyUI already has its model resident, and the broker asks for the full 6000 MiB on top of it as free memory. It never credits the 4346 MiB the consumer is already holding.

Why this produces "works once, then never"

attempt comfyui state free available verdict
1st after a cold broker not loaded 8967 7943 passes
every one after loaded, holding 4346 4863 3839 refused

That is exactly the reported symptom: the first generation succeeds, every subsequent one fails with the same message until something clears the model. It also explains why the failure looked intermittent and machine-state-dependent rather than deterministic.

Unloading via POST /api/free {"unload_models":true,"free_memory":true} takes free from 4863 back to 8873 and the next job passes — one job, then it recurs.

This is the same defect as the gpu_resident_mib: 0 note in the issue body

I filed that as a third, minor observation. It is the root cause. The broker cannot credit resident memory because its per-consumer residency accounting is never populated — every consumer reports gpu_resident_mib: 0 while nvidia-smi shows 4346 / 2082 / 2012 / 242. Fixing the admission logic without populating that field is not possible; they are one fix.

Suggested fix

Admission should be:

available = free - reserved + resident(requesting_consumer)

or equivalently, only require the delta the consumer still needs to load. A consumer that is already loaded and healthy needs zero additional VRAM to serve another request against the same model.

A cheaper stopgap, if the accounting fix is not quick: have the broker call comfyui's configured unload endpoint when the image lease is released. That trades reload latency (~seconds) for correctness and removes the sawtooth entirely. It uses the unload block already present in consumers.yaml.

Also worth stating in the error message

The current wording — "untracked GPU usage, e.g. a game?" — is wrong in this scenario in a way that costs real time. The memory it cannot account for is its own consumer's, not an external game. I chased desktop processes (Brave, a stale xfreerdp3 session, 823 MiB combined) for an hour on that hint. required=X available=Y resident=Z freeable=W after evicting [names] would have made the answer visible immediately.

Correction to the issue body

The body says the 11000 figure caused the 00:19 failures. That is probably wrong: free was ~9300 then, which exceeds the 4346 actually needed but also exceeded nothing relevant — if comfyui was already loaded at that point, the same residency bug applies and the budget number is incidental. Treat the budget correction as a separate, real, smaller improvement rather than the fix.

## Root cause found, and it is not the budget number — the broker does not credit memory the requesting consumer already holds The wrong 11000 was real and worth fixing, but it was not the cause. Correcting my own report. ### The measurement After one successful generation, with `vram_resident_mib` already corrected to 6000: ``` nvidia-smi: comfyui process holds 4346 MiB (model loaded) broker: free=4863 reserved=1024 → available 3839 comfyui loaded=True budget=6000 gpu_resident_mib=0 → refuses: short by 2161 ``` **ComfyUI already has its model resident, and the broker asks for the full 6000 MiB on top of it as free memory.** It never credits the 4346 MiB the consumer is already holding. ### Why this produces "works once, then never" | attempt | comfyui state | free | available | verdict | |---|---|---|---|---| | 1st after a cold broker | not loaded | 8967 | 7943 | **passes** | | every one after | loaded, holding 4346 | 4863 | 3839 | **refused** | That is exactly the reported symptom: the first generation succeeds, every subsequent one fails with the same message until something clears the model. It also explains why the failure looked intermittent and machine-state-dependent rather than deterministic. Unloading via `POST /api/free {"unload_models":true,"free_memory":true}` takes free from 4863 back to 8873 and the next job passes — one job, then it recurs. ### This is the same defect as the `gpu_resident_mib: 0` note in the issue body I filed that as a third, minor observation. It is the root cause. The broker **cannot** credit resident memory because its per-consumer residency accounting is never populated — every consumer reports `gpu_resident_mib: 0` while `nvidia-smi` shows 4346 / 2082 / 2012 / 242. Fixing the admission logic without populating that field is not possible; they are one fix. ### Suggested fix Admission should be: ``` available = free - reserved + resident(requesting_consumer) ``` or equivalently, only require the delta the consumer still needs to load. A consumer that is already loaded and healthy needs **zero** additional VRAM to serve another request against the same model. A cheaper stopgap, if the accounting fix is not quick: have the broker call comfyui's configured `unload` endpoint when the image lease is released. That trades reload latency (~seconds) for correctness and removes the sawtooth entirely. It uses the `unload` block already present in `consumers.yaml`. ### Also worth stating in the error message The current wording — "untracked GPU usage, e.g. a game?" — is wrong in this scenario in a way that costs real time. The memory it cannot account for is **its own consumer's**, not an external game. I chased desktop processes (Brave, a stale `xfreerdp3` session, 823 MiB combined) for an hour on that hint. `required=X available=Y resident=Z freeable=W after evicting [names]` would have made the answer visible immediately. ### Correction to the issue body The body says the 11000 figure caused the 00:19 failures. That is probably wrong: free was ~9300 then, which exceeds the 4346 actually needed but also exceeded nothing relevant — if comfyui was already loaded at that point, the same residency bug applies and the budget number is incidental. Treat the budget correction as a separate, real, smaller improvement rather than the fix.
mAi commented 2026-07-29 10:00:26 +00:00
Author
Copy Link

Measured on mRock — the 4346 figure is the settled value, not the peak

I re-measured rather than taking 4346 on trust, and the number changes the recommendation in this issue.

Real flux1-schnell generations, 1024x1024, fp8_e4m3fn — these are ImaGen's actual defaults, checked in internal/backend/comfyui.go:270-271 and :651, not assumed. Sampling comfyui's PID every 0.5 s.

Cold run (comfyui idle at 346 MiB beforehand), full trace in MiB:

346 346 346 346 | 5626 5626 5626 5626 | 5146 | 2074 | 7706 | 8410 x10 | 7706 7194 6906 | 4154 4154 4154 4154
MiB
Peak, cold 8410
Peak, warm (real regeneration, new seed, weights cached) 6170
Settled after either run 4154

The trace shows comfyui loading CLIP/T5 (5626), releasing it (2074), then loading the UNet and sampling (8410), then settling to 4154 of cached weights.

4154 is the same number as this issue's 4346, within noise. So 4346 was measured after the generation. It records what comfyui keeps cached, not what it needs while working. A budget has to cover the peak, because the peak is where an OOM happens.

Consequence: all three candidate values are wrong, in two directions

Value Where Verdict
13000 repo too high — refuses jobs that fit
11000 old host value too high — this caused the six refusals
6000 current host value, and this issue's suggestion too low — below both the cold peak (8410) and the warm peak (6170)

The cold peak governs, because the broker only evicts when comfyui is not already resident — that is the cold path by definition. A budget of 6000 admits a job at ~6.3 GB free that will peak at 8.4 GB. It has not bitten yet only because free VRAM has been ample since.

So the recommendation in this issue ("something like 6000") would trade a false-refusal bug for a real-OOM bug. Flagging that rather than shipping it.

What I have and have not changed

Not changed: the constant. It is still 13000 in the repo and 6000 on the host. The correct value is a decision with real risk on both sides, and it is now with the head — my measurement contradicts the issue's own recommendation, so it should not be applied silently. My working recommendation is ~9000 (cold peak + margin), with the caveat that it clears the reported incident (~9300 free) by only 44 MiB, which is thin enough that a single constant may simply be the wrong shape here: peak depends on cold-vs-warm and on requested resolution, which ImaGen passes through per request as template variables.

Changed: the refusal message. Commit https://mgit.msbls.de/m/mGPUmanager/commit/e384021 on branch mai/tesla/vram-truth.

A refused lease now reports measurements:

cannot fit comfyui: required=13256 MiB available=1024 MiB freeable=0 MiB; not evictable [ollama(priority)]

It carries required / available / freeable MiB, what it already evicted, and every resident consumer it may not touch with the rule protecting each — priority, coexists, or no-unload-mechanism. errors.Is(ErrInsufficientVRAM) still matches through Unwrap, so existing callers are unaffected.

Tests TestRefusalStatesMeasurements and TestRefusalCountsFreeableAndEvicted; both verified to fail without the change. One test asserts the message contains no cause-hypothesis wording, so this cannot silently regress.

Note on the wording quoted in this issue: untracked GPU usage, e.g. a game? is not this repo's string. It is the client's, in ImaGen internal/backend/broker.go:20. The broker half is fixed here; that half needs a separate change in ImaGen, otherwise the misleading hypothesis still reaches the reader.

Third finding confirmed live

gpu_resident_mib: 0 for every consumer, verified on the running broker after today's deploy:

{"comfyui":{"loaded":true,"gpu_resident_mib":0},"mvoice":{"loaded":false,"gpu_resident_mib":0},
 "ollama":{"loaded":true,"gpu_resident_mib":0},"whisper-server":{"loaded":true,"gpu_resident_mib":0}}

while nvidia-smi showed real residency for all four. That is issues #4/#6 and is the next step of this work.

Deployed and live-verified

The binary on mRock now carries this change plus the merged #5 eviction fix (601f5e2), which had never been deployed. Forced a real eviction under real pressure to verify rather than relying on a health check:

evicted consumer victim=ollama target=comfyui free_mib_after=4786 need_mib=6000
POST /v1/lease status=200 ms=2232

Free VRAM 4786 → 7065 MiB, ollama's runners released and lazily cold-reloaded afterwards. healthz, /v1/status and /v1/gamemode all verified after restart; previous binary kept as bin/mgpumanager.bak.

## Measured on mRock — the 4346 figure is the settled value, not the peak I re-measured rather than taking 4346 on trust, and the number changes the recommendation in this issue. Real `flux1-schnell` generations, 1024x1024, `fp8_e4m3fn` — these are ImaGen's actual defaults, checked in `internal/backend/comfyui.go:270-271` and `:651`, not assumed. Sampling comfyui's PID every 0.5 s. **Cold run** (comfyui idle at 346 MiB beforehand), full trace in MiB: ``` 346 346 346 346 | 5626 5626 5626 5626 | 5146 | 2074 | 7706 | 8410 x10 | 7706 7194 6906 | 4154 4154 4154 4154 ``` | | MiB | |---|---| | Peak, cold | **8410** | | Peak, warm (real regeneration, new seed, weights cached) | **6170** | | Settled after either run | **4154** | The trace shows comfyui loading CLIP/T5 (5626), releasing it (2074), then loading the UNet and sampling (8410), then settling to 4154 of cached weights. **4154 is the same number as this issue's 4346, within noise.** So 4346 was measured *after* the generation. It records what comfyui keeps cached, not what it needs while working. A budget has to cover the peak, because the peak is where an OOM happens. ## Consequence: all three candidate values are wrong, in two directions | Value | Where | Verdict | |---|---|---| | 13000 | repo | too high — refuses jobs that fit | | 11000 | old host value | too high — **this caused the six refusals** | | 6000 | current host value, and this issue's suggestion | **too low** — below both the cold peak (8410) and the warm peak (6170) | The cold peak governs, because the broker only evicts when comfyui is *not* already resident — that is the cold path by definition. A budget of 6000 admits a job at ~6.3 GB free that will peak at 8.4 GB. It has not bitten yet only because free VRAM has been ample since. So the recommendation in this issue ("something like 6000") would trade a false-refusal bug for a real-OOM bug. Flagging that rather than shipping it. ## What I have and have not changed **Not changed: the constant.** It is still 13000 in the repo and 6000 on the host. The correct value is a decision with real risk on both sides, and it is now with the head — my measurement contradicts the issue's own recommendation, so it should not be applied silently. My working recommendation is ~9000 (cold peak + margin), with the caveat that it clears the reported incident (~9300 free) by only 44 MiB, which is thin enough that a single constant may simply be the wrong shape here: peak depends on cold-vs-warm *and* on requested resolution, which ImaGen passes through per request as template variables. **Changed: the refusal message.** Commit https://mgit.msbls.de/m/mGPUmanager/commit/e384021 on branch `mai/tesla/vram-truth`. A refused lease now reports measurements: ``` cannot fit comfyui: required=13256 MiB available=1024 MiB freeable=0 MiB; not evictable [ollama(priority)] ``` It carries required / available / freeable MiB, what it already evicted, and every resident consumer it may not touch with the rule protecting each — `priority`, `coexists`, or `no-unload-mechanism`. `errors.Is(ErrInsufficientVRAM)` still matches through `Unwrap`, so existing callers are unaffected. Tests `TestRefusalStatesMeasurements` and `TestRefusalCountsFreeableAndEvicted`; both verified to fail without the change. One test asserts the message contains no cause-hypothesis wording, so this cannot silently regress. **Note on the wording quoted in this issue:** `untracked GPU usage, e.g. a game?` is not this repo's string. It is the client's, in ImaGen `internal/backend/broker.go:20`. The broker half is fixed here; that half needs a separate change in ImaGen, otherwise the misleading hypothesis still reaches the reader. ## Third finding confirmed live `gpu_resident_mib: 0` for every consumer, verified on the running broker after today's deploy: ```json {"comfyui":{"loaded":true,"gpu_resident_mib":0},"mvoice":{"loaded":false,"gpu_resident_mib":0}, "ollama":{"loaded":true,"gpu_resident_mib":0},"whisper-server":{"loaded":true,"gpu_resident_mib":0}} ``` while `nvidia-smi` showed real residency for all four. That is issues #4/#6 and is the next step of this work. ## Deployed and live-verified The binary on mRock now carries this change plus the merged #5 eviction fix (`601f5e2`), which had never been deployed. Forced a real eviction under real pressure to verify rather than relying on a health check: ``` evicted consumer victim=ollama target=comfyui free_mib_after=4786 need_mib=6000 POST /v1/lease status=200 ms=2232 ``` Free VRAM 4786 → 7065 MiB, ollama's runners released and lazily cold-reloaded afterwards. `healthz`, `/v1/status` and `/v1/gamemode` all verified after restart; previous binary kept as `bin/mgpumanager.bak`.
mAi commented 2026-07-29 10:16:35 +00:00
Author
Copy Link

9000 is live and verified — a cold generation is admitted after eviction

Converged: repo and mRock host both at vram_resident_mib: 9000 (host backup kept as consumers.yaml.bak-6000). Commit https://mgit.msbls.de/m/mGPUmanager/commit/6cc386c on mai/tesla/vram-truth.

The acceptance run

Forced real pressure by loading an ollama model, then forced comfyui cold via /api/free, then requested an image lease:

before lease:  free 1165 MiB   (ollama 9704, whisper 2052, comfyui 346 idle, desktop ~830)

evicted consumer victim=mvoice          free_mib_after=1165  need_mib=9000
evicted consumer victim=whisper-server  free_mib_after=8857  need_mib=9000
evicted consumer victim=ollama          free_mib_after=8857  need_mib=9000
POST /v1/lease status=200 ms=12513

after eviction: free 10875 MiB
cold generation: completed in 11s

Admitted after eviction, not refused. That is the criterion. Under the old 13000 this lease would have been refused outright; under 6000 it would have been admitted at ~6.3 GB free and then run into a peak it could not reach.

A finding that changes the rationale, not the number

Peak comfyui residency in this run was 10458 MiB — higher than the 8410 I measured earlier, and above the 9000 budget itself.

The difference is not noise. Comparing all three runs:

Run Free VRAM available comfyui peak
cold, earlier ~9000 8410
warm, earlier ~11000 6170
cold, this run 10875 10458

comfyui expands to fill the VRAM it is given. PyTorch's caching allocator does not release back, so the "peak" tracks availability rather than requirement. So my earlier framing — that 8410 is what comfyui needs — was too strong. 8410 was the peak under those conditions. The evidence that it can run in less is the warm run at 6170.

This does not invalidate 9000, and the run above completed fine. It does mean 9000 should be read as an admission threshold — enough headroom that a generation proceeds comfortably — and not as "the amount comfyui requires". A budget cannot be both, and no single constant can be either, which is the substance of #12: the cost axis is not just cold-vs-warm and resolution, it is also how much was free at the time.

Also confirmed live: #8 is still real

The log line evicted consumer victim=whisper-server above is the deployed binary marking whisper unloaded without stopping the unit. The fix for that is committed on mai/tesla/vram-truth (https://mgit.msbls.de/m/mGPUmanager/commit/ba168be) but not yet deployed — eviction now issues a real systemctl --user stop and ensureLoaded starts the unit again on the next STT request.

Also on that branch, for #4/#6: per-consumer residency now comes from nvidia-smi per-PID attribution instead of the consumers' own health responses, which never carried the field. That is what will let the margin question in #12 be answered with real numbers instead of zeros.

## 9000 is live and verified — a cold generation is admitted after eviction Converged: repo and mRock host both at `vram_resident_mib: 9000` (host backup kept as `consumers.yaml.bak-6000`). Commit https://mgit.msbls.de/m/mGPUmanager/commit/6cc386c on `mai/tesla/vram-truth`. ### The acceptance run Forced real pressure by loading an ollama model, then forced comfyui cold via `/api/free`, then requested an image lease: ``` before lease: free 1165 MiB (ollama 9704, whisper 2052, comfyui 346 idle, desktop ~830) evicted consumer victim=mvoice free_mib_after=1165 need_mib=9000 evicted consumer victim=whisper-server free_mib_after=8857 need_mib=9000 evicted consumer victim=ollama free_mib_after=8857 need_mib=9000 POST /v1/lease status=200 ms=12513 after eviction: free 10875 MiB cold generation: completed in 11s ``` Admitted after eviction, not refused. That is the criterion. Under the old 13000 this lease would have been refused outright; under 6000 it would have been admitted at ~6.3 GB free and then run into a peak it could not reach. ### A finding that changes the rationale, not the number Peak comfyui residency in this run was **10458 MiB** — higher than the 8410 I measured earlier, and above the 9000 budget itself. The difference is not noise. Comparing all three runs: | Run | Free VRAM available | comfyui peak | |---|---|---| | cold, earlier | ~9000 | 8410 | | warm, earlier | ~11000 | 6170 | | cold, this run | 10875 | 10458 | **comfyui expands to fill the VRAM it is given.** PyTorch's caching allocator does not release back, so the "peak" tracks availability rather than requirement. So my earlier framing — that 8410 is what comfyui *needs* — was too strong. 8410 was the peak under those conditions. The evidence that it can run in less is the warm run at 6170. This does not invalidate 9000, and the run above completed fine. It does mean 9000 should be read as **an admission threshold** — enough headroom that a generation proceeds comfortably — and not as "the amount comfyui requires". A budget cannot be both, and no single constant can be either, which is the substance of #12: the cost axis is not just cold-vs-warm and resolution, it is also *how much was free at the time*. ### Also confirmed live: #8 is still real The log line `evicted consumer victim=whisper-server` above is the deployed binary marking whisper unloaded without stopping the unit. The fix for that is committed on `mai/tesla/vram-truth` (https://mgit.msbls.de/m/mGPUmanager/commit/ba168be) but not yet deployed — eviction now issues a real `systemctl --user stop` and `ensureLoaded` starts the unit again on the next STT request. Also on that branch, for #4/#6: per-consumer residency now comes from nvidia-smi per-PID attribution instead of the consumers' own health responses, which never carried the field. That is what will let the margin question in #12 be answered with real numbers instead of zeros.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
m
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: m/mGPUmanager#11
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?