docs: Phase 6 plan re-baseline against live mBrian schema + m's answers

m answered all 11 §10 questions; every inventor pick confirmed.
m's overriding directive: "keep the database simple so it remains
easily modifiable."

Head verified the live mBrian schema after m's answers — original §3
was built off stale db/001_initial_schema.sql. Three of the six asks
turned out already-satisfied:

- MB-A (edges.metadata jsonb) — already added in db/010, GIN-indexed,
  used by migs 039/040. Drop the ask.
- MB-C (project type) — already in live schema, mig 033 confirms.
  Drop the ask.
- MB-D (per-user slug uniqueness) — already enforced by idx_nodes_slug
  in db/001. Drop the ask.

Plus 'area' as a separate mBrian type is killed per m's "keep it
simple": areas reuse type=['project'] with metadata.projax.kind='area'.
Zero DDL.

Remaining mBrian-side artifact compresses to ONE [schema] convention
node under a new [topic] projax-integration hub, plus mBrian-side
ownership of the one-shot data-migration script (per m's "mbrian must
own the migration").

Re-sequenced §8: six slices.
  0 (projax snapshot helper) → A (mBrian [schema] node + script run)
  → B (projax read-path adapter) → C (projax write-path)
  → D (mai bridge worker) → E (drop projax tables).

CalDAV/Gitea integrations stay where they are (m's Q3=(a)). No slice
F needed in the original sense.

§2 + §2.1 + §7 + §9 + §10 + §14 updated. §3 fully rewritten.

No code changes; this branch ships docs only. Slice 0 is the smallest
first projax-side step but waits for head's greenlight after the
m/mBrian issue is filed.
This commit is contained in:
mAi
2026-05-29 13:56:50 +02:00
parent b3e7183478
commit a5b0971b9d

View File

@@ -1,12 +1,20 @@
# mBrian-as-backend migration — Phase 6 design
**Status**: Phase A design (this doc).
**Status**: Phase A design — re-baselined against live mBrian schema (2026-05-29).
**Branch**: `mai/kahn/phase-6a-mbrian-design`.
**Author**: kahn (inventor), 2026-05-29.
**Source decision** (m, issue m/projax#5, 12:43 2026-05-29): Option A — full backend migration. *"I think we need the project-management element inside of mBrian for it to be the complete 2nd Brain experience. The data itself is not too important yet."*
**m's overriding directive** (2026-05-29 via head): *"keep the database simple so it remains easily modifiable."*
**Constraint**: data-loss tolerant on the 47 current `projax.items`.
**m's answers on §10 (2026-05-29)**: every inventor pick confirmed.
> Q1=reuse 'project' / Q2=(b) handler bridge / Q3=(a) clients projax-side / Q4=(a) file Gitea on m/mBrian via otto/head — m: *"mbrian must own the migration"* / Q5=(a) views stay projax-resident / Q6=(a) per-user slug / Q7=(a) hard-cut / Q8=(a) tags in metadata / Q9=(a) projax-side cycle detection / Q10=(a) keep projax MCP via adapter / Q11=keep `projax_origin` audit metadata.
**Re-baseline note**: §3's original ask was built off a stale `db/001_initial_schema.sql` read. Head verified the live mBrian schema after m's answers. Three of the six asks (MB-A, MB-C, MB-D) turned out already-satisfied — `edges.metadata` exists since `db/010_flexsiebels_compat.sql`, `'project'` type exists since `db/033`, the per-user slug unique index ships in `db/001`. The remaining mBrian-side artifact is small. §3 + §8 now reflect that. The big shift: **mBrian owns the one-shot data-migration script** — that's what "mbrian must own the migration" means — while projax owns the read+write rewiring on its own side afterward.
---
## §1 — Diagnosis
@@ -32,7 +40,7 @@ End-state contract:
| projax column | mBrian destination | notes |
|---|---|---|
| `id` (uuid) | `nodes.id` | new uuids on migration; legacy ids never round-trip |
| `kind` (text[]) | `nodes.type` | direct shape match; projax `'project'` becomes mBrian `'project'` (already exists per mig 030); add `'area'` if missing |
| `kind` (text[]) | `nodes.type` | direct shape match; projax `'project'` becomes mBrian `'project'` (already in live schema, mig 033). **Areas keep `type=['project']` + `metadata.projax.kind='area'`** — per m's "keep the database simple" directive, no new mBrian type. Zero DDL. |
| `title` | `nodes.title` | 1:1 |
| `slug` | `nodes.slug` | mBrian = unique per user; projax = unique per parent — see §2.1 |
| `paths` (text[]) | derived from `child_of` edges + `nodes.path` cache | DAG resolution via edge walk; see §2.2 |
@@ -52,11 +60,13 @@ End-state contract:
| `updated_at` | `nodes.updated_at` | trigger-maintained on both sides |
| `deleted_at` | `nodes.deleted_at` | 1:1 |
### §2.1 — Slug uniqueness
### §2.1 — Slug uniqueness (settled)
projax enforces slug uniqueness **per parent** (a `paliad` slug can exist under both `dev` and `work`). mBrian enforces slug uniqueness **per user** (one `paliad` total). For a multi-parent item like `paliad` living under both `dev` and `work`, mBrian today only allows one `paliad` node — which is actually what we want: a single canonical node connected to both parents via `child_of` edges. The DAG-as-multiple-paths view is a render-time concept; the storage is one node.
projax today enforces slug uniqueness **per parent**. mBrian's live schema has `CREATE UNIQUE INDEX idx_nodes_slug ON mbrian.nodes (user_id, slug)` — uniqueness **per user**. Per m's Q6=(a), projax adopts mBrian's model: one `paliad` node total, connected to both `dev` and `work` via two `child_of` edges. The DAG-as-multiple-paths view is a render-time concept; the storage is one node.
The projax handlers' itemwrite validator (Phase 5c) loses its per-parent slug rule and gains an uniqueness-per-user check. This is **stricter** — m can't have two different "paliad" projects under different roots. Flag for Q6.
projax handlers' itemwrite validator (Phase 5c) loses its per-parent slug rule, gains a per-user check (against the projax-managed subset of nodes). This is **stricter** — m can't have two different "paliad" projects under different roots. Settled per m's answer.
**Pre-migration dedup**: the 47-item migration script (which lives mBrian-side, see §3+§7) scans for slug collisions across the projax dataset and folds collisions into one node with multiple `child_of` edges. Skip-with-log on anything weirder.
### §2.2 — paths array vs single path
@@ -91,29 +101,45 @@ mBrian's `edges` table today has no `metadata jsonb` column — only `rel`, `not
---
## §3 — mBrian-side requirements
## §3 — mBrian-side requirements (re-baselined against live schema)
Schema fragments mBrian needs to register. Each is a separate migration on the mBrian side, written by mBrian's coder. These are the cross-repo asks:
Head verified the live mBrian schema after m's answers. Three of the original six asks turned out already-satisfied. What's actually needed reduces to one [schema] convention node + ownership of the one-shot data-migration script. Per m's Q4=(a), this lands as a Gitea issue on `m/mBrian` with the "blocks projax phase 6" tag; head files it.
**MB-A — Add `edges.metadata jsonb NOT NULL DEFAULT '{}'`** (per §2.2). Backfill is no-op; new column starts empty. Optional GIN index if projax queries by metadata content (not in v1 — projax queries by `rel` only).
### Already satisfied (no DDL needed)
**MB-B — Register projax-specific edge relations.** mBrian has no enum on `edges.rel`; new values just appear. Document the projax-namespaced rels (`projax-caldav-list`, `projax-gitea-repo`, `projax-gitea-issue`, `projax-mai-project`, `projax-url`, `projax-doc`) in a schema-node so mBrian's tooling knows they belong to projax. Concrete deliverable: a `[schema]` node `projax-edge-relations` under a `[topic]` hub `projax-integration` (created same migration).
| original ask | live-schema status |
|---|---|
| MB-A — `edges.metadata jsonb` column | **Already exists** — added in `db/010_flexsiebels_compat.sql`: `ALTER TABLE mbrian.edges ADD COLUMN IF NOT EXISTS metadata jsonb NOT NULL DEFAULT '{}'` plus GIN `idx_edges_metadata`. Already used by mig 039/040. projax link payloads land here directly. |
| MB-C — `'project'` type registration | **Already exists** — confirmed in `db/033` + inbox tests. m's Q1=(a) reuses it. |
| MB-C — `'area'` type registration | **NOT needed** — per m's "keep the database simple," areas reuse `type=['project']` with `metadata.projax.kind='area'`. Zero DDL. |
| MB-D — per-user slug uniqueness | **Already enforced**`CREATE UNIQUE INDEX idx_nodes_slug ON mbrian.nodes (user_id, slug)` in `db/001`. Handles the bulk migration as-is, modulo the pre-write dedup pass in the script (§7). |
| MB-E — read MCP coverage | **Confirmed** by head — type-array filter, edge query by `rel` + source/target, FTS search all present in mBrian's MCP today. Optional bulk "node + outbound edges" endpoint may improve adapter perf, but v1 ships without it. |
| MB-F — write MCP coverage | **Confirmed** by head — create_node, update_node, soft-delete, create_edge, delete_edge all present. |
**MB-C — Optional: register projax types as known types.** mBrian's `'project'` type already exists. Add `'area'` if missing (projax's top-level area concept). Document the projax type set in another `[schema]` node so future readers know `'mai-managed'` means "this projax node mirrors a mai.projects row."
### Remaining mBrian-side artifact
**MB-D Confirm no slug-uniqueness blockers.** projax migration creates ~47 nodes with potentially-colliding slugs (e.g. two `paliad`-rooted DAG paths today are ONE node post-migration; need to dedupe pre-write). mBrian-head to confirm the per-user unique index doesn't choke on bulk insert ordering.
**MB-B — projax-integration `[schema]` convention node.** One new mBrian node, no DDL. Lives under a new `[topic]` hub `projax-integration`. Documents:
**MB-E — Read-only API surface.** projax read-path (§4) calls mBrian. We need:
- A way to filter nodes by `type` array containment (mBrian already exposes via `list_nodes(type='...')`).
- Edge query by `rel` + `source_id` or `target_id`.
- Trigram / FTS search across title + content_md (mBrian already has this).
- Optionally: a bulk endpoint that returns nodes + their outbound edges in one call (projax's tree-render path needs ~all nodes + all edges).
1. The projax edge relations: `child_of` (already in use everywhere), `projax-caldav-list`, `projax-gitea-repo`, `projax-gitea-issue`, `projax-mai-project`, `projax-url`, `projax-doc`. Each entry: rel name + the metadata jsonb shape (e.g. `projax-caldav-list` carries `{url: text}`).
2. The projax type usage: `'project'` for both projects and areas; `metadata.projax.kind` distinguishes (`area` vs default `project`). `'mai-managed'` as a co-type marker for nodes mirroring `mai.projects` rows.
3. The projax metadata shape: `metadata.projax.{status, tags, management, public, timeline_exclude, start_time, end_time, kind}` — the subset of projax columns that don't have a first-class mBrian counterpart.
4. A pointer to `projax_origin` audit metadata (set per migrated node, per m's Q11=keep).
Confirm MCP surface coverage. If anything's missing, mBrian-coder adds it.
mBrian-side coder writes this node by creating it via mBrian's editor or MCP. No migration file needed.
**MB-F — Write API surface.** projax write-path (§5) calls mBrian. We need create_node, update_node, soft-delete, create_edge, delete_edge — all already exist in mBrian's MCP.
### mBrian owns the data-migration script
Cross-repo coordination shape: this lands as a Gitea issue on `m/mBrian` repo, batched as one ticket with sub-tasks for AF. The issue links back to this plan + projax/#5.
Per m's directive "mbrian must own the migration," the one-shot script that creates the 47 nodes + their edges lives in `m/mBrian` (likely `scripts/migrate-from-projax.ts` or similar — mBrian's stack picks). projax-side provides:
- A frozen snapshot of `projax.items` + `projax.item_links` rows (CSV or JSON dump produced by a projax-side helper).
- The mapping rules from §2 + §2.2 in a form mBrian-side can implement against (this plan doc is the canonical source).
- A spot-check checklist (5 representative items) for post-migration validation.
The script's blast radius lives on mBrian's side; projax-side blocks on its successful run before slice C kicks off.
### Cross-repo coordination shape
One Gitea issue on `m/mBrian` (filed by head), tagged "blocks projax phase 6". The issue body covers MB-B + script ownership + the snapshot-handoff protocol. Body draft delivered to head with this re-baseline (see Phase A workflow §14).
---
@@ -199,73 +225,99 @@ This is **Q2** for m.
---
## §7 — Migration mechanics
## §7 — Migration mechanics (mBrian-owned)
Per m's loss-tolerance signal: hard-cut. One script, run once, with a clear blast radius.
Per m's Q7=(a) hard-cut + Q4=(a) "mbrian must own the migration": the one-shot script lives in `m/mBrian`. projax-side provides the input snapshot + the rules in this doc; mBrian-side owns the execution.
Script outline (Go, lives in `cmd/migrate-mbrian/main.go`):
### projax-side input snapshot
1. Connect to msupabase as `postgres`.
2. Read every row from `projax.items` where `deleted_at IS NULL`.
3. For each row:
a. Generate new mBrian uuid (or preserve old uuid — projax uuids don't collide with mBrian's; we can preserve, but the migration script picks).
b. INSERT into `mbrian.nodes` with type/title/slug/aliases/metadata mapped per §2. Add `metadata.projax_origin: <old_id>` so a future audit can reconcile.
c. For each `parent_id`, INSERT `mbrian.edges (source=new_node_id, target=parent_new_id, rel='child_of')`. Parent uuids resolved via a two-pass walk: first pass creates all nodes, second pass writes edges.
4. For each `projax.item_links` row:
a. Translate `ref_type` → mBrian edge `rel` per §2.2 table.
b. INSERT `mbrian.edges` with `metadata` carrying the structured payload.
5. For each `projax.views` row (5j paliad-shape views): see Q5.
6. Smoke check: count(mbrian.nodes WHERE metadata->>'projax_origin' is not null) == count(projax.items WHERE deleted_at IS NULL).
7. DON'T drop projax.items + item_links in the same migration. Drop happens in slice E after stable read+write.
A helper command in `cmd/projax-snapshot/main.go` produces a `projax_snapshot.json` containing every live `projax.items` row + every `projax.item_links` row, shaped for direct consumption by the mBrian-side script. One file, deterministic, round-trippable. Ships in slice 0 (the snapshot handoff, see §8).
Idempotency: the script checks `metadata.projax_origin` on each insert to avoid duplicates on re-run.
### mBrian-side script outline (for the m/mBrian issue body)
Lossy bits (acceptable per m's stance): the projax `paths` array isn't preserved — it's recomputed from edges. The Phase 1.5 mai.projects mirror rows: the bridge worker handles re-sync after migration (Q2).
1. Load `projax_snapshot.json`.
2. Two-pass: pass 1 creates every node; pass 2 writes every edge (parent edges + item_links → projax-* edges).
3. For each item:
a. New mBrian uuid OR preserve the projax uuid (mBrian-side picks; either works given m's Q11 audit metadata is the durable reference).
b. INSERT into `mbrian.nodes` with `type=['project']` (or `['project']` + co-type per `kind`), `title`, `slug`, `aliases`, `metadata={projax: {...}, projax_origin: <old_id>}`.
c. Where projax had multiple paths (same node under multiple parents), DEDUPE by slug — one node, multiple `child_of` edges.
4. For each parent edge: INSERT `mbrian.edges (source=new_id, target=parent_new_id, rel='child_of')`.
5. For each item_links row: INSERT `mbrian.edges` with `rel='projax-<ref_type>'` and `metadata` carrying the structured payload per §2.2.
6. For projax.views (5j): NOT migrated — per m's Q5=(a), the views table stays projax-resident.
7. Smoke check: count(mbrian.nodes WHERE metadata->>'projax_origin' is not null) == count(items in snapshot).
8. Hand off to projax with the new uuid map (`{old_uuid: new_uuid}`) so projax-side caches can warm.
### Idempotency
Pre-flight: the script checks `metadata.projax_origin` and skips already-migrated origins on re-run. m can re-run safely if the script aborts mid-way.
### Lossy bits (acceptable per m's stance)
- `paths text[]` array is not preserved — projax-side adapter recomputes from edges per §4.
- mai.projects mirror rows: per Q2=(b), a handler-layer bridge worker re-syncs after migration; the Phase 1.5 trigger pair stays disabled.
### Blast-radius containment
mBrian-side runs the script with triggers paused, smoke-checks the count + spot-checks the 5 representative items in projax's checklist, then commits + signals projax-side to start slice C (read-path).
---
## §8 — Implementation slicing
## §8 — Implementation slicing (re-baselined)
Slices AF, with hard cross-repo coordination on A. Each slice independently shippable.
Six slices. The big shift from the original draft: the mBrian-side ask compresses to one [schema] convention node + one migration script (both mBrian-owned per m's Q4). Slice 0 is a small projax-side helper that ships the snapshot. The hard gate is the migration landing — projax-side B reads it as the trigger to start.
- **A. mBrian schema-fragment** — mBrian-side. Adds `edges.metadata`, registers projax edge relations + types in schema-nodes, confirms read+write MCP coverage. Lands as a Gitea issue on `m/mBrian`, coordinated through §9.
- **0. projax-side snapshot helper** — `cmd/projax-snapshot/main.go`. Dumps live `projax.items` + `projax.item_links` to `projax_snapshot.json`. Ships first; minimal risk; deliverable mBrian needs.
- **B. Data migration script** — projax-side. `cmd/migrate-mbrian/main.go`. Runs once against the 47 items + their links + the 5j views (per Q5). Test on a scratch mBrian dataset before the real migration.
- **A. mBrian-side: [schema] convention node + data-migration script** — m/mBrian owns. The [schema] node lives under a new `[topic]` hub `projax-integration`. The script consumes the snapshot from slice 0 and writes 47ish nodes + their edges per §7. mBrian-side post-flight: smoke-check count + spot-check 5 items per the projax checklist.
- **C. Read-path replacement** — projax-side. `store/` package rewired to mBrian. The Item struct stays; method bodies rewrite. All UI + aggregator tests stay green (they only see Item). Adapter caches a per-request snapshot to avoid N+1 mBrian calls.
- **B. projax-side read-path adapter** — projax-side. `store/` package rewired against mBrian's MCP / SQL surface. The `Item` struct stays; method bodies rewrite. All UI + aggregator tests stay green (they only see Item shape). Per-request snapshot cache to avoid N+1 calls. Reads-only soak before slice C.
- **D. Write-path replacement** — projax-side. Every handler + MCP write rewires. itemwrite validator updates for slug-uniqueness semantics.
- **C. projax-side write-path** — projax-side. Every handler + MCP write rewires through the adapter to mBrian. itemwrite validator updates for the per-user slug rule (Q6). Cycle detection on the in-memory closure (Q9).
- **E. Drop projax tables** — projax-side. After stable read+write on mBrian for one shift, drop `projax.items` + `projax.item_links`. Migration `0018_drop_projax_items.sql`.
- **D. mai.projects bridge worker** — projax-side (Q2=(b)). Disable the Phase 1.5 trigger pair; ship a small worker that observes mai.projects writes + reflects them into mBrian, and vice versa. Decoupled, killable.
- **F. Integrations disposition** — depending on Q2's answer. If (b), build the bridge worker. If (c), coordinate with mai-side to migrate workers/tasks FKs.
- **E. Drop `projax.items` + `projax.item_links`** — projax-side. Migration `0018_drop_projax_items.sql`. Triggers off after one shift's stable read+write soak on mBrian. `projax.views` stays (Q5).
Dependency graph:
```
A ──→ B ──→ C ──→ D ──→ E
↘ F (parallel after D)
0 (projax snapshot) ──→ A (mBrian [schema] node + migration script run)
B (projax read-path) ──→ C (projax write-path)
├──→ D (mai bridge worker)
E (drop projax tables)
```
A is the gate. C and D can ship together if the test surface stays green; otherwise C ships first (read-path) and D follows after one shift's worth of read-only soak.
Slice 0 unblocks A. A is mBrian-owned and the hard gate for everything else. B → C can ship together if green; otherwise B-first soak.
CalDAV / Gitea integrations stay where they are (Q3=(a)) — no slice F needed in the original sense.
---
## §9 — Cross-repo coordination
## §9 — Cross-repo coordination (settled)
mBrian is m's actual second brain. Schema changes carry blast radius. The protocol:
Per m's Q4=(a) + his words *"mbrian must own the migration"*:
1. **mBrian's head identity**. Looking at the m/mBrian repo: m manages it directly today (no mBrian/head worker registered in mai). Per global Channel Routing rule (only `otto/head` writes to m directly), the projax coordination request routes through `otto/head` who relays to m and back. Confirm with **Q4**.
1. **Protocol**: file a Gitea issue on `m/mBrian` with "blocks projax phase 6" tag. Routed via otto/head per global Channel Routing. Head files it; kahn drafts the body.
2. **The schema-fragment ask**. One delegation message to head with the §3 MB-A through MB-F items batched. Head decides whether to: file as an issue on `m/mBrian` directly, route through otto/head for m to dispatch a mBrian worker, or coordinate paired implementation.
2. **Ownership split**:
- mBrian-side owns: the `[schema]` convention node (MB-B) + the one-shot data-migration script.
- projax-side owns: the snapshot helper (slice 0), the read-path adapter (slice B), the write-path (slice C), the mai bridge (slice D), the table drop (slice E).
3. **Sequencing**. mBrian-side A must land + deploy before projax-side B can run. Recommend filing the schema request as `m/mBrian` issue with a clear "blocks projax phase 6" tag.
3. **Sequencing**: slice 0 produces the snapshot → mBrian-side A consumes it + runs the migration → mBrian-side signals back → projax-side starts B. The Gitea issue is the durable trace; the delegation reply chain is the real-time signal.
4. **Design-doc sharing**. This plan stays in `m/projax`. m/mBrian gets a pointer issue with the relevant §2+§3 excerpts.
4. **Design-doc sharing**: this plan stays in `m/projax`. The m/mBrian issue body (drafted alongside this re-baseline, delivered to head) excerpts §2 (schema mapping), §3 (the one [schema] node ask), §7 (the script outline), and the spot-check checklist.
---
## §10 — Open questions for head delegation
## §10 — Open questions (all answered 2026-05-29)
All 11 questions resolved. m confirmed every inventor pick. Section retained as the historical record + so a future hand can audit the decision rationale.
The 8 from issue #5 plus what surfaced during this survey.
@@ -375,6 +427,8 @@ Per the migration script, every migrated node carries `metadata.projax_origin =
## §14 — Status
- **Phase A (this doc)**: drafted by kahn, 2026-05-29. Awaiting m's answers on §10 via head delegation, AND cross-repo coordination via head with mBrian.
- **Phase B (coder)**: blocked on (1) m's sign-off on §10 + (2) mBrian-side schema-fragment landed (slice A complete + deployed).
- **No code changes** in this branch beyond this doc. Slice A is mBrian-side; projax-side slices BF wait on it.
- **Phase A (this doc)**: drafted by kahn 2026-05-29, re-baselined same day against live mBrian schema after m's 11 answers landed. All §10 questions resolved.
- **m/mBrian Gitea issue**: body drafted; head files it under "blocks projax phase 6" tag.
- **Phase B (projax-side coder)**: blocked on (1) slice 0 snapshot helper ships + (2) mBrian-side migration runs + signals back. NO coder flip yet.
- **Slice 0 (projax-side snapshot helper)**: scoped, not yet built. Smallest first-step on projax-side; ready when head greenlights.
- **No code changes** in this branch beyond this doc.