feat(phase 1.5): tags + management + DAG + mai.projects sync
Big task. Five migrations, full store + web rewrite, and a model upgrade
that turns the parent_id tree into a parent_ids[] DAG.
Schema (db/migrations)
- 0006_tags_management_unify: adds tags + management text[] (GIN-indexed),
collapses the area/project distinction (kind keeps the slot but 'area'
is no longer a special value), drops the structural rules from the
path trigger so root projects + non-root projects are both legal.
- 0007_backfill_mai_projects: one-shot, idempotent — for every row in
mai.projects without a 'mai-project' item_link, create a projax.items
row under a heuristic-chosen area (mhealth→health, msports/manjin→
sports, kanzlai/hlckm/work/mworkrepo/paliad or HL/* repo→work,
mhome→home, default→dev), insert the item_link, and tag the row
management=['mai']. Also flips management='mai' on any already-linked
pre-Phase-1.5 promotions.
- 0008_mai_projects_sync: bidirectional triggers. sync_to_mai runs as
projax_admin and writes mai.projects directly (after the operator-run
grant + RLS policy widening — documented in the migration header).
sync_from_mai is SECURITY DEFINER so writes by the mai role fan out
into projax.items. pg_trigger_depth() + projax.in_sync GUC keep the
cycle suppressed. Slug stays the join key for new rows; the
item_link pointer survives renames.
- 0009_items_unified_simplify: view collapses to a thin projection over
projax.items now that mai.projects is a derived projection.
- 0010_multi_parent: parent_id → parent_ids uuid[], path → paths text[].
compute_item_paths walks via parents' precomputed paths (no recursive
CTE in the hot path; cycle detection uses one). New triggers:
items_check_slug_collision (multi-parent uniqueness),
items_after_delete (manual cascade since arrays don't carry FK).
Trigger refresh_item_paths_recursive does parent-first DFS over
descendants, guarded by projax.refreshing_paths GUC.
Go store + handlers
- Item gains ParentIDs []string + Paths []string. PrimaryPath /
OtherPaths helpers feed the detail breadcrumb. Source always
'projax' now; SourceRefDeref still surfaces the mai-id pointer.
- Update / Reparent / Create take ParentIDs []string. AddParent helper
for the multi-parent UI's "also list under" action.
- GetByPath uses '$1 = any(paths)' so /i/work.paliad and /i/dev.paliad
resolve to the same row.
- buildForest renders a multi-parent item under each of its parents
(duplicated nodes in distinct branches). Tag-filter prune is
branch-preserving.
Templates
- detail.tmpl: multi-select parents, tags + management chip inputs,
"Also at: …" breadcrumb for multi-parent items.
- new.tmpl: same multi-select + chip inputs.
- tree.tmpl: tag-filter chip bar, "×N" badge on multi-parent rows,
management chips visible on every row.
- classify.tmpl: re-parent workflow (no more promote-to-projax — the
bidirectional sync removed the dichotomy).
Tests (DB + HTTP, all skip without env)
- TestMultiParentResolvesBothPaths inserts an item with two parents,
asserts both inherited paths.
- TestSlugCollisionUnderCommonParent refuses a sibling clash.
- TestMultiParentBothPathsRouteToSameRow HTTP-level: /i/dev.X and
/i/work.X both 200, same row.
- TestReparentRoundTrip rewritten for parent_ids[] semantics.
- TestPathTriggerNestAndRename / Reparent rewritten to query paths[].
Docs (docs/design.md)
- §2 rewritten: items in a DAG, no area/project distinction.
- §3 schema: parent_ids + paths + tags + management + indices.
- §3.1 path-trigger overhaul incl. cycle detection via recursive CTE
and slug-collision-under-common-parent guard.
- §3.2 view simplified.
- §3.4 NEW: mai.projects bidirectional sync, including the manual
prereq.
- §4.1 + §4.2: classify becomes re-parent, tags+management UI section.
mai head start / mai hire / mai status / mai instruct keep working
because mai.projects retains its FK-target shape; the projax sync just
mirrors the row in lock-step.
This commit is contained in:
182
docs/design.md
182
docs/design.md
@@ -12,40 +12,34 @@ projax is m's personal data backbone for self-management — areas of life, proj
|
||||
|
||||
## 2. Model
|
||||
|
||||
### 2.1 Two kinds, freely nestable
|
||||
### 2.1 Items in a DAG
|
||||
|
||||
```
|
||||
area ─┐
|
||||
├─ project ─┐
|
||||
│ ├─ project ─┐ (sub-projects allowed, any depth)
|
||||
│ │ └─ …
|
||||
└─ task (external ref)
|
||||
```
|
||||
Phase 1.5 collapsed the area/project structural distinction. Every node is an `item`; the `kind` array is kept for forward-compatibility (future: milestone, event, person, …) but `area` is no longer a special value. The seven seeded roots are just items with `parent_ids = '{}'`.
|
||||
|
||||
- **Area** — a container without start/end. Long-running domain of life. Areas live at the root (`parent_id = NULL`). Examples: `dev`, `sports`, `home`, `work`, `health`, `finances`, `social`.
|
||||
- **Project** — a bounded effort with (usually) a start and an end. Lives under an area, or under another project. Sub-projects nest to arbitrary depth (`home.spring-clean.bathroom.tiles` is fine if that's how m thinks). Examples: `home.spring-clean`, `dev.prjx`, `sports.giro-okt`.
|
||||
The hierarchy is a **directed acyclic graph**, not a tree: each item has zero or more parents, and the same item can surface under multiple branches. `work.paliad` and `dev.paliad` resolve to the same row. PER citations can use any valid path.
|
||||
|
||||
- **Item** — a node in the DAG. Examples: `dev`, `home.spring-clean`, `work.paliad`, `paliad.note`. The `kind` column carries `['project']` today; we may layer other types as needs arise.
|
||||
- **Task** — atomic work item. **Lives outside projax** (CalDAV todos, Gitea issues, `mai.tasks`). projax references and aggregates them; it does not own them.
|
||||
|
||||
Areas and projects share one table (`projax.items`) distinguished by the `kind` array column. The tree (`parent_id`) is unconstrained on depth; the only structural rules are:
|
||||
Structural rules:
|
||||
|
||||
- An area's `parent_id` must be `NULL` (areas are roots).
|
||||
- A project's `parent_id` must point to an area or another project (no project at root).
|
||||
- No cycles (enforced by the path trigger).
|
||||
|
||||
Tasks are referenced via `projax.item_links`.
|
||||
- No cycles (enforced by `items_before_write` + `compute_item_paths` recursive-CTE ancestor closure).
|
||||
- An item's slug must be unique among its siblings under any common parent (enforced by `items_check_slug_collision` BEFORE trigger, with the partial unique index `items_root_slug_uniq` covering the root case).
|
||||
- Soft delete via `deleted_at`. Hard delete cascades through `items_after_delete`, which scrubs the deleted id from every descendant's `parent_ids` array.
|
||||
|
||||
### 2.2 Identity & naming
|
||||
|
||||
- `id uuid` — canonical, immutable.
|
||||
- `slug text` — local-only segment (no dots). Renameable freely. Examples: `prjx`, `spring-clean`, `upc.deadlines` would be split as parent slug `upc` + child slug `deadlines`.
|
||||
- `path text` — full dot-joined path computed from parent walk. Cached column maintained by trigger; not the source of truth.
|
||||
- Slug convention: lowercase, vowel-elided where natural (`prjx`, `mai`, `mbrn` if m wishes), kebab-allowed for multi-word leaves (`spring-clean`).
|
||||
- `slug text` — local-only segment (no dots). Renameable freely. Examples: `prjx`, `spring-clean`. Multi-word leaves use kebab-case.
|
||||
- `parent_ids uuid[]` — zero or more parent ids. Root items have `parent_ids = '{}'`.
|
||||
- `paths text[]` — full dot-joined paths, one per ancestor lineage. Trigger-maintained from `parent_ids` + `slug`. Lookup via `'<path>' = ANY(paths)`.
|
||||
- Slug convention: lowercase, vowel-elided where natural (`prjx`, `mai`, `mbrn`), kebab-allowed for multi-word leaves (`spring-clean`).
|
||||
- Aliases: `aliases text[]` keeps old slugs searchable after rename.
|
||||
- `tags text[]` — free-vocabulary cross-cutting labels (`work`, `dev`, `home`, `tech`, …). GIN-indexed. No fixed vocabulary.
|
||||
- `management text[]` — how the project is run: `self`, `mai`, `external`. An item can carry multiple modes. Empty array means "no specific management mode declared".
|
||||
|
||||
### 2.3 Lifecycle (thin)
|
||||
|
||||
For projects only — areas don't have lifecycle.
|
||||
|
||||
```
|
||||
active → done → archived
|
||||
```
|
||||
@@ -54,8 +48,8 @@ That's it. Free-text in `content_md` covers the nuance ("waiting on Brian," "pau
|
||||
|
||||
### 2.4 Relationships
|
||||
|
||||
- **Tree** (parent/child within projax): `items.parent_id uuid` self-FK. Areas have `parent_id = NULL`. Projects point at their area or another project. Arbitrary nesting depth.
|
||||
- **External refs** (`projax.item_links`): each row links an `item_id` to a typed external resource — caldav-todo, gitea-issue, mai-task, mai-project, mbrian-node, etc. Used both for aggregating tasks and for soft cross-references.
|
||||
- **Tree-as-DAG** (parent/child within projax): `items.parent_ids uuid[]`. Root items have `parent_ids = '{}'`. Any item may name multiple parents — the same row then appears under each branch with paths inherited from each lineage.
|
||||
- **External refs** (`projax.item_links`): each row links an `item_id` to a typed external resource — `caldav-todo`, `gitea-issue`, `github-repo`, `mai-task`, `mai-project`, `mbrian-node`, `url`, etc. Used both for aggregating tasks and for soft cross-references.
|
||||
|
||||
## 3. Schema (Postgres, msupabase, schema `projax`)
|
||||
|
||||
@@ -70,27 +64,34 @@ create table projax.items (
|
||||
path text not null, -- computed, e.g. 'home.spring-clean'
|
||||
parent_id uuid references projax.items(id) on delete restrict,
|
||||
content_md text default '',
|
||||
aliases text[] default '{}',
|
||||
aliases text[] not null default '{}',
|
||||
metadata jsonb not null default '{}'::jsonb,
|
||||
status text not null default 'active', -- active | done | archived (projects only)
|
||||
status text not null default 'active', -- active | done | archived
|
||||
pinned boolean not null default false,
|
||||
archived boolean not null default false,
|
||||
start_time timestamptz,
|
||||
end_time timestamptz,
|
||||
parent_ids uuid[] not null default '{}',
|
||||
paths text[] not null default '{}',
|
||||
tags text[] not null default '{}',
|
||||
management text[] not null default '{}',
|
||||
created_at timestamptz not null default now(),
|
||||
updated_at timestamptz not null default now(),
|
||||
deleted_at timestamptz,
|
||||
unique (parent_id, slug)
|
||||
deleted_at timestamptz
|
||||
);
|
||||
|
||||
create index items_path_idx on projax.items (path);
|
||||
create index items_kind_idx on projax.items using gin (kind);
|
||||
create index items_parent_idx on projax.items (parent_id);
|
||||
create index items_paths_idx on projax.items using gin (paths);
|
||||
create index items_parent_ids_idx on projax.items using gin (parent_ids);
|
||||
create index items_kind_idx on projax.items using gin (kind);
|
||||
create index items_tags_idx on projax.items using gin (tags);
|
||||
create index items_management_idx on projax.items using gin (management);
|
||||
create unique index items_root_slug_uniq
|
||||
on projax.items (slug) where cardinality(parent_ids) = 0;
|
||||
|
||||
create table projax.item_links (
|
||||
id uuid primary key default gen_random_uuid(),
|
||||
item_id uuid not null references projax.items(id) on delete cascade,
|
||||
ref_type text not null, -- 'caldav-todo' | 'gitea-issue' | 'mai-task' | 'mai-project' | 'mbrian-node' | 'url' | ...
|
||||
ref_type text not null, -- 'caldav-todo' | 'gitea-issue' | 'github-repo' | 'mai-task' | 'mai-project' | 'mbrian-node' | 'url' | ...
|
||||
ref_id text not null, -- opaque external identifier
|
||||
rel text not null default 'contains', -- 'contains' | 'related' | 'blocked-by' | 'derived-from'
|
||||
note text,
|
||||
@@ -103,81 +104,92 @@ create index item_links_item_idx on projax.item_links (item_id);
|
||||
create index item_links_ref_idx on projax.item_links (ref_type, ref_id);
|
||||
```
|
||||
|
||||
### 3.1 The `path` trigger
|
||||
### 3.1 Path triggers (multi-parent)
|
||||
|
||||
`path` is maintained by trigger on insert/update: walks `parent_id` to the root, joins slugs with `.`. Recomputed for the subtree when a parent is renamed or re-parented. Keeps queries cheap.
|
||||
`paths` is a `text[]` maintained by `compute_item_paths(parent_ids, slug, self_id)`:
|
||||
|
||||
### 3.2 The mai.projects adapter view
|
||||
- For root items (`parent_ids = '{}'`), `paths = [slug]`.
|
||||
- Otherwise, look up every parent's `paths`, append `.<slug>` to each, dedupe and sort. The recursion is implicit — parents' paths are kept up to date by the same trigger, so children just consume the precomputed prefixes.
|
||||
- Cycle detection: the function rejects when `self_id` appears anywhere in the recursive ancestor closure (`WITH RECURSIVE closure ...`). Plus a defensive direct-self-parent check.
|
||||
|
||||
mai.projects stays untouched. projax surfaces it in the unified item stream via a read-only view:
|
||||
Two BEFORE triggers and two AFTER triggers cooperate:
|
||||
|
||||
- `items_before_write` (BEFORE INSERT/UPDATE) — cycle guard + `new.paths := compute_item_paths(...)`.
|
||||
- `items_check_slug_collision` (BEFORE INSERT/UPDATE) — for each parent in `new.parent_ids`, refuse if another row already uses `new.slug` under that parent.
|
||||
- `items_after_reparent` (AFTER UPDATE of slug/parent_ids) — DFS over descendants via `refresh_item_paths_recursive`, parent-first ordering. A session GUC `projax.refreshing_paths` short-circuits the inner UPDATEs so the cascade fires exactly once.
|
||||
- `items_after_delete` (AFTER DELETE) — scrubs the deleted id from every other row's `parent_ids` array (we have no FK integrity on array elements; this is the manual cascade).
|
||||
|
||||
### 3.2 The `items_unified` view
|
||||
|
||||
After Phase 1.5 mai.projects is a derived projection (see §3.4), so the view collapses to a thin projection over `projax.items`:
|
||||
|
||||
```sql
|
||||
create or replace view projax.items_unified as
|
||||
create view projax.items_unified as
|
||||
select
|
||||
id,
|
||||
kind,
|
||||
title,
|
||||
slug,
|
||||
path,
|
||||
parent_id,
|
||||
content_md,
|
||||
status,
|
||||
pinned,
|
||||
archived,
|
||||
start_time,
|
||||
end_time,
|
||||
i.id, i.kind, i.title, i.slug, i.paths, i.parent_ids, i.content_md,
|
||||
i.aliases, i.metadata, i.status, i.pinned, i.archived,
|
||||
i.start_time, i.end_time,
|
||||
'projax'::text as source,
|
||||
created_at,
|
||||
updated_at
|
||||
from projax.items
|
||||
where deleted_at is null
|
||||
|
||||
union all
|
||||
|
||||
select
|
||||
('00000000-0000-0000-0000-' || substr(md5(p.id), 1, 12))::uuid as id, -- deterministic placeholder
|
||||
array['project']::text[] as kind,
|
||||
p.name as title,
|
||||
p.id as slug,
|
||||
'mai.' || p.id as path,
|
||||
null::uuid as parent_id,
|
||||
coalesce(p.goal, '') as content_md,
|
||||
case p.status
|
||||
when 'active' then 'active'
|
||||
when 'sleeping' then 'archived'
|
||||
when 'archived' then 'archived'
|
||||
else 'active'
|
||||
end as status,
|
||||
false as pinned,
|
||||
(p.status = 'archived') as archived,
|
||||
null::timestamptz as start_time,
|
||||
null::timestamptz as end_time,
|
||||
'mai.projects'::text as source,
|
||||
p.created_at,
|
||||
p.updated_at
|
||||
from mai.projects p;
|
||||
(select l.ref_id from projax.item_links l
|
||||
where l.item_id = i.id and l.ref_type = 'mai-project' limit 1) as source_ref_id,
|
||||
i.tags, i.management, i.created_at, i.updated_at
|
||||
from projax.items i
|
||||
where i.deleted_at is null;
|
||||
```
|
||||
|
||||
UI reads `items_unified`. Writes target `projax.items` only. Once m wants to fully migrate, the `mai.projects` half is dropped from the view and rows are copied across with real UUIDs + proper parent assignment.
|
||||
`source` is always `'projax'` (kept for forward compat); `source_ref_id` surfaces the `mai-project` pointer when one exists so the UI can show "mai id: foo".
|
||||
|
||||
### 3.3 Classification overlay
|
||||
|
||||
For each mai.projects row, m can later promote it into projax-native (assigning area parent, real slug, kind tweak). Until promoted it appears as a top-level orphan project tagged `source=mai.projects`. An admin page surfaces the unmapped set and lets m one-click classify.
|
||||
Items can land at root in two ways:
|
||||
|
||||
- The backfill in migration 0007 heuristic-assigned every existing mai.projects row to one of the seven seeded areas (`dev`, `sports`, `work`, `home`, `health`, `finances`, `social`). None ended up at root in this pass.
|
||||
- The reverse sync trigger (§3.4) drops every NEW mai.projects row at root with `management = ['mai']`, leaving m to classify it via `/admin/classify`.
|
||||
|
||||
`/admin/classify` surfaces items where `cardinality(parent_ids) = 0 AND 'mai' = ANY(management)`. The inline form posts to `/i/{path}/reparent` to move the item under a chosen parent without touching its other fields.
|
||||
|
||||
### 3.4 mai.projects bidirectional sync (Phase 1.5)
|
||||
|
||||
`mai.workers`, `mai.tasks`, `mai.sessions`, `mai.messages`, `mai.metrics` and `mai.pwa_head_pins` hold FKs into `mai.projects(id)`, so the table cannot be replaced by a view. Instead it becomes a **derived projection** kept in sync by triggers:
|
||||
|
||||
- **Forward** (`projax.sync_to_mai`, AFTER INSERT/UPDATE/DELETE on `projax.items`) — when an item has `'mai' = ANY(management)`, upsert/update/delete the matching `mai.projects` row. Slug stays the join key (`mai.projects.id = projax.items.slug` at creation), but FK targets cannot be renamed, so projax slug and mai id may drift after a rename; the cross-system pointer in `item_links(ref_type='mai-project')` remains stable.
|
||||
- **Reverse** (`projax.sync_from_mai`, AFTER INSERT/UPDATE/DELETE on `mai.projects`, `SECURITY DEFINER` so `mai` role writes can fan out into `projax.items` which projax_admin owns) — mirror the change into a `projax.items` row, dropping new rows at root with `management = ['mai']` so `/admin/classify` can pick them up.
|
||||
- **Cycle prevention** — both functions short-circuit when `pg_trigger_depth() > 1` (the natural recursion case) and additionally honour a `projax.in_sync` session GUC as belt-and-braces.
|
||||
|
||||
The mai.projects → projax.items reverse trigger requires manual prereqs on msupabase:
|
||||
|
||||
```sql
|
||||
GRANT INSERT, UPDATE, DELETE, TRIGGER ON mai.projects TO projax_admin;
|
||||
DROP POLICY IF EXISTS projax_write ON mai.projects;
|
||||
CREATE POLICY projax_write ON mai.projects FOR ALL TO projax_admin
|
||||
USING (true) WITH CHECK (true);
|
||||
```
|
||||
|
||||
(documented in 0008's header and the README).
|
||||
|
||||
## 4. Interfaces
|
||||
|
||||
### 4.1 Phase 1 — MVP (this build)
|
||||
### 4.1 Phase 1 + 1.5 — Web frontend (this build)
|
||||
|
||||
**Web frontend at `https://projax.msbls.de`**, single binary, served by the same Go process that talks to msupabase.
|
||||
|
||||
Pages:
|
||||
|
||||
1. **Tree view** (`/`) — collapsible tree of areas → projects, reads `items_unified`. Status badges, search bar (slug, title, alias, content_md).
|
||||
2. **Item detail** (`/i/{path}`) — full editor for projax-native items (title, slug, parent, kind, status, start/end, content_md). Read-only view for mai.projects-sourced rows with a "Promote to projax" button.
|
||||
3. **New item** (`/new?parent={path}`) — small form, prefilled with parent.
|
||||
4. **Classify orphans** (`/admin/classify`) — list of unmapped mai.projects rows, inline assign-to-area control.
|
||||
1. **Tree view** (`/`) — DAG rendering: every item appears under each of its parents. Status / management / tag chips per row. Tag filter chips at the top toggle a `?tag=<csv>` filter (an item is shown when it or any descendant carries every active tag). `×N` badge on multi-parent items shows how many paths they live at.
|
||||
2. **Item detail** (`/i/{path}`) — `{path}` matches any entry in `paths`; both `work.paliad` and `dev.paliad` resolve to the same row. The page shows the primary path plus an "Also at: …" breadcrumb for the others. Edit form supports title, slug, multi-select parents, status, tags, management, pinned/archived, content. Save POSTs to `/i/{path}`.
|
||||
3. **New item** (`/new?parent={path}`) — same form shape; the `parent` query pre-selects one parent option, m can pick more.
|
||||
4. **Classify** (`/admin/classify`) — surfaces items at root with `'mai' = ANY(management)`. Inline HTMX form sets the first parent. POSTs to `/i/{path}/reparent`.
|
||||
5. **Auth** — projax's own `/login` (mBrian pattern). Same Supabase backend, per-host cookies (no `Domain` attribute).
|
||||
|
||||
Auth: shared msupabase login (matches flexsiebels precedent), single-user m.
|
||||
### 4.2 Tags + management
|
||||
|
||||
- **Tags** (`projax.items.tags text[]`, GIN-indexed) — free vocabulary, no fixed list. Cross-cutting labels for "work-y dev things" (`['work', 'dev']`), "health priorities" (`['health']`), etc. Filter chips at `/` reveal which tag-flavoured slices exist.
|
||||
- **Management** (`projax.items.management text[]`, GIN-indexed) — declarative mode flags:
|
||||
- `mai` — bidirectional sync with `mai.projects`. Adding/removing `mai` toggles the mai.projects mirror on/off (with FK safety: removal fails if workers/tasks still reference the project).
|
||||
- `self` — m runs this manually; otto does not orchestrate.
|
||||
- `external` — owned by a third party; projax mirrors metadata only.
|
||||
|
||||
Mai.projects backfilled rows arrive with `management = ['mai']`. m can layer `self` on top without dropping mai sync.
|
||||
|
||||
### 4.2 Phase 2 — task aggregation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user