Commit Graph

4 Commits

Author SHA1 Message Date
mAi
99c9d89daa feat(backups): t-paliad-246 — Backup Mode Slice A (on-demand admin org export)
m/paliad#77 Slice A. Folds the unbuilt t-paliad-214 Slice 3 (org async
export) into a new "Backup Mode" surface gated by adminGate.

m's calls (all 4 material picks per design §2):
- Storage: local disk PALIAD_EXPORT_DIR (LocalDiskStore only)
- Format: .zip bundle (xlsx + JSON + CSV + README) — no-lock-in preserved
- paliadin_turns + paliadin_aichat_conversation: EXCLUDE structurally
- Scheduler (Slice B): nightly 03:00 UTC, env-tunable

Wiring:
- mig 123 adds paliad.backups catalog table (kind/status/storage_uri/
  size/row_counts/warnings/error/deleted_at + admin-only RLS).
- ExportService.WriteOrg + orgSheetQueries enumerate 37 entity sheets
  + 12 ref sheets; REPEATABLE READ READ ONLY tx wraps the dump for
  snapshot consistency (design §3.3).
- writeBundle + runSheetQuery refactored to take a sqlx.QueryerContext
  so both *sqlx.DB (personal/project paths, unchanged) and *sqlx.Tx
  (org snapshot path) work.
- BackupRunner orchestrates: catalog INSERT → audit INSERT
  (event_type='backup_created') → WriteOrg → ArtifactStore.Put → patch
  catalog + audit on success/failure.
- ArtifactStore interface + LocalDiskStore impl (defense-in-depth key
  validation + URI-outside-dir guard).
- Sentinel actor for scheduled runs: actor_email='system@paliad',
  actor_id=NULL — no phantom user in paliad.users.
- Admin handlers POST /api/admin/backups/run + GET list/get/download
  behind adminGate(users, …); /admin/backups page + sidebar entry +
  bilingual i18n keys.
- BackupRunner only wired when PALIAD_EXPORT_DIR is set; routes return
  503 otherwise (same shape as requireDB).

Tests: 8 pure-function tests cover registry shape (no dups, paliadin
absent both as sheet name and SQL substring, ref__* sheets unscoped,
every sheet has ORDER BY) and LocalDiskStore (round-trip, bad-key
rejection, URI-traversal rejection, mkdir on construction).

go build ./... + go test ./internal/... clean. bun run build clean.

Slice B (BackupScheduler + retention cleanup) and Slice C (UI polish)
are separate follow-ups per head's instruction.
2026-05-25 15:28:37 +02:00
mAi
8f1f88b517 feat(export): t-paliad-214 Slice 2 backend — project-subtree sync export
Adds GET /api/projects/{id}/export?direct_only=0|1 streaming a
deterministic project-subtree bundle in the same xlsx + JSON + per-sheet
CSV shape as Slice 1's personal export. 16 entity sheets per design §2:
projects + project_teams + project_partner_units + deadlines +
appointments + parties + notes (4-way polymorphism resolved) + documents
(metadata only) + project_events + approval_requests + approval_policies
(triple-source attribution with `source` column for Q4 lock-in) +
checklist_instances + partner_units (attached only) +
partner_unit_members (members of attached units only) + users_referenced
(FK-referenced users only) + system_audit_log_subset. Personal sidecars
explicitly excluded; reference sheets (proceeding_types, event_types,
deadline_rules, courts, …) ship for standalone interpretability.

§4 permission gate enforced server-side:
  - global_admin can export anything, OR
  - direct project_teams membership with responsibility ∈ {lead, member}
  - Observers + Externals + derived-only partner-unit users → 403
    bilingual ("Datenexport ist nur Team-Mitgliedern (Lead / Member)
    vorbehalten / Data export is restricted to project team members").

Cross-subtree FK detection (Q3 lock-in: keep + warn) runs one
lightweight SELECT against projects.counterclaim_of and appends one
warning row to __meta.warnings per outbound reference. Recipients can
choose to keep or strip the FK on re-import.

Filename includes 8-hex-char short-uuid disambiguator (Q5 lock-in):
paliad-export-project-<slug>-<short-uuid>-<ts>.zip — two projects with
identical titles produce different filenames even when archived
together.

Audit row in paliad.system_audit_log (no new migration — already
supports scope='project'): metadata carries root_label + root_path
(ltree) + direct_only flag (Q6 lock-in) so the audit row remains
interpretable after the project is deleted.

__meta sheet + README.txt extended to surface project-scope fields:
scope_root_label, scope_root_path, direct_only.

ExportFilename signature extended to take a rootID; Slice 1 callsite
updated to pass uuid.Nil.

8 new pure-function tests pin: sheet registry shape (24 sheets in
order), triple-source approval_policies SQL tags, direct_only narrows
subtree to root-only, no-personal-sidecars guard, attached-only
partner_units filter, shortUUIDSuffix shape, project-scope meta rows,
short-uuid filename collision avoidance.
2026-05-20 13:03:57 +02:00
mAi
f9ff7b93e8 fix(export): xlsx docProps + pane XML — Excel "repairs required" + wrong Modified date
m hit two bugs opening the Slice 1 export in Excel / Windows:

1. **Excel showed a "Repairs required" prompt** on open. Root cause:
   the SetPanes call passed only `{Freeze: true, YSplit: 1}` — the
   obvious-but-wrong shape. The resulting <pane> XML missed the
   `topLeftCell` and `activePane` attributes that Excel requires for
   a frozen-row pane (excelize's parser is permissive on re-read but
   Excel is strict). Fix: complete the Panes struct (TopLeftCell="A2",
   ActivePane="bottomLeft", Selection on bottomLeft) and surface
   SetPanes errors instead of `_ =`-ignoring them.

2. **Windows Explorer / Excel's File→Info showed Modified=2006-09-16
   ("xuri")** — excelize's hardcoded first-commit defaults. Root cause:
   buildXLSX never called SetDocProps so the canned defaults leaked.
   Fix: SetDocProps({Created, Modified} = meta.GeneratedAt;
   Creator = "Paliad (<firm>)"; Title/Description scoped per export).

3. **Bonus**: the outer-zip entry mtimes were stamped 2000-01-01 (the
   deterministic constant) so extracted files showed a Y2K Modified
   date in Explorer. Now stamped meta.GeneratedAt, which preserves
   determinism within an export (same row state + same GeneratedAt →
   same bytes, the actual m's-Q6 contract).

Also: set the active sheet to __meta (index 0) after sheet creation so
a future code path that adds/removes sheets can't leave an out-of-range
active-sheet index that would trip a separate "repairs required" path.

Regression tests in dump_export_test.go pin all three fixes by re-opening
the generated xlsx via excelize.OpenReader and asserting:
- docProps Created/Modified == meta.GeneratedAt (RFC 3339 UTC)
- docProps Creator contains "Paliad"
- xlsx bytes never contain "2006-09-16T00:00:00Z" or "<dc:creator>xuri</dc:creator>"
- sheet2/sheet3 raw XML carries topLeftCell + activePane + state=frozen
- outer-zip entries' Modified is within ±2s of GeneratedAt
- developer hatch: DUMP_EXPORT=1 writes /tmp/paliad-export-debug.{zip,xlsx}
  for opening in real Excel.
2026-05-19 13:05:54 +02:00
mAi
28c7215458 feat(export): t-paliad-214 Slice 1 backend — personal sync export endpoint + xlsx/json/csv writer
Adds GET /api/me/export streaming a deterministic .zip bundle of the
caller's RLS-visible projection (per design §2.3): projects, deadlines,
appointments, parties, notes, documents (metadata), audit events,
approval requests, checklist instances + personal sidecars (me row,
caldav config without ciphertext, views, pins, card layouts, paliadin
turns) + reference data (proceeding_types, event_types, deadline_rules,
courts, countries, holidays …) + restricted users_referenced sheet.

Bundle shape: paliad-export.xlsx + paliad-export.json + per-sheet
CSVs (UTF-8 BOM, RFC 4180) + README.txt + __meta.json. Outer zip is
byte-deterministic — sorted file list, fixed Modified time on every
entry, sorted JSON keys. Two runs at same row-state → identical bytes.

ExportService.WritePersonal owns the SQL recipe + column discovery
+ PII deny-regex (?i)secret|token|password|api[_-]?key|private[_-]?key
+ per-sheet DropColumns belt-and-braces (e.g. user_caldav_config
.password_encrypted explicitly dropped on top of the regex). Audit row
written to paliad.system_audit_log before the run, patched with
row_counts + file_size_bytes after.

Migration 102 creates paliad.system_audit_log (generic event_type +
actor_id/email + scope + scope_root + metadata jsonb). Idempotent
CREATE TABLE IF NOT EXISTS + indexes; RLS enabled with self-read +
admin-read policies. AuditService.ListEntries gains a 6th UNION branch
so the new table surfaces on /admin/audit-log.

excelize/v2 added to go.mod for xlsx generation.

Pure-function tests pin formatCellValue value-coercion, PII regex,
CSV quoting + BOM + umlaut survival, JSON shape, meta key order
stability, filename slugify, and byte-determinism of the bundle
assembly.

Design: docs/design-paliad-data-export-2026-05-19.md §7 Slice 1.
2026-05-19 12:51:52 +02:00