m/paliad
1
0
Fork 0
Code Issues 146 Pull Requests 8 Actions 1 Packages Projects Releases Wiki Activity

Paliadin SKILL.md improvements: don't confidently claim 'no deadlines' on a single failed query #62

New Issue
Open
opened 2026-05-20 12:18:01 +00:00 by mAi · 1 comment
mAi commented 2026-05-20 12:18:01 +00:00
Collaborator
Copy Link

Trigger

m 2026-05-19 16:33-16:34 (recovered from chat history 2026-05-20):

16:33:04 — m: 'Whats my next deadline? dont mind the project'
         — Paliadin: 'Keine offenen Fristen in Sicht — über alle für dich sichtbaren Akten hinweg ist…'
         (ran mcp__supabase__execute_sql but returned 0 rows — the SQL was wrong)

16:34:19 — m: 'Ja, das überrascht mich — was ist mit der Duplik am 5.6 und der KE am 7.8?'
         — Paliadin: 'Du hast völlig recht — meine vorherige Antwort war falsch. Beide Fristen sind a…'
         (ran 6 SQL queries, found the deadlines, corrected itself)

m's note 2026-05-20 14:17:

I had an issue with the chat yesterday — it told me no deadlines, not sure which sql command it first did but it was wrong... when I told it, it did it right. So maybe we need to improve the paliadin skill?!

What went wrong

First-pass SQL returned 0 rows. Paliadin concluded 'no upcoming deadlines exist' and answered with full confidence. The second pass (after m pushed back) ran 6 queries and found them — i.e. they existed all along; the first query had a wrong filter / wrong join / wrong RLS-context.

Most likely culprits (audit the SKILL.md SQL recipes):

  • status = 'pending' filter where approval_status = 'approved' was the intended gate (or vice versa).
  • Missing approval_status IN ('approved','legacy') filter that excludes pending-approval rows from the user-visible view.
  • Wrong project visibility predicate — querying paliad.deadlines without joining through paliad.projects + can_see_project(caller, project_id) returns rows the user can technically see but the SQL skipped them.
  • 'Next deadline' interpretation — due_date >= today vs due_date > today vs due_date >= now()::date AND completed_at IS NULL. One of those got the boundary wrong.
  • Aggregation pitfall — LIMIT 1 without ORDER BY due_date ASC returns an arbitrary row.

What to do (SKILL improvements)

  1. Never claim 'no rows exist' on a single query result. When the first SQL returns 0, run a broader sanity-check before concluding. The 6-query corrective pass m saw is what should happen on the first 'no deadlines' result.
  2. Canonical 'next deadline' query template in the SKILL.md that's known correct — copy-paste, no ad-hoc variations. Should include: user-visibility predicate, approval_status filter, status filter, ORDER BY, LIMIT, both completed_at IS NULL and due_date >= today.
  3. Mandatory 'show me what you found' confirmation before answering definitively — Paliadin should surface the SQL it ran + the row count + a peek at the rows, so m can spot mismatches. The 0-row reply needs to surface the query so m can challenge it without round-tripping.
  4. Verification recipe for the 'no upcoming deadlines' branch — when the user-visibility query returns 0, run a broader unfiltered query (or one with relaxed approval_status) and contrast. If the broader query also returns 0, the conclusion is honest; if not, the SKILL has a bug.
  5. m-specific edge case: m is global_admin — his RLS should expose every row. Any 'no rows' result against m's session is suspicious by default and should trigger the broader retry.

Where the SKILL lives

Source of truth (per project CLAUDE.md note): m/mAi/skills/aichat/paliadin/SKILL.md. Legacy copy: ~/.claude/skills/paliadin/SKILL.md on the host running LocalPaliadinService / RemotePaliadinService. Update both during the transition, eventually retire the legacy copy.

Out of scope

  • Reshaping the paliadin transport (aichat vs legacy) — that's m/mAi#207 territory.
  • New SQL helper functions in paliad's Postgres (the SKILL can stay self-sufficient with raw SQL).
  • LLM-tuning (model choice, temperature) — content-of-SKILL improvements only.

Role

coder direct (or m + me in conversation) — SKILL.md is a documentation artifact; the fix is a careful rewrite of the relevant query recipes + the 'don't claim no-rows' guardrail. Test by replaying m's 'next deadline' question. Group with other paliadin-skill items.

## Trigger m 2026-05-19 16:33-16:34 (recovered from chat history 2026-05-20): ``` 16:33:04 — m: 'Whats my next deadline? dont mind the project' — Paliadin: 'Keine offenen Fristen in Sicht — über alle für dich sichtbaren Akten hinweg ist…' (ran mcp__supabase__execute_sql but returned 0 rows — the SQL was wrong) 16:34:19 — m: 'Ja, das überrascht mich — was ist mit der Duplik am 5.6 und der KE am 7.8?' — Paliadin: 'Du hast völlig recht — meine vorherige Antwort war falsch. Beide Fristen sind a…' (ran 6 SQL queries, found the deadlines, corrected itself) ``` m's note 2026-05-20 14:17: > I had an issue with the chat yesterday — it told me no deadlines, not sure which sql command it first did but it was wrong... when I told it, it did it right. So maybe we need to improve the paliadin skill?! ## What went wrong First-pass SQL returned 0 rows. Paliadin concluded 'no upcoming deadlines exist' and answered with full confidence. The second pass (after m pushed back) ran 6 queries and found them — i.e. they existed all along; the first query had a wrong filter / wrong join / wrong RLS-context. Most likely culprits (audit the SKILL.md SQL recipes): - `status = 'pending'` filter where `approval_status = 'approved'` was the intended gate (or vice versa). - Missing `approval_status IN ('approved','legacy')` filter that excludes pending-approval rows from the user-visible view. - Wrong project visibility predicate — querying `paliad.deadlines` without joining through `paliad.projects` + `can_see_project(caller, project_id)` returns rows the user can technically see but the SQL skipped them. - 'Next deadline' interpretation — `due_date >= today` vs `due_date > today` vs `due_date >= now()::date AND completed_at IS NULL`. One of those got the boundary wrong. - Aggregation pitfall — `LIMIT 1` without `ORDER BY due_date ASC` returns an arbitrary row. ## What to do (SKILL improvements) 1. **Never claim 'no rows exist' on a single query result.** When the first SQL returns 0, run a broader sanity-check before concluding. The 6-query corrective pass m saw is what should happen on the first 'no deadlines' result. 2. **Canonical 'next deadline' query template** in the SKILL.md that's known correct — copy-paste, no ad-hoc variations. Should include: user-visibility predicate, approval_status filter, status filter, ORDER BY, LIMIT, both completed_at IS NULL and due_date >= today. 3. **Mandatory 'show me what you found' confirmation** before answering definitively — Paliadin should surface the SQL it ran + the row count + a peek at the rows, so m can spot mismatches. The 0-row reply needs to surface the query so m can challenge it without round-tripping. 4. **Verification recipe for the 'no upcoming deadlines' branch** — when the user-visibility query returns 0, run a broader unfiltered query (or one with relaxed approval_status) and contrast. If the broader query also returns 0, the conclusion is honest; if not, the SKILL has a bug. 5. **m-specific edge case**: m is global_admin — his RLS should expose every row. Any 'no rows' result against m's session is suspicious by default and should trigger the broader retry. ## Where the SKILL lives Source of truth (per project CLAUDE.md note): `m/mAi/skills/aichat/paliadin/SKILL.md`. Legacy copy: `~/.claude/skills/paliadin/SKILL.md` on the host running `LocalPaliadinService` / `RemotePaliadinService`. **Update both during the transition, eventually retire the legacy copy.** ## Out of scope - Reshaping the paliadin transport (aichat vs legacy) — that's m/mAi#207 territory. - New SQL helper functions in paliad's Postgres (the SKILL can stay self-sufficient with raw SQL). - LLM-tuning (model choice, temperature) — content-of-SKILL improvements only. ## Role **coder direct** (or m + me in conversation) — SKILL.md is a documentation artifact; the fix is a careful rewrite of the relevant query recipes + the 'don't claim no-rows' guardrail. Test by replaying m's 'next deadline' question. Group with other paliadin-skill items.
mAi self-assigned this 2026-05-20 12:18:01 +00:00
mAi commented 2026-05-20 13:07:26 +00:00
Author
Collaborator
Copy Link

Implementation shipped to m/mAi at commit 2a53541 on branch mai/feynman/paliadin-skill-fix (the SKILL.md source of truth lives in m/mAi post-#207). Awaiting m/mAi merge before the deployed Paliadin SKILL reflects it. Legacy mirror on the local + remote hosts updated in sync.

Implementation shipped to m/mAi at commit 2a53541 on branch `mai/feynman/paliadin-skill-fix` (the SKILL.md source of truth lives in m/mAi post-#207). Awaiting m/mAi merge before the deployed Paliadin SKILL reflects it. Legacy mirror on the local + remote hosts updated in sync.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
deferred
Parked — not pursuing now, may un-park later
done
Work completed by an agent; awaiting m to close
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
mAi m
No Assignees mAi
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: m/paliad#62
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?