m/paliad
1
0
Fork 0
Code Issues 146 Pull Requests 8 Actions 1 Packages Projects Releases Wiki Activity

feat(t-paliad-131): Phase C — search backend (matview + service + handler)

Browse Source 
Closes the search half of the unified Fristenrechner. Phase D (concept-card
UI on /tools/fristenrechner) follows in a subsequent shift.

Migration 047:
  - Seed the missing `wiedereinsetzung` concept and re-point the four
    Wiedereinsetzung trigger_events (200..203) at it. PR-7 referenced
    the slug `re-establishment-of-rights` but never seeded the concept,
    so the four cross-cutting triggers were dropping out of any concept-
    JOINing query. Per m's slug rule (Q1: shared cross-cutting concepts
    use DE slug because German term dominates HLC vocabulary).
  - Create paliad.deadline_search materialised view: UNION ALL of
    (deadline_rules joined to deadline_concepts) and (trigger_events
    joined to deadline_concepts via slug). Trigram GIN indexes on
    legal_source / concept_name_de / concept_name_en / rule_name_de /
    rule_name_en / rule_code; gin (concept_aliases) for array
    containment; UNIQUE INDEX on a synthetic row_key so refresh can
    run CONCURRENTLY.

Refresh strategy: data only mutates via migration files at server
startup, so no AFTER triggers and no pg_cron — main.go calls
services.RefreshSearchView right after db.ApplyMigrations. CONCURRENTLY
keeps reads online and stays well under 100 ms at < 1k rows.

Service `internal/services/deadline_search_service.go`:
  - Two-query pipeline per request: (1) rank concept_ids by
    GREATEST(similarity()) across name / aliases / legal_source / rule_code
    plus a 0.2 alias-hit boost; (2) load all matview rows for the top-N
    concepts and assemble per-pill JSON.
  - normalizeQuery strips legal-prefix noise (`§`, `Art.`, `Section`,
    `Rule `) so users typing `§ 82` find DE.PatG.82.1 even though the
    structured legal_source column doesn't carry the prefix.
  - FormatLegalSourceDisplay renders structured codes back to the
    pleading form HLC users expect:
        UPC.RoP.23.1   → "UPC RoP R.23(1)"
        DE.PatG.82.1   → "PatG §82(1)"
        EU.EPÜ.108     → "EPÜ Art.108"
        EU.EPC-R.79.1  → "EPC R.79(1)"
        EU.RPBA.12.1.c → "RPBA Art.12(1)(c)"
  - Drill URLs route per kind: rule pills → ?proc=…&focus=…, trigger
    pills → ?mode=event&triggerId=…

Handler `GET /api/tools/fristenrechner/search?q=&party=&proc=&source=&limit=`:
  - Returns the JSON shape from design §6.1 (cards-with-pills).
  - 503 with friendly DE message when DATABASE_URL is unset, mirroring
    the other Fristenrechner endpoints.
  - Empty q returns an empty cards array (browse surface is Phase D).

Tests:
  - Pure-Go: TestFormatLegalSourceDisplay (12 cases across all known
    prefixes) + TestNormalizeQuery (8 cases).
  - Integration (skipped without TEST_DATABASE_URL): golden table
    pinning the design's binding queries — Klageerwiderung returns the
    statement-of-defence card with UPC.RoP.23.1, DE.ZPO.276.1,
    DE.PatG.82.1, EU.EPC-R.79.1, DE.PatG.59.3 pills; "RoP 23" returns
    the same card; "§ 82" → normalized "82" → BPatG hit; Wiedereinsetzung
    returns one card with exactly 4 trigger pills (ids 200..203);
    party / source filters narrow as expected; limit cap honoured.
  - SQL semantics validated against live data via supabase MCP using a
    CTE-inlined matview definition with the slug fix simulated; results
    match the golden table.

Per design doc `docs/plans/unified-fristenrechner.md` §4.6 (matview
shape) + §6 (search ranking + API).
This commit is contained in:
m
2026-05-05 04:32:50 +02:00
parent 16c991288f
commit b45278b060
8 changed files with 1119 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

572
internal/services/deadline_search_service.go Normal file
View File

@@ -0,0 +1,572 @@
package services
import (
"context"
"database/sql"
"fmt"
"sort"
"strings"
"github.com/jmoiron/sqlx"
"github.com/lib/pq"
)
// DeadlineSearchService backs the unified Fristenrechner search bar
// (t-paliad-131 Phase C). It reads from the paliad.deadline_search
// materialised view (migration 047) and groups hits per concept, so a
// single search returns one card per legal idea (Klageerwiderung,
// Wiedereinsetzung, …) with one pill per (proceeding × rule) or per
// trigger event under that concept.
//
// Two queries per request:
// 1. Rank concept_ids by trigram similarity against name / aliases /
// legal_source / rule_code, applying optional party / proc / source
// filters.
// 2. Fetch all matview rows for those concept_ids and assemble the
// per-pill payload.
//
// See docs/plans/unified-fristenrechner.md §4.6 + §6.
type DeadlineSearchService struct {
db *sqlx.DB
}
// NewDeadlineSearchService wires the service to its DB pool.
func NewDeadlineSearchService(db *sqlx.DB) *DeadlineSearchService {
return &DeadlineSearchService{db: db}
}
// SearchOptions carries the optional facet filters from the URL query
// string. Empty strings mean "no filter on this facet".
type SearchOptions struct {
Party string
Proc string
Source string
Limit int
MaxLimit int
}
// SearchFilters is the filter echo returned to the client. nil pointer
// means the facet wasn't filtered.
type SearchFilters struct {
Party *string `json:"party"`
Proc *string `json:"proc"`
Source *string `json:"source"`
}
// SearchResponse is the JSON the API hands back. See §6.1.
type SearchResponse struct {
Query string `json:"query"`
Filters SearchFilters `json:"filters"`
Cards []ConceptCard `json:"cards"`
TotalCards int `json:"total_cards"`
TotalPills int `json:"total_pills"`
}
// ConceptCard is one search hit — a concept plus its proceeding pills.
type ConceptCard struct {
Concept ConceptSummary `json:"concept"`
MatchedAliases []string `json:"matched_aliases,omitempty"`
Score float64 `json:"score"`
Pills []Pill `json:"pills"`
}
// ConceptSummary is the concept payload inside a card.
type ConceptSummary struct {
ID string `json:"id"`
Slug string `json:"slug"`
NameDE string `json:"name_de"`
NameEN string `json:"name_en"`
Description *string `json:"description,omitempty"`
Party *string `json:"party,omitempty"`
Category string `json:"category"`
}
// PillProceeding describes the proceeding context of a rule pill. nil
// for trigger pills (cross-cutting events with no proceeding).
type PillProceeding struct {
Code string `json:"code"`
NameDE string `json:"name_de"`
NameEN string `json:"name_en"`
Jurisdiction string `json:"jurisdiction"`
}
// PillDuration is the duration spec of a rule pill. nil for trigger pills.
type PillDuration struct {
Value int `json:"value"`
Unit string `json:"unit"`
Timing *string `json:"timing,omitempty"`
}
// Pill is one row inside a concept card. Either a rule (with proceeding +
// duration) or a trigger (cross-cutting; just a code + name).
type Pill struct {
Kind string `json:"kind"`
RuleID *string `json:"rule_id,omitempty"`
TriggerEventID *int64 `json:"trigger_event_id,omitempty"`
Proceeding *PillProceeding `json:"proceeding,omitempty"`
RuleLocalCode string `json:"rule_local_code"`
RuleNameDE string `json:"rule_name_de"`
RuleNameEN string `json:"rule_name_en"`
LegalSource *string `json:"legal_source,omitempty"`
LegalSourceDisplay *string `json:"legal_source_display,omitempty"`
Duration *PillDuration `json:"duration,omitempty"`
Party string `json:"party"`
DrillURL string `json:"drill_url"`
}
// rankRow is the per-concept score row from query 1.
type rankRow struct {
ConceptID string `db:"concept_id"`
Score float64 `db:"score"`
AliasHit bool `db:"alias_hit"`
ConceptSortOrder int `db:"concept_sort_order"`
ConceptNameDE string `db:"concept_name_de"`
MatchedAliases pq.StringArray `db:"matched_aliases"`
}
// pillRow is the per-(concept, context) row from query 2.
type pillRow struct {
Kind string `db:"kind"`
ConceptID string `db:"concept_id"`
ConceptSlug string `db:"concept_slug"`
ConceptNameDE string `db:"concept_name_de"`
ConceptNameEN string `db:"concept_name_en"`
ConceptDesc sql.NullString `db:"concept_description"`
ConceptParty sql.NullString `db:"concept_party"`
ConceptCategory string `db:"concept_category"`
RuleID sql.NullString `db:"rule_id"`
TriggerEventID sql.NullInt64 `db:"trigger_event_id"`
ProceedingCode sql.NullString `db:"proceeding_code"`
ProceedingNameDE sql.NullString `db:"proceeding_name_de"`
ProceedingNameEN sql.NullString `db:"proceeding_name_en"`
Jurisdiction string `db:"jurisdiction"`
RuleLocalCode string `db:"rule_local_code"`
RuleNameDE string `db:"rule_name_de"`
RuleNameEN string `db:"rule_name_en"`
LegalSource sql.NullString `db:"legal_source"`
RuleCode sql.NullString `db:"rule_code"`
DurationValue sql.NullInt32 `db:"duration_value"`
DurationUnit sql.NullString `db:"duration_unit"`
Timing sql.NullString `db:"timing"`
EffectiveParty string `db:"effective_party"`
}
// Search runs the two-query pipeline and assembles the cards.
//
// q is the raw user input. Empty q returns an empty result set (no
// filtering across the entire matview — that's a "browse" surface
// the design doc reserves for Phase D).
func (s *DeadlineSearchService) Search(ctx context.Context, q string, opts SearchOptions) (*SearchResponse, error) {
limit := opts.Limit
if limit <= 0 {
limit = 12
}
maxLimit := opts.MaxLimit
if maxLimit <= 0 {
maxLimit = 30
}
if limit > maxLimit {
limit = maxLimit
}
resp := &SearchResponse{
Query: q,
Filters: buildFilters(opts),
Cards: []ConceptCard{},
}
qNorm := normalizeQuery(q)
if qNorm == "" {
return resp, nil
}
qLow := strings.ToLower(qNorm)
party := nullable(opts.Party)
proc := nullable(opts.Proc)
source := nullable(opts.Source)
ranks, err := s.rankConcepts(ctx, qNorm, qLow, party, proc, source, limit)
if err != nil {
return nil, err
}
if len(ranks) == 0 {
return resp, nil
}
conceptIDs := make([]string, len(ranks))
for i, r := range ranks {
conceptIDs[i] = r.ConceptID
}
pills, err := s.loadPills(ctx, conceptIDs, party, proc, source)
if err != nil {
return nil, err
}
cards, totalPills := assembleCards(ranks, pills)
resp.Cards = cards
resp.TotalCards = len(cards)
resp.TotalPills = totalPills
return resp, nil
}
func (s *DeadlineSearchService) rankConcepts(
ctx context.Context,
q, qLow string,
party, proc, source *string,
limit int,
) ([]rankRow, error) {
const sqlText = `
WITH matched AS (
SELECT
s.concept_id,
s.concept_sort_order,
s.concept_name_de,
GREATEST(
similarity(s.concept_name_de, $1) * 1.0,
similarity(s.concept_name_en, $1) * 1.0,
COALESCE(similarity(s.legal_source, $1), 0) * 0.9,
COALESCE(similarity(s.rule_code, $1), 0) * 0.9,
similarity(s.rule_name_de, $1) * 0.7,
similarity(s.rule_name_en, $1) * 0.7
) AS row_score,
EXISTS (
SELECT 1 FROM unnest(s.concept_aliases) a
WHERE lower(a) = $2 OR a % $1
) AS row_alias_hit,
ARRAY(
SELECT DISTINCT a FROM unnest(s.concept_aliases) a
WHERE lower(a) = $2 OR a % $1
) AS row_matched_aliases
FROM paliad.deadline_search s
WHERE (
s.concept_name_de % $1
OR s.concept_name_en % $1
OR s.rule_name_de % $1
OR s.rule_name_en % $1
OR (s.legal_source IS NOT NULL AND s.legal_source % $1)
OR (s.rule_code IS NOT NULL AND s.rule_code % $1)
OR EXISTS (
SELECT 1 FROM unnest(s.concept_aliases) a
WHERE lower(a) = $2 OR a % $1
)
)
AND ($3::text IS NULL OR s.effective_party = $3)
AND ($4::text IS NULL OR s.proceeding_code = $4)
AND ($5::text IS NULL OR s.legal_source LIKE $5 || '%')
)
SELECT
m.concept_id,
bool_or(m.row_alias_hit) AS alias_hit,
max(m.row_score) + CASE WHEN bool_or(m.row_alias_hit)
THEN 0.2 ELSE 0 END AS score,
min(m.concept_sort_order) AS concept_sort_order,
min(m.concept_name_de) AS concept_name_de,
-- All rows in a concept share the same aliases; min() over identical
-- text[] values is well-defined and returns one of them verbatim.
COALESCE(min(m.row_matched_aliases), ARRAY[]::text[]) AS matched_aliases
FROM matched m
GROUP BY m.concept_id
ORDER BY score DESC, concept_sort_order ASC, concept_name_de ASC
LIMIT $6
`
var rows []rankRow
if err := s.db.SelectContext(ctx, &rows, sqlText, q, qLow, party, proc, source, limit); err != nil {
return nil, fmt.Errorf("rank concepts: %w", err)
}
return rows, nil
}
func (s *DeadlineSearchService) loadPills(
ctx context.Context,
conceptIDs []string,
party, proc, source *string,
) ([]pillRow, error) {
const sqlText = `
SELECT
s.kind,
s.concept_id,
s.concept_slug,
s.concept_name_de,
s.concept_name_en,
s.concept_description,
s.concept_party,
s.concept_category,
s.rule_id,
s.trigger_event_id,
s.proceeding_code,
s.proceeding_name_de,
s.proceeding_name_en,
s.jurisdiction,
s.rule_local_code,
s.rule_name_de,
s.rule_name_en,
s.legal_source,
s.rule_code,
s.duration_value,
s.duration_unit,
s.timing,
s.effective_party
FROM paliad.deadline_search s
WHERE s.concept_id = ANY($1::uuid[])
AND ($2::text IS NULL OR s.effective_party = $2)
AND ($3::text IS NULL OR s.proceeding_code = $3)
AND ($4::text IS NULL OR s.legal_source LIKE $4 || '%')
ORDER BY s.concept_id, s.kind, s.proceeding_code NULLS LAST, s.rule_local_code
`
var rows []pillRow
if err := s.db.SelectContext(ctx, &rows, sqlText, pq.Array(conceptIDs), party, proc, source); err != nil {
return nil, fmt.Errorf("load pills: %w", err)
}
return rows, nil
}
// assembleCards groups pillRows under their ranked concept and builds
// the JSON cards in score order.
func assembleCards(ranks []rankRow, pills []pillRow) ([]ConceptCard, int) {
pillsByConcept := make(map[string][]pillRow, len(ranks))
for _, p := range pills {
pillsByConcept[p.ConceptID] = append(pillsByConcept[p.ConceptID], p)
}
cards := make([]ConceptCard, 0, len(ranks))
totalPills := 0
for _, r := range ranks {
ps := pillsByConcept[r.ConceptID]
if len(ps) == 0 {
continue
}
// First row carries the concept fields.
first := ps[0]
concept := ConceptSummary{
ID: first.ConceptID,
Slug: first.ConceptSlug,
NameDE: first.ConceptNameDE,
NameEN: first.ConceptNameEN,
Category: first.ConceptCategory,
}
if first.ConceptDesc.Valid {
concept.Description = &first.ConceptDesc.String
}
if first.ConceptParty.Valid {
concept.Party = &first.ConceptParty.String
}
cardPills := make([]Pill, 0, len(ps))
for _, p := range ps {
cardPills = append(cardPills, buildPill(p))
}
// Stable secondary order: rule pills before trigger pills, then by
// jurisdiction-ish ordering (UPC, EU, DE, cross-cutting).
sort.SliceStable(cardPills, func(i, j int) bool {
return pillSortKey(cardPills[i]) < pillSortKey(cardPills[j])
})
card := ConceptCard{
Concept: concept,
MatchedAliases: []string(r.MatchedAliases),
Score: roundScore(r.Score),
Pills: cardPills,
}
cards = append(cards, card)
totalPills += len(cardPills)
}
return cards, totalPills
}
func buildPill(p pillRow) Pill {
pill := Pill{
Kind: p.Kind,
RuleLocalCode: p.RuleLocalCode,
RuleNameDE: p.RuleNameDE,
RuleNameEN: p.RuleNameEN,
Party: p.EffectiveParty,
}
if p.RuleID.Valid {
pill.RuleID = &p.RuleID.String
}
if p.TriggerEventID.Valid {
v := p.TriggerEventID.Int64
pill.TriggerEventID = &v
}
if p.ProceedingCode.Valid {
pill.Proceeding = &PillProceeding{
Code: p.ProceedingCode.String,
NameDE: p.ProceedingNameDE.String,
NameEN: p.ProceedingNameEN.String,
Jurisdiction: p.Jurisdiction,
}
}
if p.LegalSource.Valid {
ls := p.LegalSource.String
pill.LegalSource = &ls
display := FormatLegalSourceDisplay(ls)
if display != "" {
pill.LegalSourceDisplay = &display
}
}
if p.DurationValue.Valid && p.DurationUnit.Valid {
dur := &PillDuration{
Value: int(p.DurationValue.Int32),
Unit: p.DurationUnit.String,
}
if p.Timing.Valid && p.Timing.String != "" && p.Timing.String != "after" {
t := p.Timing.String
dur.Timing = &t
}
pill.Duration = dur
}
pill.DrillURL = pillDrillURL(p)
return pill
}
func pillDrillURL(p pillRow) string {
switch p.Kind {
case "rule":
if p.ProceedingCode.Valid && p.RuleLocalCode != "" {
return "/tools/fristenrechner?proc=" + p.ProceedingCode.String + "&focus=" + p.RuleLocalCode
}
return "/tools/fristenrechner"
case "trigger":
if p.TriggerEventID.Valid {
return fmt.Sprintf("/tools/fristenrechner?mode=event&triggerId=%d", p.TriggerEventID.Int64)
}
return "/tools/fristenrechner?mode=event"
}
return "/tools/fristenrechner"
}
// pillSortKey orders pills inside a card: rule pills before triggers,
// jurisdictions in HLC working order (UPC > EU > DE > DPMA > other),
// then by rule_local_code.
func pillSortKey(p Pill) string {
kindRank := "1"
if p.Kind == "trigger" {
kindRank = "2"
}
jurRank := "9"
if p.Proceeding != nil {
switch p.Proceeding.Jurisdiction {
case "UPC":
jurRank = "1"
case "EU":
jurRank = "2"
case "DE":
jurRank = "3"
case "DPMA":
jurRank = "4"
case "cross-cutting":
jurRank = "8"
}
}
return kindRank + jurRank + p.RuleLocalCode
}
func nullable(v string) *string {
v = strings.TrimSpace(v)
if v == "" {
return nil
}
return &v
}
func buildFilters(opts SearchOptions) SearchFilters {
return SearchFilters{
Party: nullable(opts.Party),
Proc: nullable(opts.Proc),
Source: nullable(opts.Source),
}
}
// normalizeQuery strips legal-prefix noise that users naturally type but
// the structured legal_source column doesn't contain. § 82 → 82, Art.108
// → 108, RoP R.23 → RoP 23. The trigram match runs on the result.
func normalizeQuery(q string) string {
q = strings.TrimSpace(q)
if q == "" {
return ""
}
// Drop common legal prefixes (§, Art., Section, Sec., R., Rule).
// We keep meaningful tokens like "RoP", "ZPO", "PatG" because they
// help narrow the search via trigram on legal_source / rule_code.
lowers := strings.ToLower(q)
for _, prefix := range []string{"§", "art.", "art ", "section ", "sec.", "sec ", "rule "} {
for strings.HasPrefix(lowers, prefix) {
q = strings.TrimSpace(q[len(prefix):])
lowers = strings.ToLower(q)
}
}
return q
}
// roundScore truncates to 4 decimals so JSON stays compact.
func roundScore(v float64) float64 {
return float64(int(v*10000+0.5)) / 10000
}
// FormatLegalSourceDisplay renders a structured legal_source code into
// the form HLC users read in pleadings:
//
// UPC.RoP.23.1 → "UPC RoP R.23(1)"
// UPC.RoP.139 → "UPC RoP R.139"
// DE.PatG.82.1 → "PatG §82(1)"
// DE.ZPO.276.1 → "ZPO §276(1)"
// EU.EPÜ.108 → "EPÜ Art.108"
// EU.EPC-R.79.1 → "EPC R.79(1)"
// EU.RPBA.12.1.c → "RPBA Art.12(1)(c)"
//
// Returns the empty string for an empty input. Unknown jurisdictions
// fall through with the structured form preserved (caller decides
// whether to display).
func FormatLegalSourceDisplay(src string) string {
src = strings.TrimSpace(src)
if src == "" {
return ""
}
parts := strings.Split(src, ".")
if len(parts) < 3 {
// Malformed — return as-is so the caller still has something.
return src
}
code := parts[1]
rest := parts[2:]
var prefix string
switch code {
case "RoP":
prefix = "UPC RoP R."
case "PatG":
prefix = "PatG §"
case "ZPO":
prefix = "ZPO §"
case "EPÜ":
prefix = "EPÜ Art."
case "EPC-R":
prefix = "EPC R."
case "RPBA":
prefix = "RPBA Art."
default:
prefix = code + " "
}
var b strings.Builder
b.Grow(len(prefix) + len(src))
b.WriteString(prefix)
b.WriteString(rest[0])
for _, p := range rest[1:] {
b.WriteByte('(')
b.WriteString(p)
b.WriteByte(')')
}
return b.String()
}
// RefreshSearchView re-populates the materialised view. Safe to call on
// every server boot — it's a CONCURRENTLY refresh against a < 1k row
// view, well under 100 ms in practice. Called from cmd/server/main.go
// right after the migration runner finishes so search reflects any
// newly-applied seed migration.
func RefreshSearchView(ctx context.Context, db *sqlx.DB) error {
_, err := db.ExecContext(ctx, `REFRESH MATERIALIZED VIEW CONCURRENTLY paliad.deadline_search`)
if err != nil {
return fmt.Errorf("refresh deadline_search: %w", err)
}
return nil
}