diff --git a/docs/design-paliadin-tailscale-ssh-2026-05-07.md b/docs/design-paliadin-tailscale-ssh-2026-05-07.md new file mode 100644 index 0000000..15bec91 --- /dev/null +++ b/docs/design-paliadin-tailscale-ssh-2026-05-07.md @@ -0,0 +1,592 @@ +# Paliadin: route prod via Tailscale SSH to mRiver + +**Issue:** m/paliad#12 — t-paliad-151 +**Date:** 2026-05-07 +**Author:** noether (inventor) +**Supersedes nothing.** Extends `docs/design-paliadin-2026-05-07.md` (the Phase 0 PoC) with a third deployment path between "laptop-only PoC" and "Anthropic API direct". +**Related:** t-paliad-146 (PoC ship), t-paliad-150 (`friendlyErrorMessage` pattern). + +--- + +## 1. Goal + +Make Paliadin reachable from `paliad.de` (Dokploy on mLake) without losing m's Claude Code subscription, by routing each turn over Tailscale + SSH from the paliad container to mRiver, where the existing long-lived `tmux` + `claude` PoC keeps running. + +**Non-goals (v1):** + +- Multi-host failover. +- Encryption beyond SSH-over-tailnet (already E2E-encrypted by Tailscale's WireGuard layer). +- Anthropic API fallback when mRiver is offline — show a friendly error instead. +- Wake-on-LAN of mRiver. +- Multi-tenant or multi-firm variants. + +--- + +## 2. Live state — what was verified before designing + +A design built on stale facts rots fast. These were probed on 2026-05-07, not assumed from CLAUDE.md or memory: + +| Fact | How verified | Result | +|---|---|---| +| mRiver = `100.99.98.203`, has tmux + claude | this worker runs on mRiver; `tmux -V` → `tmux 3.6a`; `which claude` → `/home/m/.local/bin/claude` | confirmed | +| mLake (`100.99.98.201`) has Tailscale running | `ssh m@mlake tailscale status` | confirmed; mRiver visible as `active; direct [2a02:4780:41:3fbc::1]:41641` | +| paliad container Dockerfile is alpine:3.21 minimal, no SSH, no tailscaled | `Dockerfile` | confirmed (only `ca-certificates`) | +| paliad compose runs default Docker bridge (no `network_mode`) | `docker-compose.yml` | confirmed | +| mRiver has no `~/.ssh/authorized_keys` yet | `ls ~/.ssh/` | confirmed — file must be created in Phase A | +| `/tmp/paliadin/` does not exist on mRiver yet | `ls /tmp/paliadin` | confirmed — created on first turn (paliadin.go:185 `os.MkdirAll`) | +| `paliad-paliadin` tmux session is not currently running on mRiver | `tmux ls` | not present; the existing PoC creates it on demand | + +**Implication for design:** the paliad container needs new infrastructure on three axes — network reachability of the tailnet, an SSH client + identity, and a service-layer code path that talks to a remote tmux instead of a local one. Each axis is its own sub-design below. + +--- + +## 3. Locked decisions (m, 2026-05-07 22:35) + +m made four design-shaping calls via the inventor's `AskUserQuestion` pass. They are recorded here verbatim because every downstream choice in §4–§6 follows from them. + +| # | Question | m's choice | +|---|---|---| +| 1 | Container Tailscale shape | **`network_mode: host` on paliad** | +| 2 | SSH-to-mRiver protocol granularity | **Server-side `paliadin-shim` (one RPC per turn)** | +| 3 | Routing trigger | **Env var `PALIADIN_REMOTE_HOST` + interface split** | +| 4 | SSH private key storage | **Dokploy secret env var `PALIADIN_SSH_PRIVATE_KEY`** | + +Decision (1) was *not* the inventor's recommendation — host mode has known interaction risk with traefik (§4.2). m is overriding the recommendation; this design accepts the call and codifies a Phase A test step that gates the rollout on traefik still working under host mode. If Phase A blows up, the fallback is to revisit (1) in a follow-up issue, not to silently swap to a sidecar. + +--- + +## 4. Sub-design A — Container Tailscale shape + +### 4.1 Shape: `network_mode: host` + +paliad's container shares mLake's network namespace. `tailscale0` (mLake's tailnet interface) is directly visible from inside the container. Outbound `ssh m@100.99.98.203` reaches mRiver over the tailnet without any sidecar, userspace tailscaled, SOCKS proxy, or auth-key flow inside the container. + +```yaml +# docker-compose.yml diff +services: + web: + build: . + network_mode: host # NEW + # remove: expose: ["8080"] # host mode means port is on the host directly + environment: + - PORT=8080 + ... + # NEW Paliadin remote-routing knobs + - PALIADIN_REMOTE_HOST=${PALIADIN_REMOTE_HOST} # 100.99.98.203 + - PALIADIN_REMOTE_USER=${PALIADIN_REMOTE_USER} # m + - PALIADIN_SSH_PRIVATE_KEY=${PALIADIN_SSH_PRIVATE_KEY} + - PALIADIN_KNOWN_HOSTS=${PALIADIN_KNOWN_HOSTS} # one-line ssh-keyscan output + restart: unless-stopped +``` + +### 4.2 Trade-off accepted: traefik routing under host mode + +paliad.de's TLS is provided by Dokploy's traefik on the `dokploy-network` overlay. With `network_mode: host`, paliad is no longer attached to that overlay. Two failure modes are possible: + +- **(M1)** traefik can't discover the service via Docker DNS → 502 at the edge. +- **(M2)** traefik routes via host loopback (`http://127.0.0.1:8080` or `host.docker.internal`) and works fine. + +Recent Dokploy versions configure traefik with both `loadbalancer.server.url` and Docker labels; (M2) is the documented host-mode path. **Phase A explicitly tests this** (§7) before any code is written; if (M1) materialises, the design rolls back to the sidecar variant of decision 1 in a follow-up issue. + +Other host-mode side-effects to flag in operations: + +- paliad listens on host port 8080 directly. Any other compose service binding 8080 conflicts. +- paliad's outbound DNS uses host resolver (no Docker-internal `web` etc.). Currently fine: paliad's only network deps are external (Supabase, SMTP, GitHub raw). No service on `dokploy-network` is referenced by name. +- The container can reach **every** Tailscale node, not just mRiver. Mitigations live in §5 (key restriction) and §5.2 (`from=` clause on mRiver authorized_keys). + +### 4.3 Dockerfile diff + +```dockerfile +# Final stage adds the SSH client only. Tailscale is provided by the host. +FROM alpine:3.21 +RUN apk add --no-cache ca-certificates openssh-client # +openssh-client (~1MB) +WORKDIR /app +COPY --from=backend /paliad /app/paliad +COPY --from=frontend /app/frontend/dist /app/dist +EXPOSE 8080 +CMD ["/app/paliad"] +``` + +Image-size delta: alpine `openssh-client` is ~1.1 MB compressed — negligible. No tailscaled, no entrypoint script, no extra processes inside the container. + +### 4.4 What does NOT change + +- No Tailscale auth-key inside paliad. The container inherits the host's tailnet binding, so there is no per-container Tailscale identity to rotate. mLake's existing Tailscale auth is the only one in scope. +- No tailscaled process inside the container. +- No new sidecar container. + +--- + +## 5. Sub-design B — SSH identity, restricted shim, host-key pinning + +### 5.1 Identity: dedicated ed25519 keypair `paliad-prod` + +One keypair, generated once on mRiver during Phase A, used by every paliad-prod deploy: + +```bash +# On mRiver (Phase A bootstrap): +ssh-keygen -t ed25519 -N "" -C "paliad-prod $(date +%Y-%m-%d)" -f /tmp/paliad-prod-key +# Public key → mRiver authorized_keys (see 5.2) +# Private key → Dokploy secret store as PALIADIN_SSH_PRIVATE_KEY +shred -u /tmp/paliad-prod-key # only the encrypted/secret-stored copies survive +``` + +Rotation: regenerate, push public key to mRiver authorized_keys, update Dokploy secret, redeploy. No code change needed — paliad's startup re-reads the env var on every boot. + +The private key is delivered to the container as a multi-line env var. At process start, paliad writes it to a tmpfile so OpenSSH can use it: + +```go +// cmd/server/main.go (sketch) +func loadPaliadinSSHKey() (string, error) { + blob := os.Getenv("PALIADIN_SSH_PRIVATE_KEY") + if blob == "" { return "", nil } // remote mode disabled + f, err := os.CreateTemp("", "paliadin-id_ed25519-") + if err != nil { return "", err } + if err := os.Chmod(f.Name(), 0o600); err != nil { return "", err } + if _, err := f.WriteString(blob); err != nil { return "", err } + if err := f.Close(); err != nil { return "", err } + return f.Name(), nil // path passed to RemotePaliadinService +} +``` + +The tmpfile lives at `/tmp/paliadin-id_ed25519-` for the container's lifetime. On container restart, a fresh tmpfile is written. We never persist the key to a volume. + +### 5.2 mRiver `authorized_keys` entry + +``` +command="/home/m/.local/bin/paliadin-shim",no-pty,no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-user-rc,from="100.99.98.201" ssh-ed25519 AAAA...PUBKEY... paliad-prod +``` + +Each restriction matters: + +- `command=` — every `ssh m@mriver …` invocation runs the shim regardless of what the client asked for. The client's requested command is exposed as `$SSH_ORIGINAL_COMMAND` for the shim to dispatch on. +- `no-pty,no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-user-rc` — defence-in-depth: even if someone steals the key and bypasses the shim's argument validation, they can't get an interactive shell, can't tunnel ports, can't pivot via agent forwarding. +- `from="100.99.98.201"` — only accept connections from mLake's tailnet IP. Defends against the "container has full tailnet visibility" host-mode side-effect from §4.2: if the key leaks off mLake, it can't be replayed from another tailnet host. + +### 5.3 Host-key pinning + +`StrictHostKeyChecking=accept-new` is too loose for a long-lived production identity (one-time MITM during first connect substitutes a different key forever). Instead: + +- During Phase A, run `ssh-keyscan -t ed25519 100.99.98.203` on mLake. +- Capture the single output line. +- Store as Dokploy secret `PALIADIN_KNOWN_HOSTS`. +- At container startup, write to `/tmp/paliadin-known_hosts` chmod 644. +- Pass to OpenSSH via `-o UserKnownHostsFile=/tmp/paliadin-known_hosts -o StrictHostKeyChecking=yes`. + +If mRiver's host key ever rotates (rare; only on disk wipe / fresh OS), Phase A runs again and the secret is updated. SSH refuses to connect with a clear "host key changed" error, which surfaces as `mriver_unreachable` to the user — exactly the right blast-radius (loud failure, no silent connect to a substitute host). + +### 5.4 The shim — `paliadin-shim` + +A bash script on mRiver at `/home/m/.local/bin/paliadin-shim`. It is the **only** thing the paliad-prod key is allowed to invoke, and it dispatches on `$SSH_ORIGINAL_COMMAND`. Three RPCs: + +```bash +#!/bin/bash +# paliadin-shim — server-side RPC for paliad's remote-tmux turns. +# Invoked via authorized_keys command= with $SSH_ORIGINAL_COMMAND set. +set -euo pipefail +umask 077 + +readonly TMUX_SESSION="${PALIADIN_TMUX_SESSION:-paliad-paliadin}" +readonly RESPONSE_DIR="${PALIADIN_RESPONSE_DIR:-/tmp/paliadin}" +readonly TIMEOUT_S=60 +readonly TURN_ID_RE='^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' + +mkdir -p "$RESPONSE_DIR" + +# Parse $SSH_ORIGINAL_COMMAND. Format: " …" +read -r -a argv <<< "${SSH_ORIGINAL_COMMAND:-}" +verb="${argv[0]:-}" + +ensure_pane() { + if ! tmux has-session -t "$TMUX_SESSION" 2>/dev/null; then + tmux new-session -d -s "$TMUX_SESSION" + fi + # Find or create the @paliadin-scope=chat window. + local target="" + while read -r idx; do + scope=$(tmux show-window-option -t "$TMUX_SESSION:$idx" -v @paliadin-scope 2>/dev/null || true) + if [[ "$scope" == "chat" ]]; then target="$TMUX_SESSION:$idx"; break; fi + done < <(tmux list-windows -t "$TMUX_SESSION" -F '#{window_index}') + if [[ -z "$target" ]]; then + idx=$(tmux new-window -t "$TMUX_SESSION" -n claude-paliadin -P -F '#{window_index}' claude) + target="$TMUX_SESSION:$idx" + # Wait for claude to settle (60s bound; matches Go waitForPaneReady). + for _ in $(seq 1 120); do + pane=$(tmux capture-pane -t "$target" -p 2>/dev/null || true) + if [[ "$pane" == *"❯"* || "$pane" == *"│"* ]]; then break; fi + sleep 0.5 + done + tmux set-window-option -t "$target" @paliadin-scope chat + tmux set-window-option -t "$target" @fix-name claude-paliadin + # Bootstrap system prompt — reuses the Go service's prompt text. + # The Go side sends this via the `bootstrap` RPC on first turn instead + # of duplicating the prompt here. See §6.4. + fi + echo "$target" +} + +case "$verb" in + health) + # Liveness check — used by paliad to short-circuit when mRiver is offline. + # Returns "ok" iff tmux + claude are reachable. + tmux has-session -t "$TMUX_SESSION" 2>/dev/null \ + || tmux new-session -d -s "$TMUX_SESSION" + command -v claude >/dev/null && echo ok || { echo no-claude; exit 1; } + ;; + + bootstrap) + # First-turn-only: ensure pane exists and inject the system prompt. + # $1 = base64-encoded prompt body (avoids quoting hell). + target=$(ensure_pane) + prompt=$(printf '%s' "${argv[1]:?missing prompt}" | base64 -d) + tmux send-keys -t "$target" -l -- "$prompt" + tmux send-keys -t "$target" Enter + sleep 2 # give claude a moment to absorb + echo ok + ;; + + run-turn) + # $1 = turn_id (UUID); $2 = base64-encoded user message. + turn_id="${argv[1]:?missing turn_id}" + [[ "$turn_id" =~ $TURN_ID_RE ]] || { echo >&2 "bad turn_id"; exit 2; } + msg=$(printf '%s' "${argv[2]:?missing message}" | base64 -d) + target=$(ensure_pane) + out="$RESPONSE_DIR/$turn_id.txt" + rm -f "$out" + # Envelope matches what paliadin_prompt.go expects. + tmux send-keys -t "$target" -l -- "[PALIADIN:$turn_id] $msg" + tmux send-keys -t "$target" Enter + # Poll for the response file. Same shape as Go pollForResponse. + for _ in $(seq 1 $((TIMEOUT_S * 5))); do + if [[ -s "$out" ]]; then + sleep 0.05 # settle + cat "$out" + rm -f "$out" + exit 0 + fi + sleep 0.2 + done + echo >&2 "paliadin: response timeout after ${TIMEOUT_S}s" + exit 124 + ;; + + reset) + # /clear the conversation; next turn starts fresh. + target=$(ensure_pane) + tmux send-keys -t "$target" -l -- "/clear" + tmux send-keys -t "$target" Enter + echo ok + ;; + + *) + echo >&2 "paliadin-shim: unknown verb '$verb'" + exit 2 + ;; +esac +``` + +Why a shim instead of raw tmux-over-SSH: + +- One SSH round-trip per turn (~50 ms over tailnet) vs ~10–20 round-trips for the granular pattern. +- Argument validation lives in one place (UUID regex on turn_id, base64 for messages, fixed verb list) — easier to audit than a regex over `$SSH_ORIGINAL_COMMAND` matching `tmux send-keys …`. +- mRiver-side concerns (response polling, settle delays, pane-readiness) stay on mRiver, which is where the tmux state lives. The Go service stops caring about local file polling at all. + +--- + +## 6. Sub-design C — Service-layer integration, routing, reliability + +### 6.1 Interface split + +The current `*PaliadinService` becomes an interface with two implementations: `LocalPaliadinService` (the existing tmux code, renamed) and `RemotePaliadinService` (the new SSH code). Construction picks one at startup based on `PALIADIN_REMOTE_HOST`. + +```go +// internal/services/paliadin.go (after refactor) + +type Paliadin interface { + RunTurn(ctx context.Context, req TurnRequest) (*TurnResult, error) + ResetSession(ctx context.Context) error + ListRecentTurns(ctx context.Context, callerID uuid.UUID, limit int) ([]PaliadinTurn, error) + Stats(ctx context.Context, callerID uuid.UUID) (*PaliadinStats, error) + IsOwner(ctx context.Context, userID uuid.UUID) (bool, error) +} + +// LocalPaliadinService wraps the current tmux PoC (laptop / dev path). +type LocalPaliadinService struct { /* identical to today's PaliadinService */ } + +// RemotePaliadinService talks to a paliadin-shim over SSH on mRiver. +type RemotePaliadinService struct { + db *sqlx.DB + users *UserService + sshHost string // 100.99.98.203 + sshUser string // m + sshKeyPath string // /tmp/paliadin-id_ed25519- + knownHosts string // /tmp/paliadin-known_hosts + turnMu sync.Mutex + + // Health-check cache. + healthMu sync.Mutex + healthOK bool + healthCheckedAt time.Time +} +``` + +DB access (`ListRecentTurns`, `Stats`, `IsOwner`) is identical for both — they only read `paliad.paliadin_turns`. They live in a shared `paliadinDB` helper struct embedded in both implementations. + +### 6.2 Wiring at startup + +```go +// cmd/server/main.go (excerpt) +var paliadin services.Paliadin +remoteHost := os.Getenv("PALIADIN_REMOTE_HOST") +switch { +case remoteHost != "": + keyPath, err := loadPaliadinSSHKey() + if err != nil { log.Fatalf("paliadin: load ssh key: %v", err) } + if keyPath == "" { log.Fatalf("paliadin: PALIADIN_REMOTE_HOST set but no PALIADIN_SSH_PRIVATE_KEY") } + knownHosts, err := loadPaliadinKnownHosts() + if err != nil { log.Fatalf("paliadin: load known_hosts: %v", err) } + paliadin = services.NewRemotePaliadinService(db, userSvc, services.RemotePaliadinConfig{ + SSHHost: remoteHost, + SSHUser: cmpOr(os.Getenv("PALIADIN_REMOTE_USER"), "m"), + SSHKeyPath: keyPath, + KnownHostsPath: knownHosts, + }) + log.Printf("paliadin: remote mode → ssh %s@%s", "m", remoteHost) +case localTmuxAvailable(): + paliadin = services.NewLocalPaliadinService(db, userSvc, "", "") + log.Printf("paliadin: local tmux mode") +default: + paliadin = services.NewDisabledPaliadinService(db, userSvc) + log.Printf("paliadin: disabled (no remote host, no local tmux)") +} +``` + +`NewDisabledPaliadinService` exists today implicitly via the `ErrTmuxUnavailable` path; making it explicit gives the constructor a clear name and the handler doesn't have to special-case `nil`. + +### 6.3 SSH invocation pattern + +`RemotePaliadinService` runs every RPC through the same helper: + +```go +func (s *RemotePaliadinService) callShim(ctx context.Context, args ...string) ([]byte, error) { + sshArgs := []string{ + "-i", s.sshKeyPath, + "-o", "UserKnownHostsFile=" + s.knownHostsPath, + "-o", "StrictHostKeyChecking=yes", + "-o", "BatchMode=yes", + "-o", "ConnectTimeout=3", + "-o", "ServerAliveInterval=10", + "-o", "ServerAliveCountMax=3", + s.sshUser + "@" + s.sshHost, + "--", + } + sshArgs = append(sshArgs, args...) + c, cancel := context.WithTimeout(ctx, 70*time.Second) // shim has its own 60s; +10s for SSH overhead + defer cancel() + cmd := exec.CommandContext(c, "ssh", sshArgs...) + var stdout, stderr bytes.Buffer + cmd.Stdout = &stdout; cmd.Stderr = &stderr + if err := cmd.Run(); err != nil { + return nil, fmt.Errorf("paliadin: ssh shim %v: %w (stderr: %s)", args, err, stderr.String()) + } + return stdout.Bytes(), nil +} +``` + +`RunTurn` becomes: + +```go +func (s *RemotePaliadinService) RunTurn(ctx context.Context, req TurnRequest) (*TurnResult, error) { + s.turnMu.Lock() + defer s.turnMu.Unlock() + + if err := s.healthGate(ctx); err != nil { + return nil, err // ErrMRiverUnreachable, picked up by handler + } + + turnID := uuid.New() + started := time.Now().UTC() + if err := s.insertTurnRow(ctx, …); err != nil { return nil, err } + + // First-turn-only: bootstrap the system prompt on mRiver. Detected by + // checking whether any prior turn for this user has succeeded. + if err := s.ensureBootstrapped(ctx); err != nil { + _ = s.markTurnError(ctx, turnID, "bootstrap_failed") + return nil, err + } + + msg := sanitiseForTmux(req.UserMessage) + msgB64 := base64.StdEncoding.EncodeToString([]byte(msg)) + body, err := s.callShim(ctx, "run-turn", turnID.String(), msgB64) + if err != nil { + _ = s.markTurnError(ctx, turnID, classifySSHError(err)) + return nil, err + } + + // Same trailer-parse + audit-row writes as Local, factored into shared helper. + return s.completeTurnFromBody(ctx, turnID, started, string(body)) +} +``` + +### 6.4 System prompt bootstrap + +The local PoC calls `paliadinSystemPrompt(s.responseDir)` once when it creates the pane. The remote path needs the same hook. Two options that don't require duplicating the German prompt body to mRiver: + +- **Lazy bootstrap (chosen):** the first `RunTurn` after a paliad-prod restart sends the system prompt via `bootstrap` RPC, then runs the actual turn. Subsequent turns skip the bootstrap. State is per-process: `RemotePaliadinService.bootstrapped` boolean guarded by mutex. +- Eager bootstrap at startup is rejected — it forces every container start to wait for mRiver to be online, which couples paliad's boot to mRiver's availability. + +Lazy bootstrap means the very first turn after a paliad redeploy pays a ~3 s extra cost (claude pane spin-up + system prompt absorb). Acceptable for a single-user PoC. + +### 6.5 Health-check gating (`mriver_unreachable`) + +Every `RunTurn` first calls `healthGate(ctx)`: + +- Cached for 10 s. If last check was <10 s ago and was OK, skip the probe. +- Otherwise: `s.callShim(ctx, "health")` with a 3 s timeout. On success, set cache OK; on failure, return `ErrMRiverUnreachable`. + +Why 10 s: short enough that "I just woke my laptop" propagates inside one user retry; long enough that a busy chat doesn't probe on every turn. + +```go +var ErrMRiverUnreachable = errors.New("paliadin: mriver unreachable") + +func (s *RemotePaliadinService) healthGate(ctx context.Context) error { + s.healthMu.Lock() + defer s.healthMu.Unlock() + if s.healthOK && time.Since(s.healthCheckedAt) < 10*time.Second { + return nil + } + c, cancel := context.WithTimeout(ctx, 3*time.Second) + defer cancel() + out, err := s.callShim(c, "health") + s.healthCheckedAt = time.Now() + if err != nil || strings.TrimSpace(string(out)) != "ok" { + s.healthOK = false + return fmt.Errorf("%w: %v", ErrMRiverUnreachable, err) + } + s.healthOK = true + return nil +} +``` + +### 6.6 Friendly error code (extends t-paliad-150) + +`friendlyErrorMessage` already maps `tmux_unavailable` to a localised message. We add one new code: + +- `mriver_unreachable` → DE: *"mRiver ist offline — Paliadin nicht erreichbar. Mach mRiver an, oder nutze Paliadin lokal mit `./paliad`."* / EN: *"mRiver is offline — Paliadin can't reach it. Wake mRiver, or run Paliadin locally with `./paliad`."* + +Implementation: one new `case` in the SSE-error switch in `frontend/src/client/paliadin.ts`'s `friendlyErrorMessage`, plus matching i18n keys (`paliadin.error.mriver_unreachable.de` / `.en`). Server-side: `paliadin` HTTP handler maps `errors.Is(err, services.ErrMRiverUnreachable)` to `event: error\ndata: {"code":"mriver_unreachable","message":"..."}\n\n`. + +### 6.7 Rate limit + +A runaway loop on the paliad side could DOS the SSH connection. Cheapest cap: enforce one in-flight turn at a time via `turnMu` (already exists in the local PoC). On top of that, a rolling cap of N=20 turns/min in `RemotePaliadinService` rejects with `ErrRateLimited` (mapped to a friendly `paliadin.error.rate_limited`). PoC has one user (m); the cap is a paranoid safety, not a real throttle. + +### 6.8 What about ControlMaster? + +Decision-2's chosen path (server-side shim with one RPC per turn) makes ControlMaster optional. The shim collapses ~10 raw-tmux ops into a single SSH connect — that's already the latency win ControlMaster would buy. + +Adding it on top would save ~30–50 ms per turn but adds: + +- A persistent `~/.ssh/cm-*` socket inside the container. +- Cleanup logic on shutdown. +- A subtle interaction with the SSH BatchMode + ConnectTimeout settings. + +Verdict: skip ControlMaster in v1. If turn latency over Tailscale is measured >300 ms in practice and hot enough to matter, add it in a follow-up; the call site is one helper. + +--- + +## 7. Phasing + +### Phase A — manual proof-of-concept (no Dockerfile change yet) + +Goal: validate the round-trip end-to-end on a deployed paliad, before touching the image. + +Steps: + +1. **Generate keypair** on mRiver: `ssh-keygen -t ed25519 -N "" -C "paliad-prod" -f /tmp/paliad-prod-key`. +2. **Install shim** at `/home/m/.local/bin/paliadin-shim` (script from §5.4), `chmod 755`. +3. **Write authorized_keys** with the public key + restrictions from §5.2. +4. **Capture mRiver host key**: `ssh-keyscan -t ed25519 100.99.98.203 > /tmp/paliad-known_hosts` from mLake. +5. **Confirm host networking trade-off (§4.2):** flip the running paliad-prod compose to `network_mode: host` on a temporary branch; redeploy via Dokploy; verify `https://paliad.de/` still serves correctly via traefik. **Gate**: if traefik 502s, abort Phase A and revisit decision 1 in a follow-up issue. +6. **Smoke-test SSH from inside the container**: + ``` + docker exec -it paliad-prod sh + apk add --no-cache openssh-client # one-shot, before Dockerfile change + ssh -i /tmp/key -o UserKnownHostsFile=/tmp/known_hosts m@100.99.98.203 health + # expected: "ok" + ssh … run-turn $(uuidgen) "$(printf 'Hallo Paliadin' | base64 -w0)" + # expected: response body, then ".../uuid.txt" cleaned up + ``` +7. **Wire env vars manually** via Dokploy UI for one deploy; confirm `/paliadin` works end-to-end against mRiver. + +If Phase A passes: codify into Phase B. If it fails on (5), the design rolls back to a sidecar in a new issue (decision 1 follow-up). If it fails elsewhere, fix the shim or the SSH config; the architecture is fine. + +### Phase B — bake into Dockerfile + Dokploy secrets + +1. Dockerfile: add `openssh-client` to the final stage (§4.3). +2. compose: add `network_mode: host` and the four new env vars (§4.1). +3. Dokploy secrets: register `PALIADIN_REMOTE_HOST=100.99.98.203`, `PALIADIN_REMOTE_USER=m`, `PALIADIN_SSH_PRIVATE_KEY=...`, `PALIADIN_KNOWN_HOSTS=...`. +4. Code: refactor `PaliadinService` to the interface split (§6.1–§6.2). New file `internal/services/paliadin_remote.go`. Tests: `paliadin_remote_test.go` mocks `callShim` to verify `RunTurn` audit-row writes, error mapping, and `healthGate` caching. +5. Ship under one PR; tag t-paliad-151 done. + +### Phase C — friendly errors + monitoring + +1. `paliadin.error.mriver_unreachable` i18n keys + `friendlyErrorMessage` case (§6.6). +2. `/admin/paliadin` shows last health-probe result + last successful turn timestamp. +3. Optional: `mai-mesh` integration to surface mRiver-offline events to m on Telegram (out-of-band; not gating). + +--- + +## 8. Security review summary + +| Risk | Mitigation | +|---|---| +| Stolen private key → arbitrary SSH on mRiver | `command=` shim restriction + `from="100.99.98.201"` + ed25519 key + private key only in Dokploy secret store (encrypted at rest) | +| Stolen private key → tailnet-wide SSH from non-mLake host | `from="100.99.98.201"` clause | +| Container compromise → key extraction | Key written to tmpfile chmod 600, only root inside container can read; alpine container has no shell-on-error trampolines | +| Host-key MITM during connect | Pinned `known_hosts`; `StrictHostKeyChecking=yes` | +| Shim argument injection (e.g. via `run-turn $(rm -rf /)`) | Shim parses positional args from `$SSH_ORIGINAL_COMMAND` via `read -r -a`; never passes args to a subshell `eval`; turn_id validated by UUID regex; message body always base64-decoded into a single shell variable, never re-evaluated | +| Runaway loop → SSH flood | Single-flight `turnMu` + 20/min rolling cap | +| `network_mode: host` widens blast radius | The `command=` + `from=` restrictions on mRiver mean container compromise = "can run shim verbs against mRiver only", not "shell on mRiver" | +| PaliadinOwnerEmail bypass | Unchanged from PoC: gate is in Go (`/paliadin` 404s for any other user). Even if mRiver SSH key leaks, attacker still needs paliad session as `m@hoganlovells.com`. | + +--- + +## 9. Out-of-scope clarifications (for review) + +These were called out in the issue but the design intentionally does not solve them, to keep v1 tight. Each is acknowledged so review knows it wasn't an oversight: + +- **Wake-on-LAN of mRiver:** out of scope. v1's UX when mRiver is asleep is the friendly error from §6.6. Future work: integrate with `mai-mesh` capability fallback. +- **Multi-host failover:** out of scope. Only mRiver is targeted. +- **Anthropic API fallback when mRiver offline:** out of scope per CLAUDE.md (`ANTHROPIC_API_KEY` reserved for production-v1, unused in PoC). +- **ControlMaster:** v1 ships without; revisit if turn latency >300 ms in practice (§6.8). + +--- + +## 10. File-level deliverables (for the coder shift) + +When this design is approved and the coder shift starts, the work splits roughly into: + +- `Dockerfile` — `+openssh-client`. +- `docker-compose.yml` — `network_mode: host`, four new env entries. +- `internal/services/paliadin.go` — extract `Paliadin` interface; rename existing to `LocalPaliadinService`; pull DB-only methods (`ListRecentTurns`, `Stats`, `IsOwner`) into a shared embedded `paliadinDB` so both implementations get them for free. +- `internal/services/paliadin_remote.go` — new file: `RemotePaliadinService`, `RemotePaliadinConfig`, `callShim`, `healthGate`, `ensureBootstrapped`, `classifySSHError`, `ErrMRiverUnreachable`. +- `internal/services/paliadin_remote_test.go` — unit tests with a mocked `callShim`. +- `cmd/server/main.go` — env-var-based wiring (§6.2), `loadPaliadinSSHKey`, `loadPaliadinKnownHosts`. +- `frontend/src/client/paliadin.ts` — one `case` in `friendlyErrorMessage` for `mriver_unreachable`. +- `frontend/src/i18n.ts` — two new keys (`paliadin.error.mriver_unreachable.de` / `.en`). +- `scripts/paliadin-shim` — server-side script (§5.4); copied to mRiver during Phase A, not part of any container. +- `docs/project-status.md` — note Phase 0.5 (PoC) → Phase 0.6 (Tailscale-SSH prod route). + +No DB migrations needed — `paliad.paliadin_turns` schema already covers everything (`error_code` field already accepts free-form codes including `mriver_unreachable`). + +--- + +## 11. Open questions for review + +- **Q (m):** Phase A test step 5 expects traefik to keep working under host-mode. If a quick search confirms Dokploy's traefik can't route to host-network services without manual `loadbalancer.server.url` config, we should know before Phase A. Worth a 5-minute Dokploy doc check before merging Phase B. +- **Q (m):** Should the `paliadin-shim` script live in this repo (`scripts/paliadin-shim`) and be version-pinned, or is it a one-off that lives only on mRiver? Repo location lets us audit changes; mRiver-only keeps deploy footprint smaller. +- **Q (m):** `ANTHROPIC_API_KEY` env var reservation in compose comments — keep the comment line for production-v1, or strip it now that this design supersedes that path for the foreseeable future? + +--- + +**Inventor stops here.** No code shipped this shift. Awaiting m's go/no-go on the design before the coder shift starts.