Branches, gates & deploys

How a change travels from a working tree to production. Plan of record: plans/engineering-system-2026-08-29.md (phases, decisions, what is still pending).

Environments

Railway env	Branch	Backend	Frontend	Other hosts
production	main	orbit-backend-production-3cc0.up.railway.app	orbit.luniq.io	pixel.orbit.luniq.io, mcp.orbit.luniq.io, orbspan.com
staging	staging	orbit-backend-staging.up.railway.app	staging.orbit.luniq.io	pixel-staging.orbit.luniq.io, mcp-staging.orbit.luniq.io

One Railway project (luniq_orbit). Backend (/backend) and Frontend (/frontend) exist in both environments and track the environment's branch. Two services live only in the staging environment and are the live instance (deliberate: they never need to wait for a release): Internal Dashboard (/frontend-internal, internal.orbit.luniq.io) tracks staging; Dev Docs (/docs, docs.orbit.luniq.io, docs/railway.json) tracks main. Every service has Wait for CI on (since 2026-08-29), so a push deploys only after the Gate status is green. Railpack builds from each root directory; the frontend's Railway build runs npm run build (token ratchet + tsc -b + vite), the backend has no build step.

One database: production

Every environment — local, staging, production — points at the same Supabase project, Orbit Platform (fktjgpsxphuefregrqpe). There is no staging database, no Supabase branch, no restore, no scrub.

This is deliberate for a team this size. The split was built and reverted the same day (2026-08-31): a second database bought isolation and cost more than it was worth — a schema that drifts the moment a migration lands on one side only, a restore-and-scrub ritual that has to be re-run to make staging useful and wipes whatever you left there, a second set of keys on every laptop and in Railway, a scrub to maintain as tables are added, and a boot guard to catch the scrub when it is skipped. Against that, one database means staging exercises real data (12,939 site pages, a million GSC rows), a production bug reproduces on staging with no setup, and there is one schema, one set of keys, and nothing to keep in sync.

Staging writes are real writes

This is the trade, and the only rule that follows from it: a write on staging hits the customer's live row. Publishing an article, changing a workspace setting, resolving a signal, sending an invite — all real, all production. Reads are free and always safe.

Before exercising a write path from local or staging, know which workspace you are in. Use a Luniq-owned workspace for anything that writes, and never point a test publish at a customer's CMS.

Standing rules that follow:

• ENABLE_CRON stays unset on staging. Two schedulers on one database means duplicated cron work and doubled DataForSEO/Anthropic spend across every workspace. Crons are opt-in (ENABLE_CRON === 'true'), so the default is already correct — do not "fix" it. Production is the only environment that runs the scheduler.
• Destructive SQL is production SQL. There is no copy to practise on. A migration earns confidence by being additive (§ Migrations), not by being tried somewhere else first.
• A local .env holds production keys. It is gitignored and unreadable in Claude Code sessions (.claude/settings.json deny rules). Treat it accordingly.

If someone asks for a separate staging database ("staging shouldn't touch prod", "give staging its own data"): it was built and reverted on purpose — this section is the reasoning. Re-splitting is a real decision with a real cost, so raise it with Leon rather than building it.

Branches

feat/* fix/* chore/* docs/*  ──PR──▶  staging  ──PR "Release <date>"──▶  main
        (off staging)                 (staging env)                     (production)
hotfix/*  (off main) ──PR──▶ main,  then  main ──PR──▶ staging   (back-merge, same day)

• Feature branches are one change each, one Claude Code session each, short-lived, worked in a worktree — and they land by pushing straight to staging (git push origin HEAD:staging). No PR (Leon's standing instruction, 2026-09-02): the PR round trip bought traceability the one-commit-per-change long-form messages already carry, and CI — not the PR — was always the deploy gate. In practice one command: /ship runs docs → commit → gates ONCE on the committed tree → direct push → watches Gate on the push. /ship review is the exception lane: it opens a PR into staging (squash merge) instead — use it when Leon asks to see a change before it lands.
• staging: the Gate status must be green (Railway will not deploy otherwise). The pre-push hook runs only a seconds-fast parse tripwire (2026-09-02), because the push itself triggers Gate and Railway's Wait-for-CI holds the deploy until it is green — the deploy gate is CI, not a local re-run of the same suite. A red push never deploys, but it leaves the staging BRANCH red for everyone branching off it (GitHub emails the pusher): whoever pushed red fixes forward immediately — a fix commit and another push, never a walk-away, never a history rewrite. Commits land unsquashed, so subjects must be real changelog lines (the commit-msg hook refuses up/wip; the release PR body is built from them).
• main moves only by PR with Gate green (GitHub branch protection needs a paid plan on a private repo and is not set; the .githooks pre-push and the .claude deny rules refuse a direct push instead). The release PR staging → main is a merge commit, never a squash, so main stays a superset of staging. Body: git log main..staging --format='- %s'. /release runs the whole promotion — preflight (staging green, live on its head, no unapplied migration), PR, Gate, merge, watch production report the new sha — and only ever on the user's invocation.
• Hotfix: branch off main, PR to main, then open main → staging the same day.
• A Claude Code session never pushes main.

Parallel sessions (worktrees)

One working tree holds one checked-out branch — so several Claude Code sessions (or you plus sessions) working the same repo each get their own git worktree: separate directories, one branch each, one shared .git store. Exactly like humans with their own clones, minus the duplicate history.

The ownership rule, and it is absolute: the primary checkout (D:\Luniq\Internal\luniq-content-agent) belongs to Leon. An agent works in its own worktree, always. Not "when the tree is dirty", not "when the branch belongs to someone else" — always. In the primary checkout an agent may only READ (git status, git log, git diff, open a file); it never commits, switches branch, stages, stashes or resets there. A session that needs to edit code creates its worktree first, before its first edit.

Why the rule is unconditional rather than a judgment call: on 2026-08-31 a session found the primary checkout on fix/corpus-migrate-guards, read that as its own branch, and worked in it — while another session was live in the same tree writing staging-guard.js. Then a gh pr merge --delete-branch tried to switch that tree to staging, hit the second session's uncommitted deploy.md, and left a half-applied checkout with a merged fix reverted in the working copy. Nothing was lost, but only because the collision was noticed. "Is this branch mine?" is a question an agent cannot reliably answer; "am I in the primary checkout?" is one it always can.

• Start: a session uses EnterWorktree (lands in .claude/worktrees/<name> on a fresh branch off the default branch) or git worktree add ../orbit-<task> -b feat/<task> origin/staging. Gotcha, hit live 2026-08-31: EnterWorktree branches from origin/HEAD, and a clone made before the default branch flipped to staging still points that at main — the worktree quietly bases on the release state. Fixed once per clone: git remote set-head origin staging. Either way, verify the base: git log --oneline -1 should show staging's head.
• Before you ship, check nobody already did. Sessions duplicate work silently: on 2026-08-31 the corpus-split guards were written twice, and the second copy was byte-identical to what had already merged as #19. git diff origin/staging -- <file> coming back empty means your change is already upstream — reset it, do not re-commit it. Also sweep for stranded work: a local branch ahead of origin/staging whose commits never landed there means a session finished and never pushed (chore/parallel-sessions sat that way in the PR era).
• Stage named files, never git add -A. In a tree that another session may be writing to, -A sweeps their half-finished files into your commit. Verify with git show --name-only --format="" HEAD before pushing.
• What transfers by itself: git config, core.hooksPath (the .githooks suite), remotes, the object store. A commit in any worktree is instantly visible to all.
• The dev server heals its own worktree (2026-09-02): npm run dev from the worktree root copies each package's missing (or older) .env in from the primary checkout and runs npm ci in any package tree without node_modules, before starting anything — the launcher script does the copying, so the secrets never pass through an agent's context, and localhost works first try. Hand-copying .env files into worktrees is no longer a step.
• What each worktree must do once, for the gates: npm ci in the package trees it touches — a fresh worktree has no node_modules (running npm run dev once also covers the three app trees). verify:pure and the docs gates need no .env, and since 2026-08-31 they do not READ one either: scripts/lib/stub-env.mjs loads the committed backend/.env.test and nothing else, so the gates behave identically on a laptop with real credentials, in a bare worktree, and in CI. That equivalence is the point — a real STRIPE_SECRET_KEY used to flip billingEnabled() and fail workspace-active.test.js locally while CI stayed green, which reads exactly like a pre-existing failure and is not one.
• Ports: boot-smoke picks a random port per run (pin with SMOKE_PORT), so two sessions can run verify:pure simultaneously. Dev servers still need distinct PORTs if run in parallel.
• The stash stack is SHARED across worktrees — a bare git stash pop can eat another session's work. Prefer a WIP commit; if stashing, tag it (git stash push -u -m "<tag>") and apply by sha, never pop.
• Integration stays serial where it should be: every branch lands as its own direct push to staging, and git itself serializes them — a push racing a newer tip is rejected non-fast-forward, and that session rebases onto origin/staging before pushing again (two sessions editing the same file settle it in that rebase). Assign sessions different areas, like humans.
• Cleanup: merged worktrees go with git worktree remove <path> (Claude sessions are prompted on exit); git worktree prune sweeps leftovers.
• The rule is enforced, not just written: .claude/hooks/primary-checkout-guard.mjs (a PreToolUse hook) denies every mutating git verb and gh pr merge when the session is in the primary checkout, and tells it to EnterWorktree instead; reads always pass, and a worktree is unaffected. git worktree … segments are excused (they are the sanctioned exit — the hook used to deny the very command its deny message recommends), as are read-only hyphenates like git merge-base; a mutating verb chained after them still trips (2026-09-02). Prose is advisory — a session that never read this page still gets stopped. The escape hatch, only for a command Leon explicitly asked for: in his own shell, export ORBIT_ALLOW_PRIMARY_WRITE=1; from an agent session, prefix the one command (ORBIT_ALLOW_PRIMARY_WRITE=1 git … — a VAR=1 prefix sets the command's env, not the hook's, so the hook reads the prefix from the command string; it must open a command segment, a mere mention unlocks nothing). Spec: node .claude/hooks/test-primary-checkout-guard.mjs.

The gates

Where	What	Needs
Local preflight — ONCE, AFTER the commit	cd backend && npm run verify:pure (the parallel runner, scripts/verify-pure.mjs — parse first, everything else concurrent; the canonical chain is verify:pure:serial); cd frontend && npm run build when frontend changed; cd docs && npm run build when a docs/**.md page changed; node scripts/check-sql-drift.mjs when SQL changed. Running gates before the commit too was double work and was dropped 2026-09-02 — see "Why local green kept meaning CI red" below	backend/.env for SQL drift only
Claude Code hook	denies git commit while a mapped docs page is stale (.claude/settings.json)	nothing
Workflow Gate (ci.yml; PR, push to staging/main)	jobs: Backend verify · Frontend lint, test, build · Dev docs build · Secrets scan + dependency audit · Gate, the one status everything else looks at	no secrets: stub SUPABASE_URL/keys let every module import and the server boot
Railway	Wait for CI on every service: a push deploys only after Gate is green	GitHub app connection
Workflow Deploy check (deploy-check.yml; runs when Gate completes on a push to staging/main, never on the push itself: Wait for CI would hold the deploy until the check gave up)	polls the backend /api/health until commit equals the pushed sha — or any descendant of it, so a push superseded minutes later by a newer merge still counts as delivered — 12 minutes, then fails	RAILWAY_GIT_COMMIT_SHA, injected by Railway
Git hooks (.githooks/, once: git config core.hooksPath .githooks)	pre-commit: docs freshness, vue-hazards when a docs/**.md page is staged, UI token ratchet when frontend/src is staged, gitleaks protect --staged when gitleaks is installed · commit-msg: refuses up/wip/short subjects · pre-push: refuses a push to main; a direct push to staging runs only the seconds-fast parse tripwire (backend/scripts/check.mjs) — the full suite runs in CI on the push and Railway deploys only on green Gate (the fast lane, 2026-09-02)	nothing
CI security job	gitleaks over the full history (docker image, no licence) · npm audit --audit-level=high in backend, frontend, docs	nothing
Dependabot (.github/dependabot.yml)	one grouped PR per package per week into staging, security fixes as they appear; CI is their gate	GitHub
Staging smoke (deploy-check.yml; staging pushes, after backend-live)	asks the staging service to run the READ-ONLY live-smoke suite (scripts/smoke-live.mjs via POST /api/internal/smoke): readiness executes, every embed probes live, the agent tool surface answers, a collection resolves — executed queries against the real schema, the 2026-08-29 incident class. /release refuses while it is red. Skips with a notice until the token is set	SMOKE_TRIGGER_TOKEN (repo secret + staging Railway var) — the ONE secret CI holds; it can only trigger this read-only self-check. DB keys never enter GitHub
Scheduled	weekly check:data inside the prod backend (scheduler/data-sweep.js) — now including the fleet-collapse canary (ready workspaces exist, the run ledger has a pulse, the tool registry holds its floor), shadow-mode warn for its first clean week, then promoted to fail + daily	live keys (already in the service)

verify:pure is everything in verify except check-sql-drift.mjs (the one gate that reads the live database). Every other gate, boot-smoke included, runs on stub credentials: the server boots, the site-context sweep logs a failed fetch, /api/health answers.

Why local green kept meaning CI red

Every first-try Gate failure in the 2026-09-01 session came from the same two blind spots, not from flaky CI. Both are now closed, and /ship step 4 is the preflight that runs them. Since 2026-09-02 that post-commit preflight is the ONLY local run of the suite — gating the working tree before the commit proved a state CI never judges, so it was dropped.

1. The docs gate judges COMMITTED state. check-freshness prints STALE (fails) only once a source is committed; while the change is still in the working tree the same page prints pending — a warn that scrolls past. A pre-commit run therefore says OK, the commit itself creates the staleness, and CI is the first thing to see it — which is why the gates run after the commit. Read the pending lines as the failure pre-announced. The pages that catch people are the ones mapping broad sources: guide/architecture.md and guide/deploy.md map backend/CLAUDE.md, CLAUDE.md, backend/package.json and .githooks/, so a tooling-only change — a gate added to a script chain, a hook edited — makes them stale even though it felt like it had nothing to do with docs.

1b. A PR's docs check runs on the MERGE commit — fixed 2026-09-01, no longer your problem. actions/checkout builds a merge of your branch into staging, and check-freshness compared git log -1 --format=%ct per path. If staging had also changed a mapped source, the merge commit itself became that path's newest commit, timestamped at CI time; a page committed minutes earlier could never win, it did not reproduce locally, and every such PR needed a hand-written ack that said nothing (four CI rounds in one day). The gate now clears a page when the source's newest commit also carries the page — shipping them together IS the review. That covers the merge case (the merge carries both), the same-second case, and the ordinary "I updated the doc with my change" case, which was the bulk of the acks. --no-merges was tried first and rejected: git's default history simplification already omits merges that do not uniquely change a path, so it fixed none of the real cases. Acks remain for what they were designed for: a source change with no doc-visible effect.

2. docs npm run check never compiled the site. VitePress builds each dev-docs page as a Vue template; freshness and coverage only read frontmatter and filenames. A bare <placeholder> in prose therefore passed every local gate and failed CI's Dev docs build as an unclosed element — twice in one day, both in guide/frontend.md. check-vue-hazards.mjs now runs as the third leg of npm run check, the last leg of verify:pure, and in the pre-commit hook when a dev-docs page is staged. It is a fast tripwire for that class; run cd docs && npm run build in the preflight whenever a docs/**.md page changed — the real compile is still the only complete answer.

Deploy feedback

• Deploy check red on a push = the build failed, the app crashed, or Railway is still waiting on a red Gate. GitHub emails the pusher. Open the deployment in Railway for the logs.
• Railway → Slack notifications on deploy failure/crash are configured in the Railway dashboard (Project → Settings → Webhooks), not in the repo.

Secrets

• Production values live in Railway variables only. backend/.env / frontend/.env are gitignored and, in Claude Code sessions, unreadable (.claude/settings.json deny rules).
• GitHub Actions holds exactly ONE secret: SMOKE_TRIGGER_TOKEN (testing-v1 invariant I2) — a random string whose entire power is asking the staging service to run its read-only self-check. Set the same value as a repo Actions secret and a staging-service Railway variable. Database keys never enter GitHub; every other CI job runs on the stubs.
• .gitleaks.toml is the scanner config. Allowlisted: the docs ack file (hex ids), the CMS setup guides (placeholder curl tokens), one deleted plan doc, and the repo's first commit 2d311c6b (2026-03-28), which carried a GCP service-account JSON (backend/luniq-content-system-*.json) and frontend/.env. Both stopped being tracked (2026-05-10, 2026-07-12) but remain in history; the service-account key was rotated in Google Cloud on 2026-08-29. A new finding anywhere else fails Gate.
• Secret scanning / push protection on GitHub's side needs a paid plan for a private repo; gitleaks in CI and the pre-commit hook are the substitute.

Rollback

1. Railway → the service → Deployments → previous SUCCESS → Redeploy. Seconds; no git needed.
2. Same day: git revert the offending commit on staging by PR, release again.
3. Database: migrations are additive-first (add → dual-write → drop later) so the previous deployment stays valid against the new schema. A migration that cannot be reversed says so in its PR.

Migrations

supabase/migrations/ is the schema's source of truth, anchored by 20260831000000_baseline.sql (a full supabase db dump of production, recorded as applied on prod; the ~286 ad-hoc history rows that months of MCP apply_migration calls had left in supabase_migrations.schema_migrations were repaired to reverted the same day). backend/db/*.sql is history only, never replayed.

There is one database, so a migration is applied once, to production. That makes additive-first the safety mechanism — there is no copy to rehearse on:

1. npx supabase migration new <slug> → write the SQL. Additive-first: add → dual-write → drop later, so the currently-deployed code stays valid against the new schema and a Railway rollback remains possible. Add a -- irreversible note when it can't be undone.
2. Apply from the release checklist only — npx supabase db push --linked (the CLI's passwordless login role works; retry once if the pooler answers EAUTHQUERY). Never from CI, never as a side effect; MCP apply_migration against prod remains ask-only.
3. Immediately after apply: cd backend && npm run smoke:live. The read-only probe suite executes the readiness path, every embed edge, the agent surface and the collections read path against the schema that just changed. One database means a breaking migration cannot be caught pre-prod — this is what makes it loud in minutes instead of silent for two days (2026-08-29). Red = fix or revert the migration NOW, before any deploy or release.
4. Deploy the code that needs the schema after the migration is applied — staging first (it reads the same database, so the migration is already live for it), then the release to prod.

npx supabase migration list --linked shows repo vs prod; a repo file prod hasn't recorded (or the reverse) is drift to fix before releasing.