Health

Module id: health (frozen) · Mounted at: /api/modules/health · Empty settings row (the shell-module convention; no switches — the one per-workspace pause is agent.paused, held by the scheduler chokepoint).

Purpose

Technical site health for the two pipelines that decide whether a site can rank or be cited at all: Google (indexability, snippet eligibility) and AI-assistant crawlers (ChatGPT/Perplexity/Claude/Google-AI bot access). Produces one 0–100 score, a Google verdict + an AI verdict, and a check-grouped, resolve-tracked issue list.

Features

One weekly audit (runHealthAudit → doAudit) with independently-degrading stages, guarded by a cross-process run lock (health_audits partial unique index WHERE status='running'; 60s heartbeat, 5-min stale reclaim):

1. Site checks — robots reachable/blocks-all, AI-search-bot blocked, AI-training opt-out, sitemap missing/not-in-robots, TLS invalid, home variants, HTTP→HTTPS redirect, UA probes, DNS.
2. Full-site crawl (DataForSEO OnPage; budget = whole catalog — site_pages + blog_articles — ×1.2 headroom, clamped 100–10,000; a crawl that used every page of its budget reports crawlCoverage.finished: false even when the crawler says finished; crawl_delay 500 ms, polled up to 4 h — 2026-08-26, was 30 min, which cut slow sites off at a few hundred pages while the vendor task kept crawling and billing unread pages) — broken pages (+ inbound link sources), broken/duplicate/missing canonicals/titles/h1, orphan/deep pages, links-to-redirects, non-https, links-to-http (mixed-content, notice — the crawler's https_to_http_links is anchors, not resources), slow TTFB, large pages, no compression, param explosion, redirect loops/chains, noindex/robots-blocked pages, sitemap broken/noindex entries. Per-page checks run on 200s only — a 3xx URL is judged by the redirect checks, never as "no H1"/"not https". noindex-pages skips conventional intentional noindex (legal pages, CMS date archives, tag/category listings); robots-blocked-pages skips parameter URLs (blocking them is the param-explosion fix).
3. GSC fusion — soft-404, crawled/discovered-not-indexed, canonical mismatch, rich-results errors, nosnippet, schema-missing, hreflang-broken (uses live crawl statuses to drop stale advice).
4. Core Web Vitals — CrUX field data (origin + ≤10 key pages) is the only source of cwv-poor; a site with no field data stores a homepage PSI lab reading on site_checks.cwv.lab as a diagnostic and is not graded (see spine).
5. Rendered/mobile parity sample — csr-content-missing, render-directive-flip, mobile-parity.
6. Scoring (computeScores, muted checks excluded) + issue lifecycle sync (syncIssues upsert/resolve; only checks whose detector actually RAN can resolve).

Per-check mute escape hatch for intentional findings. The check catalog, site checks, crawl, CWV, and parity primitives live in the spine at workspace/site-audit/.

Routes

Method	Path	Purpose
GET	/:workspaceId/overview	Latest audit + running progress + trend + grouped open issues
GET	/:workspaceId/issues/:checkId	Affected pages for a check (resolved to app resources)
POST	/:workspaceId/audit	Start manual audit
POST	/:workspaceId/issues/:checkId/mute	Per-check mute toggle

Crons

• Weekly audit — 0 5 * * 2 (Tuesdays 05:00, jittered) → runHealthAudit, gated by the scheduler's fire-time automation gate — setup complete + agent not paused, decided once for every workspace cron (see crons reference); the scheduler body re-checks nothing.
• Operator trigger healthAudit ("Health audit (full site)").

Agent tools & skills

• Tool get_site_health — stored audit: score (+delta), googleVerdict (eligible/issues/blocked), aiVerdict (reachable/partial/blocked), category scores (with labels), AI-bot access table (class search/live/training), issues ranked blockers-first with samples/evidence/fix plus the catalog's why and a siteLevel flag. Params: severity ('actionable' default | 'all' — notice-level findings, 10 of the 43 checks, are now reachable), include_muted (muted groups ride along flagged muted:true — agent counts agree with the UI tile), category (one catalog category's groups only), include_history + history_limit (default 12, max 52): audit records, not a score series (2026-08-23 — audit-history.js#listAudits, every status incl. failed, newest first, { audits, total }, site_checks left out of the list read), and audit_id to read ONE past audit in full (getAudit: verdicts, categoryScores, siteChecks, pagesCrawled, trigger, stage, error, startedAt/finishedAt) — "what did the audit on date X find / compare to last month" is now answerable; open issues stay the CURRENT state. The latest audit block carries id/trigger/startedAt and the full site-check facts (tls.issuer, sitemap.url, robots.status + robots.sitemaps). The audit block is degradation-honest: running {status, stage, pagesCrawled, trigger, startedAt}, lastFailure {error, failedAt}, crawlError, crawlCoverage, cwv (source: 'field' = graded; source: 'lab' = no real-user data, lab {lcp, cls, tbt, perfScore} is a diagnostic; {skipped}), llmsTxt — a degraded/partial audit is visible as such instead of being asserted as full-site health. Samples are one ordered 3-row read per included check (B3, 2026-08-23): the old single pooled 500-row read let one check with ≥500 open rows starve every other group of samples. Issue count is the real grouped count (the description used to claim a 500 cap it never had). Zero API cost. Conventions (2026-08-23): never-audited → { empty: true, reason 'never_audited' | 'first_audit_running', message }; the handler is wrapped in guardRead, so a failed read returns { error } instead of throwing.
• Tool get_page_health (H-5) — every OPEN issue on ONE url across all checks, each with its evidence (source + cause + location) plus surface/why/fix from the catalog. URL filtering is DB-side since 2026-08-05 — one indexed equality on the url_normalized generated column (a SQL mirror of url-identity.js normalizeUrl, db/tool-coverage-migration.sql); the old client-side filter after .limit(2000) went false-clean on workspaces past 2000 open issues. include_resolved adds FIXED issues (resolved_at set) — the "was that fixed, and when" answer. The per-page slot get_item_details reserved, delivered ring-clean: this module owns the read, the agent composes the two tools. The url may arrive with any scheme / www / trailing slash — getIssuesForUrl matches on url_normalized (documented in the description since 2026-08-23).
• Tool get_health_issue_urls (H-5) — the FULL affected-URL list for one check (get_site_health samples 3), incl. linkedFrom referrers for broken links. check_id + limit (max 200) + offset + include_resolved (fixed occurrences carry resolvedAt, open ones sort first) + url_prefix (one section of the site, DB like) + sort (url | last_seen). Since 2026-08-23 (B4) getIssueOccurrences pages in the DB (range) and total is a head count over the same filters — the old read sliced a 500/1000-row fetch, so total was min(actual, 500) and truncated:false at exactly 500 read as complete. total: null = the count failed (unknown, not zero). Envelope = the shared listEnvelope (core/utils/tool-envelope.js).

Data

Writes health_audits, health_issues; reads site_pages, blog_articles, workspaces.

Key files

index.js, services/audit.js (orchestrator + run lock), services/fusion.js (GSC ground-truth checks), services/issues.js (syncIssues, mutes), services/parity-sample.js (template-aware sampler), scheduler.js, routes.js, queries.js (overview, occurrences, per-URL issues, getSiteHealth), audit-history.js (audit record shape, getAudit, listAudits), agent-tools.js.

Signals (Autopilot v2)

The manifest declares mechanical signal detectors (agent-signals.js, collected via the agent registry's signals hook, run by the nightly sweep): one evidence atom per open issue group from the latest audit (severity-mapped, muted groups skipped) plus a score-drop atom at −5 or worse. Two step-5 gates (2026-09-01): the GSC index-state checks (gsc-crawled-not-indexed/gsc-discovered-not-indexed, NOT_VISIBLE_OWNED_CHECKS) no longer mint cards — the per-page xmod:not-visible card owns each unserved page — and they are skipped WITHOUT presence, because presence on a retired fingerprint reopens its superseded row every sweep (the audit page still lists them). And an info-severity page-level check needs at least one KEY page among its occurrences to earn a card (listKeyPagePaths from the spine; gated checks stay present, an unreadable key set gates nothing) — "Missing H1: 178 pages" weighted by count alone was audit defect 5. Two 2026-08-05 fixes: blocker severity now maps to critical (the map lacked the key, so every blocker landed as info), and signals carry the catalog's real title/why (they read label/description — fields the catalog never had — so every health signal was a raw checkId with detail: null). Since 2026-08-14 each issue atom also carries plan: [issue.fix] — the catalog has written a fix for every check since it existed and only the health page ever showed it, while it is the proposed plan for 84 of the fleet's cards for the cost of reading a field. 2026-08-19 — the card carries its pages and explains them (Leon: "one uniform linked pages", "any explanation always in the markdown"): for a page-level issue the detector reads its open occurrences (getIssueOccurrences, best-effort) and hands them to the board as linkedPages (url, title, status, linkedFrom, resourceType/Id — the board caps at 100 and records the full count), and writes facts — one line per page, up to 10, for the document's What's happening: /old-post returns 404, linked from /blog/a, /blog/b and 3 more, a referrer that is itself broken reads (itself broken), and a 404 reached through a redirect reads reached through a redirect from /en), linked from /en/hub/articles/… — because redirect hops are not referrers (site-audit/crawl.js inboundLinkSources, 2026-08-19): the crawler records a redirect as a link item of type: 'redirect' whose page_from is the redirecting URL, so the 404 used to read "linked from /en)" — a URL nobody can edit. The lookup now follows each hop one level back to the anchors pointing at it (linkedFrom = real pages) and reports the hop as redirectedFrom; audit.js stores both on the occurrence evidence. (The Luniq case: an article's anchor text swallowed a ) → /en) → 301 (locale redirect, Accept-Language-dependent) → /nl/en) 404.) detail is the catalog why alone now (the "For example: …" tail is gone; example_urls stays on the evidence). The signal page no longer fetches /issues/:checkId lazily.

Every health draft is a CONDITION card since 2026-08-24 (recurring: true on the draft, stamped into evidence.recurring by upsertSignal): a human close is final for the ROW but never for the CHECK. Health fingerprints are per-check (health:<checkId>), so before this, resolving or dismissing one card permanently suppressed the whole check class for the workspace — the detector's re-emission died against the final close (the Luniq case: seven checks burned in one board-cleanup session). Now, once the closed card's condition goes quiet past the absence grace (the sweep stamps evidence_gone_since on the closed row) and the check later fires again, the closed card's fingerprint is archived (#-suffixed with the row id) and the draft mints a fresh card through the normal insert path — the person's close, attribution and outcome record stay untouched, and a new incident gets a new work cycle. Mechanism lives in agent/board/lifecycle.js (refreshKindFor → 'recur') + agent/signals.js (partition stamps human-closed recurring rows; upsert archives + re-inserts); a card closed while the detector still emits stays closed until the quiet-then-return cycle (a close ahead of measurement catching up is the same incident). A fully muted check keeps presence, so its closed card stays parked while the mute stands.

Canonical mismatch is structural since 2026-08-09 (fusion.js): when the Inspection pair gsc_google_canonical / gsc_user_canonical exists, gsc-canonical-mismatch is decided by comparing the two URLs — it survives Google rewording its coverage prose, carries { declared, googleChose } as evidence, and an equal pair suppresses a stale "duplicate" prose verdict. The substring match remains only as fallback for rows not yet re-inspected. gsc-discovered-not-indexed / gsc-crawled-not-indexed findings on site pages now carry foundVia (Google's referring URLs) + inSitemaps — the "found how?" answer they could never give.