Market

Module id: market (frozen) · Mounted at: /api/modules/market · Empty settings row (the shell-module convention). Built 2026-08-22 from plans/market-module.md (the authority — five engines on one weekly sync). The page is pure data and visualization by design: the battle plan this module computes materializes ONLY as signal cards on the board.

Purpose

The workspace's competitive field, measured weekly and stored: who the rivals are (discovery from real keyword overlap), who wins which searches (ranked-keyword head-to-head + share of voice), what rivals are building (the site atlas — sitemap + ranked-URL evolution), where demand is going (12-mo volume trends carried inside the payloads), and what to do about it (the battle plan: priority = value × winnability, honestly diagnosed write / refresh / authority).

SSOT & seams

• Competitor NAMES live in the profile settings key competitors (domain strings — the same list ai-visibility matches against and the writer excludes). market_competitors holds only the machine layer (tier, locale, evidence, lifecycle); competitors.js is the ONE write path (skill + routes + sync bootstrap) and syncs the profile key after every mutation.
• AI-answer share is composed at read, never copied: ai-visibility's snapshots already roll up per-rival mentions; the tool/page point there. No cross-module join.
• Two fields, two questions (2026-08-23). Share of voice counts a rival's WHOLE presence in the country — a French agency really does own that attention in Belgium, and filtering it out would flatter us. The battle plan and the gap cards are bounded by the languages the workspace publishes (isOffLanguage), because that is the work this site can actually do. Same rows, two reads.
• Share of voice is VISIBILITY-weighted (visibilityAt: #1 ≈ 0.28 of a search's clicks, #11 ≈ 0.015, decaying to ~0 by #100), not a top-10 count. The binary rule scored every domain in Luniq's Belgium at exactly 0.0% — a number that could never move however good the work got. It is also null, never 0%, when fewer than two domains rank in the field: a share of a set containing only yourself is not a measurement. The field size rides in market_snapshots.rollups.field_keywords so the share is always read with its base.
• Shared payload math lives in core/services/keyword-field.js (extracted from lead-audit when market became its second consumer): pickComparableRivals (size-ratio cap on full_domain_metrics, ≥2 intersections), brand-token exclusion, rankedKeywordRow, volumeTrendOf, and answeredByPath (the already-answered guard, moved from gsc — gsc re-exports it as alreadyAnsweredBy).

The weekly sync (sync.js, in-flight dedupe)

A MARKET IS A COUNTRY (2026-08-23, measured): the vendor language parameter does NOT separate languages — hln.be (a Dutch newspaper) queried in Belgium/FRENCH returns Dutch keywords; lesoir.be queried in Belgium/DUTCH returns French ones. Selling "Belgium · Dutch" and "Belgium · English" as two markets billed twice for one measurement. So a market is a country, the query language is fixed per country (marketLanguageFor, the country catalog's first official language) and never shown, and KEYWORD language is decided by us (keywordLanguage in keyword-field.js: function words + the ranking URL's slug/locale segment, abstaining when unsure). Stored sets from the old (country, language) model normalize and dedupe on read.

THE SET IS BUILT FROM SETTINGS: markets.js owns which COUNTRIES a workspace monitors — stored as the module's own markets settings key (config, not measurement; competitors are a table because they carry evidence and lifecycle, markets carry neither). reconcileMarkets(ws) is the ONE path from settings to markets: it reads serviceAreas through core's resolveBusinessMarkets, keeps only the declared ones, and calls syncDeclaredMarkets. Both the weekly sync AND the /overview route call it, so cron and page can never disagree, and "Where you sell" (moved from Content settings to Settings → Workspace → Localization on 2026-08-23 — it is workspace identity, not a writing preference) takes effect on the next page load rather than the next Monday. A market row carries source: declared rows follow the setting and are DROPPED when the country is dropped (the page shows what the workspace is set up for); manual rows — added by hand, the country being scaled into — are never swept. First run: the overview route calls requestFirstSync when any monitored market has no snapshot; that fires runMarketSync in the background (single-flight, one trigger per workspace per 6h on top of the vendor daily cap) and returns syncing: true, which the page renders as "Measuring <market>…" and polls every 20s. Adding a market by hand does the same. Cap 5 (~$0.50/wk each). getMarketFrame(ws, { market }) resolves the read's market from the monitored set (a 2056-nl key); only monitored markets are addressable. STANCE is derived, never stored: established (own ranked keywords ≥ ENTRY_THRESHOLD 5) vs entering — a market graduates on its own. The probe-based single-market pick it replaced is gone; resolveBusinessMarkets still returns each published language's biggest market as an UNDECLARED candidate, for the add-market surface.

Per workspace × monitored market (syncMarket, atlas cached across markets — a rival tracked in three markets has its sitemap read once): (1) first-run bootstrap seeds the profile key's domains as WATCH (the Luniq soak seeded three zero-overlap agencies into paid slots — a business rival is not necessarily a search rival); (2) discovery — one labsCompetitors call → suggested rows with evidence (cap 10, stale suggestions pruned), then promoteProvenProfileRivals: profile-sourced watch rows with measured intersections ≥ PROMOTE_MIN_OVERLAP (2) move into tracked while slots are free, strongest first — never demotes, never touches manual rows or suggestions; (3) keywords — own domain + tracked rivals × (labsDomainRankOverview + labsRankedKeywords limit 500), branded keywords excluded field-wide (incl. compound brands via compact match), prev_* shifted on write, replaced wholesale except pinned tracked scoreboard rows; rows of domains that left the tracked set are deleted (snapshots stay as history); (4) authority — ONE backlinksBulkRanks call (new wrapper, /v3/backlinks/bulk_ranks/live); (5) atlas — readSitemap per tracked+watch rival ∪ their ranked URLs → market_pages (first_seen/last_seen/removed_at is the evolution record; removal stamped only after a complete read — the sitemap reader's contract); (6) snapshots — per-domain weekly row: counts, visibility-weighted SoV over the union field (null with <2 ranking domains), authority, atlas totals, demand index. Cost ≈ $0.50/wk per market, attributed in dataforseo_usage.

Whose demand, whose rival (identity curation, 2026-08-23)

A country's field is the COUNTRY's — vendor-correct and commercially useless on its own. Luniq (B2B web agency, publishes nl/en, sells in BE) had "imprimerie ciney" in its Belgian field and two French agencies in its paid tracked slots. Three gates, each at the cheapest place it can be decided:

• Keywords — identity.js + core/services/keyword-relevance.js. businessContext(profile) renders the workspace's own words (companyDescription, industry, products, idealClient, visibilityTargets, languages, serviceAreas; '' below two facts, which no-ops every gate). splitByRelevance judges the field's top 200 keywords by volume in 40-keyword Haiku batches and the verdict is stored ONCE on our own snapshot's rollups.off_topic, so every reader applies it for free: getKeywordTable annotates offTopic per row, filters unless scope: 'all', and always reports the count; classifyGaps skips them (via offTopicOf(snapshots, ownDomain)), so no card proposes work nobody wants. Measurement is untouched — share of voice is computed over the whole field, because that is what a share IS. The judge is shared with lead-audit (which builds its context from a prospect's homepage instead — lead-audit/services/relevance.js) and is best-effort: any failure keeps the mechanical rows.
• How rivals are FOUND at all — discovery.js (2026-08-23, replaced the rival-of-rival chain). "Who ranks for the same keywords as you?" answers nothing when you rank for one keyword, and the chained fallback ("who competes with THEM?") filled Luniq's Belgium with French agencies three hops from anything it sells. So the question changed: search the demand we want and read page one. seedKeywords picks up to 4 searches by ALTERNATING two sources — the profile's visibilityTargets (2–6 words, never our own brand; focusAreas retired 2026-08-28) and our keywords ranked in THIS market (right country, right language, past the relevance verdict) — with keywords from other markets filling any gap. Neither source can go first alone, and both failure modes were seen live: rankings-only searched "sentinelone vs crowdstrike" for Luniq (demo articles from its test-bed days) and found Gartner and Reddit, while Matubu — a coffee roaster with excellent declared targets — spent its whole budget on other companies' brand names ("jura e8", "kaffabar") because four ranked keywords filled it first. Declared-only would miss the local phrasing a site has proven it can rank for. discoverByDemand runs each seed through serpOverview (now returning organicDomains, parsed from the response we already pay for), drops platforms (PLATFORM_HOSTS: Reddit/YouTube/G2/Clutch… — never anyone's competitor) and foreign ccTLDs, and counts who keeps appearing. A domain earns a TRACKED slot when it wins 2+ of our searches or holds the top 5 of one (qualifiesAsRival), is measurable in this market, and is within sizeCapFor(ourCount) — the same ceiling the vendor-sourced rivals answer to. Everything else lands as a suggestion with serp_hits/serp_keywords evidence. Runs ONLY when no qualified rival holds a tracked slot: ≈4 SERP reads (~$0.03) per market per week, and nothing at all once the field is healthy. Live across four configured workspaces, three industries, four countries: Luniq's Belgium ended on digidreams.be + webit.be (Belgian B2B web agencies) and its UK on axongarside.com; Matubu (coffee roaster, BE) on javry.com + javacoffee.be + mokabon.be; HRTH (recruitment, BE) on hays.be + recru.be; Neue World (agency, US+GB) on coalitiontechnologies.com/redantler.com/graphite.io and how.studio/madebyshape.co.uk/makebuild.studio. Every suggestion list came back as same-industry companies.
• What the candidate IS — core/services/business-identity.js (2026-08-23). Ranking together is not competing. The first search sweep offered webflow.com, forbes.com, dribbble.com, optimizely.com and globy.com to a B2B web agency; all five rank for "website optimization" and none is a company a client would hire instead. readBusinessContext(domain) fetches the homepage and flattens it (title/H1/meta/body excerpt — the same reader lead-audit uses on prospects, moved to core), judgeRivalry(ourBusiness, theirs) classifies the relationship in one Haiku call: competitor | platform | publisher | directory | marketplace | other | unknown. A PLATFORM IS NOT A RIVAL — it is what our buyer might use instead of hiring anyone (Webflow, Optimizely, Siteimprove), and a card telling an agency to out-rank Webflow is noise. The verdict is about the COMPANY, so it is stored on the row (rival_kind, rival_what, rival_checked), read back across every market by knownVerdicts, and each domain is judged exactly once. It gates three things: only a verified competitor is auto-promoted to tracked, only companies enter the suggestion list, and a tracked domain verified as a non-rival is REMOVED from the set (never demoted to watch — watch still means "a rival we don't share searches with"). unknown (unreachable homepage) is honest and inert: it stays, it is never promoted, and the language/relevance rule remains its backstop. manual rows are annotated, never touched. Live: optimizely.com (platform) and globy.com (B2B marketplace) left Luniq's Belgian field; digidreams.be and webit.be — actual Belgian B2B web agencies — took the slots; every profile-named rival came back competitor with a one-line description.
• Rivals, before they cost anything — tldLocationCode (core/dataforseo). A candidate whose ccTLD names another country is not a local rival (hubspot.fr, digeek.fr as "Belgian competitors"); generic TLDs (.com/.io/.agency…) claim nothing and stay eligible. Applied to both discovery paths in sync.js via localToMarket.
• Rivals, once measured — the demotion loop in syncMarket. After the field is built: of everything a tracked rival ranks for here, how much is in our published languages AND not off-topic? Zero over a whole measured field → back to watch (evidence relevant_keywords, field_keywords), and it stops seeding entry discovery. Never applied to manual rows (a person's statement outranks the measurement) and never on an empty read (no data is not a verdict). Live on Luniq: netween.be and bluepixel.be demoted on the first run, share of voice 0% → 4.7%.

recordDiscovery is now called ONCE per market with both paths' candidates — the second call used to prune the first's suggestions.

Tiers

tracked (cap 5/market — the measured field) · watch (cap 15 — site atlas only) · suggested (discovery evidence, no monitoring). A human promotes, always; caps refuse with a reason.

Routes

Every GET takes ?market=<locationCode>-<langCode> (default: the workspace's primary).

Method	Path	Purpose
GET	/:workspaceId/overview	Scorecard: standings + deltas, SoV trend, competitor set, the monitored markets with stance
GET	/:workspaceId/keywords	Head-to-head table (state/competitor/min_volume filters, enveloped)
GET	/:workspaceId/moves	Move log + atlas sections + per-domain provenance
GET/POST	/:workspaceId/markets	The monitored set / { action: add|remove|primary, market: '2056-nl' } (409 on cap)
POST	/:workspaceId/competitors	Curation: { action: track|watch|remove, domain } (409 on cap)

Crons

• Weekly sync — 0 7 * * 1 (Mondays 07:00, jittered — after backlinks at 06:00 so the authority read is fresh), schedule lane (it spends), gated at the scheduler chokepoint.
• PAUSED since 2026-08-23 (Leon). scheduler.js registers no job unless MARKET_SYNC_ENABLED=true: the page is behind a "Coming soon" flag in production (frontend/src/lib/features.ts), and a fleet-wide weekly sync spends ~$0.50–1.50 per workspace per market on a surface nobody can open. Paused by DEFAULT, not by an env var somewhere, so no deploy resumes it by forgetting a setting. The DELIBERATE paths still work and are how testing continues: the operator Run-now trigger, and the /overview route's first read for a market that has never been measured (requestFirstSync, one per workspace per 6h). test-module-crons.mjs reports market×0 while it is paused.
• Operator trigger marketSync ("Market sync (competitive field refresh)").

Agent tools & skills

• Tool get_competitor_landscape — one stored-data read, six views: scorecard (standings + set + SoV trend), gaps (THE battle plan, scored + diagnosed), movement (won/overtaken crossings), keywords (the raw table), content (the move log with provenance — atlasComplete=false suppresses removal claims), atlas (section structure/velocity). Zero API cost; every list enveloped {rows, total, truncated}.
• Skill edit_competitors (chatOnly, incremental) — track/watch/remove; the one write path, profile key kept in step.

Entering a market

A market where we rank for nothing is the one that needs the picture most, and three fixes make it possible — each found by running it against Luniq (2026-08-23):

• Cold-start discovery. labsCompetitors asks "who is similar to US?", which finds nothing where we rank for nothing. The sync therefore asks the same question about a rival that DOES rank here (we-r.co in BE·NL → six real Belgian agencies) and records the answer as suggested — weak evidence never takes a slot. Two guards: a small agency's keyword set is mostly its OWN BRAND, so the chain returns name-alikes (groovedigital.agency → groove.cm, groovelit.com, groovetechnology.com), killed by a 5-character shared-root test; and seeds are only rivals with measured presence here, strongest first, max 2.
• Three-way qualification for gaps (rivalQualified): measured overlap with us, measured presence in this market (PRESENT_MIN_KEYWORDS 20), or a person's explicit track. The overlap floor alone — right for the "v0 logo" case — silenced every entry market, where our overlap with anyone is zero by definition.
• A gap is OUR absence, not their strength (keywordState): requiring a top-10 rival read a rival at #96 as "contested" while we ranked nowhere and contested nothing. Their position feeds battleScore's winnability instead, which is where it belongs — a market whose best rival sits deep is the cheapest one to enter. The card copy follows: a deep-rival gap leads with the demand, never with "ranks #96".

Cards: market:entry-beachhead (the cheapest winnable cluster, ONE card per market — entering a market is one decision) and market:entry-transfer (a page already ranking in a same-language market, against the same search here; exact keyword match only — cross-language equivalence would be a guess).

Signals (agent-signals.js — pure classifiers in classify.js, asserted in test-autopilot-safety.mjs)

Five detectors over stored rows, the keyword-side ones PER MONITORED MARKET with market-scoped fingerprints ( — the same keyword in two markets is two fights); the atlas detectors run once per workspace over domain-keyed fingerprints (a rival's site is one site), zero marginal cost: market_movement (won/overtaken crossings ≥50/mo with refresh_article how when we own the page; SoV shift ≥3pts; demand shift ≥15%), market_gaps (gap topics ≥100/mo through the banking gates — the rival must have measured overlap ≥ GAP_MIN_RIVAL_OVERLAP with us (their demand is not ours otherwise — the "v0 logo" lesson), brand filter, already-answered catalog guard, one-gap-per-rival-page dedupe with pre-collapse presence, primary-collection stamp, write_article how, evidence.recommends_content so the sweep's territory gate judges it; authority-limited gaps cluster into ONE evidence card, never a write), market_atlas (section launches ≥5 pages/28d, counter-moves on own brand tokens with a counter-page how, slot-opens against genuinely-removed rival pages, weekly new-content roundups), market_rivals (suggested rivals ≥30 intersections — human promotes). Fingerprint families all carry family plans (board/plans.js); market joined LOCALIZED_SOURCES (cards quote the market's own keywords). Crossing/slot cards measure on query_series (the proven outcome rails; a dedicated keyword_position measurer over market_keywords history is the planned upgrade per OUTCOME_LOOP_PLAN.md).

Data

Writes market_competitors, market_snapshots, market_keywords, market_pages (all RLS-on/no-policies, service-role only — migration market_module_v1); reads workspaces, the profile settings row, spine site_pages + blog_articles (catalog guard), collections (stamp). Rival URLs live ONLY in market_pages — never in the workspace's own site tables (the corpus split's URL uniqueness).

Key files

index.js (manifest), sync.js (the one data write path), competitors.js (the one curation write path), markets.js (the monitored set + reconcileMarkets), identity.js (the workspace's business, in the judge's words), discovery.js (finding rivals by searching our own demand + the identity check), classify.js (every threshold, pure), queries.js, routes.js, scheduler.js, agent-tools.js, agent-skills.js, agent-signals.js. Frontend: frontend/src/pages/orbit/market/ (Market/Scorecard/Keywords/Moves — zero action surfaces beyond curation).