kazuma/market_screener
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
93aac355ccf768626b74b8e4a815174ac39ea74b
market_screener/CLAUDE.md
T
Sai Kiran Vella 93aac355cc phase-7_alpha: legacy code cleanup
2026-06-06 22:55:43 -04:00

39 KiB
Raw Blame History

CLAUDE.md

Guidance for working in this repository.

Overview

market-screener is a Node.js project consisting of a Fastify API server that powers the SvelteKit dashboard in the ui/ subdirectory.

Every asset is scored under two lenses:

  • Market-Adjusted — gates derived from live Yahoo benchmarks (SPY P/E, XLK P/E, XLRE yield, LQD spread). Reflects what is acceptable in today's market.
  • Fundamental — strict Graham/value-investing gates from ScoringConfig. Reflects genuine value regardless of market conditions.

The comparison produces a Signal (Strong Buy / Momentum / Speculation / Neutral / Avoid).

ES module project ("type": "module"); use import/export, not require.

Commands

npm install                                   # install dependencies
npm run dev                                   # start API server (port 3000) + SvelteKit UI (port 5173) together
npm run server                                # API server only (port 3000)
npm test                                      # run all unit tests (node:test, zero external deps)
npm run test:watch                            # watch mode — uses verbose spec reporter
npm run format                                # format all server/bin/tests with Prettier
npm run format:check                          # check formatting without writing (used in CI/pre-commit)
npm run ui:install                            # install UI dependencies (ui/ subdirectory)

npm run dev runs both the API server and the SvelteKit UI (in ui/) concurrently. Run npm run ui:install once before first use.

Project Structure

bin/
  server.ts              ← Fastify API server entry point (imports buildApp from server/app.ts)

prompts/
  catalyst-analysis.md   ← daily catalyst analysis playbook (LLM prompt + workflow)

server/
  app.ts                 ← Fastify app factory (buildApp). Registers CORS + all controllers.
                           NOTE: lives at server/app.ts, NOT inside server/controllers/.

  controllers/           ← HTTP only: parse request, call service, return response
    screener.controller.ts ← POST /api/screen, GET /api/screen/catalysts
    finance.controller.ts  ← GET /api/finance/portfolio, POST|DELETE /api/finance/holdings
    calls.controller.ts    ← CRUD for market calls + GET /api/calls/calendar
    analyze.controller.ts  ← POST /api/analyze (LLM analysis for a ticker set)

  services/              ← business logic, no HTTP or I/O concerns
    ScreenerEngine.ts    ← orchestrates: fetch → score × 2. Method: screenTickers() → ScreenerResult.
                           Accepts injected YahooFinanceClient + BenchmarkProvider + { logger } option.
    DataMapper.ts        ← normalises Yahoo payload → flat asset data object.
                           Computes: DCF intrinsic value, analyst upside, 52W movement fields,
                           grossMargin, marketCap. Uses trailingPE. Preserves negative FCF.
    RuleMerger.ts        ← merges base rules + sector overrides + MarketRegime (INFLATED mode)
    BenchmarkProvider.ts ← fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD → marketContext.
                           In-memory cache: 1 hr TTL. Resets on server restart.
    MarketRegime.ts      ← derives INFLATED gate overrides from live benchmarks + rate regime
    CatalystAnalyst.ts   ← fetches Yahoo Finance news, extracts relatedTickers. Accepts { logger }.
    LLMAnalyst.ts        ← uses AnthropicClient to analyze headlines → summary, sentiment,
                           affectedIndustries, relatedTickers. Returns null if API key not set.
    PersonalFinanceAnalyzer.ts ← net worth, cash vs investments, spending by category
    PortfolioAdvisor.ts  ← cross-references holdings with screener signals → hold/sell/add advice
    index.ts             ← barrel re-export (import services from here, not individual files)

  repositories/          ← data persistence only (JSON file read/write)
    MarketCallRepository.ts ← persists market thesis entries to market-calls.json.
                              CRUD: list/get/create/delete.
    PortfolioRepository.ts  ← read/write portfolio.json. Methods: read, upsert, remove.

  clients/               ← external API connectors, one class per third-party system
    YahooFinanceClient.ts ← wraps yahoo-finance2 v3, retry + backoff. Methods: fetchSummary,
                            fetchCalendarEvents, search. Typed via YahooFinanceLib interface.
    SimpleFINClient.ts   ← claims setup token → access URL, fetches /accounts via Basic Auth.
    AnthropicClient.ts   ← wraps Anthropic SDK. complete(system, user) → raw text response.

  models/                ← domain entity classes with metrics + display logic
    Asset.ts             ← abstract base: ticker, currentPrice, type, formatting helpers
    Stock.ts             ← metrics + _mapToStandardSector (8 sectors) + _classifyMarketCap
                           (Mega/Large/Mid/Small/Micro) + _classifyGrowth (style classification).
                           Holds: valuation, quality, risk, 52W movement, analyst consensus, DCF.
    Etf.ts               ← metrics: expenseRatio, yield, volume, fiveYearReturn, totalAssets
    Bond.ts              ← metrics: ytm, duration, creditRating, creditRatingNumeric

  scorers/               ← stateless pure scoring functions, no I/O
    StockScorer.ts       ← gate checks + weighted registry:
                           core: ROE, opMargin, margin, peg, revenue, fcf
                           expert: analyst consensus (inverted Yahoo 1–5 scale), DCF margin of safety
                           riskFlags: beta, 52W position, 52W momentum, analyst divergence, DCF divergence
    EtfScorer.ts         ← expense gate + registry (cost, yield, volume, fiveYearReturn)
    BondScorer.ts        ← credit gate + spread/duration scoring

  config/
    ScoringConfig.ts     ← CREDIT_RATING_SCALE + ScoringRules (single source of truth for all
                           gates, weights, thresholds including analyst and dcf weights)
    constants.ts         ← SIGNAL, ASSET_TYPE, SECTOR, SCORE_MODE, REGIME, SIGNAL_ORDER,
                           CAP_CATEGORY (Mega/Large/Mid/Small/Micro),
                           GROWTH_CATEGORY (High Growth/Growth/Stable/Value/Turnaround/Declining)

  types/                 ← all domain types, one file per domain
    asset.model.ts       ← Signal, AssetType, ScoreMode, ScoringRules, ScoreResult, AssetResult,
                           ScreenerResult, GateSet, WeightSet, ThresholdSet, RuleBlock, StockRules
    market.model.ts      ← RateRegime, VolatilityRegime, Benchmarks, MarketContext
    portfolio.model.ts   ← HoldingType, PortfolioHolding, PortfolioAdvice, AdviceRow
    calls.model.ts       ← TickerSnapshot, MarketCall, SnapshotEntry, CreateCallInput
    finance.model.ts     ← LLMAnalysis, CatalystStory, CalendarEvent, SimpleFINAccount,
                           SimpleFINTransaction, SimpleFINData, YahooNewsItem, YahooFinanceLib
    logger.model.ts      ← Logger
    models.model.ts      ← AssetData, StockData, StockMetrics, EtfData, EtfMetrics, BondData, BondMetrics
    repositories.model.ts ← StoreData, PortfolioData
    scorers.model.ts      ← NumVal, SanitizedMetrics, SanitizedBondMetrics
    services.model.ts     ← BenchmarkProviderOptions, InflatedOverrides, ErrorResult, RuleSet,
                            ScreenerEngineOptions, CatalystResult, MappedData, and other service shapes
    schemas.ts            ← Fastify JSON Schema objects for request body validation
    index.ts              ← re-exports all of the above (use this barrel for imports)
  types.ts               ← thin barrel: export type * from './types/index.js'

  utils/
    Chunker.ts           ← splits ticker list into batches
    logger.ts            ← noopLogger constant for silencing output in server context

ui/                      ← SvelteKit dashboard (lives inside this repo, not a separate repo)
  src/
    styles/              ← global SCSS design-token system
      app.scss           ← root file — @use all partials
      _tokens.scss       ← CSS custom properties generated from SCSS maps ($bg, $text, $blues, $signals…)
      _reset.scss        ← box-sizing reset + body base
      _layout.scss       ← shell, nav, .links, main, nav-progress, .loading-area
      _section.scss      ← .section, .section-header, .count, .mode-tabs, .error-banner
      _table.scss        ← table, thead/tbody, .table-wrap, .col-ticker, .ticker, .num, .tag
      _buttons.scss      ← button base, .btn-primary, .btn-catalyst, .btn-ghost, .btn-screen, .btn-analyze
      _badges.scss       ← .verdict-pill, .sentiment-pill, .text-* helpers (SCSS @each maps)
    lib/
      api.ts             ← backward-compat shim; re-exports from api/index.ts
      api/
        screener.ts      ← screenTickers, fetchCatalysts, analyzeTickers
        finance.ts       ← fetchPortfolio, addHolding, removeHolding, fetchMarketContext
        calls.ts         ← fetchCalls, fetchCall, createCall, deleteCall, fetchCallsCalendar
        index.ts         ← barrel re-export
      types.ts           ← re-exports shared types from $types (server/types/); UI-only types defined here
      utils.ts           ← shared pure functions: sigOrd, sorted, verdictShort, vClass, fmtPE, fmt…
      MarketContext.svelte      ← collapsible card-grid context (used in portfolio + safe-buys)
      MarketContextStrip.svelte ← horizontal chip strip (used in screener)
      AssetTable.svelte         ← STOCK/ETF/BOND section: mode tabs + Analyze + table
      AnalysisSidebar.svelte    ← LLM analysis slide-over panel
      VerdictPill.svelte        ← verdict-pill span; props: label
      SignalBadge.svelte        ← signal emoji + label badge
      Spinner.svelte            ← sm: dot-pulse | md/lg: chart-line animation
    routes/
      +page.ts           ← SvelteKit load (ssr:false) — fetches catalysts + screens on mount
      +page.svelte       ← main screener UI
      +layout.svelte     ← shell, nav, nav-progress bar, nav-overlay with Spinner
      calls/             ← market calls list + detail views
      portfolio/         ← portfolio advice view
      safe-buys/         ← filtered strong-buy view

market-calls.json        ← persisted market thesis calls (written by MarketCallRepository)
portfolio.json           ← user's holdings: ticker, shares, costBasis, source, type
.env                     ← SIMPLEFIN_ACCESS_URL or SIMPLEFIN_SETUP_TOKEN, ANTHROPIC_API_KEY

Data Flow

Yahoo Finance API
  ↓
BenchmarkProvider      — fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD
                         builds marketContext { sp500Price, riskFreeRate, vixLevel,
                         rateRegime, volatilityRegime, benchmarks { marketPE, techPE, reitYield, igSpread } }
  ↓
DataMapper             — normalises raw Yahoo payload → flat data object with type (STOCK/ETF/BOND)
                         uses trailingPE as primary; preserves negative FCF yield; infers bond duration
                         computes: DCF intrinsic value, analyst upside, 52W movement, grossMargin, marketCap
  ↓
Asset subclass         — Stock / Etf / Bond holds metrics + getDisplayMetrics()
                         Stock also derives capCategory (Mega/Large/Mid/Small/Micro)
                         and growthCategory (High Growth/Growth/Stable/Value/Turnaround/Declining)
  ↓
RuleMerger × 2        — FUNDAMENTAL mode: ScoringConfig as-is (Graham-style)
                         INFLATED mode:   sector override + MarketRegime live gate overrides
  ↓
Scorer × 2            — StockScorer / EtfScorer / BondScorer, fully stateless
  ↓
ScreenerEngine         — derives Signal from comparing both verdicts
  ↓
  └── API path:   screenTickers() → JSON (with serialized displayMetrics) → SvelteKit UI

API Routes (Fastify)

Method Path Description
GET /health Health check
POST /api/screen Screen tickers. Body: { tickers: string[] } (max 50). Returns { STOCK, ETF, BOND, ERROR, marketContext } with asset.displayMetrics pre-serialized
GET /api/screen/catalysts Yahoo news → { tickers, stories }
GET /api/finance/portfolio Portfolio advice + optional SimpleFIN data
GET /api/finance/market-context Live benchmark data only
GET /api/calls List all market calls (newest first)
GET /api/calls/:id Get one call + re-screened current prices for comparison
POST /api/analyze Fetch Yahoo news for specific tickers + run LLM analysis. Body: { tickers: string[] }. Returns { analysis }
POST /api/calls Create a market call; snapshots current prices. Body: { title, quarter, thesis, tickers[], date? }
DELETE /api/calls/:id Delete a market call
GET /api/calls/calendar Earnings + dividend calendar. Query: ?tickers=AAPL,MSFT (omit for all call tickers)

CORS is configured for CLIENT_ORIGIN env var (default http://localhost:5173).

Request body validation uses Fastify JSON Schema, defined in server/types/schemas.ts.

Scoring Modes

Mode P/E Gate (general) P/E Gate (tech) Source
FUNDAMENTAL 15x 35x ScoringConfig (true Graham)
INFLATED S&P 500 P/E × 1.5 XLK P/E × 1.3 Live SPY/XLK data

Rate regime effect on INFLATED mode:

  • HIGH rate regime: P/E multiplier compresses to 1.2× (vs 1.5× in NORMAL)
  • HIGH rate regime: REIT yield floor tightens (0.95× vs 0.85×)
  • HIGH rate regime: bond spread demand increases (0.90× vs 0.80×)
Signal Meaning
✅ Strong Buy Passes both fundamental AND inflated gates
⚡ Momentum Passes inflated, holds fundamentally
⚠️ Speculation Passes inflated, fails fundamental
🔄 Neutral Hold territory in one or both lenses
❌ Avoid Fails both

ScoringConfig Key Values

server/config/ScoringConfig.ts — single source of truth for all gates, weights, thresholds.

STOCK base gates (Fundamental mode):

  • maxPERatio: 15 — Graham's actual rule (trailing P/E)
  • maxPegGate: 1.0 — Lynch standard: PEG > 1.0 means paying full price
  • maxDebtToEquity: 1.5 — most distress starts above 2x
  • minQuickRatio: 0.8 — below this signals real liquidity stress

STOCK base weights:

  • roe: 3, fcf: 3 — primary quality signals
  • opMargin: 2, margin: 2, peg: 2, revenue: 2 — secondary factors
  • analyst: 2 — Wall Street consensus (Yahoo 1–5 scale inverted; requires ≥3 analysts)
  • dcf: 2 — DCF margin of safety (positive = undervalued; only fires when FCF > 0)

Sector overrides (structural — apply in both modes):

Sector Key difference
TECHNOLOGY D/E up to 2.0, P/E up to 35x, FCF weight raised
REIT P/E and PEG disabled (9999), scored on yield + P/FFO
FINANCIAL D/E disabled, scored on ROE + P/B, maxPriceToBook 1.5x
ENERGY FCF weight 4, yield weight 3, opMargin primary
HEALTHCARE Revenue growth primary, P/E up to 25x
COMMUNICATION FCF weight 4, P/E up to 25x (META, GOOGL, NFLX)
CONSUMER_STAPLES Margin/ROE focus, low revenue growth expectations
CONSUMER_DISCRETIONARY Revenue growth primary, P/E up to 25x

ETF gates:

  • maxExpenseRatio: 0.2% — hard gate
  • minFiveYearReturn: 8.0% — S&P long-run floor
  • minVolume: 1,000,000 ADV

BOND gates:

  • minCreditRating: 7 (BBB = investment-grade floor)
  • minSpread: 1.5% above risk-free
  • maxDuration: 7 years

MarketRegime (INFLATED overrides)

server/services/MarketRegime.ts derives gate overrides from live benchmarks and current rate regime:

Gate Formula (NORMAL rates) Formula (HIGH rates)
Stock maxPERatio SPY trailing P/E × 1.5 SPY trailing P/E × 1.2
Tech maxPERatio XLK P/E × 1.3 XLK P/E × 1.3
Tech maxPegGate XLK P/E ÷ 15 XLK P/E ÷ 15
REIT minYield XLRE yield × 0.85 XLRE yield × 0.95
Bond minSpread LQD−TNX × 0.80 LQD−TNX × 0.90
ETF maxExpenseRatio 0.75% 0.75%

Rate regime thresholds: < 2% = LOW, 2–5% = NORMAL, > 5% = HIGH (10Y Treasury yield). Known issue: sharp break at 5% can flip the scoring regime between two back-to-back requests when the 10Y hovers near the threshold. Consider smoothing or making the threshold configurable via env var.

Expert Scoring Features (Stock)

Added to DataMapper.ts (extraction) and StockScorer.ts (scoring). All data comes from Yahoo's existing modules — no extra API calls required.

DCF Intrinsic Value

Two-stage discounted cash flow model computed per stock with positive TTM FCF:

  • Stage 1: FCF per share grows at the earnings/revenue growth rate for 5 years, discounted at 9.5% (4% risk-free + 5.5% equity risk premium)
  • Stage 2: Terminal value at 2.5% perpetuity growth (Gordon Growth Model)
  • Growth rate capped at 30%, floored at -5%
  • Returns dcfIntrinsicValue ($ per share) and dcfMarginOfSafety (% undervaluation)

Scoring: ≥20% margin of safety → +dcf weight; 0–20% → +1; -20% to 0 → -1; < -20% → -dcf weight. Only fires when FCF > 0. Risk flags trigger at ±30% divergence.

Analyst Consensus

From financialData.recommendationMean (Yahoo scale: 1.0 = Strong Buy, 5.0 = Strong Sell):

  • Requires ≥3 analysts to avoid noise from thin coverage
  • ≤2.0 → full weight; ≤3.0 → +1; ≤4.0 → -1; >4.0 → -full weight
  • analystTargetPrice, analystUpside (% to target), numberOfAnalysts surfaced in display
  • Risk flags trigger at ≥25% upside or ≤-15% downside vs analyst target

52-Week Movement

Three fields replace the single 52W Pos position metric:

Field Meaning
52W Chg Total % price return over last 52 weeks (ks['52WeekChange'])
From High % current price is below 52-week high (negative = below peak)
From Low % current price is above 52-week low (positive = recovered)

Risk flags: strong uptrend (≥+50%), significant drawdown (≤-30%), >20% off 52W high.

Market Cap Segmentation

Stock._classifyMarketCap() derives capCategory from price.marketCap:

Tier Threshold
Mega Cap > $200B
Large Cap $10B – $200B
Mid Cap $2B – $10B
Small Cap $300M – $2B
Micro Cap < $300M

Growth / Style Classification

Stock._classifyGrowth() derives growthCategory from revenue growth, earnings growth, and dividend yield:

Category Condition
High Growth revenueGrowth ≥ 15% OR earningsGrowth ≥ 20%
Growth revenueGrowth 5–15%
Value revenueGrowth < 5% AND dividendYield ≥ 3%
Stable Low growth, modest or no dividend
Turnaround earningsGrowth < 0% AND revenueGrowth ≥ 0%
Declining revenueGrowth < -5%

Both Cap Tier and Style appear in getDisplayMetrics() and the screener table.

Sector Detection

Stock._mapToStandardSector() maps Yahoo Finance sector/industry strings to internal constants. Order matters — more specific matches first:

TECHNOLOGY   → "technology", "electronic", "semiconductor", "software"
REIT         → "real estate", "reit"
FINANCIAL    → "financial", "bank", "insurance", "asset management"
ENERGY       → "energy", "oil", "gas", "petroleum"
HEALTHCARE   → "health", "biotech", "pharmaceutical", "medical"
COMMUNICATION→ "communication", "media", "entertainment", "telecom"
CONSUMER_STAPLES → "consumer defensive", "consumer staples", "household", "beverage", "food"
CONSUMER_DISCRETIONARY → "consumer cyclical", "consumer discretionary", "retail", "apparel", "auto"
GENERAL      → fallback

DataMapper Notes

  • peRatio: prefers trailingPE (audited) over forwardPE (analyst estimate, ~10-15% optimistic)
  • FCF yield: freeCashflow !== 0 (not > 0) — negative FCF preserved so cash-burning companies fail the gate, not silently skip it
  • grossMargin: financialData.grossMargins * 100 — exposed in display; not yet a scoring factor
  • marketCap: price.marketCap — used for cap tier classification
  • analystRating: financialData.recommendationMean (1=Strong Buy, 5=Strong Sell). targetMeanPrice and numberOfAnalystOpinions also extracted
  • 52W movement: defaultKeyStatistics['52WeekChange'] for annual return; From High/From Low computed from fiftyTwoWeekHigh/fiftyTwoWeekLow
  • DCF growth rate: uses earningsGrowth (TTM decimal) first, falls back to revenueGrowth * 0.7. Capped at 30%, floored at -5%
  • Bond duration: inferred from category string ("Short-Term" → 2y, "Intermediate" → 5y, "Long" → 18y, default 6y). Yahoo does not expose effective duration in the modules we fetch
  • D/E ratio: Yahoo returns financialData.debtToEquity as a percentage (e.g. 175.56 for 1.7556×); dividing by 100 normalises to the standard ratio
  • Quick ratio: falls back to currentRatio when missing — known methodological issue; current ratio includes inventory and can overstate liquidity for retailers/manufacturers

Missing Data Convention

  • Missing metrics use null (not 0) in _sanitize. Gate checks skip null rather than auto-failing.
  • pegRatio falls back to trailingPE / earningsGrowth when Yahoo doesn't provide it.
  • DCF, analyst, and 52W scoring factors are all optional — they activate only when the underlying data is non-null.

Logger Injection Pattern

Classes that produce output accept an optional { logger } constructor option so they work cleanly in server context. Pass noopLogger from server/utils/logger.ts to silence all output.

Affected: ScreenerEngine, BenchmarkProvider, CatalystAnalyst, SimpleFINClient, LLMAnalyst.

// CLI (default) — writes to stdout
new ScreenerEngine()

// Server — fully silent
new ScreenerEngine({ logger: noopLogger })

SimpleFIN Auth Flow

  1. User gets a Setup Token from https://beta-bridge.simplefin.org
  2. SimpleFINClient.init() base64-decodes it → POSTs once to claim Access URL
  3. onAccessUrlClaimed callback is called with the URL — CLI uses saveAccessUrlToEnv(), server stores elsewhere
  4. All subsequent requests use Access URL with Authorization: Basic header (not embedded in URL)

portfolio.json Format

{
  "holdings": [
    { "ticker": "AAPL",    "shares": 10,   "costBasis": 150.00, "source": "Robinhood", "type": "stock"  },
    { "ticker": "VOO",     "shares": 8,    "costBasis": 380.00, "source": "Vanguard",  "type": "etf"    },
    { "ticker": "BTC-USD", "shares": 0.25, "costBasis": 45000,  "source": "Coinbase",  "type": "crypto" }
  ]
}

type values: stock, etf, crypto. Crypto is priced via Yahoo (BTC-USD style) but not fundamentally scored.

Tests

Uses Node's built-in test runner (node:test + node:assert/strict) — no test framework needed.

tests/
  ScoringConfig.test.js    ← gate values (P/E 15x, PEG 1.0, QuickRatio 0.8), sector overrides
  RuleMerger.test.js       ← FUNDAMENTAL vs INFLATED modes, sector merging
  MarketRegime.test.js     ← inflated overrides including HIGH/NORMAL rate regime variants
  StockScorer.test.js      ← gate failures, scoring labels, risk flags
  EtfScorer.test.js        ← expense gate, volume penalty, 5Y return scoring
  BondScorer.test.js       ← credit gate, spread/duration scoring, unit handling
  DataMapper.test.js       ← type detection, PEG computation, trailing PE preference,
                             negative FCF, ETF volume, bond duration inference
  PortfolioAdvisor.test.js ← position gain/loss calc, advice signal mapping, BRK.B normalisation
  LLMAnalyst.test.js       ← markdown fence stripping, JSON parse correctness

Pre-commit hook runs lint-staged (Prettier) then npm test. Pre-push hook runs npm test. Test output uses the built-in spec reporter.

Key unit: ytm in Bond.metrics is stored as a percentage (e.g. 6.5 = 6.5%). BondScorer._sanitize divides by 100 before spread calculation.

Coverage gaps (known):

  • MarketCallRepository.ts — no tests; CRUD against market-calls.json is untested
  • LLMAnalyst.test.js — tests a local copy of the fence-stripping regex rather than importing from source; will silently drift if the regex changes
  • API controllers (server/controllers/) — no integration tests; covered implicitly by manual testing only
  • Expert scoring features (analyst, DCF, 52W) — not yet covered in StockScorer.test.js
  • UI components — not tested at the unit level

Architecture Guide

This section is the single reference for where code lives and how to add features. Read this before touching any file.

Server layer map

Folder Role Rule
server/app.ts Fastify bootstrap — wires DI, registers controllers No business logic
server/controllers/ HTTP only — parse request, call service, return response No business logic, no file I/O, no external API calls
server/services/ Business logic and orchestration No HTTP concerns (req/reply), no direct file I/O
server/repositories/ Data persistence — JSON file read/write No business logic; one class per data file
server/clients/ External API connectors — one class per third-party system No business logic; only I/O and protocol handling
server/models/ Domain entity classes — hold metrics and getDisplayMetrics() No I/O; pure data + formatting
server/scorers/ Stateless pure scoring functions No I/O, no state; score(metrics, rules, marketContext) only
server/config/ Constants and scoring gates/weights No logic; change numbers here, not in scorers
server/types/ TypeScript interfaces and types No logic; one *.model.ts per domain
server/utils/ Shared pure utilities No domain knowledge
bin/ CLI entry points Call into services only; the only place process.exit() is allowed

UI layer map

Path Role
ui/src/routes/*/+page.ts SvelteKit load functions — fetch data for the page
ui/src/routes/*/+page.svelte Route component — composition only; minimal logic
ui/src/lib/api/screener.ts Fetch wrappers for /api/screen*
ui/src/lib/api/finance.ts Fetch wrappers for /api/finance/*
ui/src/lib/api/calls.ts Fetch wrappers for /api/calls/*
ui/src/lib/stores/ Reactive state shared across components (Svelte 5 runes)
ui/src/lib/components/ Generic shared components (VerdictPill, SignalBadge, Spinner)
ui/src/lib/portfolio/ Portfolio-specific components
ui/src/lib/calls/ Calls-specific components
ui/src/lib/types.ts Re-exports server types via $types alias; UI-only types defined here
ui/src/lib/utils.ts Pure formatting and sorting helpers
ui/src/styles/ SCSS partials — global design tokens, layout, tables, forms

Where to put a new type

  • Shared domain type (used by server + UI): server/types/<domain>.model.ts and re-export from server/types/index.ts
  • UI-only type (component state, display shape): ui/src/lib/types.ts
  • Private implementation detail used only within one file: inline in that file

Where to put new code — decision table

What you're adding Where it goes
New API endpoint server/controllers/<domain>.controller.ts + register in server/app.ts
Business logic for that endpoint New method in server/services/<Domain>.ts
Call to a new external API New class in server/clients/<Service>Client.ts
New data stored in a JSON file New class in server/repositories/<Domain>Repository.ts
New scoring rule or gate value server/config/ScoringConfig.ts
New market regime override server/services/MarketRegime.ts → getInflatedOverrides()
New stock metric (mapped from Yahoo) server/services/DataMapper.ts → mapStockData() + StockData/StockMetrics interfaces in server/types/models.model.ts + server/models/Stock.ts constructor + getDisplayMetrics()
New scoring factor server/config/ScoringConfig.ts (add weight + threshold) + server/scorers/StockScorer.ts (add factor to array)
New UI page ui/src/routes/<name>/+page.ts + +page.svelte
New UI fetch call ui/src/lib/api/<domain>.ts + re-export from api/index.ts
Reactive state shared by >1 component ui/src/lib/stores/<domain>.store.ts
New shared UI component ui/src/lib/components/ (generic) or domain subfolder
New global style ui/src/styles/_<domain>.scss + @use in app.scss

Conventions

  • Asset type (uppercased: STOCK / ETF / BOND) is the routing key across DataMapper, model classes, SCORERS map, and ScoringRules. Keep it consistent everywhere.
  • Prefer adjusting ScoringConfig or MarketRegime over hardcoding numbers in scorers.
  • BenchmarkProvider caches for 1 hour in memory — cache is lost on server restart. A persistent cache is planned (see Phase 8).
  • All entry points live in bin/. Do not add logic there — they call into services/ and controllers.
  • Never call process.exit() inside server/ — only bin/ may do that.
  • Class instances don't survive JSON.stringify. Call getDisplayMetrics() server-side before returning from API routes (see serializeAssets() in screener.controller.ts).
  • Controllers use constructor injection — dependencies are wired in server/app.ts, not created inside handlers.
  • The $types alias in the UI resolves to server/types/ — use it instead of duplicating type definitions.
  • Ticker normalisation (BRK.B → BRK-B) currently only happens in FinanceController.normalizeYahoo(). Submitting BRK.B directly to /api/screen will fail. Fix target: move normalisation into YahooFinanceClient.fetchSummary().

Adding a new scoring metric — step-by-step

When adding a new data point that flows from Yahoo → scoring:

  1. Extract field in DataMapper.mapStockData() from the Yahoo payload
  2. Add field to StockData and StockMetrics interfaces in server/types/models.model.ts
  3. Add field to Stock constructor assignment and getDisplayMetrics() in server/models/Stock.ts
  4. Add weight + threshold to ScoringConfig.ts base weights/thresholds (and any relevant sector overrides)
  5. Add the field to SanitizedMetrics and _sanitize() in StockScorer.ts
  6. Add a factor entry to the factors array in StockScorer.score()
  7. Add a test case to StockScorer.test.js

Warning: The scorer.score(asset.metrics as never, ...) cast in ScreenerEngine._process() bypasses TypeScript for the scorer dispatch. If you add a field to StockMetrics but forget to add it to SanitizedMetrics, the compiler will not catch it — the scorer will silently receive undefined. Always update _sanitize() when adding metrics.

Architecture Roadmap

Phases 1–7 ✅ COMPLETE

All phases of the original roadmap are done. Summary of what was delivered:

  • Phase 1–4: CLI cleanup, shared utilities, TypeScript migration, SCSS design tokens
  • Phase 5: +page.svelte decomposed into AssetTable, AnalysisSidebar, MarketContextStrip, VerdictPill
  • Phase 6: Full TypeScript conversion across server/ and bin/
  • Phase 7a–7b: Type domain split, API module split
  • Phase 7f: Server layer restructured to layered architecture (controllers / services / repositories / clients / models / scorers)
  • Phase 7g: Controllers converted to classes with DI, types moved to domain files, YahooFinanceClient properly typed

Pending UI work (Phase 7c–7e):

  • 7c: Decompose portfolio/+page.svelte (751 lines) into AddHoldingForm, InlineEditRow, AdviceTable, AccountsTable; decompose calls/+page.svelte (385 lines) into CallForm, CallCard, CalendarSection
  • 7d: Add ui/src/lib/stores/ layer — screener.store.ts, portfolio.store.ts
  • 7e: Extract inline <style> blocks from portfolio, calls, and AnalysisSidebar into _forms.scss, _sidebar.scss, _calls.scss, _portfolio.scss

Phase 8 — Server Hardening & Quality

Priority order. Complete earlier items before starting later ones.

8a — Fix as never scorer dispatch in ScreenerEngine

scorer.score(asset.metrics as never, ...) bypasses TypeScript. Replace with a properly typed discriminated union or per-type dispatch so the compiler can verify that StockMetrics matches StockScorer.score()'s signature. This is the highest-risk cast in the scoring pipeline.

8b — Inject dependencies into ScreenerEngine and PortfolioAdvisor

Both classes self-construct their YahooFinanceClient and BenchmarkProvider in the constructor, making unit testing impossible without monkey-patching. Target:

export class ScreenerEngine {
  constructor(
    private readonly client: YahooFinanceClient,
    private readonly benchmarkProvider: BenchmarkProvider,
    { logger }: ScreenerEngineOptions = {},
  ) {}
}

Wire in server/app.ts. This unblocks proper service-layer unit tests.

8c — Controller integration tests

Add one Fastify inject() smoke test per route using a fixture for ScreenerEngine.screenTickers(). Catches schema validation regressions and response shape changes without needing live Yahoo access. Target: tests/screener.controller.test.js, tests/calls.controller.test.js.

8d — Repository tests

MarketCallRepository has zero test coverage. Add tests/MarketCallRepository.test.js using a temp file path (inject via constructor or env var) to test list, create, delete, and concurrent-write safety.

8e — Ticker normalisation in YahooFinanceClient

BRK.B → BRK-B normalisation lives only in FinanceController. Move it to YahooFinanceClient.fetchSummary() so it applies to all callers including /api/screen.

async fetchSummary(ticker: string, ...): Promise<any> {
  const normalized = ticker.replace(/\./g, '-');
  return await this.lib.quoteSummary(normalized, { modules: YAHOO_MODULES });
}

8f — Persistent benchmark cache

BenchmarkProvider's 1-hour cache is in-memory only — cold start after every restart adds 2–4s latency to the first request. Write the cached MarketContext to .benchmark-cache.json (or a single-row SQLite table). Read it on boot; only re-fetch if stale.

8g — Rate limiting + API key auth

Add @fastify/rate-limit on /api/screen and /api/analyze (e.g. 10 req/min per IP). Add a simple Authorization: Bearer <key> check against an API_KEY env var as middleware in server/app.ts. Both are single-digit line additions.

8h — Extract CalendarService

CallsController.calendar() is 80+ lines of inline event construction, date parsing, and sorting — all inside a controller method. Extract to server/services/CalendarService.ts to make it testable and keep the controller under 50 lines.

8i — SQLite migration for repositories

Both market-calls.json and portfolio.json use writeFileSync with no concurrency guard. Two concurrent writes within the same event loop tick will lose one write. Replace with better-sqlite3 for both repositories: concurrent-write safe, atomic transactions, no extra infrastructure. At current portfolio sizes the footprint is trivial.

8j — Cache CatalystAnalyst results

CatalystAnalyst.run() fires fresh Yahoo news queries on every /api/screen/catalysts call. Cache the result for 15 minutes. A new CatalystAnalyst instance is also created on each call inside ScreenerController.catalysts() — hoist it to a class-level singleton wired in server/app.ts.

8k — Add expert scoring tests

Update StockScorer.test.js to cover the three new scoring factors: analyst consensus scoring (including the numberOfAnalysts < 3 guard), DCF margin of safety scoring (positive/negative/null cases), and the new 52W risk flags.

Adding a New Asset Type

  1. Create a subclass of Asset in server/models/ with a flat metrics object and getDisplayMetrics().
  2. Add a per-type entry (gates / weights / thresholds) to ScoringRules in server/config/ScoringConfig.ts.
  3. Add inflated overrides in server/services/MarketRegime.ts → getInflatedOverrides().
  4. Create a Scorer in server/scorers/ exposing score(metrics, rules, marketContext).
  5. Add a mapper branch in server/services/DataMapper.ts.
  6. Wire into server/services/ScreenerEngine.ts: add case in _buildAsset, entry in SCORERS map.
  7. Add the new type to serializeAssets() in server/controllers/screener.controller.ts.

Clean Architecture Pattern (Server-Side)

Core Principles

  1. Types in server/types/ — Domain shapes only; one *.model.ts per domain
  2. Schemas in server/types/schemas.ts — Fastify JSON Schema objects for request body validation
  3. Class-Based Implementation — All .ts files (except types/schemas) are classes
  4. No Inline Interfaces — All types in server/types/; private-only shapes may stay inline
  5. Module-Level Cleanliness — No module-level constants or functions outside classes
  6. Direct Imports — Import directly from source files; use server/types/index.ts barrel for types
  7. Service barrel — server/services/index.ts re-exports all services so controllers import from one path

Implementation Pattern

Controllers (Classes with DI)

export class ScreenerController {
  constructor(private readonly engine: ScreenerEngine) {}

  register(app: FastifyInstance): void {
    app.post('/api/screen', { schema: screenSchema }, this.screen.bind(this));
  }

  private async screen(req: FastifyRequest) {
    const tickers = (req.body as { tickers: string[] }).tickers.map(t => t.toUpperCase());
    return this.engine.screenTickers(tickers);
  }
}

Services (Instance methods, static helpers)

export class BenchmarkProvider {
  private static readonly TTL_MS = 60 * 60 * 1000;

  async getMarketContext(): Promise<MarketContext> { ... }

  private static rateRegime(rate: number): string {
    return rate < 2 ? 'LOW' : rate <= 5 ? 'NORMAL' : 'HIGH';
  }
}

Scorers (Static-only classes)

export class StockScorer {
  static score(metrics: StockMetrics, rules: ScoringRules): ScoreResult { ... }
  private static _sanitize(m: StockMetrics): SanitizedMetrics { ... }
}

Import Guidelines

// ✅ CORRECT
import { ScreenerEngine } from '../services/ScreenerEngine';   // direct for implementations
import { ScreenerEngine } from '../services';                  // via barrel also acceptable
import type { StockMetrics } from '../types';                  // always use types barrel
import { screenSchema } from '../types/schemas';               // direct for schemas

// ❌ WRONG
import type { StockMetrics } from '../types/models.model';     // use barrel instead

Adding a New API Endpoint

  1. Define types → server/types/<domain>.model.ts
  2. Define schema → server/types/schemas.ts
  3. Create service → server/services/<Domain>Service.ts
  4. Wire controller → server/controllers/<domain>.controller.ts
  5. Register → server/app.ts
Reference in New Issue View Git Blame Copy Permalink