# CLAUDE.md Guidance for working in this repository. ## Overview `market-screener` is a Node.js project with two modes: 1. **CLI** — screens stocks, ETFs, and bonds via `npm start`, generates HTML reports 2. **Fastify API server** — powers the SvelteKit dashboard in the `ui/` subdirectory Every asset is scored under two lenses: - **Market-Adjusted** — gates derived from live Yahoo benchmarks (SPY P/E, XLK P/E, XLRE yield, LQD spread). Reflects what is acceptable in today's market. - **Fundamental** — strict Graham/value-investing gates from `ScoringConfig`. Reflects genuine value regardless of market conditions. The comparison produces a **Signal** (Strong Buy / Momentum / Speculation / Neutral / Avoid). ES module project (`"type": "module"`); use `import`/`export`, not `require`. --- ## Commands ```bash npm install # install dependencies npm run dev # start API server (port 3000) + SvelteKit UI (port 5173) together npm run server # API server only (port 3000) npm start # CLI: Yahoo news → catalyst tickers → screener-report.html npm start -- watch # CLI: default watchlist npm start -- AAPL MSFT VOO # CLI: specific tickers npm run finance # CLI: portfolio advice + SimpleFIN → finance-report.html npm test # run all unit tests (node:test, zero external deps) npm run test:watch # watch mode — uses verbose spec reporter npm run format # format all server/bin/tests with Prettier npm run format:check # check formatting without writing (used in CI/pre-commit) npm run ui:install # install UI dependencies (ui/ subdirectory) ``` `npm run dev` runs both the API server and the SvelteKit UI (in `ui/`) concurrently. Run `npm run ui:install` once before first use. --- ## Project Structure ``` bin/ screen.js ← CLI screener entry point finance.js ← CLI personal finance entry point import-portfolio.js ← broker CSV importer server.js ← Fastify API server entry point scripts/ summary-reporter.js ← custom node:test reporter (silent on pass, summary line at end) prompts/ catalyst-analysis.md ← daily catalyst analysis playbook (LLM prompt + workflow) server/ config/ ScoringConfig.js ← CREDIT_RATING_SCALE + ScoringRules (single source of truth) constants.js ← SIGNAL, ASSET_TYPE, SECTOR, SCORE_MODE, REGIME, SIGNAL_ORDER market/ ← Yahoo Finance data layer YahooClient.js ← wraps yahoo-finance2 v3, retry + backoff BenchmarkProvider.js ← fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD → marketContext MarketRegime.js ← derives INFLATED gate overrides from live benchmarks + rate regime screener/ ← core screening domain ScreenerEngine.js ← orchestrates: fetch → score × 2. Methods: screenTickers() (pure data), screenWithProgress() (CLI with stdout). Accepts { logger } option. DataMapper.js ← normalises Yahoo payload → flat asset data object NOTE: uses trailingPE (not forwardPE). Preserves negative FCF. Infers bond duration from category string. Maps ETF volume. RuleMerger.js ← merges base rules + sector overrides + MarketRegime (INFLATED mode) Chunker.js ← splits ticker list into batches assets/ Asset.js ← abstract base: ticker, currentPrice, type, formatting helpers Stock.js ← metrics + _mapToStandardSector (8 sectors detected) Etf.js ← metrics: expenseRatio, yield, volume, fiveYearReturn, totalAssets Bond.js ← metrics: ytm, duration, creditRating, creditRatingNumeric scorers/ StockScorer.js ← gate checks + weighted registry (ROE, opMargin, margin, peg, rev, fcf) EtfScorer.js ← expense gate + registry (cost, yield, volume, fiveYearReturn) BondScorer.js ← credit gate + spread/duration scoring analyst/ CatalystAnalyst.js ← fetches Yahoo Finance news, extracts relatedTickers. Accepts { logger }. LLMAnalyst.js ← uses Claude Haiku (ANTHROPIC_API_KEY) to analyze headlines → summary, sentiment (BULLISH/NEUTRAL/BEARISH), affectedIndustries, relatedTickers. Returns null gracefully if API key is not set. Accepts { logger }. calls/ MarketCallStore.js ← persists quarterly market thesis entries to market-calls.json. Each call stores: title, quarter, date, thesis, tickers[], snapshot{} (price + signal per ticker at creation time). CRUD: list/get/create/delete. finance/ clients/ SimpleFINClient.js ← claims setup token → access URL, fetches /accounts via Basic Auth header (NOT embedded credentials in URL). Accepts { logger, onAccessUrlClaimed }. PersonalFinanceAnalyzer.js ← net worth, cash vs investments, spending by category PortfolioAdvisor.js ← cross-references holdings with screener signals → hold/sell/add advice reporters/ HtmlReporter.js ← render() → HTML string (server), generate() → writes file (CLI) FinanceReporter.js ← render() → HTML string (server), generate() → writes file (CLI) server/ app.js ← Fastify app factory (buildApp). Registers CORS + routes. routes/ screener.js ← POST /api/screen, GET /api/screen/catalysts Serializes asset.getDisplayMetrics() before JSON response. finance.js ← GET /api/finance/portfolio, GET /api/finance/market-context calls.js ← CRUD for market calls + GET /api/calls/calendar (earnings/dividend events) ui/ ← SvelteKit dashboard (lives inside this repo, not a separate repo) src/ styles/ ← global SCSS design-token system (Phase 4) app.scss ← root file — @use all partials _tokens.scss ← CSS custom properties generated from SCSS maps ($bg, $text, $blues, $signals…) _reset.scss ← box-sizing reset + body base _layout.scss ← shell, nav, .links, main, nav-progress, .loading-area _section.scss ← .section, .section-header, .count, .mode-tabs, .error-banner _table.scss ← table, thead/tbody, .table-wrap, .col-ticker, .ticker, .num, .tag _buttons.scss ← button base, .btn-primary, .btn-catalyst, .btn-ghost, .btn-screen, .btn-analyze _badges.scss ← .verdict-pill, .sentiment-pill, .text-* helpers (SCSS @each maps) lib/ api.js ← typed fetch wrappers for all API routes utils.ts ← shared pure functions: sigOrd, sorted, verdictShort, vClass, fmtPE, fmt… MarketContext.svelte ← collapsible card-grid context (used in portfolio + safe-buys) MarketContextStrip.svelte ← horizontal chip strip (used in screener — Phase 5) AssetTable.svelte ← STOCK/ETF/BOND section: mode tabs + Analyze + table (Phase 5) AnalysisSidebar.svelte ← LLM analysis slide-over panel (Phase 5) VerdictPill.svelte ← verdict-pill span; props: label (Phase 5) SignalBadge.svelte ← signal emoji + label badge Spinner.svelte ← sm: dot-pulse | md/lg: chart-line animation routes/ +page.js ← SvelteKit load (ssr:false) — fetches catalysts + screens on mount (Phase 5) +page.svelte ← main screener UI (~230 lines after Phase 5 decomposition) +layout.svelte ← shell, nav, nav-progress bar, nav-overlay with Spinner calls/ ← market calls list + detail views portfolio/ ← portfolio advice view safe-buys/ ← filtered strong-buy view market-calls.json ← persisted market thesis calls (written by MarketCallStore) portfolio.json ← user's holdings: ticker, shares, costBasis, source, type .env ← SIMPLEFIN_ACCESS_URL or SIMPLEFIN_SETUP_TOKEN, ANTHROPIC_API_KEY ``` --- ## Data Flow ``` Yahoo Finance API ↓ BenchmarkProvider — fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD builds marketContext { sp500Price, riskFreeRate, vixLevel, rateRegime, volatilityRegime, benchmarks { marketPE, techPE, reitYield, igSpread } } ↓ DataMapper — normalises raw Yahoo payload → flat data object with type (STOCK/ETF/BOND) uses trailingPE as primary; preserves negative FCF yield; infers bond duration ↓ Asset subclass — Stock / Etf / Bond holds metrics + getDisplayMetrics() ↓ RuleMerger × 2 — FUNDAMENTAL mode: ScoringConfig as-is (Graham-style) INFLATED mode: sector override + MarketRegime live gate overrides ↓ Scorer × 2 — StockScorer / EtfScorer / BondScorer, fully stateless ↓ ScreenerEngine — derives Signal from comparing both verdicts ↓ ├── CLI path: screenWithProgress() → HtmlReporter.generate() → screener-report.html └── API path: screenTickers() → JSON (with serialized displayMetrics) → SvelteKit UI ``` --- ## API Routes (Fastify) | Method | Path | Description | |---|---|---| | GET | `/health` | Health check | | POST | `/api/screen` | Screen tickers. Body: `{ tickers: string[] }`. Returns `{ STOCK, ETF, BOND, ERROR, marketContext }` with `asset.displayMetrics` pre-serialized | | GET | `/api/screen/catalysts` | Yahoo news → `{ tickers, stories }` | | GET | `/api/finance/portfolio` | Portfolio advice + optional SimpleFIN data | | GET | `/api/finance/market-context` | Live benchmark data only | | GET | `/api/calls` | List all market calls (newest first) | | GET | `/api/calls/:id` | Get one call + re-screened current prices for comparison | | POST | `/api/analyze` | Fetch Yahoo news for specific tickers + run LLM analysis. Body: `{ tickers: string[] }`. Returns `{ analysis }` | | POST | `/api/calls` | Create a market call; snapshots current prices. Body: `{ title, quarter, thesis, tickers[], date? }` | | DELETE | `/api/calls/:id` | Delete a market call | | GET | `/api/calls/calendar` | Earnings + dividend calendar. Query: `?tickers=AAPL,MSFT` (omit for all call tickers) | CORS is configured for `CLIENT_ORIGIN` env var (default `http://localhost:5173`). --- ## Scoring Modes | Mode | P/E Gate (general) | P/E Gate (tech) | Source | |---|---|---|---| | FUNDAMENTAL | 15x | 35x | ScoringConfig (true Graham) | | INFLATED | S&P 500 P/E × 1.5 | XLK P/E × 1.3 | Live SPY/XLK data | **Rate regime effect on INFLATED mode:** - HIGH rate regime: P/E multiplier compresses to 1.2× (vs 1.5× in NORMAL) - HIGH rate regime: REIT yield floor tightens (0.95× vs 0.85×) - HIGH rate regime: bond spread demand increases (0.90× vs 0.80×) | Signal | Meaning | |---|---| | ✅ Strong Buy | Passes both fundamental AND inflated gates | | ⚡ Momentum | Passes inflated, holds fundamentally | | ⚠️ Speculation | Passes inflated, fails fundamental | | 🔄 Neutral | Hold territory in one or both lenses | | ❌ Avoid | Fails both | --- ## ScoringConfig Key Values `server/config/ScoringConfig.js` — single source of truth for all gates, weights, thresholds. **STOCK base gates (Fundamental mode):** - `maxPERatio: 15` — Graham's actual rule (trailing P/E) - `maxPegGate: 1.0` — Lynch standard: PEG > 1.0 means paying full price - `maxDebtToEquity: 1.5` — most distress starts above 2x - `minQuickRatio: 0.8` — below this signals real liquidity stress **Sector overrides** (structural — apply in both modes): | Sector | Key difference | |---|---| | TECHNOLOGY | D/E up to 2.0, P/E up to 35x, FCF weight raised | | REIT | P/E and PEG disabled (9999), scored on yield + P/FFO | | FINANCIAL | D/E disabled, scored on ROE + P/B, maxPriceToBook 1.5x | | ENERGY | FCF weight 4, yield weight 3, opMargin primary | | HEALTHCARE | Revenue growth primary, P/E up to 25x | | COMMUNICATION | FCF weight 4, P/E up to 25x (META, GOOGL, NFLX) | | CONSUMER_STAPLES | Margin/ROE focus, low revenue growth expectations | | CONSUMER_DISCRETIONARY | Revenue growth primary, P/E up to 25x | **ETF gates:** - `maxExpenseRatio: 0.2%` — hard gate - `minFiveYearReturn: 8.0%` — S&P long-run floor - `minVolume: 1,000,000` ADV **BOND gates:** - `minCreditRating: 7` (BBB = investment-grade floor) - `minSpread: 1.5%` above risk-free - `maxDuration: 7` years --- ## MarketRegime (INFLATED overrides) `server/market/MarketRegime.js` derives gate overrides from live benchmarks and current rate regime: | Gate | Formula (NORMAL rates) | Formula (HIGH rates) | |---|---|---| | Stock maxPERatio | SPY trailing P/E × 1.5 | SPY trailing P/E × 1.2 | | Tech maxPERatio | XLK P/E × 1.3 | XLK P/E × 1.3 | | Tech maxPegGate | XLK P/E ÷ 15 | XLK P/E ÷ 15 | | REIT minYield | XLRE yield × 0.85 | XLRE yield × 0.95 | | Bond minSpread | LQD−TNX × 0.80 | LQD−TNX × 0.90 | | ETF maxExpenseRatio | 0.75% | 0.75% | --- ## Sector Detection `Stock._mapToStandardSector()` maps Yahoo Finance `sector`/`industry` strings to internal constants. Order matters — more specific matches first: ``` TECHNOLOGY → "technology", "electronic", "semiconductor", "software" REIT → "real estate", "reit" FINANCIAL → "financial", "bank", "insurance", "asset management" ENERGY → "energy", "oil", "gas", "petroleum" HEALTHCARE → "health", "biotech", "pharmaceutical", "medical" COMMUNICATION→ "communication", "media", "entertainment", "telecom" CONSUMER_STAPLES → "consumer defensive", "consumer staples", "household", "beverage", "food" CONSUMER_DISCRETIONARY → "consumer cyclical", "consumer discretionary", "retail", "apparel", "auto" GENERAL → fallback ``` --- ## DataMapper Notes - **peRatio**: prefers `trailingPE` (audited) over `forwardPE` (analyst estimate, ~10-15% optimistic) - **FCF yield**: `freeCashflow !== 0` (not `> 0`) — negative FCF preserved so cash-burning companies fail the gate, not silently skip it - **Bond duration**: inferred from category string ("Short-Term" → 2y, "Intermediate" → 5y, "Long" → 18y, default 6y). Yahoo does not expose effective duration in the modules we fetch. - **ETF volume**: `summaryDetail.averageVolume` — was missing before, causing the `-2` liquidity penalty on every ETF --- ## Missing Data Convention - Missing metrics use `null` (not `0`) in `_sanitize`. Gate checks skip `null` rather than auto-failing. - `pegRatio` falls back to `trailingPE / earningsGrowth` when Yahoo doesn't provide it. - `quickRatio` falls back to `currentRatio` when missing. --- ## Logger Injection Pattern Classes that produce output accept an optional `{ logger }` constructor option so they work cleanly in server context: ```js // CLI (default) — writes to stdout new ScreenerEngine() // Server — fully silent new ScreenerEngine({ logger: { write: () => {}, log: () => {}, warn: () => {} } }) ``` Affected: `ScreenerEngine`, `BenchmarkProvider`, `CatalystAnalyst`, `SimpleFINClient`, `LLMAnalyst`. --- ## Reporter Pattern Both reporters have two methods: ```js reporter.render(...) // → HTML string (use in server route responses) reporter.generate(...) // → writes file to disk, returns path (use in CLI) ``` --- ## SimpleFIN Auth Flow 1. User gets a Setup Token from https://beta-bridge.simplefin.org 2. `SimpleFINClient.init()` base64-decodes it → POSTs once to claim Access URL 3. `onAccessUrlClaimed` callback is called with the URL — CLI uses `saveAccessUrlToEnv()`, server stores elsewhere 4. All subsequent requests use Access URL with `Authorization: Basic` header (not embedded in URL) --- ## portfolio.json Format ```json { "holdings": [ { "ticker": "AAPL", "shares": 10, "costBasis": 150.00, "source": "Robinhood", "type": "stock" }, { "ticker": "VOO", "shares": 8, "costBasis": 380.00, "source": "Vanguard", "type": "etf" }, { "ticker": "BTC-USD", "shares": 0.25, "costBasis": 45000, "source": "Coinbase", "type": "crypto" } ] } ``` `type` values: `stock`, `etf`, `crypto`. Crypto is priced via Yahoo (BTC-USD style) but not fundamentally scored. --- ## Tests Uses Node's built-in test runner (`node:test` + `node:assert/strict`) — no test framework needed. ``` tests/ ScoringConfig.test.js ← gate values (P/E 15x, PEG 1.0, QuickRatio 0.8), sector overrides RuleMerger.test.js ← FUNDAMENTAL vs INFLATED modes, sector merging MarketRegime.test.js ← inflated overrides including HIGH/NORMAL rate regime variants StockScorer.test.js ← gate failures, scoring labels, risk flags EtfScorer.test.js ← expense gate, volume penalty, 5Y return scoring BondScorer.test.js ← credit gate, spread/duration scoring, unit handling DataMapper.test.js ← type detection, PEG computation, trailing PE preference, negative FCF, ETF volume, bond duration inference PortfolioAdvisor.test.js ← _position gain/loss calc, _advice signal mapping, BRK.B dot-notation normalisation LLMAnalyst.test.js ← markdown fence stripping, JSON parse correctness ``` Pre-commit hook runs `lint-staged` (Prettier) then `npm test`. Pre-push hook runs `npm test`. Test output: silent on pass, shows only failures + one summary line (`scripts/summary-reporter.js`). **Key unit:** `ytm` in `Bond.metrics` is stored as a percentage (e.g. `6.5` = 6.5%). `BondScorer._sanitize` divides by 100 before spread calculation. **Coverage gaps (known):** - `MarketCallStore.js` — no tests; CRUD against `market-calls.json` is untested - `LLMAnalyst.test.js` — tests a local copy of the fence-stripping regex rather than importing from source; will silently drift if the regex changes - API routes (`server/server/routes/`) — no integration tests; covered implicitly by manual testing only - UI components — not tested at the unit level (Phases 4–5 are pure UI; no server logic changed) --- ## Conventions - Asset `type` (uppercased) is the routing key across DataMapper, asset classes, `SCORERS` map, and ScoringRules. - Prefer adjusting `ScoringConfig` or `MarketRegime` over hardcoding numbers in scorers. - BenchmarkProvider caches for 1 hour — restart the server to force a fresh fetch. - All entry points live in `bin/`. Do not add logic to entry points — they call into `server/`. - `bin/server.js` starts Fastify; `server/server/` contains all route logic. - **Never** call `process.exit()` inside `server/` — only `bin/` may do that. - Class instances don't survive `JSON.stringify`. Call `getDisplayMetrics()` server-side before returning from API routes (see `server/server/routes/screener.js` `serializeAssets()`). --- ## Architecture Roadmap Planned improvements in priority order. Do not start a later phase before completing earlier ones. ### Phase 1 — Cleanup ✅ COMPLETE All items completed. Additional features delivered alongside cleanup: **Cleanup done:** - Deleted root-level `finance.js`, `import-portfolio.js`, `markdown.md` - Deleted `server/server/routes/analyze.js` (orphaned route file) - Removed dead `analysis` state, `analysisOpen` state, and "🤖 AI Market Analysis" panel from `+page.svelte` - Fixed `.gitignore` — `portfolio.json`, `market-calls.json`, `.env` are now excluded from git **Features added during Phase 1:** - `POST /api/analyze` — per-tab LLM analysis with sidebar (✦ Analyze button on each asset section) - `POST /api/finance/holdings` + `DELETE /api/finance/holdings/:ticker` — add/edit/delete holdings via UI - Portfolio page: inline row editing, optimistic UI updates, sortable columns, collapsible market context with tooltips, P&L summary card tooltips - Holdings can be added/edited/deleted via the portfolio UI (manual entry replaces CSV importer) - `BRK.B` dot-notation tickers now normalised to Yahoo Finance format (`BRK.B → BRK-B`) - Market graph drawing-line animation replaces generic spinner (lg/md); dot-pulse for sm (buttons) - Portfolio page loads client-side (`$effect`) to avoid blocking navigation - Catalyst page auto-loads on mount; LLM analysis only runs on explicit ✦ Analyze click **Pending (deferred to later):** - LLM Analysis button on portfolio page (analyse holdings against current news) ### Phase 2 — Extract Shared Utilities ✅ COMPLETE **Done:** - Created `ui/src/lib/utils.ts` — typed shared pure functions: `sigOrd`, `sorted`, `verdictShort`, `vClass`, `fmtPE`, `fmt`, `fmtShort`, `glClass`, `advClass`. Exports `Signal` type. - Created `server/server/utils/logger.js` — shared `noopLogger` constant, imported by `screener.js`, `app.js`, `finance.js`, and `calls.js` - Added TypeScript support to `ui/` — `tsconfig.json` extending SvelteKit's generated config, `typescript` and `svelte-check` added as dev dependencies - All three pages (`+page.svelte`, `safe-buys/+page.svelte`, `portfolio/+page.svelte`) now import from `$lib/utils.js` instead of duplicating logic ### Phase 3 — Rename `src/` → `server/` ✅ COMPLETE **Done:** - Renamed `src/` to `server/` — `src/server/` is now `server/server/` - Updated all import paths in `bin/`, `tests/`, and `CLAUDE.md` ### Phase 4 — SCSS Migration ✅ COMPLETE **Done:** - Created `ui/src/styles/` with 7 partials + `app.scss` root (imported by `+layout.svelte`) - `_tokens.scss` uses SCSS maps (`$bg`, `$borders`, `$text`, `$blues`, `$signals`) with `@each` loops to emit CSS custom properties — adding a token is one line in the map - `_badges.scss` uses `@each` + `map.get` for verdict/sentiment color variants and `.text-*` helpers; shared `%pill-base` placeholder - `_buttons.scss` uses `%btn-disabled` / `%btn-inline-flex` placeholders + nested `&:hover` / `&:disabled` - `_section.scss`, `_table.scss`, `_layout.scss` use SCSS nesting throughout - `.vpill` (safe-buys) unified with `.verdict-pill` (screener) — inconsistency resolved - All component `