Kazuma
cd74497de6
fix: restore ScoringConfig improvements lost in refactor commit docs: rewrite README and CLAUDE.md to reflect current architecture code-format code fixes
232 lines
11 KiB
Markdown
232 lines
11 KiB
Markdown
# CLAUDE.md
|
||
|
||
Guidance for working in this repository.
|
||
|
||
## Overview
|
||
|
||
`market-screener` is a Node.js CLI tool that screens stocks, ETFs, and bonds by fetching live data from Yahoo Finance and scoring each asset under two lenses:
|
||
|
||
- **Market-Adjusted** — gates derived from live Yahoo benchmarks (SPY P/E, XLK P/E, XLRE yield, LQD spread). Reflects what is acceptable in today's inflated market.
|
||
- **Fundamental** — strict Graham/value-investing style gates from `ScoringConfig`. Reflects genuine value regardless of market conditions.
|
||
|
||
The comparison produces a **Signal** (Strong Buy / Momentum / Speculation / Neutral / Avoid).
|
||
|
||
ES module project (`"type": "module"`); use `import`/`export`, not `require`.
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
npm install # install dependencies (yahoo-finance2, dotenv, prettier)
|
||
npm start # Yahoo news → catalyst tickers → screener-report.html
|
||
npm start -- watch # default watchlist
|
||
npm start -- AAPL MSFT VOO # specific tickers
|
||
npm run finance # portfolio advice + SimpleFIN → finance-report.html
|
||
npm run import-portfolio -- holdings.csv # import Robinhood/Vanguard CSV into portfolio.json
|
||
npm test # run all unit tests (node:test, zero deps)
|
||
npm run test:watch # watch mode during development
|
||
npm run format # format all src/bin/tests with Prettier
|
||
npm run format:check # check formatting without writing (CI)
|
||
```
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
bin/
|
||
screen.js ← market screener entry point
|
||
finance.js ← personal finance entry point
|
||
import-portfolio.js ← broker CSV importer
|
||
|
||
prompts/
|
||
catalyst-analysis.md ← daily catalyst analysis playbook (LLM prompt + workflow)
|
||
|
||
src/
|
||
config/
|
||
ScoringConfig.js ← CREDIT_RATING_SCALE + ScoringRules (single source of truth)
|
||
|
||
market/ ← Yahoo Finance data layer
|
||
YahooClient.js ← wraps yahoo-finance2 v3, retry + backoff
|
||
BenchmarkProvider.js ← fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD → marketContext
|
||
MarketRegime.js ← derives INFLATED gate overrides from live benchmarks
|
||
|
||
screener/ ← core screening domain
|
||
ScreenerEngine.js ← orchestrates: fetch → score × 2 → HTML report
|
||
DataMapper.js ← normalises Yahoo payload → flat asset data object
|
||
RuleMerger.js ← merges base rules + sector overrides + MarketRegime (INFLATED mode)
|
||
Chunker.js ← splits ticker list into batches
|
||
assets/
|
||
Asset.js ← abstract base: ticker, currentPrice, type, formatting helpers
|
||
Stock.js ← metrics: peRatio, pegRatio, priceToBook, ROE, opMargin, etc.
|
||
Etf.js ← metrics: expenseRatio, yield, totalAssets
|
||
Bond.js ← metrics: ytm, duration, creditRating, creditRatingNumeric
|
||
scorers/
|
||
StockScorer.js ← gate checks + weighted registry (ROE, opMargin, margin, peg, rev, fcf)
|
||
EtfScorer.js ← expense gate + registry (cost, yield, volume)
|
||
BondScorer.js ← credit gate + spread/duration scoring
|
||
|
||
analyst/
|
||
CatalystAnalyst.js ← fetches Yahoo Finance news, extracts relatedTickers
|
||
|
||
finance/
|
||
clients/
|
||
SimpleFINClient.js ← claims setup token → access URL, fetches /accounts
|
||
PersonalFinanceAnalyzer.js ← net worth, cash vs investments, spending by category
|
||
PortfolioAdvisor.js ← cross-references holdings with screener signals → hold/sell/add advice
|
||
PortfolioImporter.js ← parses Robinhood/Vanguard/Fidelity CSV → merges into portfolio.json
|
||
|
||
reporters/
|
||
HtmlReporter.js ← generates screener-report.html (tabbed inflated/fundamental views)
|
||
FinanceReporter.js ← generates finance-report.html (net worth, portfolio, spending)
|
||
|
||
portfolio.json ← user's holdings: ticker, shares, costBasis, source, type
|
||
```
|
||
|
||
## Data Flow
|
||
|
||
```
|
||
Yahoo Finance API
|
||
↓
|
||
BenchmarkProvider — fetches ^GSPC, ^TNX, ^VIX, SPY, XLK, XLRE, LQD
|
||
builds marketContext { sp500Price, riskFreeRate, vixLevel,
|
||
rateRegime, volatilityRegime, benchmarks { marketPE, techPE, reitYield, igSpread } }
|
||
↓
|
||
DataMapper — normalises raw Yahoo payload → flat data object with type (STOCK/ETF/BOND)
|
||
computes: pFFO proxy, FCF yield, computed PEG, 52-week position
|
||
↓
|
||
Asset subclass — Stock / Etf / Bond holds metrics, formats display
|
||
↓
|
||
RuleMerger × 2 — FUNDAMENTAL mode: ScoringConfig as-is (Graham-style)
|
||
INFLATED mode: sector override + MarketRegime live gate overrides
|
||
↓
|
||
Scorer × 2 — StockScorer / EtfScorer / BondScorer, fully stateless
|
||
↓
|
||
ScreenerEngine — derives Signal from comparing both verdicts
|
||
↓
|
||
HtmlReporter — screener-report.html: signal summary + two tabbed tables per asset class
|
||
```
|
||
|
||
## Scoring Modes
|
||
|
||
| Mode | P/E Gate (general) | Source |
|
||
|---|---|---|
|
||
| FUNDAMENTAL | 20x | ScoringConfig (Graham) |
|
||
| INFLATED | S&P 500 P/E × 1.5 | Live SPY data |
|
||
|
||
| Signal | Meaning |
|
||
|---|---|
|
||
| ✅ Strong Buy | Passes both fundamental AND inflated gates |
|
||
| ⚡ Momentum | Passes inflated, holds fundamentally |
|
||
| ⚠️ Speculation | Passes inflated, fails fundamental |
|
||
| 🔄 Neutral | Hold territory in one or both lenses |
|
||
| ❌ Avoid | Fails both |
|
||
|
||
## ScoringConfig Structure
|
||
|
||
`src/config/ScoringConfig.js` exports two things:
|
||
|
||
- `CREDIT_RATING_SCALE` — `{ AAA: 10, AA: 9, ..., D: 1 }`. Used by Bond.js and BondScorer.
|
||
- `ScoringRules` — per-type `{ gates, weights, thresholds }` + `SECTOR_OVERRIDE` map for STOCK.
|
||
|
||
Sector overrides are structural (apply in both modes). MarketRegime overrides valuation gates in INFLATED mode only.
|
||
|
||
## Sector Notes
|
||
|
||
- **TECHNOLOGY** — `maxDebtToEquity: 2.0` (mega-cap tech borrows for buybacks), `minQuickRatio: 0.8` (AAPL runs ~0.9). P/E and PEG gates inflated from XLK live data.
|
||
- **REIT** — P/E and PEG disabled (9999). Scored on dividend yield and P/FFO proxy. All base weights (margin, peg, revenue) explicitly zeroed out.
|
||
- **FINANCIAL** — P/E, PEG, D/E disabled. Scored on ROE + Price-to-Book. Quick ratio gate lowered (0.1).
|
||
|
||
## MarketRegime (INFLATED overrides)
|
||
|
||
`src/market/MarketRegime.js` derives gate overrides from live benchmarks:
|
||
|
||
| Gate | Formula |
|
||
|---|---|
|
||
| Stock maxPERatio | SPY trailing P/E × 1.5 |
|
||
| Tech maxPERatio | XLK P/E × 1.3 |
|
||
| Tech maxPegGate | XLK P/E ÷ 15 |
|
||
| REIT minYield | XLRE dividend yield × 0.85 |
|
||
| Bond minSpread | LQD−TNX spread × 0.80 |
|
||
| ETF maxExpenseRatio | 0.75% (structural loosening) |
|
||
|
||
## Missing Data Convention
|
||
|
||
- Missing metrics use `null` (not `0`) in `_sanitize`. Gate checks skip `null` values rather than auto-failing.
|
||
- `pegRatio` falls back to `trailingPE / earningsGrowth` when Yahoo doesn't provide it.
|
||
- `quickRatio` falls back to `currentRatio` when missing.
|
||
|
||
## SimpleFIN Auth Flow
|
||
|
||
1. User gets a Setup Token from https://beta-bridge.simplefin.org
|
||
2. `SimpleFINClient.init()` base64-decodes it → POSTs once to claim URL → receives Access URL
|
||
3. Access URL is auto-appended to `.env` as `SIMPLEFIN_ACCESS_URL`
|
||
4. All subsequent requests use Access URL directly (setup token is one-time use)
|
||
|
||
## portfolio.json Format
|
||
|
||
```json
|
||
{
|
||
"holdings": [
|
||
{ "ticker": "AAPL", "shares": 10, "costBasis": 150.00, "source": "Robinhood", "type": "stock" },
|
||
{ "ticker": "VOO", "shares": 8, "costBasis": 380.00, "source": "Vanguard", "type": "etf" },
|
||
{ "ticker": "BTC-USD", "shares": 0.25, "costBasis": 45000, "source": "Coinbase", "type": "crypto" }
|
||
]
|
||
}
|
||
```
|
||
|
||
`type` values: `stock`, `etf`, `crypto`. Crypto is priced via Yahoo (BTC-USD style) but not fundamentally scored.
|
||
|
||
## Tests
|
||
|
||
Uses Node's built-in test runner (`node:test` + `node:assert/strict`) — no test framework to install.
|
||
|
||
```
|
||
tests/
|
||
ScoringConfig.test.js ← CREDIT_RATING_SCALE, gate values, sector overrides
|
||
RuleMerger.test.js ← FUNDAMENTAL vs INFLATED modes, sector merging
|
||
MarketRegime.test.js ← inflated override formulas per asset type and sector
|
||
StockScorer.test.js ← gate failures, scoring labels, risk flags
|
||
EtfScorer.test.js ← expense gate, score tiers
|
||
BondScorer.test.js ← credit gate, spread/duration scoring, unit handling
|
||
DataMapper.test.js ← type detection, PEG computation, null convention
|
||
PortfolioAdvisor.test.js ← _position gain/loss calc, _advice signal mapping
|
||
```
|
||
|
||
**Key unit to remember:** `ytm` in `Bond.metrics` is stored as a percentage (e.g. `6.5` = 6.5%). `BondScorer._sanitize` divides by 100 before spread calculation. `minSpread` in `ScoringConfig` is also in percentage form (e.g. `1.0` = 1%).
|
||
|
||
## Conventions
|
||
|
||
- Asset `type` (uppercased) is the routing key across DataMapper, asset classes, `SCORERS` map in ScreenerEngine, and ScoringRules.
|
||
- Prefer adjusting `ScoringConfig` or `MarketRegime` over hardcoding numbers in scorers.
|
||
- BenchmarkProvider caches for 1 hour — restart the process to force a fresh fetch during development.
|
||
- All entry points live in `bin/`. Do not add logic to entry points — they call into `src/`.
|
||
|
||
## Planned Enhancements
|
||
|
||
### 1. Rate Sensitivity Flag (no new fetches needed)
|
||
Flag assets exposed to rate changes using `riskFreeRate` and `rateRegime`:
|
||
- Long-duration bonds (duration > 7) in HIGH rate regime → warn
|
||
- REITs with high D/E in HIGH rate regime → warn
|
||
- Growth stocks with negative FCF in HIGH rate regime → warn
|
||
|
||
### 2. 52-Week Positioning (data already mapped)
|
||
`week52High` and `week52Low` are in asset metrics. Add interpretation:
|
||
- `> 85%` position + high P/E = crowded/momentum trade
|
||
- `< 15%` position + passing fundamental gates = potential opportunity
|
||
|
||
### 3. Sector Rotation Cues (3–4 new fetches)
|
||
Add XLF, XLE, XLV, XLI to BenchmarkProvider. Compare each sector ETF's 1-year return to S&P 500 to identify leading/lagging sectors.
|
||
|
||
### 4. Client/Server Architecture
|
||
Planned split into:
|
||
- **Server** (Fastify + BullMQ) — API endpoints, webhook-driven news monitoring, WebSocket for live updates
|
||
- **Client** (SvelteKit) — interactive dashboard replacing the HTML reports
|
||
- LLM integration (Claude Haiku) wired into the server for deeper catalyst analysis
|
||
|
||
## Adding a New Asset Type
|
||
|
||
1. Create a subclass of `Asset` in `src/screener/assets/` with a flat `metrics` object and `getDisplayMetrics()`.
|
||
2. Add a per-type entry (`gates` / `weights` / `thresholds`) to `ScoringRules` in `ScoringConfig.js`.
|
||
3. Add inflated overrides in `MarketRegime.getInflatedOverrides()`.
|
||
4. Create a Scorer in `src/screener/scorers/` exposing `score(metrics, rules, marketContext)`.
|
||
5. Add a mapper in `DataMapper.js`.
|
||
6. Wire into `ScreenerEngine`: add `case` in `_buildAsset`, entry in `SCORERS` map.
|