kazuma/market_screener
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase-1: optimize code

Browse Source
This commit is contained in:
Kazuma
2026-06-04 01:32:05 -04:00
parent cd74497de6
commit 3513024fc6
58 changed files with 7380 additions and 850 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+152 -90
View File
@@ -1,17 +1,22 @@
# Market Screener & Personal Finance Assistant
# Market Screener
A Node.js CLI tool that screens stocks, ETFs, bonds, and crypto using live Yahoo Finance data. It scores each asset under two lenses — **Market-Adjusted** (what's acceptable in today's inflated market) and **Fundamental** (Graham-style strict value investing) — and gives you an honest signal by comparing both.
A Node.js stock screener and personal finance assistant. Screens stocks, ETFs, and bonds using live Yahoo Finance data and scores each asset under two lenses — **Market-Adjusted** (what's acceptable in today's market) and **Fundamental** (strict Graham value-investing) — then compares both to give you an honest signal.
It also connects to your brokerage accounts via **SimpleFIN** to track net worth, spending, and give hold/sell/add advice on your actual portfolio.
Comes with a **Fastify API server** and a companion **SvelteKit dashboard** (`../market-screener-ui`) for an interactive UI, or use it as a CLI to generate HTML reports.
---
## Quick Start
```bash
# API + Dashboard (recommended)
npm install
cp .env.example .env # add SIMPLEFIN_SETUP_TOKEN if you have SimpleFIN
npm start # screen today's catalyst tickers from Yahoo news
cd ../market-screener-ui && npm install && cd ../market_screener
npm run dev # starts API on :3000 + UI on :5173
# open http://localhost:5173
# CLI only
npm start # screen today's news catalyst tickers → screener-report.html
```
---
@@ -20,39 +25,50 @@ npm start # screen today's catalyst tickers from Yahoo news
| Command | What it does |
|---|---|
| `npm start` | Fetches today's market news, extracts catalyst tickers, screens them |
| `npm start -- watch` | Screens the default watchlist instead |
| `npm start -- AAPL MSFT VOO` | Screens specific tickers |
| `npm run finance` | Portfolio advice + SimpleFIN account data → `finance-report.html` |
| `npm run import-portfolio -- file.csv` | Imports Robinhood/Vanguard/Fidelity CSV into `portfolio.json` |
| `npm test` | Runs all unit tests (51 tests, zero external dependencies) |
| `npm run test:watch` | Re-runs tests on file changes during development |
| `npm run format` | Formats all source files with Prettier |
| `npm run format:check` | Checks formatting without writing (useful in CI) |
Both commands generate self-contained HTML reports that open in any browser.
| `npm run dev` | Start API server (port 3000) + SvelteKit UI (port 5173) together |
| `npm run server` | Start API server only |
| `npm start` | CLI: fetch today's market news, extract tickers, screen them |
| `npm start -- watch` | CLI: screen the default watchlist |
| `npm start -- AAPL MSFT VOO` | CLI: screen specific tickers |
| `npm run finance` | CLI: portfolio advice + SimpleFIN → `finance-report.html` |
| `npm run import-portfolio -- file.csv` | Import Robinhood/Vanguard/Fidelity CSV into `portfolio.json` |
| `npm test` | Run all 61 unit tests |
| `npm run test:watch` | Re-run tests on file changes |
| `npm run format` | Format all source files with Prettier |
---
## How the Screener Works
Every asset is scored twice:
Every asset is scored **twice** under different rule sets:
**Market-Adjusted** — gates derived from live Yahoo Finance benchmarks:
- Stock P/E gate = S&P 500 P/E (via SPY) × 1.5
- Tech P/E gate = XLK sector P/E × 1.3
- REIT min yield = XLRE dividend yield × 0.85
- Bond min spread = LQD − TNX live spread × 0.80
### Market-Adjusted mode
Gates derived from live Yahoo Finance benchmarks — reflects what is acceptable in today's market:
**Fundamental** — strict Graham/value-investing gates from `src/config/ScoringConfig.js`:
- Stock P/E < 20x, PEG < 1.5
- Bond spread > 1.0% above risk-free rate
| Gate | Formula |
|---|---|
| Stock P/E | S&P 500 P/E (via SPY) × 1.5× (or × 1.2× in HIGH rate regime) |
| Tech P/E | XLK sector P/E × 1.3× |
| REIT min yield | XLRE dividend yield × 0.85× |
| Bond min spread | LQD − TNX spread × 0.80× |
The comparison produces a **Signal**:
### Fundamental mode
Strict Graham/value-investing gates — reflects genuine value regardless of market conditions:
| Gate | Value | Rationale |
|---|---|---|
| Stock P/E | < 15× | Graham's actual rule (trailing earnings) |
| Stock PEG | < 1.0 | Lynch standard: PEG > 1.0 = paying full price |
| D/E ratio | < 1.5× | Distress typically starts above 2× |
| Quick ratio | > 0.8 | Below 0.8 = real liquidity stress |
| Bond spread | > 1.5% | Must clear risk-free rate meaningfully |
| Bond duration | < 7 years | Rate sensitivity management |
### Signals
| Signal | Meaning |
|---|---|
| ✅ Strong Buy | Passes both — genuinely good value |
| ✅ Strong Buy | Passes both lenses — genuinely good value |
| ⚡ Momentum | Passes market-adjusted, holds fundamentally |
| ⚠️ Speculation | Passes market-adjusted, fails fundamental — priced for perfection |
| 🔄 Neutral | Hold territory in one or both lenses |
@@ -60,6 +76,38 @@ The comparison produces a **Signal**:
---
## Sector Overrides
Sector-specific rules apply in both modes (not just inflated):
| Sector | Key adjustments |
|---|---|
| **Technology** | P/E up to 35×, D/E up to 2.0 (buybacks), FCF weight raised |
| **REIT** | P/E/PEG disabled, scored on dividend yield + P/FFO proxy |
| **Financial** | D/E disabled, scored on ROE (≥12%) + P/B (< 1.5×) |
| **Energy** | FCF primary signal (weight 4), dividend yield scored |
| **Healthcare** | Revenue growth primary, P/E up to 25× (R&D burn) |
| **Communication** | FCF primary (META/GOOGL platforms), P/E up to 25× |
| **Consumer Staples** | Margin/ROE focus, low revenue growth expectations (2–5%) |
| **Consumer Discretionary** | Revenue growth primary, P/E up to 25× |
---
## API Server
```
GET /health → { status: "ok" }
POST /api/screen → screen tickers
body: { tickers: string[] }
GET /api/screen/catalysts → Yahoo news → { tickers, stories }
GET /api/finance/portfolio → portfolio advice + net worth
GET /api/finance/market-context → live benchmark data
```
Set `CLIENT_ORIGIN` env var to allow a different CORS origin (default: `http://localhost:5173`).
---
## Personal Finance
Edit `portfolio.json` with your holdings (or import from a broker CSV):
@@ -74,15 +122,13 @@ Edit `portfolio.json` with your holdings (or import from a broker CSV):
}
```
`npm run finance` screens your holdings, fetches crypto prices, and generates hold/sell/add advice based on the screener signal crossed with your gain/loss position.
`npm run finance` (CLI) or `GET /api/finance/portfolio` (API) screens your holdings and cross-references the screener signal with your gain/loss position to give hold/sell/add advice.
### SimpleFIN (optional)
Connects to your real bank and brokerage accounts for net worth, balances, and 30-day spending breakdown.
### SimpleFIN (optional — live bank/brokerage balances)
1. Get your setup token from [beta-bridge.simplefin.org](https://beta-bridge.simplefin.org)
2. Add to `.env`: `SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...`
3. Run `npm run finance` — the Access URL is claimed and saved automatically
3. On first run the Access URL is claimed and saved to `.env` automatically
### Importing broker holdings
@@ -91,73 +137,89 @@ npm run import-portfolio -- ~/Downloads/robinhood_holdings.csv
npm run import-portfolio -- ~/Downloads/vanguard_holdings.csv
```
Broker is auto-detected from CSV headers. Running multiple imports merges them into `portfolio.json`.
Broker is auto-detected from CSV headers. Multiple imports merge into `portfolio.json`.
---
## Project Structure
```
├── bin/
│ ├── screen.js # Market screener entry point
│ ├── finance.js # Personal finance entry point
│ └── import-portfolio.js # Broker CSV importer
│
├── prompts/
│ └── catalyst-analysis.md # Daily catalyst analysis playbook
│
├── src/
│ ├── config/
│ │ └── ScoringConfig.js # All scoring gates, weights, thresholds
│ │
│ ├── market/ # Yahoo Finance data layer
│ │ ├── YahooClient.js
│ │ ├── BenchmarkProvider.js
│ │ └── MarketRegime.js # Derives inflated gate overrides from live data
│ │
│ ├── screener/ # Core screening domain
│ │ ├── ScreenerEngine.js
│ │ ├── DataMapper.js
│ │ ├── RuleMerger.js
│ │ ├── Chunker.js
│ │ ├── assets/ # Stock, Etf, Bond data containers
│ │ └── scorers/ # StockScorer, EtfScorer, BondScorer
│ │
│ ├── analyst/
│ │ └── CatalystAnalyst.js # Extracts tickers from Yahoo Finance news
│ │
│ ├── finance/
│ │ ├── clients/
│ │ │ └── SimpleFINClient.js
│ │ ├── PersonalFinanceAnalyzer.js
│ │ ├── PortfolioAdvisor.js
│ │ └── PortfolioImporter.js
│ │
│ └── reporters/
│ ├── HtmlReporter.js # screener-report.html
│ └── FinanceReporter.js # finance-report.html
│
├── portfolio.json # Your holdings (edit this)
└── .env # SIMPLEFIN_SETUP_TOKEN / SIMPLEFIN_ACCESS_URL
bin/
screen.js CLI screener entry point
finance.js CLI personal finance entry point
import-portfolio.js Broker CSV importer
server.js Fastify API server entry point
scripts/
summary-reporter.js Custom node:test reporter (silent pass, shows failures + summary)
src/
config/
ScoringConfig.js All gates, weights, thresholds (single source of truth)
constants.js Shared enums: SIGNAL, ASSET_TYPE, SECTOR, SCORE_MODE, REGIME
market/
YahooClient.js Wraps yahoo-finance2 v3 with retry + backoff
BenchmarkProvider.js Fetches live benchmarks → marketContext (1-hour cache)
MarketRegime.js Derives inflated gate overrides from live data + rate regime
screener/
ScreenerEngine.js Orchestrates fetch → score × 2.
screenTickers() → pure data (server/CLI)
screenWithProgress() → with stdout progress (CLI only)
DataMapper.js Normalises Yahoo payload → flat asset objects
Uses trailingPE (not forwardPE). Preserves negative FCF.
RuleMerger.js Merges base rules + sector overrides + MarketRegime
assets/ Stock, Etf, Bond data containers
scorers/ StockScorer, EtfScorer, BondScorer (stateless)
analyst/
CatalystAnalyst.js Extracts tickers from Yahoo Finance news headlines
finance/
clients/
SimpleFINClient.js Auth + account fetching via Basic Auth header
PersonalFinanceAnalyzer.js Net worth, cash vs investments, spending
PortfolioAdvisor.js Hold/sell/add advice per holding
PortfolioImporter.js Parses Robinhood/Vanguard/Fidelity CSV
reporters/
HtmlReporter.js render() → string | generate() → file (CLI)
FinanceReporter.js render() → string | generate() → file (CLI)
server/
app.js Fastify app factory (buildApp)
routes/
screener.js POST /api/screen, GET /api/screen/catalysts
finance.js GET /api/finance/portfolio, GET /api/finance/market-context
```
---
## Metrics Scored per Stock
## Environment Variables
| Metric | Source | Why it matters |
|---|---|---|
| P/E ratio | Yahoo forwardPE / trailingPE | Valuation |
| PEG ratio | Yahoo or computed (trailingPE ÷ earningsGrowth) | Valuation vs growth |
| Price-to-Book | Yahoo | Graham's primary value metric |
| ROE | Yahoo returnOnEquity | Buffett's primary quality metric |
| Operating margin | Yahoo operatingMargins | Pricing power |
| Net profit margin | Yahoo profitMargins | Bottom-line profitability |
| Revenue growth | Yahoo revenueGrowth | Top-line momentum |
| FCF yield | Computed (freeCashflow ÷ market cap) | Cash generation quality |
| Debt/Equity | Yahoo debtToEquity | Balance sheet risk |
| Quick ratio | Yahoo quickRatio (falls back to currentRatio) | Liquidity |
| Beta | Yahoo beta | Market sensitivity |
| 52-week position | Yahoo fiftyTwoWeekHigh/Low | Momentum / opportunity flag |
```bash
# .env
SIMPLEFIN_ACCESS_URL=https://user:pass@beta-bridge.simplefin.org/simplefin
# or on first run:
SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...
Sector overrides apply: REIT scores on yield + P/FFO, FINANCIAL on ROE + P/B, TECHNOLOGY with realistic D/E tolerance.
# Optional server config
PORT=3000
HOST=0.0.0.0
CLIENT_ORIGIN=http://localhost:5173 # CORS allowed origin for SvelteKit UI
```
---
## Testing
61 unit tests, no external test framework:
```bash
npm test # summary output: "✅ 61 tests: 61 passed (0.02s)"
npm run test:watch # verbose spec output for development
```
Pre-commit: Prettier (auto-format staged files) + full test suite.
Pre-push: full test suite.