phase-1: optimize code
This commit is contained in:
saikiranvella
@@ -1,58 +1,225 @@
|
||||
# Financial Screener & Personal Finance Assistant
|
||||
# Market Screener
|
||||
|
||||
## Project Overview
|
||||
A Node.js stock screener and personal finance assistant. Screens stocks, ETFs, and bonds using live Yahoo Finance data and scores each asset under two lenses — **Market-Adjusted** (what's acceptable in today's market) and **Fundamental** (strict Graham value-investing) — then compares both to give you an honest signal.
|
||||
|
||||
This project is a modular, rule-based financial analysis engine designed to evaluate assets and manage personal investment portfolios. It separates data acquisition, strategy configuration, and evaluation logic to provide actionable investment insights.
|
||||
Comes with a **Fastify API server** and a companion **SvelteKit dashboard** (`../market-screener-ui`) for an interactive UI, or use it as a CLI to generate HTML reports.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Structure
|
||||
## Quick Start
|
||||
|
||||
### 1. Data Pipeline (`/src/data/`)
|
||||
```bash
|
||||
# API + Dashboard (recommended)
|
||||
npm install
|
||||
cd ../market-screener-ui && npm install && cd ../market_screener
|
||||
npm run dev # starts API on :3000 + UI on :5173
|
||||
# open http://localhost:5173
|
||||
|
||||
- **Fetcher:** Handles API communication (e.g., Yahoo Finance).
|
||||
- **Mapper:** Normalizes disparate API responses into a unified flat object structure.
|
||||
- **Asset Models (`/models/`):** Defines common properties for `Stock`, `Etf`, and `Bond`.
|
||||
|
||||
### 2. Logic & Configuration (`/src/config/` & `/src/utils/`)
|
||||
|
||||
- **`ScoringConfig.js`:** Houses all thresholds, gates, and weights.
|
||||
- **`RuleMerger.js`:** Dynamically applies sector-specific overrides to base rules.
|
||||
|
||||
### 3. Evaluation & Personal Assistant (`/src/engine/` & `/src/assistant/`)
|
||||
|
||||
- **`ScoringEngine.js`:** Orchestrates evaluation, applying market context and sector overrides.
|
||||
- **`PortfolioManager.js` (NEW):** Tracks individual holdings, cost basis, and performance metrics.
|
||||
- **`AdvisorModule.js` (NEW):** Provides personalized suggestions based on screening results and portfolio health.
|
||||
- **`EventMonitor.js` (NEW):** Tracks calendar events (Earnings Calls) to trigger alerts.
|
||||
# CLI only
|
||||
npm start # screen today's news catalyst tickers → screener-report.html
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Diagram
|
||||
## Commands
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `npm run dev` | Start API server (port 3000) + SvelteKit UI (port 5173) together |
|
||||
| `npm run server` | Start API server only |
|
||||
| `npm start` | CLI: fetch today's market news, extract tickers, screen them |
|
||||
| `npm start -- watch` | CLI: screen the default watchlist |
|
||||
| `npm start -- AAPL MSFT VOO` | CLI: screen specific tickers |
|
||||
| `npm run finance` | CLI: portfolio advice + SimpleFIN → `finance-report.html` |
|
||||
| `npm run import-portfolio -- file.csv` | Import Robinhood/Vanguard/Fidelity CSV into `portfolio.json` |
|
||||
| `npm test` | Run all 61 unit tests |
|
||||
| `npm run test:watch` | Re-run tests on file changes |
|
||||
| `npm run format` | Format all source files with Prettier |
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
## How the Screener Works
|
||||
|
||||
### Phase 1: Core Engine & Soft Scoring
|
||||
Every asset is scored **twice** under different rule sets:
|
||||
|
||||
- **Soft Scoring System:** Transition from "Hard Gates" to a weighted point-based system.
|
||||
- **Market Context Integration:** Automate the `marketContext` parameter by fetching real-time 10Y Treasury Yields.
|
||||
### Market-Adjusted mode
|
||||
Gates derived from live Yahoo Finance benchmarks — reflects what is acceptable in today's market:
|
||||
|
||||
### Phase 2: Personal Finance Features
|
||||
| Gate | Formula |
|
||||
|---|---|
|
||||
| Stock P/E | S&P 500 P/E (via SPY) × 1.5× (or × 1.2× in HIGH rate regime) |
|
||||
| Tech P/E | XLK sector P/E × 1.3× |
|
||||
| REIT min yield | XLRE dividend yield × 0.85× |
|
||||
| Bond min spread | LQD − TNX spread × 0.80× |
|
||||
|
||||
- **Personal Portfolio Tracking:** Implement a `PortfolioManager` to track custom user holdings, monitor unrealized P&L, and calculate weightings relative to total assets.
|
||||
- **Automated Financial Coaching:** Develop an `AdvisorModule` that analyzes the portfolio and provides suggestions (e.g., "Reduce exposure to High-Debt REITs," or "Rebalance to increase Technology allocation").
|
||||
- **Earnings Call Notification System:** \* Integrate an earnings calendar API.
|
||||
- Implement a polling or webhook service to monitor for upcoming calls.
|
||||
- Add a notification service (Email, Push, or CLI log) to alert the user 24 hours prior to a scheduled earnings call.
|
||||
### Fundamental mode
|
||||
Strict Graham/value-investing gates — reflects genuine value regardless of market conditions:
|
||||
|
||||
### Phase 3: Infrastructure & Intelligence
|
||||
| Gate | Value | Rationale |
|
||||
|---|---|---|
|
||||
| Stock P/E | < 15× | Graham's actual rule (trailing earnings) |
|
||||
| Stock PEG | < 1.0 | Lynch standard: PEG > 1.0 = paying full price |
|
||||
| D/E ratio | < 1.5× | Distress typically starts above 2× |
|
||||
| Quick ratio | > 0.8 | Below 0.8 = real liquidity stress |
|
||||
| Bond spread | > 1.5% | Must clear risk-free rate meaningfully |
|
||||
| Bond duration | < 7 years | Rate sensitivity management |
|
||||
|
||||
- **Caching Layer:** Use local JSON caching to reduce API overhead.
|
||||
- **Sentiment Analysis:** Integrate news-scrapers to weight "Buy" signals based on recent headlines.
|
||||
- **Backtesting Module:** Run historical simulations to test strategy performance.
|
||||
### Signals
|
||||
|
||||
| Signal | Meaning |
|
||||
|---|---|
|
||||
| ✅ Strong Buy | Passes both lenses — genuinely good value |
|
||||
| ⚡ Momentum | Passes market-adjusted, holds fundamentally |
|
||||
| ⚠️ Speculation | Passes market-adjusted, fails fundamental — priced for perfection |
|
||||
| 🔄 Neutral | Hold territory in one or both lenses |
|
||||
| ❌ Avoid | Fails both |
|
||||
|
||||
---
|
||||
|
||||
_Maintained by: AI Collaborator_
|
||||
## Sector Overrides
|
||||
|
||||
Sector-specific rules apply in both modes (not just inflated):
|
||||
|
||||
| Sector | Key adjustments |
|
||||
|---|---|
|
||||
| **Technology** | P/E up to 35×, D/E up to 2.0 (buybacks), FCF weight raised |
|
||||
| **REIT** | P/E/PEG disabled, scored on dividend yield + P/FFO proxy |
|
||||
| **Financial** | D/E disabled, scored on ROE (≥12%) + P/B (< 1.5×) |
|
||||
| **Energy** | FCF primary signal (weight 4), dividend yield scored |
|
||||
| **Healthcare** | Revenue growth primary, P/E up to 25× (R&D burn) |
|
||||
| **Communication** | FCF primary (META/GOOGL platforms), P/E up to 25× |
|
||||
| **Consumer Staples** | Margin/ROE focus, low revenue growth expectations (2–5%) |
|
||||
| **Consumer Discretionary** | Revenue growth primary, P/E up to 25× |
|
||||
|
||||
---
|
||||
|
||||
## API Server
|
||||
|
||||
```
|
||||
GET /health → { status: "ok" }
|
||||
POST /api/screen → screen tickers
|
||||
body: { tickers: string[] }
|
||||
GET /api/screen/catalysts → Yahoo news → { tickers, stories }
|
||||
GET /api/finance/portfolio → portfolio advice + net worth
|
||||
GET /api/finance/market-context → live benchmark data
|
||||
```
|
||||
|
||||
Set `CLIENT_ORIGIN` env var to allow a different CORS origin (default: `http://localhost:5173`).
|
||||
|
||||
---
|
||||
|
||||
## Personal Finance
|
||||
|
||||
Edit `portfolio.json` with your holdings (or import from a broker CSV):
|
||||
|
||||
```json
|
||||
{
|
||||
"holdings": [
|
||||
{ "ticker": "AAPL", "shares": 10, "costBasis": 150.00, "source": "Robinhood", "type": "stock" },
|
||||
{ "ticker": "VOO", "shares": 8, "costBasis": 380.00, "source": "Vanguard", "type": "etf" },
|
||||
{ "ticker": "BTC-USD", "shares": 0.25, "costBasis": 45000, "source": "Coinbase", "type": "crypto" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`npm run finance` (CLI) or `GET /api/finance/portfolio` (API) screens your holdings and cross-references the screener signal with your gain/loss position to give hold/sell/add advice.
|
||||
|
||||
### SimpleFIN (optional — live bank/brokerage balances)
|
||||
|
||||
1. Get your setup token from [beta-bridge.simplefin.org](https://beta-bridge.simplefin.org)
|
||||
2. Add to `.env`: `SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...`
|
||||
3. On first run the Access URL is claimed and saved to `.env` automatically
|
||||
|
||||
### Importing broker holdings
|
||||
|
||||
```bash
|
||||
npm run import-portfolio -- ~/Downloads/robinhood_holdings.csv
|
||||
npm run import-portfolio -- ~/Downloads/vanguard_holdings.csv
|
||||
```
|
||||
|
||||
Broker is auto-detected from CSV headers. Multiple imports merge into `portfolio.json`.
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
bin/
|
||||
screen.js CLI screener entry point
|
||||
finance.js CLI personal finance entry point
|
||||
import-portfolio.js Broker CSV importer
|
||||
server.js Fastify API server entry point
|
||||
|
||||
scripts/
|
||||
summary-reporter.js Custom node:test reporter (silent pass, shows failures + summary)
|
||||
|
||||
src/
|
||||
config/
|
||||
ScoringConfig.js All gates, weights, thresholds (single source of truth)
|
||||
constants.js Shared enums: SIGNAL, ASSET_TYPE, SECTOR, SCORE_MODE, REGIME
|
||||
|
||||
market/
|
||||
YahooClient.js Wraps yahoo-finance2 v3 with retry + backoff
|
||||
BenchmarkProvider.js Fetches live benchmarks → marketContext (1-hour cache)
|
||||
MarketRegime.js Derives inflated gate overrides from live data + rate regime
|
||||
|
||||
screener/
|
||||
ScreenerEngine.js Orchestrates fetch → score × 2.
|
||||
screenTickers() → pure data (server/CLI)
|
||||
screenWithProgress() → with stdout progress (CLI only)
|
||||
DataMapper.js Normalises Yahoo payload → flat asset objects
|
||||
Uses trailingPE (not forwardPE). Preserves negative FCF.
|
||||
RuleMerger.js Merges base rules + sector overrides + MarketRegime
|
||||
assets/ Stock, Etf, Bond data containers
|
||||
scorers/ StockScorer, EtfScorer, BondScorer (stateless)
|
||||
|
||||
analyst/
|
||||
CatalystAnalyst.js Extracts tickers from Yahoo Finance news headlines
|
||||
|
||||
finance/
|
||||
clients/
|
||||
SimpleFINClient.js Auth + account fetching via Basic Auth header
|
||||
PersonalFinanceAnalyzer.js Net worth, cash vs investments, spending
|
||||
PortfolioAdvisor.js Hold/sell/add advice per holding
|
||||
PortfolioImporter.js Parses Robinhood/Vanguard/Fidelity CSV
|
||||
|
||||
reporters/
|
||||
HtmlReporter.js render() → string | generate() → file (CLI)
|
||||
FinanceReporter.js render() → string | generate() → file (CLI)
|
||||
|
||||
server/
|
||||
app.js Fastify app factory (buildApp)
|
||||
routes/
|
||||
screener.js POST /api/screen, GET /api/screen/catalysts
|
||||
finance.js GET /api/finance/portfolio, GET /api/finance/market-context
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# .env
|
||||
SIMPLEFIN_ACCESS_URL=https://user:pass@beta-bridge.simplefin.org/simplefin
|
||||
# or on first run:
|
||||
SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...
|
||||
|
||||
# Optional server config
|
||||
PORT=3000
|
||||
HOST=0.0.0.0
|
||||
CLIENT_ORIGIN=http://localhost:5173 # CORS allowed origin for SvelteKit UI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
61 unit tests, no external test framework:
|
||||
|
||||
```bash
|
||||
npm test # summary output: "✅ 61 tests: 61 passed (0.02s)"
|
||||
npm run test:watch # verbose spec output for development
|
||||
```
|
||||
|
||||
Pre-commit: Prettier (auto-format staged files) + full test suite.
|
||||
Pre-push: full test suite.
|
||||
|
||||
Reference in New Issue
Block a user