market_screener/README.md

# Market Screener

A personal stock screener and portfolio tracker. Scores stocks, ETFs, and bonds under two lenses — **Market-Adjusted** (what's acceptable in today's market) and **Fundamental** (strict Graham value-investing) — then compares them to produce an actionable signal. Comes with a live SvelteKit dashboard.

---

## Table of Contents

- [Developer Setup](#developer-setup)
- [Environment Variables](#environment-variables)
- [Commands](#commands)
- [Running Tests](#running-tests)
- [Project Structure](#project-structure)
- [User Guide](#user-guide)
- [API Testing with Bruno](#api-testing-with-bruno)

---

## Developer Setup

### Prerequisites

- **Node.js 20+** (v22 recommended)
- **npm 10+**

**Check your versions:**
```bash
node --version   # Should output v20.x.x or higher
npm --version    # Should output 10.x.x or higher
```

**Not on Node 20+?** See [NODE_VERSION_FIX.md](./NODE_VERSION_FIX.md) for upgrade instructions.

### Install

```bash
# Install server dependencies
npm install

# Install UI dependencies (first time only)
npm run ui:install
```

### Start

```bash
npm run dev
```

This starts both the API server on **port 3000** and the SvelteKit UI on **port 5173** concurrently. Open [http://localhost:5173](http://localhost:5173).

To run the API server alone:

```bash
npm run server
```

---

## Environment Variables

Create a `.env` file in the project root. None are required to run the app — it works with Yahoo Finance data out of the box. Optional keys unlock additional features.

### `ANTHROPIC_API_KEY` — LLM news analysis *(optional)*

Powers the **Analyze** button on each screener section. Without this key the button is disabled.

1. Go to [console.anthropic.com](https://console.anthropic.com)
2. Create an API key under **API Keys**
3. Add to `.env`:

```env
ANTHROPIC_API_KEY=sk-ant-...
```

### `SIMPLEFIN_SETUP_TOKEN` — Live bank/brokerage balances *(optional)*

Powers the personal finance section of the Portfolio page (net worth, account balances, spending breakdown).

1. Go to [beta-bridge.simplefin.org](https://beta-bridge.simplefin.org) and create a Setup Token
2. Add to `.env`:

```env
SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...
```

On first request the token is claimed automatically and the resulting Access URL is saved back to `.env` as `SIMPLEFIN_ACCESS_URL`. Subsequent restarts use the Access URL directly.

### `API_KEY` — Bearer token auth *(optional)*

When set, every API route requires `Authorization: Bearer <key>`. Useful when the server is exposed to a network. `/health` and OPTIONS preflight are exempt.

```env
API_KEY=your-secret-key
```

### `CLIENT_ORIGIN` — CORS allowed origin *(optional)*

Defaults to `http://localhost:5173`. Change if the UI is served from a different origin.

```env
CLIENT_ORIGIN=https://yourdomain.com
```

### `EDGAR_USER_AGENT` — SEC filings poller *(recommended)*

The news pipeline polls SEC EDGAR for 8-K / SC 13D / S-4 / DEFM14A filings.
The SEC requires a descriptive User-Agent with contact info:

```env
EDGAR_USER_AGENT=market-screener/1.0 you@example.com
```

### `DISCORD_WEBHOOK_URL` — Daily digest alerts *(optional)*

The daily change digest (`npm run digest:daily`) posts signal flips + their
news catalysts to Discord. Create: channel → Settings → Integrations →
Webhooks → New Webhook → copy URL. Paste it RAW (no quotes, no escaping).
Forum channels are supported (each digest becomes a dated post).
Test with `npm run discord:test`.

```env
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
```

### `NEWS_PRWIRE_FEEDS` — Override press-release RSS feeds *(optional)*

Comma-separated RSS URLs. Defaults to GlobeNewswire + PR Newswire. Only
needed if a default feed goes stale or you want to add one.

### `NEWS_POLL` — Disable in-server news polling *(optional)*

Set `NEWS_POLL=off` if you prefer running `npm run news:poll` from cron
instead of polling inside the server (EDGAR 10 min, PR-wire 15 min).

### Complete `.env` example

```env
ANTHROPIC_API_KEY=sk-ant-...
SIMPLEFIN_SETUP_TOKEN=aHR0cHM6Ly...
API_KEY=optional-secret
CLIENT_ORIGIN=http://localhost:5173
EDGAR_USER_AGENT=market-screener/1.0 you@example.com
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
```

---

## Commands

| Command | Description |
|---|---|
| `npm run dev` | Start API (port 3000) + UI (port 5173) together |
| `npm run server` | Start API server only |
| `npm run ui:install` | Install UI dependencies (first time / after `git pull`) |
| `npm test` | Run all unit + integration tests |
| `npm run test:watch` | Watch mode — re-run on file changes |
| `npm run typecheck` | TypeScript type check without emitting |
| `npm run format` | Format all source files with Prettier |
| `npm run format:check` | Check formatting without writing (used in CI) |
| `npm run lint` | Run ESLint on all TypeScript files |
| `npm run lint:fix` | Auto-fix ESLint issues |
| `npm run screen:daily` | Screen watchlist + holdings, write signal snapshots (cron at market close) |
| `npm run news:poll` | One-shot news poll: EDGAR + PR wires → news DB (cron alternative) |
| `npm run digest:daily` | Daily change digest: signal flips + catalysts → terminal/Discord (run after screen:daily) |
| `npm run discord:test` | Send a fake digest to verify the Discord webhook |

**Recommended cron (weekdays, market close):**

```
30 16 * * 1-5  cd /path/to/market_screener && npm run screen:daily && npm run digest:daily
```

---

## Running Tests

```bash
npm test
```

Uses Node's built-in `node:test` runner — no external framework. **114 test cases** across 9 files cover:

| Test File | Tests | Coverage |
|-----------|-------|----------|
| `app.test.ts` | 9 | App bootstrap, CORS, health endpoints |
| `screener-controller.test.ts` | 10 | `/api/screen` endpoints |
| `screener-engine.test.ts` | 11 | Screening orchestration logic |
| `stock-scorer.test.ts` | 13 | Stock valuation gates |
| `etf-scorer.test.ts` | 17 | ETF fund gates |
| `bond-scorer.test.ts` | 16 | Bond credit analysis |
| `portfolio-advisor.test.ts` | 12 | Portfolio advice logic |
| `portfolio-controller.test.ts` | 12 | Portfolio endpoints |
| `calls-controller.test.ts` | 14 | Market calls endpoints |

### Pre-Commit & Pre-Push Hooks

On `git commit`, the **pre-commit hook** automatically:

1. **Formats** all files with Prettier
2. **Lints & fixes** staged files with ESLint
3. **Runs tests** to catch errors early

On `git push`, the **pre-push hook** runs tests again for safety.

---

## Project Structure

**Phase 9: Domain-Driven Architecture** (completed)

```
bin/
  server.ts              API server entry point

server/
  app.ts                 Fastify app factory — wires DI, rate limiting, auth hook
  domains/               Domain-driven structure (shared, screener, portfolio, calls, finance)
    shared/              Infrastructure & cross-domain utilities
      adapters/          YahooFinanceClient, AnthropicClient, SimpleFINClient
      services/          BenchmarkProvider, CatalystAnalyst, LLMAnalyst
      entities/          Asset, Stock, Etf, Bond
      persistence/       MarketCallRepository, PortfolioRepository
      config/            ScoringConfig (gates/weights), constants
      scoring/           MarketRegime, scoring overrides
      types/             TypeScript interfaces (one file per domain)
    screener/            Stock/ETF/Bond filtering & scoring
      ScreenerEngine.ts  Orchestrates: fetch → score × 2 (fundamental + inflated)
      scorers/           StockScorer, EtfScorer, BondScorer
      transform/         DataMapper, RuleMerger
    portfolio/           Holdings management & investment advice
      PortfolioAdvisor.ts  Cross-references holdings with screener signals
    calls/               Market call tracking & earnings calendar
      CalendarService.ts  Earnings calendar logic
    finance/             Portfolio metrics & reporting

ui/
  src/
    routes/              SvelteKit pages: /, /portfolio, /calls, /safe-buys
    lib/
      components/        Shared UI components organized by domain
      stores/            Svelte 5 reactive stores
      api/               Fetch wrappers for each API domain
    styles/              Global SCSS design tokens and partials

tests/                   Unit + integration tests (9 files, 114 test cases)
                         Controllers, services, scorers fully covered

portfolio.json           Your holdings (gitignored — create manually or via the UI)
market-calls.json        Persisted market thesis calls (gitignored)
.benchmark-cache.json    Benchmark data cache — survives server restart (gitignored)
```

See **[CLAUDE.md](./CLAUDE.md)** for detailed architecture and **[PHASES.md](./PHASES.md)** for the complete roadmap.

---

## User Guide

### Screener tab

The main view. On load it automatically fetches today's financial news, extracts the most-mentioned tickers, and screens them.

#### Market context strip

The row of chips at the top shows live benchmark data fetched from Yahoo Finance:

| Chip | Meaning |
|---|---|
| 10Y | 10-year Treasury yield — the risk-free rate |
| VIX | Volatility index — market fear gauge |
| S&P | S&P 500 index price |
| S&P P/E | Trailing P/E of the S&P 500 (via SPY) |
| Tech P/E | Trailing P/E of the tech sector (via XLK) |
| REIT Yld | REIT dividend yield (via XLRE) |
| IG Sprd | Investment-grade bond spread above risk-free (LQD − TNX) |
| Rates | Rate regime: LOW / NORMAL / HIGH (based on 10Y yield) |
| Vol | Volatility regime: LOW / NORMAL / HIGH (based on VIX) |

The rate regime affects how strict the Market-Adjusted gates are — in a HIGH rate environment the P/E multiplier compresses and bond spreads tighten.

#### Signal Summary table

Quick overview of all screened tickers with their signal, market-adjusted verdict, fundamental verdict, market cap tier, and growth style.

#### Per-asset detail tables

Expand each section (STOCK / ETF / BOND) for full metrics: P/E, PEG, ROE, margins, FCF yield, D/E, analyst consensus, DCF intrinsic value, 52-week movement, and more.

#### Analyze button

Runs an Anthropic LLM over the latest Yahoo Finance news for assets in that section. Returns a sentiment summary, affected industries, and related tickers to watch. Requires `ANTHROPIC_API_KEY`.

#### Search tickers

Click **Search tickers** to screen any custom list — type tickers comma or space separated and press Enter or click Screen.

#### Signals explained

| Signal | What it means |
|---|---|
| ✅ Strong Buy | Passes both Market-Adjusted AND Fundamental gates — genuine value at current prices |
| ⚡ Momentum | Passes Market-Adjusted, holds fundamentally — good in the current market but not a bargain |
| ⚠️ Speculation | Passes Market-Adjusted, fails Fundamental — priced for perfection, high risk |
| 🔄 Neutral | Borderline in one or both lenses — hold, no clear edge |
| ❌ Avoid | Fails both lenses |

#### How scoring works

Every asset is scored twice:

**Market-Adjusted** gates move with the market. The stock P/E gate = SPY trailing P/E × 1.5 (compresses to × 1.2 in a HIGH rate regime). Tech P/E = XLK P/E × 1.3. This reflects what the market is currently willing to pay.

**Fundamental** gates are fixed Graham/value-investing standards that never change:

| Gate | Threshold | Rationale |
|---|---|---|
| Stock P/E | < 15× | Graham's actual rule |
| Stock PEG | < 1.0 | Lynch: PEG > 1.0 = paying full price |
| D/E ratio | < 1.5× | Distress typically starts above 2× |
| Quick ratio | > 0.8 | Below 0.8 = real liquidity stress |

Sector overrides apply in both modes — e.g. tech stocks allow P/E up to 35× and D/E up to 2.0, REITs are scored on yield rather than P/E.

---

### Portfolio tab

Track your holdings and get hold/sell/add advice cross-referenced with screener signals.

**Adding holdings** — click **+ Add Holding** and fill in ticker, shares, cost basis, asset type, and source broker. Holdings are saved to `portfolio.json` on disk.

**Inline editing** — click the ✎ pencil icon on any row to edit shares, cost basis, type, or source directly in the table.

**Advice column** — each holding is screened live and the signal is combined with your gain/loss position:

| Situation | Advice |
|---|---|
| ✅ Strong Buy signal | Hold & Add |
| ⚡ Momentum + > 30% gain | Consider partial profit-taking |
| ⚠️ Speculation + > 20% gain | Reduce position |
| ❌ Avoid signal + in profit | Sell (Take Profits) |
| ❌ Avoid signal + at a loss | Sell (Cut Loss) |
| Crypto | Hold / Review position (no fundamental scoring) |

**Personal finance section** *(requires SimpleFIN)* — when configured, the page also shows net worth, total assets vs liabilities, cash vs investments ratio, monthly income/spend, account balances, and a spending breakdown by category for the last 30 days.

---

### Market Calls tab

Record and track quarterly investment theses from the day you make the call.

**Creating a call** — click **＋ New Call** and fill in:
- **Title** — e.g. "Q3 2025 — Rate pivot & tech rotation"
- **Quarter** — the quarter this thesis applies to
- **Thesis** — the macro reasoning behind the call (min 10 characters)
- **Tickers** — the assets you're watching for this thesis

When saved, the current price and signal for each ticker are snapshotted automatically.

**Viewing performance** — click any call card to see the current price and signal for each ticker alongside the original snapshot, so you can measure how the thesis played out.

**Calendar** — shows upcoming earnings dates, ex-dividend dates, and dividend payment dates for all tickers across your active calls.

---

### Safe Buys tab

A filtered view showing only tickers with a **✅ Strong Buy** signal across both lenses. A quick watchlist of assets passing the strictest criteria in the current market.

---

### API rate limits

`/api/screen`, `/api/screen/catalysts`, and `/api/analyze` are capped at **10 requests per minute** per IP. All other routes allow 60 per minute.

---

## API Testing with Bruno

### What is Bruno?

[Bruno](https://www.usebruno.com/) is a lightweight, open-source API client that's Git-friendly and perfect for testing REST APIs. It stores collections as plain text files instead of JSON blobs, making them easy to version control and collaborate on.

### Installing Bruno

#### macOS (via Homebrew)
```bash
brew install bruno
```

#### macOS (Direct Download)
1. Visit [usebruno.com/downloads](https://www.usebruno.com/downloads)
2. Download the macOS version
3. Drag `Bruno.app` to Applications folder

#### Windows
1. Visit [usebruno.com/downloads](https://www.usebruno.com/downloads)
2. Download the Windows installer (.exe)
3. Run the installer and follow the prompts
4. Or via Chocolatey: `choco install bruno`

#### Linux (Ubuntu/Debian)
```bash
# Add Bruno repository
curl -1sLf 'https://dl.usebruno.com/install.sh' | sudo bash

# Install
sudo apt-get install bruno
```

#### Linux (Fedora/RHEL)
```bash
curl -1sLf 'https://dl.usebruno.com/install.sh' | sudo bash
sudo dnf install bruno
```

### Installing Bruno CLI (brucli)

For running tests from the command line without the GUI:

#### macOS
```bash
brew install brucli
```

#### Windows
```bash
choco install bruno-cli
```

#### Linux
```bash
curl -1sLf 'https://dl.usebruno.com/install.sh' | sudo bash
```

### Importing the API Collection

#### Method 1: Import via Bruno GUI (Easiest)

1. **Open Bruno**
2. **File → Import Collection**
3. **Select** `api_collections/market-screener.postman_collection.json`
4. **Choose location** where to save the converted collection (e.g., `api_collections/market-screener`)
5. **Click Import** — Bruno automatically converts and structures the collection

#### Method 2: Import via Bruno CLI

```bash
# Navigate to the project root
cd market-screener

# Import the Postman collection
bru import api_collections/market-screener.postman_collection.json -o api_collections/market-screener

# Output: Collection imported to api_collections/market-screener/
```

#### Method 3: Convert Postman to Bruno Format (Manual)

If you prefer to manually convert the collection:

```bash
# Install conversion dependencies (if needed)
pip install requests

# Run the conversion script
python3 api_collections/convert_postman_to_bruno.py \
  api_collections/market-screener.postman_collection.json \
  api_collections/market-screener
```

### Running Tests

#### Via Bruno GUI

1. **Open the imported collection** in Bruno
2. **Set the `baseUrl` variable** (default: `http://localhost:3000`)
3. **Click the Play button** to run all tests
4. **View results** for each request in the UI

#### Via Bruno CLI (brucli)

```bash
# Navigate to the collection directory
cd api_collections/market-screener

# Run all tests in the collection
bru run

# Run with specific environment
bru run --env local

# Run with output format
bru run --output json > test-results.json

# Run specific test file
bru run "Screener/Screen - Mixed.bru"
```

### Collection Structure

After import, you'll have:

```
api_collections/market-screener/
├── bruno.json                    # Collection metadata
├── Health/
│   └── Health Check.bru
├── Screener/
│   ├── Screen - Mixed.bru
│   ├── Screen - Tech Stocks.bru
│   ├── Screen - REIT.bru
│   ├── Validation empty tickers.bru
│   ├── Validation 50 plus tickers.bru
│   └── Get Catalysts.bru
├── Market Context/
│   └── Get Market Context.bru
├── Portfolio/
│   ├── Add Holding AAPL.bru
│   ├── Add Holding VOO.bru
│   ├── Add Holding BTC-USD.bru
│   ├── Add Holding Validation.bru
│   ├── Get Portfolio.bru
│   ├── Remove Holding AAPL.bru
│   └── Remove Holding Non-existent.bru
├── Market Calls/
│   ├── List Calls.bru
│   ├── Create Market Call.bru
│   ├── Get Call by ID.bru
│   ├── Get Call Non-existent.bru
│   ├── Get Earnings Calendar.bru
│   ├── Get Calendar Specific Tickers.bru
│   ├── Create Call Validation.bru
│   ├── Delete Call.bru
│   └── Delete Call Already Deleted.bru
└── LLM Analysis/
    ├── Analyze Tickers.bru
    └── Analyze Validation.bru
```

### Configuration

#### Setting Variables

Variables are stored in `bruno.json` and can be overridden per request:

**Default variables:**
- `baseUrl`: `http://localhost:3000`
- `callId`: (auto-populated by Create Market Call request)

To change variables in the GUI:
1. Right-click collection → **Settings**
2. Click **Variables** tab
3. Edit `baseUrl` or other variables
4. Click **Save**

#### Environment Files

Create a `.env.bruno` file in the collection directory for local overrides:

```env
baseUrl=http://localhost:3000
apiKey=your-secret-key
```

### Common Workflows

#### 1. Test the full API flow

```bash
cd api_collections/market-screener
bru run
```

#### 2. Test just the Screener endpoints

```bash
cd api_collections/market-screener
bru run "Screener"
```

#### 3. Test and save results

```bash
cd api_collections/market-screener
bru run --output json > test-results-$(date +%Y%m%d).json
```

#### 4. Continuous testing (while developing)

```bash
# Terminal 1: Run the API server
npm run dev

# Terminal 2: Watch and run tests every 5 seconds
cd api_collections/market-screener
watch -n 5 'bru run'
```

### Troubleshooting

#### "You can run only at the root of a collection" error

Make sure you're in the correct directory:

```bash
# ❌ Wrong — project root
cd market-screener
bru run

# ✅ Correct — collection root
cd api_collections/market-screener
bru run
```

#### Variables not found

Verify variable names in `bruno.json`:

```bash
# Check variables
cat api_collections/market-screener/bruno.json | grep -A 10 "vars"
```

#### Tests failing with "undefined" errors

Common causes:
- Variable name mismatch (case-sensitive)
- Server not running on the expected port
- Port conflict (try `lsof -i :3000` to check)

### Postman vs Bruno

| Feature | Postman | Bruno |
|---------|---------|-------|
| **Download Size** | ~380MB | ~50MB |
| **Collection Format** | Single JSON blob | Plain text `.bru` files |
| **Git-Friendly** | ❌ Binary | ✅ Text-based, diffable |
| **API Response** | UI-only | CLI + GUI |
| **Cost** | Free tier + paid | ✅ Completely free |
| **IDE Integration** | None | Can edit `.bru` files directly |

### References

- **Bruno Docs**: [docs.usebruno.com](https://docs.usebruno.com)
- **Bruno GitHub**: [github.com/usebruno/bruno](https://github.com/usebruno/bruno)
- **Postman Collection**: `api_collections/market-screener.postman_collection.json`