market_screener/DATABASE_SECURITY.md

Database Security & Hardening Guide

Executive Summary

Your codebase is currently safe from SQL injection because it uses better-sqlite3's parameterized queries correctly. However, the new abstraction layers below provide:

1. Type-safe query construction (QueryBuilder)
2. Audit logging for compliance (QueryAudit)
3. Statement caching for performance (DatabaseConnection)
4. Transaction support for atomic operations
5. Clear separation of concerns between data access and business logic

---

Current Safety Assessment

✅ SQL Injection: Safe
Your code uses parameterized queries (? placeholders) throughout:

// SAFE — all values in parameter array
this.db.prepare('SELECT * FROM holdings WHERE ticker = ?').get(id);

// SAFE — INSERT uses parameters
this.db.prepare('INSERT INTO holdings (...) VALUES (?, ?, ?, ?, ?)').run(
  ticker, shares, costBasis, type, source
);

The better-sqlite3 library handles parameter binding internally — user input never touches the SQL string.

---

New Architecture: QueryBuilder + DatabaseConnection

Problem Solved

While your code is secure, it has several maintainability issues:

1. Hardcoded SQL strings scattered across repositories
2. No audit trail — impossible to trace mutations for compliance
3. No statement caching — compiler recompiles the same queries repeatedly
4. No type safety — column names are strings, easy to typo
5. Mixed concerns — repositories call .prepare() directly; hard to add logging/caching globally

Solution: Three Layers

┌─────────────────────────────────────────────────┐
│ Controllers / Services (business logic)         │
└────────────────┬────────────────────────────────┘
                 │
                 ↓
┌─────────────────────────────────────────────────┐
│ DatabaseConnection (timing, logging, caching)   │
├─────────────────────────────────────────────────┤
│ - Execute queries via QueryBuilder              │
│ - Log to QueryAudit                             │
│ - Cache prepared statements                     │
│ - Measure execution time                        │
│ - Support transactions                          │
└────────────────┬────────────────────────────────┘
                 │
                 ↓
┌─────────────────────────────────────────────────┐
│ QueryBuilder (type-safe, column-validated)      │
├─────────────────────────────────────────────────┤
│ - Whitelist column/table names                  │
│ - Build SQL with validated identifiers          │
│ - Keep all user input in parameter array        │
│ - Fluent API for clarity                        │
└────────────────┬────────────────────────────────┘
                 │
                 ↓
┌─────────────────────────────────────────────────┐
│ QueryAudit (compliance trail)                   │
├─────────────────────────────────────────────────┤
│ - Log every query: timestamp, SQL, params       │
│ - Track READ / WRITE / DELETE actions           │
│ - Measure performance                           │
│ - Generate audit reports                        │
└────────────────┬────────────────────────────────┘
                 │
                 ↓
┌─────────────────────────────────────────────────┐
│ better-sqlite3 (SQLite execution)               │
│ (parameterized → injection-safe)                │
└─────────────────────────────────────────────────┘

---

Usage Examples

QueryBuilder — Type-Safe Query Construction

All column and table names are validated against a whitelist. User input stays in the parameter array.

SELECT

// Safe: columns validated, params isolated
const qb = new QueryBuilder('holdings')
  .select(['ticker', 'shares', 'cost_basis'])
  .where('type = ? AND shares > ?', ['stock', 10])
  .orderBy('ticker', 'ASC')
  .limit(100);

const rows = db.all(qb);
// Equivalent SQL: SELECT ticker, shares, cost_basis FROM holdings 
//   WHERE type = ? AND shares > ? ORDER BY ticker ASC LIMIT 100
// Params: ['stock', 10]

INSERT

const qb = new QueryBuilder('holdings')
  .insert(['ticker', 'shares', 'cost_basis', 'type', 'source'], 
    ['AAPL', 100, 15000, 'stock', 'Manual']);

db.run(qb);
// Equivalent SQL: INSERT INTO holdings (ticker, shares, cost_basis, type, source) 
//   VALUES (?, ?, ?, ?, ?)
// Params: ['AAPL', 100, 15000, 'stock', 'Manual']

UPDATE

const qb = new QueryBuilder('holdings')
  .update({ shares: 150, cost_basis: 22500 })
  .where('ticker = ?', ['AAPL']);

db.run(qb);
// Equivalent SQL: UPDATE holdings SET shares = ?, cost_basis = ? WHERE ticker = ?
// Params: [150, 22500, 'AAPL']

DELETE

const qb = new QueryBuilder('holdings')
  .delete()
  .where('ticker = ?', ['AAPL']);

db.run(qb);
// Equivalent SQL: DELETE FROM holdings WHERE ticker = ?
// Params: ['AAPL']

DatabaseConnection — Unified Data Access

Wraps better-sqlite3 with logging, caching, and audit trails.

// In server/app.ts
import BetterSqlite3 from 'better-sqlite3';
import { DatabaseConnection, QueryAudit } from './server/db';

const betterSqlite3Db = new BetterSqlite3('./market-screener.db');
const audit = new QueryAudit();
const db = new DatabaseConnection(betterSqlite3Db, { audit, logSlowQueries: 100 });

// Pass `db` to repositories, not the raw better-sqlite3 instance
app.register(ScreenerController, { db });

QueryAudit — Compliance Logging

Automatically logs all queries with timestamps, performance metrics, and parameters.

// In your app
const audit = new QueryAudit(async (entry) => {
  // Optional: send to compliance logger, file, or remote service
  if (entry.action === 'WRITE' && entry.error) {
    console.error(`WRITE failed at ${entry.timestamp}: ${entry.error}`);
  }
});

const db = new DatabaseConnection(betterSqlite3Db, { audit });

// Later: inspect the audit trail
db.printAudit();
// Output:
// === Query Audit Report ===
// Total entries: 42
// Showing last 100 entries:
// 
// [2026-06-05T12:34:56.789Z] READ ✓ (1.23ms) — 5 rows
//   SQL: SELECT ticker, shares, cost_basis FROM holdings WHERE type = ? ORDER BY ticker ASC
//   Params: ["stock"]
// 
// [2026-06-05T12:34:57.456Z] WRITE ✓ (0.89ms) — 1 rows
//   SQL: INSERT INTO holdings (ticker, shares, ...) VALUES (?, ?, ...)
//   Params: ["AAPL", 100, 15000, "stock", "Manual"]

---

Migration Path: Refactor Repositories

Update your repositories to use the new DatabaseConnection and QueryBuilder.

Before (Current)

export class MarketCallRepository {
  constructor(private readonly db: Db) {}

  list(): MarketCall[] {
    const rows = this.db
      .prepare('SELECT * FROM market_calls ORDER BY created_at DESC')
      .all() as CallRow[];
    return rows.map(MarketCallRepository.toCall);
  }

  get(id: string): MarketCall | null {
    const row = this.db
      .prepare('SELECT * FROM market_calls WHERE id = ?')
      .get(id) as CallRow | undefined;
    return row ? MarketCallRepository.toCall(row) : null;
  }
}

After (Hardened)

import { DatabaseConnection, QueryBuilder } from '../db';

export class MarketCallRepository {
  constructor(private readonly db: DatabaseConnection) {}

  list(): MarketCall[] {
    const qb = new QueryBuilder('market_calls')
      .select(['id', 'title', 'quarter', 'date', 'thesis', 'tickers', 'snapshot', 'created_at'])
      .orderBy('created_at', 'DESC');
    
    const rows = this.db.all<CallRow>(qb);
    return rows.map(MarketCallRepository.toCall);
  }

  get(id: string): MarketCall | null {
    const qb = new QueryBuilder('market_calls')
      .select(['id', 'title', 'quarter', 'date', 'thesis', 'tickers', 'snapshot', 'created_at'])
      .where('id = ?', [id]);
    
    const row = this.db.get<CallRow>(qb);
    return row ? MarketCallRepository.toCall(row) : null;
  }

  create({ title, quarter, date, thesis, tickers, snapshot }: CreateCallInput): MarketCall {
    const call = {
      id: randomUUID(),
      title,
      quarter,
      date: date ?? new Date().toISOString().slice(0, 10),
      thesis,
      tickers: tickers ?? [],
      snapshot: snapshot ?? {},
      createdAt: new Date().toISOString(),
    };

    const qb = new QueryBuilder('market_calls')
      .insert(
        ['id', 'title', 'quarter', 'date', 'thesis', 'tickers', 'snapshot', 'created_at'],
        [call.id, call.title, call.quarter, call.date, call.thesis, 
         JSON.stringify(call.tickers), JSON.stringify(call.snapshot), call.createdAt],
      );

    this.db.run(qb);
    return call;
  }

  delete(id: string): boolean {
    const qb = new QueryBuilder('market_calls')
      .delete()
      .where('id = ?', [id]);

    const changes = this.db.run(qb);
    return changes > 0;
  }

  private static toCall(row: CallRow): MarketCall {
    return {
      id: row.id,
      title: row.title,
      quarter: row.quarter,
      date: row.date,
      thesis: row.thesis,
      tickers: JSON.parse(row.tickers),
      snapshot: JSON.parse(row.snapshot),
      createdAt: row.created_at,
    };
  }
}

Key improvements:

1. Explicit columns — Only SELECT the columns you need (better for indexing)
2. Audit trail — Every query is logged automatically
3. Type safety — QueryBuilder validates column names at compile time (via TypeScript)
4. Performance — Prepared statements are cached
5. Clarity — Fluent API makes queries self-documenting

---

Whitelist of Safe Columns

The QueryBuilder validates all column/table names against a whitelist to prevent injection via identifiers:

Holdings Table

• ticker
• shares
• cost_basis
• type
• source

Market Calls Table

• id
• title
• quarter
• date
• thesis
• tickers
• snapshot
• created_at

Adding New Columns

When you add a new column:

1. Update the DDL in server/db/index.ts
2. Add the column name to SAFE_COLUMNS in QueryBuilder.ts
3. Update the relevant domain type in server/types/
4. Update the repository to select/insert the new column

Example: Adding updated_at to market_calls

// 1. Update DDL
const DDL = `
  CREATE TABLE IF NOT EXISTS market_calls (
    ...
    created_at TEXT NOT NULL,
    updated_at TEXT NOT NULL   -- NEW
  );
`;

// 2. Update QueryBuilder.ts
const SAFE_COLUMNS = new Set([
  // ... existing columns
  'updated_at',  // NEW
]);

// 3. Update types
export interface MarketCall {
  // ... existing fields
  updatedAt: string;  // NEW
}

// 4. Update repository
const qb = new QueryBuilder('market_calls')
  .select(['id', 'title', 'quarter', 'date', 'thesis', 'tickers', 'snapshot', 'created_at', 'updated_at'])  // ADDED
  .where('id = ?', [id]);

---

Performance: Statement Caching

DatabaseConnection automatically caches prepared statements. The first execution of a query compiles it; subsequent executions reuse the compiled statement.

// First call: compiles the statement (1.5ms overhead)
const qb1 = new QueryBuilder('holdings').select(['ticker', 'shares']).where('type = ?', ['stock']);
db.all(qb1); // ~1.5ms

// Second call: reuses the cached statement (0.1ms)
const qb2 = new QueryBuilder('holdings').select(['ticker', 'shares']).where('type = ?', ['etf']);
db.all(qb2); // ~0.1ms (same SQL template)

Cache key is the complete SQL string. If you generate different SQL, it creates a new cached statement.

---

Transactions: Atomic Operations

Use db.transaction() to execute multiple queries as a single atomic unit. If any query fails, all are rolled back.

db.transaction(() => {
  // Create a market call
  const qb1 = new QueryBuilder('market_calls')
    .insert(['id', 'title', ...], [callId, 'Q4 Earnings', ...]);
  db.run(qb1);

  // Add related tickers as separate records (if you had a separate table)
  for (const ticker of tickers) {
    const qb2 = new QueryBuilder('call_tickers')
      .insert(['call_id', 'ticker'], [callId, ticker]);
    db.run(qb2);
  }

  // If ANY query fails, BOTH are rolled back
  // If all succeed, both are committed
});

---

Audit Trail: Compliance & Debugging

The QueryAudit class tracks every database operation automatically.

Built-in Features

const audit = db.getAudit();

// Get the last 100 queries
const recent = audit.recent(100);

// Filter by action type
const writes = audit.byAction(AuditAction.WRITE);

// Generate a human-readable report
console.log(audit.report());

Custom Callback

Send audit entries to a logging service or file:

const audit = new QueryAudit(async (entry) => {
  if (entry.action === 'WRITE') {
    // Log all mutations to your compliance logger
    await complianceLogger.log({
      timestamp: entry.timestamp,
      action: entry.action,
      sql: entry.sql,
      params: entry.params,
      rowsAffected: entry.rowsAffected,
    });
  }
});

const db = new DatabaseConnection(betterSqlite3Db, { audit });

---

Slow Query Logging

By default, queries slower than 100ms are logged to console.warn:

const db = new DatabaseConnection(betterSqlite3Db, { logSlowQueries: 100 });
// Output:
// [SLOW QUERY] 234.56ms
//   SELECT ticker, shares, cost_basis FROM holdings WHERE type = ? ORDER BY ticker ASC

Adjust the threshold based on your needs:

new DatabaseConnection(betterSqlite3Db, { logSlowQueries: 50 });  // warn on >50ms
new DatabaseConnection(betterSqlite3Db, { logSlowQueries: 5000 }); // warn on >5s

---

Common Pitfalls & How to Avoid Them

❌ DON'T: Hardcode user input in SQL

// NEVER DO THIS
const ticker = getUserInput(); // e.g. "AAPL'; DROP TABLE holdings; --"
const qb = new QueryBuilder('holdings')
  .select(['ticker', 'shares'])
  .where(`ticker = '${ticker}'`); // SQL INJECTION!

✅ DO: Use parameter placeholders

// ALWAYS DO THIS
const ticker = getUserInput();
const qb = new QueryBuilder('holdings')
  .select(['ticker', 'shares'])
  .where('ticker = ?', [ticker]); // User input is a PARAMETER

❌ DON'T: Use string concatenation for column names

// NEVER DO THIS
const sortCol = getUserInput(); // e.g. "ticker; DELETE FROM holdings; --"
const qb = new QueryBuilder('holdings')
  .select(['ticker', 'shares'])
  .orderBy(`${sortCol}`); // COLUMN NAME INJECTION!

✅ DO: Column names come from your code, not user input

// ALWAYS DO THIS
const sortCol = getUserInput(); // e.g. "ticker"
const ALLOWED_SORT_COLS = ['ticker', 'shares', 'type'];

if (!ALLOWED_SORT_COLS.includes(sortCol)) {
  throw new Error('Invalid sort column');
}

const qb = new QueryBuilder('holdings')
  .select(['ticker', 'shares'])
  .orderBy(sortCol); // Whitelist prevents injection

---

Testing

The new abstractions make testing easier:

import { DatabaseConnection, QueryBuilder, QueryAudit } from '../db';
import BetterSqlite3 from 'better-sqlite3';

describe('MarketCallRepository', () => {
  let db: DatabaseConnection;
  let repo: MarketCallRepository;

  beforeEach(() => {
    // Use in-memory SQLite for tests
    const rawDb = new BetterSqlite3(':memory:');
    rawDb.exec(DDL); // Initialize schema
    db = new DatabaseConnection(rawDb);
    repo = new MarketCallRepository(db);
  });

  it('should insert and retrieve a call', () => {
    const call = repo.create({
      title: 'Q4 Earnings',
      quarter: 'Q4',
      thesis: 'FANG tech breakout',
      tickers: ['GOOGL', 'META', 'NVDA'],
    });

    expect(call.id).toBeDefined();

    const retrieved = repo.get(call.id);
    expect(retrieved).toEqual(call);

    // Verify the audit trail
    const audit = db.getAudit();
    const writes = audit.byAction(AuditAction.WRITE);
    expect(writes.length).toBeGreaterThan(0);
  });
});

---

Summary

Feature	Before	After
SQL injection protection	✅ Parameterized queries	✅ Parameterized + column whitelist
Audit trail	❌ None	✅ QueryAudit with timestamp & params
Performance	⚠️ No statement caching	✅ Automatic statement cache
Type safety	⚠️ String column names	✅ Validated at build time
Testing	⚠️ Hard to mock	✅ Testable via DatabaseConnection
Transactions	⚠️ Manual raw DB calls	✅ db.transaction()
Slow query logging	❌ None	✅ Automatic > 100ms warning

---

Next Steps

1. Review the three new files:

   • server/db/QueryBuilder.ts — Query construction
   • server/db/QueryAudit.ts — Audit logging
   • server/db/DatabaseConnection.ts — Unified access

2. Update server/app.ts to create and wire DatabaseConnection

3. Refactor repositories to use QueryBuilder and DatabaseConnection (see migration examples above)

4. Add tests for repositories using in-memory SQLite

5. Deploy with confidence — you now have audit trails and safeguards