Operator's guide
How SignalForge works
Turn market-research theories into signal modules, let an AI compose them into experiments, and simulate them against real history — with statistical honesty enforced in code, not by good intentions.
01The one idea
Search hard enough through market data and you will always find a pattern that looks profitable. Most are noise you got lucky on — the more combinations you try, the more certain you are to be fooled. This is multiple-hypothesis testing, the number-one way quant researchers deceive themselves.
SignalForge's entire design exists to make that self-deception structurally impossible. You cannot see a result without also seeing how many things were tried to find it, a Sharpe ratio mathematically deflated for that search, and an out-of-sample track record.
The mechanism, in one line
Every experiment writes a row to a tamper-proof trial ledger before it runs. The trial count feeds the Deflated Sharpe Ratio, which discounts performance for how much you searched. Nothing touches capital until it survives a quarantine on live, unseen data.
02The pipeline
Six stages, each a module. Data flows in order; the honesty checks sit between simulation and anything you're allowed to believe.
Market data
Prices, macro series, uncertainty indices, fund flows, news — each datum stamped with when it happened and when you could have known it.
02 · computeSignals
Each research theory is a module that emits one normalized signal with a confidence and a regime context, on a shared contract.
03 · simulateTrials
Purged walk-forward backtests with pessimistic retail costs. Every run is a ledgered trial — no ledger row, no simulation.
04 · exploreOrchestrator
A deterministic sweep plus an AI agent compose signal combinations across regimes and must attach a mechanism story to anything surfaced.
05 · quarantinePatterns
Discovered patterns are paper-traded on live data. Promotion toward capital is decided by code, not by you.
06 · reportReports
Weekly write-ups: current signals, families searched, and what earned — or lost — its place, deflated stats attached.
03The screens
What each page in the left nav is for — click any card to open it.
Dashboard
Home. At-a-glance widgets — latest signals, recent activity. Where / redirects after login.
/dataData
What's been ingested: price bars, macro and index series, coverage and freshness. Populated by pnpm backfill.
/signalsSignals
Current output of each signal family — value, confidence, and regime context — from the ingested data.
/orchestratorOrchestrator
Kick off a sweep or AI-driven exploration that composes families into experiments across historical regimes.
/trialsTrials
The ledger. Every experiment ever run, raw vs. deflated Sharpe, trial count, and out-of-sample window.
/patternsPatterns
The quarantine pen. Patterns moving through candidate → quarantined → live-eligible, criteria shown.
/proposalsProposals
New signal modules the AI has proposed from new data. A human review gate promotes them into the codebase.
/reportsReports
Generated weekly reports — the narrative layer over the ledger. Read these; don't stare at raw trials.
/chatChat
AI chat with retrieval over your research context. Anthropic for chat, OpenAI for embeddings (both wired).
04The promotion lifecycle
A pattern cannot jump straight to “trade this.” It climbs a ladder, and each rung is gated by code-enforced criteria — you cannot promote something by hand because you like it.
Found in a sweep
Surfaced by the orchestrator with a deflated Sharpe and a mechanism story. Interesting, unproven, in-sample only.
Paper-traded live
Runs forward on genuinely unseen data. The honest out-of-sample test — the one number that can't be overfit.
Earned its place
Survived quarantine on the code's terms. Only now is it a pattern you may act on. It can still be demoted.
05Signal families
Each family is one research theory as a self-contained module. Three are live; two are planned. Adding a family is a module, not a rewrite.
| Family | Theory | Data | Status |
|---|---|---|---|
| uncertainty | Economic & geopolitical policy uncertainty (Baker–Bloom–Davis EPU, Caldara–Iacoviello GPR) | Published free indices | live |
| flows | Inelastic Markets Hypothesis (Gabaix & Koijen) — flows move prices | ICI / ETF fund flows, buybacks | live |
| narrative | Narrative economics (Shiller) + LLM text analysis | GDELT / news, embeddings | live |
| networks | Production networks / granular origins (Acemoglu) | Filings, trade data | later |
| ml-pricing | ML asset pricing (Gu–Kelly–Xiu) | Derived from market data | later |
06Honesty ground rules
Non-negotiable and enforced by the code, not by discipline. They are the reason to trust anything this tool tells you.
No simulation without a ledger row first. A trial must be recorded before it can run — the search is on the record before you see the result.
No pattern shown without its receipts. Trial count, deflated Sharpe, and out-of-sample window travel with every result. No view hides them.
Promotion is code-enforced. candidate → quarantined → live-eligible advances only on met criteria — never by manual override.
Costs are pessimistic by default. Retail frictions are assumed high, so live disappointment is the exception, not the rule.
Point-in-time correctness. Every datum stores event_time and available_time; sims read only what was knowable as of the simulated date. No lookahead.
07Operating it
Three commands do the heavy lifting. Run them from the project directory.
pnpm devStart the app (serves on port 3012).pnpm backfill [--target=…] [--only=SYM] [--from=DATE]Ingest real history. Targets: bars, macro, index, news, flows, or all. Re-run after integration tests (they wipe shared tables).pnpm demoDrive the full loop end-to-end: sweep → deflated stats → quarantine → report. Fastest way to populate the dashboards.08Data & keys
The budget is “free to cheap.” The deterministic engine needs zero keys; keys only buy the AI layer and richer data. Paid upgrades are adapter swaps, never rewrites.
| Source / key | Powers | Cost | Status |
|---|---|---|---|
| Yahoo (default) | Price bars — the market-data spine | Free, keyless | active |
| FRED | Macro series for uncertainty | Free (keyless CSV) | not needed |
| ANTHROPIC_API_KEY | LLM orchestrator, AI chat | Paid, usage-based | wired |
| OPENAI_API_KEY | RAG embeddings for chat | Paid, usage-based | wired |
| Tiingo | Alternate price source | Free tier / ~$10–30 mo | unused |
09Glossary
- Deflated Sharpe Ratio (DSR)
- A Sharpe ratio adjusted downward for how many strategies you tried and for non-normal returns (Bailey & López de Prado). The more you searched, the more it discounts.
- Trial ledger
- An append-only record (sim_trials) of every experiment, so the trial count feeding the DSR can't be quietly reset — the antidote to self-deception.
- Purged walk-forward validation
- Backtesting that trains on the past and tests on the future in rolling windows, purging data near the boundary so information can't leak backward.
- Block bootstrap
- Resampling returns in contiguous blocks (not single days) to preserve autocorrelation, giving honest confidence intervals for time-series performance.
- event_time vs. available_time
- When something happened vs. when you could first have known it. Sims read only available_time ≤ as_of, so no backtest peeks at unpublished data.