Compare commits
15
Commits
470e243526
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 jarno
|
03a724f7ea | ||
|
jarno
|
1faca2cdda | ||
|
jarno
|
c1c02e2b0d | ||
|
jarno
|
900b70d6df | ||
|
jarno
|
2165c4269d | ||
|
jarno
|
3b30a83f2e | ||
|
jarno
|
f7827fb086 | ||
|
jarno
|
c67eb82bba | ||
|
jarno
|
465e09fc82 | ||
|
jarno
|
fddcc9190a | ||
|
jarno
|
a699bfc0fb | ||
|
jarno
|
dde7e81bf8 | ||
|
jarno
|
7943f48d47 | ||
|
jarno
|
e2eb5dd574 | ||
|
jarno
|
8d3e663e71 |
+12
@@ -0,0 +1,12 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.venv/
|
||||
dist/
|
||||
build/
|
||||
*.egg-info/
|
||||
|
||||
data/ibkr/
|
||||
data/training/
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Project Purpose
|
||||
|
||||
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot will use the IBKR API to facilitate trades, starting with paper trading and allowing a carefully controlled path to real trading later.
|
||||
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot uses the Alpaca Markets API for market data and paper trading, with any future live trading behind a carefully controlled path.
|
||||
|
||||
## Current Direction
|
||||
|
||||
@@ -19,8 +19,10 @@ Keep these areas loosely separated in code and documentation. Avoid coupling the
|
||||
|
||||
- Use Python for implementation.
|
||||
- Use `mise` to control the Python version.
|
||||
- Use `uv` to manage Python packages and virtual environments.
|
||||
- The local Python version is pinned in `.mise.toml`.
|
||||
- Prefer commands run through `mise exec -- ...` when the Python environment matters.
|
||||
- Prefer `mise exec -- uv run ...` for project Python commands once dependencies are synced.
|
||||
- Git is the version control system for this project. A remote will be added later.
|
||||
|
||||
## Safety And Trading Constraints
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Trading Bot
|
||||
|
||||
Python trading bot project using machine learning to decide which assets should be held. The bot is intended to trade through the IBKR API, beginning with paper trading and potentially supporting real trades later behind explicit safeguards.
|
||||
Python trading bot project using machine learning to decide which assets should be held. The bot currently uses the Alpaca Markets API for market data and paper trading, with any future live trading kept behind explicit safeguards.
|
||||
|
||||
## Project Status
|
||||
|
||||
This repository is at the planning and scaffolding stage. The machine learning model, trading parameters, asset universe, and risk rules will be designed later.
|
||||
This repository now has a working prototype flow for Alpaca market data, dataset generation, XGBoost training, prediction, and paper-trading rebalancing. The strategy is still intentionally narrow: it currently focuses on `SPY` exposure using a small market-regime feature set.
|
||||
|
||||
## Main Parts
|
||||
|
||||
@@ -15,21 +15,76 @@ This repository is at the planning and scaffolding stage. The machine learning m
|
||||
|
||||
## Environment
|
||||
|
||||
Use `mise` to manage Python.
|
||||
Use `mise` to manage Python and `uv` to manage Python packages.
|
||||
|
||||
```sh
|
||||
mise install
|
||||
mise exec -- python --version
|
||||
mise exec -- uv sync
|
||||
mise exec -- uv run python --version
|
||||
```
|
||||
|
||||
The current local Python version is pinned in `.mise.toml`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Runtime dependencies are declared in `pyproject.toml`.
|
||||
|
||||
- `alpaca-py` for Alpaca market data and trading clients.
|
||||
- `pandas` for tabular candle data handling.
|
||||
- `pyarrow` for Parquet file support.
|
||||
- `xgboost` and `scikit-learn` for model training and evaluation.
|
||||
- `python-dotenv` for loading local Alpaca credentials from `.env`.
|
||||
|
||||
## Trading Safety
|
||||
|
||||
The default target is paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
|
||||
The default target is Alpaca paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
|
||||
|
||||
Do not commit secrets such as IBKR credentials, account identifiers, API tokens, or private configuration.
|
||||
Do not commit secrets such as Alpaca API keys, account identifiers, API tokens, or private configuration.
|
||||
|
||||
## Current Commands
|
||||
|
||||
Fetch Alpaca daily candles:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
Build the training dataset and train the current model:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
|
||||
```
|
||||
|
||||
Run the current Alpaca paper-trading rebalance flow:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/models/trade.py \
|
||||
--paper \
|
||||
--fetch-recent-data \
|
||||
--model-path models/spy_xgb_v1.json \
|
||||
--metadata-path models/spy_xgb_v1_meta.json
|
||||
```
|
||||
|
||||
The main Python entry points are:
|
||||
|
||||
- `src/trading_bot/data/train_pipeline.py` for dataset generation and model training.
|
||||
- `src/trading_bot/models/trade.py` for Alpaca account inspection and SPY rebalancing.
|
||||
- `src/trading_bot/ui/dashboard.py` for the read-only performance dashboard.
|
||||
|
||||
Serve the read-only dashboard:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
See [docs/README.md](docs/README.md) for the initial architecture notes and decision log.
|
||||
|
||||
The current data collection note is [docs/data-fetcher.md](docs/data-fetcher.md), and the training dataset contract is [docs/training-dataset.md](docs/training-dataset.md).
|
||||
|
||||
The read-only dashboard design is documented in [docs/read-only-ui.md](docs/read-only-ui.md).
|
||||
|
||||
The first dataset generation notebook is [notebooks/spy_direction_dataset.ipynb](notebooks/spy_direction_dataset.ipynb).
|
||||
|
||||
Manual test instructions for the Alpaca fetcher are in [docs/manual-test/README.md](docs/manual-test/README.md).
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"symbols": {
|
||||
"SPY": "SPY",
|
||||
"VIX": "VIXY",
|
||||
"TLT": "TLT",
|
||||
"USO": "USO"
|
||||
},
|
||||
"fractions": {
|
||||
"train": 0.7,
|
||||
"validation": 0.15,
|
||||
"test": 0.15
|
||||
},
|
||||
"model": {
|
||||
"n_estimators": 300,
|
||||
"max_depth": 1,
|
||||
"learning_rate": 0.01,
|
||||
"subsample": 0.7,
|
||||
"colsample_bytree": 0.7,
|
||||
"early_stopping_rounds": 20,
|
||||
"random_state": 42,
|
||||
"eval_metric": "logloss"
|
||||
}
|
||||
}
|
||||
+28
-17
@@ -2,7 +2,7 @@
|
||||
|
||||
## Objective
|
||||
|
||||
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses the IBKR API to facilitate trades. The first supported trading mode should be paper trading.
|
||||
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses Alpaca Markets for market data and broker execution. The first supported trading mode is paper trading.
|
||||
|
||||
## Planned Modules
|
||||
|
||||
@@ -10,6 +10,12 @@ Build a Python-based trading bot that uses machine learning to determine which a
|
||||
|
||||
Responsible for acquiring and storing market, asset, and any future feature data needed for model training and evaluation.
|
||||
|
||||
The current data tool is an Alpaca daily candle fetcher. It fetches open, high, low, close, and volume data for a ticker and date range, then persists that data to a ticker-named Parquet file. See [data-fetcher.md](data-fetcher.md).
|
||||
|
||||
The initial supervised training dataset is documented in [training-dataset.md](training-dataset.md). It derives market-regime features from `SPY`, a volatility proxy, `TLT`, and `USO`, then labels whether `SPY` closes higher five trading days later. The checked-in configuration currently maps the volatility input to `VIXY` for Alpaca data availability.
|
||||
|
||||
Parquet files are partitioned by ticker, not by date.
|
||||
|
||||
Open decisions:
|
||||
|
||||
- asset universe;
|
||||
@@ -23,10 +29,11 @@ Open decisions:
|
||||
|
||||
Responsible for building datasets, training models, evaluating candidates, and writing versioned model artifacts.
|
||||
|
||||
The main entry point is `src/trading_bot/data/train_pipeline.py`. It reads raw Alpaca Parquet files from `data/alpaca/daily`, builds `data/training/spy_direction_5d.parquet`, trains an XGBoost classifier, and writes model artifacts to `models/`.
|
||||
|
||||
Open decisions:
|
||||
|
||||
- prediction target;
|
||||
- feature set;
|
||||
- model family;
|
||||
- validation strategy;
|
||||
- evaluation metrics;
|
||||
@@ -36,11 +43,13 @@ Open decisions:
|
||||
|
||||
### Trading Bot
|
||||
|
||||
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the IBKR API to place or simulate orders.
|
||||
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the Alpaca trading API to place paper-trading orders.
|
||||
|
||||
The main entry point is `src/trading_bot/models/trade.py`. It loads a model and metadata, optionally refreshes recent Alpaca market data, estimates a target `SPY` exposure from the model probability, cancels open Alpaca orders, and submits a day market order when the desired rebalance exceeds the configured minimum notional amount. Pass `--model-path models/spy_xgb_v1.json --metadata-path models/spy_xgb_v1_meta.json` to trade with artifacts produced by the current training pipeline.
|
||||
|
||||
Initial expectations:
|
||||
|
||||
- paper trading first;
|
||||
- Alpaca paper trading first;
|
||||
- real trading later only behind explicit configuration;
|
||||
- clear logging of model version, signals, target holdings, generated orders, and broker responses;
|
||||
- separation between signal generation, portfolio construction, and broker execution.
|
||||
@@ -51,32 +60,34 @@ Open decisions:
|
||||
- position sizing;
|
||||
- risk limits;
|
||||
- cash handling;
|
||||
- order types;
|
||||
- order types and time-in-force choices;
|
||||
- failed order handling;
|
||||
- market hours behavior;
|
||||
- manual override behavior.
|
||||
|
||||
### Read-Only Web UI
|
||||
|
||||
Optional future module for checking status without controlling trading behavior.
|
||||
Optional module for checking status without controlling trading behavior.
|
||||
|
||||
Possible scope:
|
||||
The initial implementation is documented in [read-only-ui.md](read-only-ui.md).
|
||||
It serves one backend-rendered page, reads Alpaca account state with `alpaca-py`,
|
||||
and caches snapshots in SQLite so normal page loads do not repeatedly call Alpaca.
|
||||
|
||||
- current holdings;
|
||||
- latest signals;
|
||||
- recent orders;
|
||||
- account summary;
|
||||
- model version;
|
||||
- bot health and logs.
|
||||
Current scope:
|
||||
|
||||
- three-month account performance summary;
|
||||
- account value chart with a `SPY` S&P 500 proxy comparison;
|
||||
- cash and equity portfolio table;
|
||||
- completed trades table.
|
||||
|
||||
## Near-Term Priorities
|
||||
|
||||
1. Decide the initial project package structure.
|
||||
2. Add Python packaging and dependency management.
|
||||
2. Keep Python packaging and dependency management current with `uv`.
|
||||
3. Add a minimal configuration system.
|
||||
4. Define interfaces for data collection, model artifacts, and broker execution.
|
||||
5. Add tests for the core trading decision boundaries before connecting real broker behavior.
|
||||
4. Harden interfaces for data collection, model artifacts, and Alpaca broker execution.
|
||||
5. Expand tests around trading decision boundaries, portfolio sizing, stale data handling, and broker API boundaries.
|
||||
|
||||
## Decisions Deferred
|
||||
|
||||
The model, features, labels, asset selection rules, risk management rules, and trading cadence are intentionally deferred for later discussion.
|
||||
Broader model design, asset selection rules, risk management rules, live-trading gates, and trading cadence are intentionally deferred for later discussion.
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Data Fetcher Design
|
||||
|
||||
## Initial Goal
|
||||
|
||||
Create a Python module/tool that fetches daily candlestick data from the Alpaca Market Data API for a specific ticker and date range.
|
||||
|
||||
The intended example workflow is:
|
||||
|
||||
- ticker: `SPY`;
|
||||
- date range: `2026-06-01` to `2026-06-30`;
|
||||
- bar size: one trading day;
|
||||
- fields: open, high, low, close, volume;
|
||||
- output: one Parquet file named for the ticker, such as `SPY.parquet`.
|
||||
|
||||
## Current Implementation
|
||||
|
||||
The current implementation is intentionally small:
|
||||
|
||||
- ticker passed as a required command line argument;
|
||||
- end date passed with `--end-date YYYYMMDD`, defaulting to yesterday;
|
||||
- end date can also be derived with `--end-date-from-parquet`;
|
||||
- duration passed with `--duration`, defaulting to `1 W`;
|
||||
- uses the Alpaca Market Data API through `alpaca-py`;
|
||||
- reads credentials from `--api-key` / `--secret-key`, `ALPACA_API_KEY` / `ALPACA_SECRET_KEY`, or `.env`;
|
||||
- writes candles to a symbol-named Parquet file;
|
||||
- appends to an existing symbol file and keeps one row per date.
|
||||
|
||||
Run it with:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
To override the requested range:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
```
|
||||
|
||||
To fetch backward from the oldest date already stored in the symbol file:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
|
||||
```
|
||||
|
||||
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
|
||||
|
||||
This expects Alpaca API credentials to be available via CLI arguments, environment variables, or `.env`.
|
||||
|
||||
By default, output is written to `data/alpaca/daily/SPY.parquet`. Use `--output-dir` to choose another directory.
|
||||
|
||||
Manual test instructions are in [manual-test/README.md](manual-test/README.md).
|
||||
|
||||
## Intended Future Behavior
|
||||
|
||||
Later, this tool should broaden configuration around data source and normalization choices.
|
||||
|
||||
Decided storage behavior:
|
||||
|
||||
- Parquet files are partitioned by ticker, not by date.
|
||||
- Each ticker should have its own Parquet file, such as `SPY.parquet`.
|
||||
- The trading date is used as the row key for merges.
|
||||
- Re-fetching a date replaces the existing row for that date in that symbol's file.
|
||||
|
||||
Candidate output schema:
|
||||
|
||||
| Column | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `date` | date index | Trading session date |
|
||||
| `symbol` | string | Asset ticker |
|
||||
| `open` | float | Daily open price |
|
||||
| `high` | float | Daily high price |
|
||||
| `low` | float | Daily low price |
|
||||
| `close` | float | Daily close price |
|
||||
| `volume` | integer | Daily traded volume |
|
||||
|
||||
Open decisions:
|
||||
|
||||
- how to handle adjusted versus unadjusted prices;
|
||||
- how to handle missing sessions, market holidays, and Alpaca API rate limits;
|
||||
- whether to add explicit feed selection, adjustment settings, or data entitlement checks.
|
||||
@@ -0,0 +1,79 @@
|
||||
# Manual Test Instructions
|
||||
|
||||
## Alpaca Daily Fetcher
|
||||
|
||||
This test checks the Alpaca data fetcher and confirms it writes a symbol-named Parquet file.
|
||||
|
||||
The fetcher currently requests:
|
||||
|
||||
- symbol: provided as a command line argument;
|
||||
- end date: provided with `--end-date YYYYMMDD`, defaulting to yesterday;
|
||||
- end date can also be derived with `--end-date-from-parquet`;
|
||||
- duration: provided with `--duration`, defaulting to `1 W`;
|
||||
- bar size: `1 day`;
|
||||
- data source: Alpaca stock bars through `alpaca-py`;
|
||||
- output: a Parquet file in `data/alpaca/daily/`.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Create or confirm Alpaca API credentials.
|
||||
2. Provide credentials through `ALPACA_API_KEY` and `ALPACA_SECRET_KEY`, a local `.env`, or CLI arguments.
|
||||
3. Sync Python dependencies:
|
||||
|
||||
```sh
|
||||
mise exec -- uv sync
|
||||
```
|
||||
|
||||
## Run The Test
|
||||
|
||||
From the repository root, run:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
To fetch a specific Alpaca range, pass an end date and duration:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
```
|
||||
|
||||
To fetch backward from the oldest date already stored in `data/alpaca/daily/SPY.parquet`:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
|
||||
```
|
||||
|
||||
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
|
||||
|
||||
## Expected Output
|
||||
|
||||
The tool should first print the request range:
|
||||
|
||||
```text
|
||||
Fetching SPY daily candles ending 2025-06-05 for duration 1 M via Alpaca API
|
||||
```
|
||||
|
||||
If the request succeeds, it should print how many candles were fetched and where the merged Parquet file was written:
|
||||
|
||||
```text
|
||||
Fetched 21 daily candles for SPY.
|
||||
Wrote 21 total daily rows to data/alpaca/daily/SPY.parquet
|
||||
```
|
||||
|
||||
Exact row counts depend on the requested date range and market calendar.
|
||||
|
||||
The tool should then write or update:
|
||||
|
||||
```text
|
||||
data/alpaca/daily/SPY.parquet
|
||||
```
|
||||
|
||||
If the Parquet file already exists, rows from the latest fetch are merged into it. The trading date is used as the row key, so a symbol file keeps only one row for each date.
|
||||
|
||||
## Common Issues
|
||||
|
||||
- Missing credentials: set `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` or pass them with CLI flags.
|
||||
- No historical bars: confirm the symbol, requested range, and Alpaca market data permissions.
|
||||
- Authentication or entitlement errors: confirm the keys belong to the intended Alpaca account and data plan.
|
||||
- Rate-limit errors: wait before retrying or reduce repeated requests.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Read-Only Performance UI
|
||||
|
||||
## Goal
|
||||
|
||||
Provide a single read-only page for checking Alpaca paper account performance and recent trading activity without exposing any trading controls.
|
||||
|
||||
## Requested Feature List
|
||||
|
||||
- Show the last three months of account performance in the header as text, or a shorter range when less data is available.
|
||||
- Show account value over time as the main body chart.
|
||||
- Include an S&P 500 comparison line in the chart. The initial implementation uses `SPY` daily candles as the S&P 500 proxy because Alpaca stock market data exposes it through the same API path.
|
||||
- Show a portfolio table below the chart with cash plus equity positions.
|
||||
- Show a completed trades table at the end of the page.
|
||||
- Use Python and `alpaca-py` in the backend.
|
||||
- Serve the page directly from the backend as a single page with no user options.
|
||||
- Cache fetched data in SQLite and refresh periodically, initially every 12 hours.
|
||||
|
||||
## Design Notes
|
||||
|
||||
The UI is intentionally separate from the trading and training modules. It reads account state from Alpaca, stores normalized snapshots in SQLite, and renders HTML from the backend. It does not place, replace, or cancel orders.
|
||||
|
||||
The first cache path is `data/ui/performance.sqlite`. On each page request, the server refreshes data only when the cache is older than the configured refresh interval. If Alpaca refresh fails but cached data exists, the page can still render stale data with a warning.
|
||||
|
||||
## Backend Data
|
||||
|
||||
The dashboard stores:
|
||||
|
||||
- account equity history for up to three months;
|
||||
- normalized S&P 500 proxy values from `SPY`;
|
||||
- the latest cash and equity holdings snapshot;
|
||||
- recently closed orders that have a filled quantity.
|
||||
|
||||
## Run Command
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
|
||||
```
|
||||
|
||||
The server defaults to `127.0.0.1:8000` and reads `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` from the environment or local `.env`.
|
||||
@@ -0,0 +1,132 @@
|
||||
# Training Dataset Design
|
||||
|
||||
## Initial Goal
|
||||
|
||||
Build a supervised learning dataset from stored Alpaca daily candle data. The first dataset predicts whether `SPY` closes higher five trading days after the observation date.
|
||||
|
||||
The current implementation is `src/trading_bot/data/train_pipeline.py`, which combines the original dataset notebook and XGBoost training notebook into one runnable script. The earlier notebook, [../notebooks/spy_direction_dataset.ipynb](../notebooks/spy_direction_dataset.ipynb), remains useful background for inspecting the initial feature design.
|
||||
|
||||
## Raw Data
|
||||
|
||||
Raw daily candles are stored under:
|
||||
|
||||
```text
|
||||
data/alpaca/daily
|
||||
```
|
||||
|
||||
Storage expectations:
|
||||
|
||||
- one Parquet file per symbol;
|
||||
- file name is the symbol, such as `SPY.parquet`;
|
||||
- each row contains one daily candle for that symbol;
|
||||
- expected columns are `date`, `symbol`, `open`, `high`, `low`, `close`, and `volume`.
|
||||
|
||||
The initial dataset requires at least these logical inputs:
|
||||
|
||||
- `SPY`;
|
||||
- volatility proxy, currently `VIXY` in `config/train_config.json`;
|
||||
- `TLT`;
|
||||
- `USO`.
|
||||
|
||||
## Observation Rows
|
||||
|
||||
Each dataset row represents one trading date. Feature values are derived from data available on or before that date. The target is derived from `SPY` closing price five trading days after that date.
|
||||
|
||||
Rows should be dropped when any required feature or target value cannot be computed.
|
||||
|
||||
## Input Features
|
||||
|
||||
| Feature | Description |
|
||||
| --- | --- |
|
||||
| `SPY_ret_5` | `SPY` fractional return over the previous 5 trading days. |
|
||||
| `SPY_ret_20` | `SPY` fractional return over the previous 20 trading days. |
|
||||
| `SPY_dist_sma50` | `SPY` distance from its 50 trading day simple moving average, divided by that average. |
|
||||
| `VIX_change_5` | `VIX` fractional change over the previous 5 trading days. |
|
||||
| `VIX_rank_20` | `VIX` percentile rank within a rolling 20 trading day window. |
|
||||
| `TLT_ret_10` | `TLT` fractional return over the previous 10 trading days. |
|
||||
| `USO_ret_5` | `USO` fractional return over the previous 5 trading days. |
|
||||
| `SPY_TLT_ratio_ret` | Fractional 5 trading day change in the `SPY` to `TLT` close-price ratio. |
|
||||
|
||||
Fractional percentage changes should follow pandas `pct_change` convention. For example, a five percent return is represented as `0.05`.
|
||||
|
||||
## Output Target
|
||||
|
||||
The target column is a binary indicator of `SPY` forward return over the next five trading days.
|
||||
|
||||
| Target | Description |
|
||||
| --- | --- |
|
||||
| `spy_up_5d` | `1.0` when `SPY` closes above today's close five trading days later; `0.0` when `SPY` closes below today's close five trading days later; `0.5` when the future close equals today's close. |
|
||||
|
||||
The original dataset design used `0.5` for unchanged prices as an explicit neutral target. The current training pipeline drops unchanged `0.5` rows before training so the XGBoost model remains a binary classifier.
|
||||
|
||||
## Data Alignment
|
||||
|
||||
The dataset should use trading-day alignment rather than calendar-day alignment. A five day lookback or forecast horizon means five available trading sessions for the relevant symbol, not five calendar days.
|
||||
|
||||
`SPY` should define the primary observation calendar because the first target is an `SPY` forward-return label. Other symbols should be joined to the `SPY` observation dates after their own features are computed.
|
||||
|
||||
## Notebook Feature Sketch
|
||||
|
||||
The first notebook should use pandas transformations close to this shape:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
df["SPY_ret_5"] = df["SPY_close"].pct_change(5)
|
||||
df["SPY_ret_20"] = df["SPY_close"].pct_change(20)
|
||||
|
||||
sma_50 = df["SPY_close"].rolling(50).mean()
|
||||
df["SPY_dist_sma50"] = (df["SPY_close"] - sma_50) / sma_50
|
||||
|
||||
df["VIX_change_5"] = df["VIX_close"].pct_change(5)
|
||||
df["VIX_rank_20"] = df["VIX_close"].rolling(20).rank(pct=True)
|
||||
|
||||
df["TLT_ret_10"] = df["TLT_close"].pct_change(10)
|
||||
df["USO_ret_5"] = df["USO_close"].pct_change(5)
|
||||
df["SPY_TLT_ratio_ret"] = (df["SPY_close"] / df["TLT_close"]).pct_change(5)
|
||||
|
||||
spy_forward_close = df["SPY_close"].shift(-5)
|
||||
df["spy_up_5d"] = np.select(
|
||||
[
|
||||
spy_forward_close > df["SPY_close"],
|
||||
spy_forward_close < df["SPY_close"],
|
||||
],
|
||||
[1.0, 0.0],
|
||||
default=0.5,
|
||||
)
|
||||
|
||||
df_model = df.dropna().copy()
|
||||
```
|
||||
|
||||
## Dataset Splits
|
||||
|
||||
Training, validation, and test splits should be chronological:
|
||||
|
||||
- oldest rows for training;
|
||||
- newer rows for validation;
|
||||
- newest rows for the final test set.
|
||||
|
||||
A split indicator column such as `split` is useful in the dataset artifact for auditability and reproducibility. It should be treated as metadata, not as a model input feature. The model training code should build `X` from the explicit feature column list and exclude metadata columns such as `date`, `split`, raw close prices, and the target.
|
||||
|
||||
## Training Pipeline
|
||||
|
||||
Run the current pipeline from the repository root:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
|
||||
```
|
||||
|
||||
The script reads configuration from `config/train_config.json`, writes the dataset to `data/training/spy_direction_5d.parquet`, and saves model artifacts to:
|
||||
|
||||
- `models/spy_xgb_v1.json`;
|
||||
- `models/spy_xgb_v1_meta.json`.
|
||||
|
||||
The metadata file stores the training base probability, feature column list, last trained date, and model configuration.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
- exact adjusted versus unadjusted close handling;
|
||||
- whether same-day volatility proxy, `TLT`, and `USO` values are acceptable for the intended trading decision timing;
|
||||
- exact train, validation, and test date boundaries or split percentages;
|
||||
- output dataset schema metadata and versioning.
|
||||
@@ -0,0 +1,164 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"id": "c8a08105",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"from alpaca.trading.client import TradingClient\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"\n",
|
||||
"# Set paper=True for paper trading (sandbox), paper=False for live trading\n",
|
||||
"trading_client = TradingClient(\n",
|
||||
" api_key=os.getenv(\"ALPACA_API_KEY\"),\n",
|
||||
" secret_key=os.getenv(\"ALPACA_SECRET_KEY\"),\n",
|
||||
" paper=True,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"id": "49b14380",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No open positions.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"positions = trading_client.get_all_positions()\n",
|
||||
"\n",
|
||||
"if not positions:\n",
|
||||
" print(\"No open positions.\")\n",
|
||||
"else:\n",
|
||||
" print(\"Current Portfolio Positions:\")\n",
|
||||
" for pos in positions:\n",
|
||||
" print(\n",
|
||||
" f\"Symbol: {pos.symbol:<5} | \"\n",
|
||||
" f\"Qty: {pos.qty:<5} | \"\n",
|
||||
" f\"Avg Entry Price: ${float(pos.avg_entry_price):.2f} | \"\n",
|
||||
" f\"Current Price: ${float(pos.current_price):.2f} | \"\n",
|
||||
" f\"Unrealized P/L: ${float(pos.unrealized_pl):.2f}\"\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"id": "c032239c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"99997.95 99997.95\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"account = trading_client.get_account()\n",
|
||||
"print(\n",
|
||||
" account.cash,\n",
|
||||
" account.equity\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"id": "c759e40f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No open orders found.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from alpaca.trading.requests import GetOrdersRequest\n",
|
||||
"from alpaca.trading.enums import QueryOrderStatus\n",
|
||||
"\n",
|
||||
"# Request only open orders\n",
|
||||
"request_params = GetOrdersRequest(status=QueryOrderStatus.OPEN)\n",
|
||||
"open_orders = trading_client.get_orders(filter=request_params)\n",
|
||||
"\n",
|
||||
"if not open_orders:\n",
|
||||
" print(\"No open orders found.\")\n",
|
||||
"else:\n",
|
||||
" print(f\"Found {len(open_orders)} open order(s):\")\n",
|
||||
" for order in open_orders:\n",
|
||||
" print(\n",
|
||||
" f\"ID: {order.id} | Symbol: {order.symbol} | \"\n",
|
||||
" f\"Side: {order.side} | Qty: {order.qty} | Status: {order.status}\"\n",
|
||||
" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"id": "091cf061",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Submitted Order ID: 4838c16d-7a12-4dc3-80cb-7c5d1dbecda3 | Status: OrderStatus.ACCEPTED\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from alpaca.trading.requests import MarketOrderRequest\n",
|
||||
"from alpaca.trading.enums import OrderSide, TimeInForce\n",
|
||||
"\n",
|
||||
"# Define a market buy order for 10 shares of SPY\n",
|
||||
"market_order_data = MarketOrderRequest(\n",
|
||||
" symbol=\"SPY\",\n",
|
||||
" notional=100.0,\n",
|
||||
" side=OrderSide.BUY,\n",
|
||||
" time_in_force=TimeInForce.DAY,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Submit the order\n",
|
||||
"order = trading_client.submit_order(order_data=market_order_data)\n",
|
||||
"\n",
|
||||
"print(f\"Submitted Order ID: {order.id} | Status: {order.status}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.15"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,308 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "038bf2ce",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import warnings\n",
|
||||
"from pathlib import Path\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"# Source of truth for model inputs (must match training order exactly)\n",
|
||||
"FEATURE_COLUMNS = [\n",
|
||||
" \"SPY_ret_5\",\n",
|
||||
" \"SPY_ret_20\",\n",
|
||||
" \"SPY_dist_sma50\",\n",
|
||||
" \"VIX_change_5\",\n",
|
||||
" \"VIX_rank_20\",\n",
|
||||
" \"TLT_ret_10\",\n",
|
||||
" \"USO_ret_5\",\n",
|
||||
" \"SPY_TLT_ratio_ret\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"DEFAULT_SYMBOLS = {\n",
|
||||
" \"SPY\": \"SPY\",\n",
|
||||
" \"VIXY\": \"VIX\", # Change to \"VIX\": \"VIX\" if using raw VIX Parquet\n",
|
||||
" \"TLT\": \"TLT\",\n",
|
||||
" \"USO\": \"USO\",\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def load_raw_close_prices(\n",
|
||||
" raw_data_dir: Path, symbols: dict[str, str]\n",
|
||||
") -> pd.DataFrame:\n",
|
||||
" \"\"\"Reads raw Parquet files and merges close prices into a single inner-joined DataFrame.\"\"\"\n",
|
||||
" frames = []\n",
|
||||
" for symbol, alias in symbols.items():\n",
|
||||
" path = raw_data_dir / f\"{symbol}.parquet\"\n",
|
||||
" if not path.exists():\n",
|
||||
" raise FileNotFoundError(f\"Missing raw data file: {path}\")\n",
|
||||
"\n",
|
||||
" frame = pd.read_parquet(path)\n",
|
||||
" if \"date\" in frame.columns:\n",
|
||||
" frame = frame.set_index(\"date\")\n",
|
||||
"\n",
|
||||
" frame.index = pd.to_datetime(frame.index)\n",
|
||||
" frame = frame.sort_index()\n",
|
||||
" frames.append(frame[[\"close\"]].rename(columns={\"close\": f\"{alias}_close\"}))\n",
|
||||
"\n",
|
||||
" return pd.concat(frames, axis=1, join=\"inner\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def compute_features(prices: pd.DataFrame) -> pd.DataFrame:\n",
|
||||
" \"\"\"Computes engineered features from raw merged price history.\"\"\"\n",
|
||||
" df = prices.copy()\n",
|
||||
"\n",
|
||||
" # Target asset features\n",
|
||||
" df[\"SPY_ret_5\"] = df[\"SPY_close\"].pct_change(5)\n",
|
||||
" df[\"SPY_ret_20\"] = df[\"SPY_close\"].pct_change(20)\n",
|
||||
"\n",
|
||||
" sma_50 = df[\"SPY_close\"].rolling(50).mean()\n",
|
||||
" df[\"SPY_dist_sma50\"] = (df[\"SPY_close\"] - sma_50) / sma_50\n",
|
||||
"\n",
|
||||
" # Volatility / Market Stress\n",
|
||||
" df[\"VIX_change_5\"] = df[\"VIX_close\"].pct_change(5)\n",
|
||||
" df[\"VIX_rank_20\"] = df[\"VIX_close\"].rolling(20).rank(pct=True)\n",
|
||||
"\n",
|
||||
" # Macro & Relative ratios\n",
|
||||
" df[\"TLT_ret_10\"] = df[\"TLT_close\"].pct_change(10)\n",
|
||||
" df[\"USO_ret_5\"] = df[\"USO_close\"].pct_change(5)\n",
|
||||
" df[\"SPY_TLT_ratio_ret\"] = (df[\"SPY_close\"] / df[\"TLT_close\"]).pct_change(5)\n",
|
||||
"\n",
|
||||
" return df[FEATURE_COLUMNS]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def get_latest_inference_features(\n",
|
||||
" raw_data_dir: Path,\n",
|
||||
" symbols: dict[str, str] | None = None,\n",
|
||||
" max_age_days: int = 1,\n",
|
||||
") -> pd.DataFrame:\n",
|
||||
" \"\"\"Loads raw prices, computes features, verifies date freshness,\n",
|
||||
"\n",
|
||||
" and returns the latest single row for model prediction.\n",
|
||||
" \"\"\"\n",
|
||||
" symbols = symbols or DEFAULT_SYMBOLS\n",
|
||||
"\n",
|
||||
" # 1. Load prices & compute rolling features\n",
|
||||
" prices = load_raw_close_prices(raw_data_dir, symbols)\n",
|
||||
" features = compute_features(prices).dropna()\n",
|
||||
"\n",
|
||||
" if features.empty:\n",
|
||||
" raise ValueError(\n",
|
||||
" \"Not enough historical rows to compute 50-day rolling window features.\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # 2. Extract latest available row as a 1-row DataFrame\n",
|
||||
" latest_row = features.iloc[[-1]]\n",
|
||||
" latest_date = latest_row.index[0]\n",
|
||||
"\n",
|
||||
" # 3. Check data freshness and raise a warning if stale\n",
|
||||
" now = pd.Timestamp.now()\n",
|
||||
" latest_date_naive = (\n",
|
||||
" latest_date.tz_localize(None)\n",
|
||||
" if latest_date.tz is not None\n",
|
||||
" else latest_date\n",
|
||||
" )\n",
|
||||
" days_old = (now.floor(\"D\") - latest_date_naive.floor(\"D\")).days\n",
|
||||
"\n",
|
||||
" if days_old > max_age_days:\n",
|
||||
" warnings.warn(\n",
|
||||
" f\"STALE DATA WARNING: Latest feature row is from {latest_date.strftime('%Y-%m-%d')} \"\n",
|
||||
" f\"({days_old} day(s) old). Update raw Parquet files before executing trades.\",\n",
|
||||
" UserWarning,\n",
|
||||
" stacklevel=2,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return latest_row\n",
|
||||
"\n",
|
||||
"def get_target_exposure(\n",
|
||||
" p_pred: float, p_base: float, sensitivity: float = 5.0\n",
|
||||
") -> float:\n",
|
||||
" \"\"\"Maps predicted probability to a target portfolio equity allocation (0.0 to 1.0).\n",
|
||||
"\n",
|
||||
" - p_pred == p_base --> 50% Target Exposure (Neutral)\n",
|
||||
" - p_pred > p_base --> Scale up toward 100% (Bullish)\n",
|
||||
" - p_pred < p_base --> Scale down toward 0% (Bearish / Cash)\n",
|
||||
" \"\"\"\n",
|
||||
" # Calculate deviation from the historical average\n",
|
||||
" delta = p_pred - p_base\n",
|
||||
"\n",
|
||||
" # Base target allocation is 50% equity / 50% cash\n",
|
||||
" base_allocation = 0.50\n",
|
||||
"\n",
|
||||
" # Sensitivity controls how aggressively probability changes alter allocation\n",
|
||||
" # e.g., a +0.08 delta * 5.0 = +0.40 -> 90% Equity Allocation\n",
|
||||
" target_allocation = base_allocation + (delta * sensitivity)\n",
|
||||
"\n",
|
||||
" # Clamp bounds strictly between 0% (full cash) and 100% (full SPY)\n",
|
||||
" return float(np.clip(target_allocation, 0.0, 1.0))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "0b3d2c25",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Date: 2026-07-27 | Prob: 0.5843 | Base: 0.5830\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0.5064256139268726"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"import xgboost as xgb\n",
|
||||
"\n",
|
||||
"# 1. Load model and metadata\n",
|
||||
"model = xgb.XGBClassifier()\n",
|
||||
"model.load_model(\"models/spy_xgb_v1.json\")\n",
|
||||
"\n",
|
||||
"with open(\"models/spy_xgb_v1_meta.json\", \"r\") as f:\n",
|
||||
" meta = json.load(f)\n",
|
||||
"\n",
|
||||
"# 2. Fetch latest features from raw parquet files\n",
|
||||
"RAW_DATA_DIR = Path(\"../data/ibkr/daily\")\n",
|
||||
"X_latest = get_latest_inference_features(RAW_DATA_DIR, max_age_days=1)\n",
|
||||
"\n",
|
||||
"# 3. Predict probability\n",
|
||||
"p_pred = float(model.predict_proba(X_latest[meta[\"feature_cols\"]])[0, 1])\n",
|
||||
"p_base = meta[\"p_base\"]\n",
|
||||
"\n",
|
||||
"print(\n",
|
||||
" f\"Date: {X_latest.index[0].date()} | Prob: {p_pred:.4f} | Base: {p_base:.4f}\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"get_target_exposure(p_pred, p_base, sensitivity=5.0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "3f823fa6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>SPY_ret_5</th>\n",
|
||||
" <th>SPY_ret_20</th>\n",
|
||||
" <th>SPY_dist_sma50</th>\n",
|
||||
" <th>VIX_change_5</th>\n",
|
||||
" <th>VIX_rank_20</th>\n",
|
||||
" <th>TLT_ret_10</th>\n",
|
||||
" <th>USO_ret_5</th>\n",
|
||||
" <th>SPY_TLT_ratio_ret</th>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>date</th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>2026-07-27</th>\n",
|
||||
" <td>-0.004043</td>\n",
|
||||
" <td>0.013855</td>\n",
|
||||
" <td>-0.007938</td>\n",
|
||||
" <td>0.007065</td>\n",
|
||||
" <td>0.75</td>\n",
|
||||
" <td>-0.00262</td>\n",
|
||||
" <td>-0.005976</td>\n",
|
||||
" <td>-0.002378</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 VIX_rank_20 \\\n",
|
||||
"date \n",
|
||||
"2026-07-27 -0.004043 0.013855 -0.007938 0.007065 0.75 \n",
|
||||
"\n",
|
||||
" TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret \n",
|
||||
"date \n",
|
||||
"2026-07-27 -0.00262 -0.005976 -0.002378 "
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"X_latest"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "476fd7fd",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.15"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"p_base": 0.5819477434679335,
|
||||
"feature_cols": [
|
||||
"VIX_rank_20",
|
||||
"TLT_ret_10",
|
||||
"USO_ret_5",
|
||||
"SPY_TLT_ratio_ret",
|
||||
"SPY_ret_5",
|
||||
"SPY_ret_20",
|
||||
"SPY_dist_sma50"
|
||||
],
|
||||
"last_trained_date": "2026-07-29 00:00:00"
|
||||
}
|
||||
@@ -0,0 +1,848 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# SPY Direction Training Dataset\n",
|
||||
"\n",
|
||||
"Build the first model-ready dataset from IBKR daily candle Parquets. The target is whether `SPY` closes above, below, or unchanged from today's close five trading days later."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"The notebook is deliberately plain pandas so feature logic remains easy to inspect. `VIX_INPUT_SYMBOL` defaults to `VIXY` because that is the VIX-related Parquet currently present locally; switch it to `VIX` after fetching a `VIX.parquet` file."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(PosixPath('/home/jarno/repos/trading-bot'),\n",
|
||||
" PosixPath('/home/jarno/repos/trading-bot/data/alpaca/daily'),\n",
|
||||
" PosixPath('/home/jarno/repos/trading-bot/data/training/spy_direction_5d.parquet'))"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from pathlib import Path\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def find_project_root(start: Path | None = None) -> Path:\n",
|
||||
" current = (start or Path.cwd()).resolve()\n",
|
||||
" for candidate in [current, *current.parents]:\n",
|
||||
" if (candidate / \"pyproject.toml\").exists():\n",
|
||||
" return candidate\n",
|
||||
" raise RuntimeError(\"Could not find project root containing pyproject.toml\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"PROJECT_ROOT = find_project_root()\n",
|
||||
"#RAW_DATA_DIR = PROJECT_ROOT / \"data\" / \"ibkr\" / \"daily\"\n",
|
||||
"RAW_DATA_DIR = PROJECT_ROOT / \"data\" / \"alpaca\" / \"daily\"\n",
|
||||
"OUTPUT_PATH = PROJECT_ROOT / \"data\" / \"training\" / \"spy_direction_5d.parquet\"\n",
|
||||
"\n",
|
||||
"SPY_SYMBOL = \"SPY\"\n",
|
||||
"VIX_INPUT_SYMBOL = \"VIXY\"\n",
|
||||
"TLT_SYMBOL = \"TLT\"\n",
|
||||
"USO_SYMBOL = \"USO\"\n",
|
||||
"\n",
|
||||
"TRAIN_FRACTION = 0.70\n",
|
||||
"VALIDATION_FRACTION = 0.15\n",
|
||||
"TEST_FRACTION = 0.15\n",
|
||||
"\n",
|
||||
"PROJECT_ROOT, RAW_DATA_DIR, OUTPUT_PATH"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Load Close Prices\n",
|
||||
"\n",
|
||||
"Each symbol file is read, sorted by trading date, and reduced to a single close-price column. `SPY` defines the observation calendar through the inner join."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<class 'pandas.DataFrame'>\n",
|
||||
"DatetimeIndex: 1258 entries, 2021-08-02 to 2026-08-05\n",
|
||||
"Data columns (total 4 columns):\n",
|
||||
" # Column Non-Null Count Dtype \n",
|
||||
"--- ------ -------------- ----- \n",
|
||||
" 0 SPY_close 1258 non-null float64\n",
|
||||
" 1 VIX_close 1258 non-null float64\n",
|
||||
" 2 TLT_close 1258 non-null float64\n",
|
||||
" 3 USO_close 1258 non-null float64\n",
|
||||
"dtypes: float64(4)\n",
|
||||
"memory usage: 49.1 KB\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>SPY_close</th>\n",
|
||||
" <th>VIX_close</th>\n",
|
||||
" <th>TLT_close</th>\n",
|
||||
" <th>USO_close</th>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>date</th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-08-02</th>\n",
|
||||
" <td>437.59</td>\n",
|
||||
" <td>25.68</td>\n",
|
||||
" <td>150.67</td>\n",
|
||||
" <td>49.18</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-08-03</th>\n",
|
||||
" <td>441.15</td>\n",
|
||||
" <td>24.40</td>\n",
|
||||
" <td>150.75</td>\n",
|
||||
" <td>48.85</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-08-04</th>\n",
|
||||
" <td>438.98</td>\n",
|
||||
" <td>24.38</td>\n",
|
||||
" <td>151.06</td>\n",
|
||||
" <td>47.20</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-08-05</th>\n",
|
||||
" <td>441.76</td>\n",
|
||||
" <td>23.74</td>\n",
|
||||
" <td>150.29</td>\n",
|
||||
" <td>48.10</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-08-06</th>\n",
|
||||
" <td>442.49</td>\n",
|
||||
" <td>23.14</td>\n",
|
||||
" <td>147.78</td>\n",
|
||||
" <td>47.57</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" SPY_close VIX_close TLT_close USO_close\n",
|
||||
"date \n",
|
||||
"2021-08-02 437.59 25.68 150.67 49.18\n",
|
||||
"2021-08-03 441.15 24.40 150.75 48.85\n",
|
||||
"2021-08-04 438.98 24.38 151.06 47.20\n",
|
||||
"2021-08-05 441.76 23.74 150.29 48.10\n",
|
||||
"2021-08-06 442.49 23.14 147.78 47.57"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def load_close(symbol: str, alias: str | None = None) -> pd.DataFrame:\n",
|
||||
" alias = alias or symbol\n",
|
||||
" path = RAW_DATA_DIR / f\"{symbol}.parquet\"\n",
|
||||
" if not path.exists():\n",
|
||||
" raise FileNotFoundError(f\"Missing raw data file: {path}\")\n",
|
||||
"\n",
|
||||
" frame = pd.read_parquet(path)\n",
|
||||
" if \"date\" in frame.columns:\n",
|
||||
" frame = frame.set_index(\"date\")\n",
|
||||
" if \"close\" not in frame.columns:\n",
|
||||
" raise ValueError(f\"{path} does not contain a close column\")\n",
|
||||
"\n",
|
||||
" frame = frame.copy()\n",
|
||||
" frame.index = pd.to_datetime(frame.index)\n",
|
||||
" frame.index.name = \"date\"\n",
|
||||
" frame = frame.sort_index()\n",
|
||||
" return frame[[\"close\"]].rename(columns={\"close\": f\"{alias}_close\"})\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"prices = pd.concat(\n",
|
||||
" [\n",
|
||||
" load_close(SPY_SYMBOL, \"SPY\"),\n",
|
||||
" load_close(VIX_INPUT_SYMBOL, \"VIX\"),\n",
|
||||
" load_close(TLT_SYMBOL, \"TLT\"),\n",
|
||||
" load_close(USO_SYMBOL, \"USO\"),\n",
|
||||
" ],\n",
|
||||
" axis=1,\n",
|
||||
" join=\"inner\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"prices.info()\n",
|
||||
"prices.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Generate Features And Target\n",
|
||||
"\n",
|
||||
"Feature values use only current and prior rows. The target looks five trading rows ahead in `SPY_close`; unchanged future prices are encoded as `0.5`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>SPY_ret_5</th>\n",
|
||||
" <th>SPY_ret_20</th>\n",
|
||||
" <th>SPY_dist_sma50</th>\n",
|
||||
" <th>VIX_change_5</th>\n",
|
||||
" <th>VIX_rank_20</th>\n",
|
||||
" <th>TLT_ret_10</th>\n",
|
||||
" <th>USO_ret_5</th>\n",
|
||||
" <th>SPY_TLT_ratio_ret</th>\n",
|
||||
" <th>spy_up_5d</th>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>date</th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-10-11</th>\n",
|
||||
" <td>0.014114</td>\n",
|
||||
" <td>-0.026625</td>\n",
|
||||
" <td>-0.018145</td>\n",
|
||||
" <td>-0.090869</td>\n",
|
||||
" <td>0.20</td>\n",
|
||||
" <td>-0.033135</td>\n",
|
||||
" <td>0.031015</td>\n",
|
||||
" <td>0.038908</td>\n",
|
||||
" <td>1.0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-10-12</th>\n",
|
||||
" <td>0.001201</td>\n",
|
||||
" <td>-0.023752</td>\n",
|
||||
" <td>-0.020386</td>\n",
|
||||
" <td>-0.074091</td>\n",
|
||||
" <td>0.10</td>\n",
|
||||
" <td>-0.001041</td>\n",
|
||||
" <td>0.008628</td>\n",
|
||||
" <td>-0.001303</td>\n",
|
||||
" <td>1.0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-10-13</th>\n",
|
||||
" <td>0.000644</td>\n",
|
||||
" <td>-0.028356</td>\n",
|
||||
" <td>-0.016597</td>\n",
|
||||
" <td>-0.081006</td>\n",
|
||||
" <td>0.05</td>\n",
|
||||
" <td>0.006928</td>\n",
|
||||
" <td>0.036928</td>\n",
|
||||
" <td>-0.005897</td>\n",
|
||||
" <td>1.0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-10-14</th>\n",
|
||||
" <td>0.008754</td>\n",
|
||||
" <td>-0.010443</td>\n",
|
||||
" <td>-0.000214</td>\n",
|
||||
" <td>-0.096759</td>\n",
|
||||
" <td>0.05</td>\n",
|
||||
" <td>0.010809</td>\n",
|
||||
" <td>0.026192</td>\n",
|
||||
" <td>-0.011991</td>\n",
|
||||
" <td>1.0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2021-10-15</th>\n",
|
||||
" <td>0.018294</td>\n",
|
||||
" <td>0.010127</td>\n",
|
||||
" <td>0.007213</td>\n",
|
||||
" <td>-0.079882</td>\n",
|
||||
" <td>0.05</td>\n",
|
||||
" <td>-0.002202</td>\n",
|
||||
" <td>0.030287</td>\n",
|
||||
" <td>-0.003823</td>\n",
|
||||
" <td>1.0</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 VIX_rank_20 \\\n",
|
||||
"date \n",
|
||||
"2021-10-11 0.014114 -0.026625 -0.018145 -0.090869 0.20 \n",
|
||||
"2021-10-12 0.001201 -0.023752 -0.020386 -0.074091 0.10 \n",
|
||||
"2021-10-13 0.000644 -0.028356 -0.016597 -0.081006 0.05 \n",
|
||||
"2021-10-14 0.008754 -0.010443 -0.000214 -0.096759 0.05 \n",
|
||||
"2021-10-15 0.018294 0.010127 0.007213 -0.079882 0.05 \n",
|
||||
"\n",
|
||||
" TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret spy_up_5d \n",
|
||||
"date \n",
|
||||
"2021-10-11 -0.033135 0.031015 0.038908 1.0 \n",
|
||||
"2021-10-12 -0.001041 0.008628 -0.001303 1.0 \n",
|
||||
"2021-10-13 0.006928 0.036928 -0.005897 1.0 \n",
|
||||
"2021-10-14 0.010809 0.026192 -0.011991 1.0 \n",
|
||||
"2021-10-15 -0.002202 0.030287 -0.003823 1.0 "
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"df = prices.copy()\n",
|
||||
"\n",
|
||||
"df[\"SPY_ret_5\"] = df[\"SPY_close\"].pct_change(5)\n",
|
||||
"df[\"SPY_ret_20\"] = df[\"SPY_close\"].pct_change(20)\n",
|
||||
"\n",
|
||||
"sma_50 = df[\"SPY_close\"].rolling(50).mean()\n",
|
||||
"df[\"SPY_dist_sma50\"] = (df[\"SPY_close\"] - sma_50) / sma_50\n",
|
||||
"\n",
|
||||
"df[\"VIX_change_5\"] = df[\"VIX_close\"].pct_change(5)\n",
|
||||
"df[\"VIX_rank_20\"] = df[\"VIX_close\"].rolling(20).rank(pct=True)\n",
|
||||
"\n",
|
||||
"df[\"TLT_ret_10\"] = df[\"TLT_close\"].pct_change(10)\n",
|
||||
"df[\"USO_ret_5\"] = df[\"USO_close\"].pct_change(5)\n",
|
||||
"df[\"SPY_TLT_ratio_ret\"] = (df[\"SPY_close\"] / df[\"TLT_close\"]).pct_change(5)\n",
|
||||
"\n",
|
||||
"spy_forward_close = df[\"SPY_close\"].shift(-5)\n",
|
||||
"df[\"spy_up_5d\"] = np.nan\n",
|
||||
"df.loc[spy_forward_close > df[\"SPY_close\"], \"spy_up_5d\"] = 1.0\n",
|
||||
"df.loc[spy_forward_close < df[\"SPY_close\"], \"spy_up_5d\"] = 0.0\n",
|
||||
"df.loc[spy_forward_close == df[\"SPY_close\"], \"spy_up_5d\"] = 0.5\n",
|
||||
"\n",
|
||||
"FEATURE_COLUMNS = [\n",
|
||||
" \"SPY_ret_5\",\n",
|
||||
" \"SPY_ret_20\",\n",
|
||||
" \"SPY_dist_sma50\",\n",
|
||||
" \"VIX_change_5\",\n",
|
||||
" \"VIX_rank_20\",\n",
|
||||
" \"TLT_ret_10\",\n",
|
||||
" \"USO_ret_5\",\n",
|
||||
" \"SPY_TLT_ratio_ret\",\n",
|
||||
"]\n",
|
||||
"TARGET_COLUMN = \"spy_up_5d\"\n",
|
||||
"\n",
|
||||
"df_model = df[FEATURE_COLUMNS + [TARGET_COLUMN]].dropna().copy()\n",
|
||||
"df_model.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Chronological Splits\n",
|
||||
"\n",
|
||||
"Splits are assigned by row order after feature/target cleanup: oldest rows for training, newer rows for validation, newest rows for test. The `split` column is metadata and should not be used as a model input."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>rows</th>\n",
|
||||
" <th>start_date</th>\n",
|
||||
" <th>end_date</th>\n",
|
||||
" <th>target_mean</th>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>split</th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" <th></th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>train</th>\n",
|
||||
" <td>842</td>\n",
|
||||
" <td>2021-10-11</td>\n",
|
||||
" <td>2025-02-18</td>\n",
|
||||
" <td>0.581948</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>validation</th>\n",
|
||||
" <td>180</td>\n",
|
||||
" <td>2025-02-19</td>\n",
|
||||
" <td>2025-11-04</td>\n",
|
||||
" <td>0.638889</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>test</th>\n",
|
||||
" <td>182</td>\n",
|
||||
" <td>2025-11-05</td>\n",
|
||||
" <td>2026-07-29</td>\n",
|
||||
" <td>0.576923</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" rows start_date end_date target_mean\n",
|
||||
"split \n",
|
||||
"train 842 2021-10-11 2025-02-18 0.581948\n",
|
||||
"validation 180 2025-02-19 2025-11-04 0.638889\n",
|
||||
"test 182 2025-11-05 2026-07-29 0.576923"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def assign_chronological_splits(\n",
|
||||
" frame: pd.DataFrame,\n",
|
||||
" train_fraction: float,\n",
|
||||
" validation_fraction: float,\n",
|
||||
" test_fraction: float,\n",
|
||||
") -> pd.Series:\n",
|
||||
" total_fraction = train_fraction + validation_fraction + test_fraction\n",
|
||||
" if not np.isclose(total_fraction, 1.0):\n",
|
||||
" raise ValueError(f\"Split fractions must sum to 1.0, got {total_fraction}\")\n",
|
||||
"\n",
|
||||
" n_rows = len(frame)\n",
|
||||
" train_end = int(n_rows * train_fraction)\n",
|
||||
" validation_end = train_end + int(n_rows * validation_fraction)\n",
|
||||
"\n",
|
||||
" split = pd.Series(index=frame.index, dtype=\"object\")\n",
|
||||
" split.iloc[:train_end] = \"train\"\n",
|
||||
" split.iloc[train_end:validation_end] = \"validation\"\n",
|
||||
" split.iloc[validation_end:] = \"test\"\n",
|
||||
" return split\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"df_model[\"split\"] = assign_chronological_splits(\n",
|
||||
" df_model,\n",
|
||||
" TRAIN_FRACTION,\n",
|
||||
" VALIDATION_FRACTION,\n",
|
||||
" TEST_FRACTION,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"split_summary = df_model.groupby(\"split\", sort=False).agg(\n",
|
||||
" rows=(TARGET_COLUMN, \"size\"),\n",
|
||||
" start_date=(TARGET_COLUMN, lambda values: values.index.min().date()),\n",
|
||||
" end_date=(TARGET_COLUMN, lambda values: values.index.max().date()),\n",
|
||||
" target_mean=(TARGET_COLUMN, \"mean\"),\n",
|
||||
")\n",
|
||||
"split_summary"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Inspect And Save\n",
|
||||
"\n",
|
||||
"`model_input_columns` is the source of truth for columns that are allowed into the model. The saved Parquet includes `date`, features, target, and split metadata."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Rows: 1,204\n",
|
||||
"Feature columns: ['SPY_ret_5', 'SPY_ret_20', 'SPY_dist_sma50', 'VIX_change_5', 'VIX_rank_20', 'TLT_ret_10', 'USO_ret_5', 'SPY_TLT_ratio_ret']\n",
|
||||
"Target column: spy_up_5d\n",
|
||||
"Training matrix shape: (842, 8)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>SPY_ret_5</th>\n",
|
||||
" <th>SPY_ret_20</th>\n",
|
||||
" <th>SPY_dist_sma50</th>\n",
|
||||
" <th>VIX_change_5</th>\n",
|
||||
" <th>VIX_rank_20</th>\n",
|
||||
" <th>TLT_ret_10</th>\n",
|
||||
" <th>USO_ret_5</th>\n",
|
||||
" <th>SPY_TLT_ratio_ret</th>\n",
|
||||
" <th>spy_up_5d</th>\n",
|
||||
" <th>split</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>count</th>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>unique</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>3</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>top</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>train</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>freq</th>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>842</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>mean</th>\n",
|
||||
" <td>0.002464</td>\n",
|
||||
" <td>0.009816</td>\n",
|
||||
" <td>0.010973</td>\n",
|
||||
" <td>0.017820</td>\n",
|
||||
" <td>0.404506</td>\n",
|
||||
" <td>-0.004103</td>\n",
|
||||
" <td>0.004938</td>\n",
|
||||
" <td>0.004838</td>\n",
|
||||
" <td>0.589701</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>std</th>\n",
|
||||
" <td>0.023044</td>\n",
|
||||
" <td>0.043515</td>\n",
|
||||
" <td>0.037287</td>\n",
|
||||
" <td>0.299050</td>\n",
|
||||
" <td>0.337773</td>\n",
|
||||
" <td>0.028469</td>\n",
|
||||
" <td>0.052606</td>\n",
|
||||
" <td>0.027586</td>\n",
|
||||
" <td>0.492092</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>min</th>\n",
|
||||
" <td>-0.114962</td>\n",
|
||||
" <td>-0.123975</td>\n",
|
||||
" <td>-0.141995</td>\n",
|
||||
" <td>-0.387709</td>\n",
|
||||
" <td>0.050000</td>\n",
|
||||
" <td>-0.092880</td>\n",
|
||||
" <td>-0.196652</td>\n",
|
||||
" <td>-0.117208</td>\n",
|
||||
" <td>0.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>25%</th>\n",
|
||||
" <td>-0.009749</td>\n",
|
||||
" <td>-0.016557</td>\n",
|
||||
" <td>-0.008704</td>\n",
|
||||
" <td>-0.056606</td>\n",
|
||||
" <td>0.100000</td>\n",
|
||||
" <td>-0.023139</td>\n",
|
||||
" <td>-0.025950</td>\n",
|
||||
" <td>-0.010029</td>\n",
|
||||
" <td>0.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>50%</th>\n",
|
||||
" <td>0.003749</td>\n",
|
||||
" <td>0.015639</td>\n",
|
||||
" <td>0.017755</td>\n",
|
||||
" <td>-0.018430</td>\n",
|
||||
" <td>0.300000</td>\n",
|
||||
" <td>-0.004429</td>\n",
|
||||
" <td>0.004869</td>\n",
|
||||
" <td>0.005508</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>75%</th>\n",
|
||||
" <td>0.015950</td>\n",
|
||||
" <td>0.038167</td>\n",
|
||||
" <td>0.038124</td>\n",
|
||||
" <td>0.030815</td>\n",
|
||||
" <td>0.750000</td>\n",
|
||||
" <td>0.014188</td>\n",
|
||||
" <td>0.032295</td>\n",
|
||||
" <td>0.021358</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>max</th>\n",
|
||||
" <td>0.082843</td>\n",
|
||||
" <td>0.157566</td>\n",
|
||||
" <td>0.088105</td>\n",
|
||||
" <td>3.846154</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>0.091322</td>\n",
|
||||
" <td>0.327273</td>\n",
|
||||
" <td>0.129204</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 VIX_rank_20 \\\n",
|
||||
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204.000000 \n",
|
||||
"unique NaN NaN NaN NaN NaN \n",
|
||||
"top NaN NaN NaN NaN NaN \n",
|
||||
"freq NaN NaN NaN NaN NaN \n",
|
||||
"mean 0.002464 0.009816 0.010973 0.017820 0.404506 \n",
|
||||
"std 0.023044 0.043515 0.037287 0.299050 0.337773 \n",
|
||||
"min -0.114962 -0.123975 -0.141995 -0.387709 0.050000 \n",
|
||||
"25% -0.009749 -0.016557 -0.008704 -0.056606 0.100000 \n",
|
||||
"50% 0.003749 0.015639 0.017755 -0.018430 0.300000 \n",
|
||||
"75% 0.015950 0.038167 0.038124 0.030815 0.750000 \n",
|
||||
"max 0.082843 0.157566 0.088105 3.846154 1.000000 \n",
|
||||
"\n",
|
||||
" TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret spy_up_5d split \n",
|
||||
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204 \n",
|
||||
"unique NaN NaN NaN NaN 3 \n",
|
||||
"top NaN NaN NaN NaN train \n",
|
||||
"freq NaN NaN NaN NaN 842 \n",
|
||||
"mean -0.004103 0.004938 0.004838 0.589701 NaN \n",
|
||||
"std 0.028469 0.052606 0.027586 0.492092 NaN \n",
|
||||
"min -0.092880 -0.196652 -0.117208 0.000000 NaN \n",
|
||||
"25% -0.023139 -0.025950 -0.010029 0.000000 NaN \n",
|
||||
"50% -0.004429 0.004869 0.005508 1.000000 NaN \n",
|
||||
"75% 0.014188 0.032295 0.021358 1.000000 NaN \n",
|
||||
"max 0.091322 0.327273 0.129204 1.000000 NaN "
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"model_input_columns = FEATURE_COLUMNS\n",
|
||||
"model_target_column = TARGET_COLUMN\n",
|
||||
"\n",
|
||||
"X_train = df_model.loc[df_model[\"split\"] == \"train\", model_input_columns]\n",
|
||||
"y_train = df_model.loc[df_model[\"split\"] == \"train\", model_target_column]\n",
|
||||
"\n",
|
||||
"print(f\"Rows: {len(df_model):,}\")\n",
|
||||
"print(f\"Feature columns: {model_input_columns}\")\n",
|
||||
"print(f\"Target column: {model_target_column}\")\n",
|
||||
"print(f\"Training matrix shape: {X_train.shape}\")\n",
|
||||
"\n",
|
||||
"df_model.describe(include=\"all\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(PosixPath('/home/jarno/repos/trading-bot/data/training/spy_direction_5d.parquet'),\n",
|
||||
" (1204, 11),\n",
|
||||
" date SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 \\\n",
|
||||
" 0 2021-10-11 0.014114 -0.026625 -0.018145 -0.090869 \n",
|
||||
" 1 2021-10-12 0.001201 -0.023752 -0.020386 -0.074091 \n",
|
||||
" 2 2021-10-13 0.000644 -0.028356 -0.016597 -0.081006 \n",
|
||||
" 3 2021-10-14 0.008754 -0.010443 -0.000214 -0.096759 \n",
|
||||
" 4 2021-10-15 0.018294 0.010127 0.007213 -0.079882 \n",
|
||||
" \n",
|
||||
" VIX_rank_20 TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret spy_up_5d split \n",
|
||||
" 0 0.20 -0.033135 0.031015 0.038908 1.0 train \n",
|
||||
" 1 0.10 -0.001041 0.008628 -0.001303 1.0 train \n",
|
||||
" 2 0.05 0.006928 0.036928 -0.005897 1.0 train \n",
|
||||
" 3 0.05 0.010809 0.026192 -0.011991 1.0 train \n",
|
||||
" 4 0.05 -0.002202 0.030287 -0.003823 1.0 train )"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)\n",
|
||||
"dataset_to_save = df_model.reset_index()\n",
|
||||
"dataset_to_save.to_parquet(OUTPUT_PATH, index=False)\n",
|
||||
"\n",
|
||||
"OUTPUT_PATH, dataset_to_save.shape, dataset_to_save.head()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.15"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,28 @@
|
||||
[project]
|
||||
name = "trading-bot"
|
||||
version = "0.1.0"
|
||||
description = "Python trading bot using machine learning and the Alpaca Markets API."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.12"
|
||||
dependencies = [
|
||||
"alpaca-py>=0.43.5",
|
||||
"matplotlib>=3.11.1",
|
||||
"pandas>=2.3.0",
|
||||
"pyarrow>=20.0.0",
|
||||
"pytest>=9.1.1",
|
||||
"python-dotenv>=1.2.2",
|
||||
"scikit-learn>=1.9.0",
|
||||
"xgboost>=3.2.0",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"ipykernel>=7.3.0",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
package = true
|
||||
|
||||
[build-system]
|
||||
requires = ["uv_build>=0.8.0,<0.9.0"]
|
||||
build-backend = "uv_build"
|
||||
@@ -0,0 +1 @@
|
||||
"""Trading bot package."""
|
||||
@@ -0,0 +1 @@
|
||||
"""Training data collection tools."""
|
||||
@@ -0,0 +1,255 @@
|
||||
"""Shared Alpaca daily candle helpers used by both the CLI fetcher and inference refreshes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime, time, timedelta
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pandas as pd
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load environment variables from .env file if available
|
||||
load_dotenv()
|
||||
|
||||
DEFAULT_OUTPUT_DIR = Path("data/alpaca/daily")
|
||||
DEFAULT_DURATION = "1 W"
|
||||
EASTERN_TZ = ZoneInfo("America/New_York")
|
||||
DURATION_PATTERN = re.compile(r"^(\d+)\s*([DWMY])$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DailyCandle:
|
||||
"""Daily OHLCV market data for one trading session."""
|
||||
|
||||
trading_day: date
|
||||
open: float
|
||||
high: float
|
||||
low: float
|
||||
close: float
|
||||
volume: int
|
||||
|
||||
|
||||
def default_end_date() -> date:
|
||||
"""Return yesterday's date in the US/Eastern market timezone."""
|
||||
|
||||
return datetime.now(EASTERN_TZ).date() - timedelta(days=1)
|
||||
|
||||
|
||||
def current_market_date() -> date:
|
||||
"""Return today's date in the US/Eastern market timezone."""
|
||||
|
||||
return datetime.now(EASTERN_TZ).date()
|
||||
|
||||
|
||||
def parse_end_date(value: str) -> date:
|
||||
"""Parse an end date in YYYYMMDD format."""
|
||||
|
||||
try:
|
||||
return datetime.strptime(value, "%Y%m%d").date()
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError(
|
||||
"end date must use YYYYMMDD format, such as 20250605"
|
||||
) from exc
|
||||
|
||||
|
||||
def parse_duration(value: str) -> str:
|
||||
"""Normalize and validate a duration string like '1 W' or '1 M'."""
|
||||
|
||||
normalized = " ".join(value.upper().split())
|
||||
if not DURATION_PATTERN.fullmatch(normalized):
|
||||
raise argparse.ArgumentTypeError(
|
||||
"duration must look like '1 D', '1 W', '1 M', or '1 Y'"
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def duration_to_start_date(end_date: date, duration: str) -> date:
|
||||
"""Calculate the start date based on end date and a duration string."""
|
||||
|
||||
match = DURATION_PATTERN.match(duration)
|
||||
if not match:
|
||||
raise ValueError(f"Invalid duration format: {duration}")
|
||||
|
||||
amount = int(match.group(1))
|
||||
unit = match.group(2)
|
||||
|
||||
if unit == "D":
|
||||
return end_date - timedelta(days=amount)
|
||||
if unit == "W":
|
||||
return end_date - timedelta(weeks=amount)
|
||||
if unit == "M":
|
||||
return end_date - timedelta(days=amount * 30)
|
||||
if unit == "Y":
|
||||
return end_date - timedelta(days=amount * 365)
|
||||
|
||||
raise ValueError(f"Unsupported duration unit: {unit}")
|
||||
|
||||
|
||||
def fetch_daily_candles(
|
||||
symbol: str,
|
||||
end_date: date,
|
||||
duration: str,
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
) -> list[DailyCandle]:
|
||||
"""Fetch daily candles for a symbol using Alpaca Market Data API."""
|
||||
|
||||
try:
|
||||
from alpaca.data.historical import StockHistoricalDataClient
|
||||
from alpaca.data.requests import StockBarsRequest
|
||||
from alpaca.data.timeframe import TimeFrame
|
||||
except ImportError as exc:
|
||||
raise SystemExit(
|
||||
"Missing dependency: alpaca-py. Install it via 'pip install alpaca-py' "
|
||||
"before running the script."
|
||||
) from exc
|
||||
|
||||
start_date = duration_to_start_date(end_date, duration)
|
||||
start_dt = datetime.combine(start_date, time.min, tzinfo=EASTERN_TZ)
|
||||
end_dt = datetime.combine(end_date, time.max, tzinfo=EASTERN_TZ)
|
||||
|
||||
client = StockHistoricalDataClient(api_key=api_key, secret_key=secret_key)
|
||||
|
||||
request_params = StockBarsRequest(
|
||||
symbol_or_symbols=symbol,
|
||||
timeframe=TimeFrame.Day,
|
||||
start=start_dt,
|
||||
end=end_dt,
|
||||
)
|
||||
|
||||
bars = client.get_stock_bars(request_params)
|
||||
|
||||
if not bars or symbol not in bars.data:
|
||||
print(f"Alpaca returned no historical bars for {symbol}.")
|
||||
return []
|
||||
|
||||
candles: list[DailyCandle] = []
|
||||
for bar in bars[symbol]:
|
||||
trading_day = bar.timestamp.astimezone(EASTERN_TZ).date()
|
||||
candles.append(
|
||||
DailyCandle(
|
||||
trading_day=trading_day,
|
||||
open=float(bar.open),
|
||||
high=float(bar.high),
|
||||
low=float(bar.low),
|
||||
close=float(bar.close),
|
||||
volume=int(bar.volume),
|
||||
)
|
||||
)
|
||||
|
||||
return candles
|
||||
|
||||
|
||||
def candles_to_frame(symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
|
||||
"""Convert candles to a date-indexed dataframe ready for Parquet storage."""
|
||||
|
||||
rows = [
|
||||
{
|
||||
"date": candle.trading_day,
|
||||
"symbol": symbol,
|
||||
"open": candle.open,
|
||||
"high": candle.high,
|
||||
"low": candle.low,
|
||||
"close": candle.close,
|
||||
"volume": candle.volume,
|
||||
}
|
||||
for candle in candles
|
||||
]
|
||||
frame = pd.DataFrame.from_records(rows)
|
||||
if frame.empty:
|
||||
return pd.DataFrame(
|
||||
columns=["symbol", "open", "high", "low", "close", "volume"],
|
||||
index=pd.Index([], name="date"),
|
||||
)
|
||||
|
||||
frame["date"] = pd.to_datetime(frame["date"]).dt.date
|
||||
return frame.set_index("date")
|
||||
|
||||
|
||||
def read_existing_candles(path: Path) -> pd.DataFrame:
|
||||
"""Read an existing candle Parquet file as a date-indexed dataframe."""
|
||||
|
||||
if not path.exists():
|
||||
return pd.DataFrame(
|
||||
columns=["symbol", "open", "high", "low", "close", "volume"],
|
||||
index=pd.Index([], name="date"),
|
||||
)
|
||||
|
||||
frame = pd.read_parquet(path)
|
||||
if "date" in frame.columns:
|
||||
frame["date"] = pd.to_datetime(frame["date"]).dt.date
|
||||
frame = frame.set_index("date")
|
||||
|
||||
frame.index = pd.to_datetime(frame.index).date
|
||||
frame.index.name = "date"
|
||||
return frame
|
||||
|
||||
|
||||
def oldest_stored_date_or_today(path: Path) -> date:
|
||||
"""Return the oldest stored candle date, or today if no data exists yet."""
|
||||
|
||||
existing = read_existing_candles(path)
|
||||
if existing.empty:
|
||||
return current_market_date()
|
||||
return min(existing.index)
|
||||
|
||||
|
||||
def write_candles(path: Path, symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
|
||||
"""Append candles to a ticker Parquet file, keeping one row per date."""
|
||||
|
||||
existing = read_existing_candles(path)
|
||||
fetched = candles_to_frame(symbol, candles)
|
||||
combined = pd.concat([existing, fetched])
|
||||
if not combined.empty:
|
||||
combined = combined[~combined.index.duplicated(keep="last")]
|
||||
combined = combined.sort_index()
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
combined.to_parquet(path, index=True)
|
||||
return combined
|
||||
|
||||
|
||||
def print_candles(symbol: str, candles: list[DailyCandle]) -> None:
|
||||
"""Print candles in a compact table."""
|
||||
|
||||
print("date,symbol,open,high,low,close,volume")
|
||||
for candle in candles:
|
||||
print(
|
||||
f"{candle.trading_day.isoformat()},"
|
||||
f"{symbol},"
|
||||
f"{candle.open:.2f},"
|
||||
f"{candle.high:.2f},"
|
||||
f"{candle.low:.2f},"
|
||||
f"{candle.close:.2f},"
|
||||
f"{candle.volume}"
|
||||
)
|
||||
|
||||
|
||||
def refresh_recent_market_data(
|
||||
output_dir: Path | str,
|
||||
symbols: dict[str, str] | None = None,
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
duration: str = DEFAULT_DURATION,
|
||||
) -> None:
|
||||
"""Refresh the latest weekly Alpaca candle parquet files for the provided symbols."""
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
symbols = symbols or {}
|
||||
|
||||
for symbol in symbols.keys():
|
||||
output_path = output_dir / f"{symbol}.parquet"
|
||||
candles = fetch_daily_candles(
|
||||
symbol=symbol,
|
||||
end_date=default_end_date(),
|
||||
duration=duration,
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
)
|
||||
print(f"Fetched {len(candles)} daily candles for {symbol}.")
|
||||
write_candles(output_path, symbol, candles)
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Alpaca daily candle fetcher CLI wrapper.
|
||||
|
||||
The reusable implementation lives in the shared library module so the same
|
||||
fetching and parquet persistence logic can be reused by the prediction flow.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import (
|
||||
DEFAULT_DURATION,
|
||||
DEFAULT_OUTPUT_DIR,
|
||||
DailyCandle,
|
||||
candles_to_frame,
|
||||
current_market_date,
|
||||
default_end_date,
|
||||
duration_to_start_date,
|
||||
fetch_daily_candles,
|
||||
oldest_stored_date_or_today,
|
||||
parse_duration,
|
||||
parse_end_date,
|
||||
print_candles,
|
||||
read_existing_candles,
|
||||
write_candles,
|
||||
)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""Parse command line arguments."""
|
||||
|
||||
parser = argparse.ArgumentParser(description="Fetch daily Alpaca candles.")
|
||||
parser.add_argument("symbol", help="Ticker symbol to fetch, such as SPY.")
|
||||
end_date_group = parser.add_mutually_exclusive_group()
|
||||
end_date_group.add_argument(
|
||||
"--end-date",
|
||||
type=parse_end_date,
|
||||
help="Request end date in YYYYMMDD format. Defaults to yesterday.",
|
||||
)
|
||||
end_date_group.add_argument(
|
||||
"--end-date-from-parquet",
|
||||
action="store_true",
|
||||
help=(
|
||||
"Use the oldest date from the symbol Parquet file as the request "
|
||||
"end date. Defaults to today if the file is missing or empty."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--duration",
|
||||
type=parse_duration,
|
||||
default=DEFAULT_DURATION,
|
||||
help=(
|
||||
"Duration string, such as '1 W', '1 M', or '1 Y'. "
|
||||
f"Defaults to {DEFAULT_DURATION}."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_OUTPUT_DIR,
|
||||
help=f"Directory for Parquet files. Defaults to {DEFAULT_OUTPUT_DIR}.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--api-key",
|
||||
default=os.getenv("ALPACA_API_KEY"),
|
||||
help="Alpaca API Key ID (defaults to ALPACA_API_KEY env var or .env).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--secret-key",
|
||||
default=os.getenv("ALPACA_SECRET_KEY"),
|
||||
help="Alpaca Secret Key (defaults to ALPACA_SECRET_KEY env var or .env).",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Run the daily candle fetcher."""
|
||||
|
||||
args = parse_args()
|
||||
symbol = args.symbol.upper()
|
||||
output_path = args.output_dir / f"{symbol}.parquet"
|
||||
end_date = (
|
||||
oldest_stored_date_or_today(output_path)
|
||||
if args.end_date_from_parquet
|
||||
else args.end_date or default_end_date()
|
||||
)
|
||||
|
||||
print(
|
||||
f"Fetching {symbol} daily candles ending {end_date:%Y-%m-%d} "
|
||||
f"for duration {args.duration} via Alpaca API"
|
||||
)
|
||||
candles = fetch_daily_candles(
|
||||
symbol,
|
||||
end_date,
|
||||
args.duration,
|
||||
api_key=args.api_key,
|
||||
secret_key=args.secret_key,
|
||||
)
|
||||
print(f"Fetched {len(candles)} daily candles for {symbol}.")
|
||||
stored = write_candles(output_path, symbol, candles)
|
||||
print(f"Wrote {len(stored)} total daily rows to {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,207 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate training dataset and train an XGBoost model using a JSON config.
|
||||
|
||||
This script combines the logic from notebooks/spy_direction_dataset.ipynb
|
||||
and notebooks/train-xboost.ipynb into a single runnable script.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import xgboost as xgb
|
||||
from sklearn.metrics import classification_report, accuracy_score
|
||||
|
||||
|
||||
def find_project_root(start: Path | None = None) -> Path:
|
||||
current = (start or Path.cwd()).resolve()
|
||||
for candidate in [current, *current.parents]:
|
||||
if (candidate / "pyproject.toml").exists():
|
||||
return candidate
|
||||
raise RuntimeError("Could not find project root containing pyproject.toml")
|
||||
|
||||
|
||||
def load_close(raw_dir: Path, symbol: str, alias: str | None = None) -> pd.DataFrame:
|
||||
alias = alias or symbol
|
||||
path = raw_dir / f"{symbol}.parquet"
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Missing raw data file: {path}")
|
||||
|
||||
frame = pd.read_parquet(path)
|
||||
if "date" in frame.columns:
|
||||
frame = frame.set_index("date")
|
||||
if "close" not in frame.columns:
|
||||
raise ValueError(f"{path} does not contain a close column")
|
||||
|
||||
frame = frame.copy()
|
||||
frame.index = pd.to_datetime(frame.index)
|
||||
frame.index.name = "date"
|
||||
frame = frame.sort_index()
|
||||
return frame[["close"]].rename(columns={"close": f"{alias}_close"})
|
||||
|
||||
|
||||
def assign_chronological_splits(frame: pd.DataFrame, train_fraction: float, validation_fraction: float, test_fraction: float) -> pd.Series:
|
||||
total_fraction = train_fraction + validation_fraction + test_fraction
|
||||
if not np.isclose(total_fraction, 1.0):
|
||||
raise ValueError(f"Split fractions must sum to 1.0, got {total_fraction}")
|
||||
|
||||
n_rows = len(frame)
|
||||
train_end = int(n_rows * train_fraction)
|
||||
validation_end = train_end + int(n_rows * validation_fraction)
|
||||
|
||||
split = pd.Series(index=frame.index, dtype="object")
|
||||
split.iloc[:train_end] = "train"
|
||||
split.iloc[train_end:validation_end] = "validation"
|
||||
split.iloc[validation_end:] = "test"
|
||||
return split
|
||||
|
||||
|
||||
def build_dataset(raw_data_dir: Path, output_path: Path, symbols: dict, fractions: dict) -> pd.DataFrame:
|
||||
prices = pd.concat(
|
||||
[
|
||||
load_close(raw_data_dir, symbols["SPY"], "SPY"),
|
||||
load_close(raw_data_dir, symbols["VIX"] , "VIX"),
|
||||
load_close(raw_data_dir, symbols["TLT"] , "TLT"),
|
||||
load_close(raw_data_dir, symbols["USO"] , "USO"),
|
||||
],
|
||||
axis=1,
|
||||
join="inner",
|
||||
)
|
||||
|
||||
df = prices.copy()
|
||||
df["SPY_ret_5"] = df["SPY_close"].pct_change(5)
|
||||
df["SPY_ret_20"] = df["SPY_close"].pct_change(20)
|
||||
|
||||
sma_50 = df["SPY_close"].rolling(50).mean()
|
||||
df["SPY_dist_sma50"] = (df["SPY_close"] - sma_50) / sma_50
|
||||
|
||||
df["VIX_change_5"] = df["VIX_close"].pct_change(5)
|
||||
df["VIX_rank_20"] = df["VIX_close"].rolling(20).rank(pct=True)
|
||||
|
||||
df["TLT_ret_10"] = df["TLT_close"].pct_change(10)
|
||||
df["USO_ret_5"] = df["USO_close"].pct_change(5)
|
||||
df["SPY_TLT_ratio_ret"] = (df["SPY_close"] / df["TLT_close"]).pct_change(5)
|
||||
|
||||
spy_forward_close = df["SPY_close"].shift(-5)
|
||||
df["spy_up_5d"] = np.nan
|
||||
df.loc[spy_forward_close > df["SPY_close"], "spy_up_5d"] = 1.0
|
||||
df.loc[spy_forward_close < df["SPY_close"], "spy_up_5d"] = 0.0
|
||||
df.loc[spy_forward_close == df["SPY_close"], "spy_up_5d"] = 0.5
|
||||
|
||||
FEATURE_COLUMNS = [
|
||||
"SPY_ret_5",
|
||||
"SPY_ret_20",
|
||||
"SPY_dist_sma50",
|
||||
"VIX_change_5",
|
||||
"VIX_rank_20",
|
||||
"TLT_ret_10",
|
||||
"USO_ret_5",
|
||||
"SPY_TLT_ratio_ret",
|
||||
]
|
||||
TARGET_COLUMN = "spy_up_5d"
|
||||
|
||||
df_model = df[FEATURE_COLUMNS + [TARGET_COLUMN]].dropna().copy()
|
||||
|
||||
# Drop unchanged targets (0.5) to keep binary classification
|
||||
df_model = df_model[df_model[TARGET_COLUMN] != 0.5].copy()
|
||||
df_model[TARGET_COLUMN] = df_model[TARGET_COLUMN].astype(int)
|
||||
|
||||
df_model["split"] = assign_chronological_splits(
|
||||
df_model,
|
||||
fractions["train"],
|
||||
fractions["validation"],
|
||||
fractions["test"],
|
||||
)
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
dataset_to_save = df_model.reset_index()
|
||||
dataset_to_save.to_parquet(output_path, index=False)
|
||||
|
||||
return df_model, FEATURE_COLUMNS, TARGET_COLUMN
|
||||
|
||||
|
||||
def train_model(df_model: pd.DataFrame, feature_cols: list, target_col: str, config: dict, models_dir: Path):
|
||||
train_df = df_model[df_model["split"] == "train"]
|
||||
val_df = df_model[df_model["split"] == "validation"]
|
||||
test_df = df_model[df_model["split"] == "test"]
|
||||
|
||||
X_train, y_train = train_df[feature_cols], train_df[target_col]
|
||||
X_val, y_val = val_df[feature_cols], val_df[target_col]
|
||||
X_test, y_test = test_df[feature_cols], test_df[target_col]
|
||||
|
||||
clf_params = dict(config)
|
||||
# Ensure early_stopping_rounds present as int
|
||||
|
||||
model = xgb.XGBClassifier(**clf_params)
|
||||
|
||||
eval_set = [(X_train, y_train), (X_val, y_val)]
|
||||
fit_kwargs = {"eval_set": eval_set, "verbose": False}
|
||||
|
||||
model.fit(X_train, y_train, **fit_kwargs)
|
||||
|
||||
print(f"Best iteration: {getattr(model, 'best_iteration', None)}")
|
||||
evals = model.evals_result()
|
||||
if "validation_1" in evals and "logloss" in evals["validation_1"]:
|
||||
val_loss = evals["validation_1"]["logloss"]
|
||||
print(f"Starting Validation Loss: {val_loss[0]:.4f}")
|
||||
print(f"Final Validation Loss: {val_loss[-1]:.4f}")
|
||||
|
||||
# Evaluate on test set
|
||||
y_pred = model.predict(X_test)
|
||||
print("\n--- Final Holdout Test Performance ---")
|
||||
print(f"Test Set Accuracy: {accuracy_score(y_test, y_pred):.2%}\n")
|
||||
print(classification_report(y_test, y_pred))
|
||||
|
||||
# Save model and metadata
|
||||
models_dir.mkdir(parents=True, exist_ok=True)
|
||||
model_path = models_dir / "spy_xgb_v1.json"
|
||||
meta_path = models_dir / "spy_xgb_v1_meta.json"
|
||||
model.save_model(str(model_path))
|
||||
|
||||
metadata = {
|
||||
"p_base": float(y_train.mean()),
|
||||
"feature_cols": feature_cols,
|
||||
"last_trained_date": str(df_model.index.max().date()),
|
||||
"config": config,
|
||||
}
|
||||
with open(meta_path, "w") as f:
|
||||
json.dump(metadata, f, indent=2)
|
||||
|
||||
print(f"Saved model to {model_path}")
|
||||
|
||||
|
||||
def main():
|
||||
project_root = find_project_root()
|
||||
|
||||
# Defaults
|
||||
raw_data_dir = project_root / "data" / "alpaca" / "daily"
|
||||
output_path = project_root / "data" / "training" / "spy_direction_5d.parquet"
|
||||
models_dir = project_root / "models"
|
||||
config_path = project_root / "config" / "train_config.json"
|
||||
|
||||
# Load config
|
||||
if not config_path.exists():
|
||||
raise FileNotFoundError(f"Config file not found: {config_path}")
|
||||
with open(config_path, "r") as f:
|
||||
config = json.load(f)
|
||||
|
||||
symbols = config.get("symbols", {"SPY": "SPY", "VIX": config.get("vix_symbol", "VIXY"), "TLT": "TLT", "USO": "USO"})
|
||||
fractions = config.get("fractions", {"train": 0.7, "validation": 0.15, "test": 0.15})
|
||||
|
||||
print("Building dataset...")
|
||||
df_model, feature_cols, target_col = build_dataset(raw_data_dir, output_path, symbols, fractions)
|
||||
|
||||
print(f"Rows: {len(df_model):,}")
|
||||
print(f"Feature columns: {feature_cols}")
|
||||
print(f"Target column: {target_col}")
|
||||
|
||||
split_counts = df_model.groupby("split").size()
|
||||
print("Split counts:")
|
||||
print(split_counts.to_string())
|
||||
|
||||
print("\nTraining model...")
|
||||
model_config = config.get("model", {})
|
||||
train_model(df_model, feature_cols, target_col, model_config, models_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,11 @@
|
||||
"""Model-related modules for the trading bot."""
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_SYMBOLS",
|
||||
"FEATURE_COLUMNS",
|
||||
"PredictionResult",
|
||||
"compute_features",
|
||||
"get_latest_inference_features",
|
||||
"load_raw_close_prices",
|
||||
"predict_latest_probability",
|
||||
]
|
||||
@@ -0,0 +1,277 @@
|
||||
"""Prediction pipeline for the initial trading bot prototype.
|
||||
|
||||
This module mirrors the notebook prototype by loading raw daily OHLCV parquet
|
||||
files, deriving the engineered feature set used during training, and using the
|
||||
trained XGBoost model's predict_proba() method to emit a probability for the
|
||||
latest available observation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import warnings
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import xgboost as xgb
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import refresh_recent_market_data
|
||||
|
||||
# Load environment variables from .env file if available
|
||||
load_dotenv()
|
||||
|
||||
DEFAULT_DATA_DIR = Path("data/alpaca/daily")
|
||||
|
||||
FEATURE_COLUMNS = [
|
||||
"SPY_ret_5",
|
||||
"SPY_ret_20",
|
||||
"SPY_dist_sma50",
|
||||
"VIX_change_5",
|
||||
"VIX_rank_20",
|
||||
"TLT_ret_10",
|
||||
"USO_ret_5",
|
||||
"SPY_TLT_ratio_ret",
|
||||
]
|
||||
|
||||
DEFAULT_SYMBOLS = {
|
||||
"SPY": "SPY",
|
||||
"VIXY": "VIX",
|
||||
"TLT": "TLT",
|
||||
"USO": "USO",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PredictionResult:
|
||||
"""Container for the latest model prediction."""
|
||||
|
||||
prediction_date: pd.Timestamp
|
||||
probability: float
|
||||
base_probability: float
|
||||
feature_columns: list[str]
|
||||
|
||||
|
||||
def load_raw_close_prices(
|
||||
raw_data_dir: Path | str, symbols: dict[str, str] | None = None
|
||||
) -> pd.DataFrame:
|
||||
"""Load raw parquet close prices and merge them into a single frame."""
|
||||
|
||||
symbols = symbols or DEFAULT_SYMBOLS
|
||||
raw_data_dir = Path(raw_data_dir)
|
||||
|
||||
frames: list[pd.DataFrame] = []
|
||||
for symbol, alias in symbols.items():
|
||||
path = raw_data_dir / f"{symbol}.parquet"
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Missing raw data file: {path}")
|
||||
|
||||
frame = pd.read_parquet(path)
|
||||
if "date" in frame.columns:
|
||||
frame = frame.set_index("date")
|
||||
|
||||
frame.index = pd.to_datetime(frame.index)
|
||||
frame = frame.sort_index()
|
||||
frames.append(frame[["close"]].rename(columns={"close": f"{alias}_close"}))
|
||||
|
||||
return pd.concat(frames, axis=1, join="inner")
|
||||
|
||||
|
||||
def compute_features(prices: pd.DataFrame) -> pd.DataFrame:
|
||||
"""Compute the engineered feature matrix used for inference."""
|
||||
|
||||
df = prices.copy()
|
||||
|
||||
df["SPY_ret_5"] = df["SPY_close"].pct_change(5)
|
||||
df["SPY_ret_20"] = df["SPY_close"].pct_change(20)
|
||||
|
||||
sma_50 = df["SPY_close"].rolling(50).mean()
|
||||
df["SPY_dist_sma50"] = (df["SPY_close"] - sma_50) / sma_50
|
||||
|
||||
df["VIX_change_5"] = df["VIX_close"].pct_change(5)
|
||||
df["VIX_rank_20"] = df["VIX_close"].rolling(20).rank(pct=True)
|
||||
|
||||
df["TLT_ret_10"] = df["TLT_close"].pct_change(10)
|
||||
df["USO_ret_5"] = df["USO_close"].pct_change(5)
|
||||
df["SPY_TLT_ratio_ret"] = (df["SPY_close"] / df["TLT_close"]).pct_change(5)
|
||||
|
||||
return df[FEATURE_COLUMNS]
|
||||
|
||||
|
||||
def get_latest_inference_features(
|
||||
raw_data_dir: Path | str,
|
||||
symbols: dict[str, str] | None = None,
|
||||
max_age_days: int = 1,
|
||||
) -> tuple[pd.Timestamp, pd.DataFrame]:
|
||||
"""Return the latest row of features and its observation date."""
|
||||
|
||||
prices = load_raw_close_prices(raw_data_dir, symbols)
|
||||
features = compute_features(prices).dropna()
|
||||
if features.empty:
|
||||
raise ValueError(
|
||||
"Not enough historical rows to compute 50-day rolling window features."
|
||||
)
|
||||
|
||||
latest_row = features.iloc[[-1]]
|
||||
latest_date = latest_row.index[0]
|
||||
|
||||
now = pd.Timestamp.now()
|
||||
latest_date_naive = (
|
||||
latest_date.tz_localize(None)
|
||||
if latest_date.tz is not None
|
||||
else latest_date
|
||||
)
|
||||
days_old = (now.floor("D") - latest_date_naive.floor("D")).days
|
||||
|
||||
if days_old > max_age_days:
|
||||
warnings.warn(
|
||||
f"STALE DATA WARNING: Latest feature row is from {latest_date.strftime('%Y-%m-%d')} "
|
||||
f"({days_old} day(s) old). Update raw Parquet files before executing trades.",
|
||||
UserWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
|
||||
return latest_date, latest_row
|
||||
|
||||
|
||||
def load_model(model_path: Path | str) -> xgb.XGBClassifier:
|
||||
"""Load the serialized XGBoost classifier from disk."""
|
||||
|
||||
model = xgb.XGBClassifier()
|
||||
model.load_model(str(model_path))
|
||||
return model
|
||||
|
||||
|
||||
def load_feature_metadata(metadata_path: Path | str) -> dict[str, Any]:
|
||||
"""Load the model metadata JSON file."""
|
||||
|
||||
with Path(metadata_path).open("r", encoding="utf-8") as handle:
|
||||
return json.load(handle)
|
||||
|
||||
|
||||
def predict_latest_probability(
|
||||
model_path: Path | str,
|
||||
metadata_path: Path | str,
|
||||
raw_data_dir: Path | str,
|
||||
symbols: dict[str, str] | None = None,
|
||||
max_age_days: int = 1,
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
fetch_recent_data: bool = True,
|
||||
) -> PredictionResult:
|
||||
"""Return the latest predicted probability for the next SPY move."""
|
||||
|
||||
if fetch_recent_data:
|
||||
refresh_recent_market_data(
|
||||
output_dir=raw_data_dir,
|
||||
symbols=symbols or DEFAULT_SYMBOLS,
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
)
|
||||
|
||||
model = load_model(model_path)
|
||||
metadata = load_feature_metadata(metadata_path)
|
||||
latest_date, latest_features = get_latest_inference_features(
|
||||
raw_data_dir=raw_data_dir,
|
||||
symbols=symbols,
|
||||
max_age_days=max_age_days,
|
||||
)
|
||||
|
||||
feature_columns = list(metadata.get("feature_cols", FEATURE_COLUMNS))
|
||||
model_input = latest_features[feature_columns]
|
||||
probabilities = model.predict_proba(model_input)
|
||||
probability = float(probabilities[0, 1])
|
||||
base_probability = float(metadata.get("p_base", 0.5))
|
||||
|
||||
return PredictionResult(
|
||||
prediction_date=latest_date,
|
||||
probability=probability,
|
||||
base_probability=base_probability,
|
||||
feature_columns=feature_columns,
|
||||
)
|
||||
|
||||
|
||||
def get_target_exposure(
|
||||
p_pred: float, p_base: float, sensitivity: float = 5.0
|
||||
) -> float:
|
||||
"""Maps predicted probability to a target portfolio equity allocation (0.0 to 1.0).
|
||||
|
||||
- p_pred == p_base --> 50% Target Exposure (Neutral)
|
||||
- p_pred > p_base --> Scale up toward 100% (Bullish)
|
||||
- p_pred < p_base --> Scale down toward 0% (Bearish / Cash)
|
||||
"""
|
||||
# Calculate deviation from the historical average
|
||||
delta = p_pred - p_base
|
||||
|
||||
# Base target allocation is 50% equity / 50% cash
|
||||
base_allocation = 0.50
|
||||
|
||||
# Sensitivity controls how aggressively probability changes alter allocation
|
||||
# e.g., a +0.08 delta * 5.0 = +0.40 -> 90% Equity Allocation
|
||||
target_allocation = base_allocation + (delta * sensitivity)
|
||||
|
||||
# Clamp bounds strictly between 0% (full cash) and 100% (full SPY)
|
||||
return float(np.clip(target_allocation, 0.0, 1.0))
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""Parse command line arguments for local prediction runs."""
|
||||
|
||||
parser = argparse.ArgumentParser(description="Predict the latest SPY direction probability.")
|
||||
parser.add_argument(
|
||||
"--model-path",
|
||||
type=Path,
|
||||
default=Path("notebooks/models/spy_xgb_v1.json"),
|
||||
help="Path to the XGBoost model artifact.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--metadata-path",
|
||||
type=Path,
|
||||
default=Path("notebooks/models/spy_xgb_v1_meta.json"),
|
||||
help="Path to the model metadata JSON file.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--data-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_DATA_DIR,
|
||||
help=f"Directory containing the raw Parquet market data files. Defaults to {DEFAULT_DATA_DIR}.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--api-key",
|
||||
default=os.getenv("ALPACA_API_KEY"),
|
||||
help="Alpaca API Key ID (defaults to ALPACA_API_KEY env var or .env).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--secret-key",
|
||||
default=os.getenv("ALPACA_SECRET_KEY"),
|
||||
help="Alpaca Secret Key (defaults to ALPACA_SECRET_KEY env var or .env).",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Run the latest prediction from the command line."""
|
||||
|
||||
args = parse_args()
|
||||
result = predict_latest_probability(
|
||||
model_path=args.model_path,
|
||||
metadata_path=args.metadata_path,
|
||||
raw_data_dir=args.data_dir,
|
||||
api_key=args.api_key,
|
||||
secret_key=args.secret_key,
|
||||
fetch_recent_data=True,
|
||||
)
|
||||
print(
|
||||
f"Date: {result.prediction_date.date()} | Prob: {result.probability:.4f} | "
|
||||
f"Base: {result.base_probability:.4f}"
|
||||
)
|
||||
print(f"Exposure: {get_target_exposure(result.probability, result.base_probability, sensitivity=5.0):.2f}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,439 @@
|
||||
"""Portfolio and open order inspection for Alpaca trading."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from alpaca.trading.client import TradingClient
|
||||
from alpaca.trading.enums import OrderSide, OrderType, QueryOrderStatus, TimeInForce
|
||||
from alpaca.trading.models import Order, Position, TradeAccount
|
||||
from alpaca.trading.requests import GetOrdersRequest, OrderRequest
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from trading_bot.models.prediction import (
|
||||
DEFAULT_DATA_DIR,
|
||||
get_target_exposure,
|
||||
predict_latest_probability,
|
||||
)
|
||||
|
||||
load_dotenv()
|
||||
|
||||
DEFAULT_SYMBOL = "SPY"
|
||||
DEFAULT_MODEL_PATH = "notebooks/models/spy_xgb_v1.json"
|
||||
DEFAULT_METADATA_PATH = "notebooks/models/spy_xgb_v1_meta.json"
|
||||
DEFAULT_MIN_ORDER_DOLLARS = 25.0
|
||||
DEFAULT_SENSITIVITY = 25.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AlpacaOrderSummary:
|
||||
symbol: str
|
||||
side: str
|
||||
qty: float
|
||||
filled_qty: float
|
||||
unfilled_qty: float
|
||||
limit_price: float | None
|
||||
order_type: str
|
||||
status: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AlpacaPortfolioSummary:
|
||||
cash: float
|
||||
spy_quantity: float
|
||||
spy_market_value: float
|
||||
spy_avg_entry_price: float | None
|
||||
current_cash_to_spy_ratio: float | None
|
||||
projected_cash_to_spy_ratio: float | None
|
||||
open_spy_order_count: int
|
||||
estimated_spy_price: float | None
|
||||
|
||||
|
||||
def _string_to_float(value: Any, default: float = 0.0) -> float:
|
||||
if value is None:
|
||||
return default
|
||||
if isinstance(value, (float, int)):
|
||||
return float(value)
|
||||
if isinstance(value, str):
|
||||
value = value.strip()
|
||||
if value == "":
|
||||
return default
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def create_alpaca_trading_client(
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
paper: bool = True,
|
||||
) -> TradingClient:
|
||||
if api_key is None or secret_key is None:
|
||||
raise ValueError(
|
||||
"Alpaca API key and secret key must be provided via CLI arguments or environment variables."
|
||||
)
|
||||
|
||||
return TradingClient(api_key, secret_key, paper=paper)
|
||||
|
||||
|
||||
def _extract_order_limit_price(order: Order) -> float | None:
|
||||
value = order.limit_price
|
||||
if value is None:
|
||||
return None
|
||||
price = _string_to_float(value, default=0.0)
|
||||
return price if price is not None and price > 0 else None
|
||||
|
||||
|
||||
def _build_order_summary(order: Order) -> AlpacaOrderSummary:
|
||||
symbol = str(order.symbol).upper()
|
||||
side = str(order.side).lower()
|
||||
qty = _string_to_float(order.qty or "0.0")
|
||||
filled_qty = _string_to_float(order.filled_qty or "0.0")
|
||||
limit_price = _extract_order_limit_price(order)
|
||||
order_type = order.type or OrderType.MARKET
|
||||
status = order.status or QueryOrderStatus.OPEN
|
||||
|
||||
return AlpacaOrderSummary(
|
||||
symbol=symbol,
|
||||
side=side,
|
||||
qty=qty,
|
||||
filled_qty=filled_qty,
|
||||
unfilled_qty=max(qty - filled_qty, 0.0),
|
||||
limit_price=limit_price,
|
||||
order_type=order_type,
|
||||
status=status,
|
||||
)
|
||||
|
||||
|
||||
def _find_spy_position(positions: list[Position], symbol: str = DEFAULT_SYMBOL) -> tuple[float, float, float | None]:
|
||||
normalized = symbol.upper()
|
||||
for position in positions:
|
||||
position_symbol = position.symbol.upper()
|
||||
if position_symbol != normalized:
|
||||
continue
|
||||
|
||||
quantity = _string_to_float(position.qty or "0.0")
|
||||
market_value = _string_to_float(position.market_value or "0.0")
|
||||
avg_entry_price = _string_to_float(position.avg_entry_price)
|
||||
return quantity, market_value, avg_entry_price
|
||||
|
||||
return 0.0, 0.0, None
|
||||
|
||||
|
||||
def _safe_ratio(numerator: float, denominator: float) -> float | None:
|
||||
if denominator <= 0:
|
||||
return None
|
||||
return numerator / denominator
|
||||
|
||||
|
||||
def get_current_spy_exposure(cash: float, spy_market_value: float) -> float | None:
|
||||
total_equity = cash + spy_market_value
|
||||
if total_equity <= 0:
|
||||
return None
|
||||
return float(spy_market_value / total_equity)
|
||||
|
||||
|
||||
def calculate_spy_order_notional(
|
||||
cash: float,
|
||||
spy_market_value: float,
|
||||
target_exposure: float,
|
||||
min_order_dollars: float = DEFAULT_MIN_ORDER_DOLLARS,
|
||||
) -> float | None:
|
||||
total_equity = cash + spy_market_value
|
||||
if total_equity <= 0:
|
||||
return None
|
||||
|
||||
current_exposure = get_current_spy_exposure(cash, spy_market_value)
|
||||
if current_exposure is None:
|
||||
return None
|
||||
|
||||
dollar_delta = (target_exposure - current_exposure) * total_equity
|
||||
if abs(dollar_delta) < min_order_dollars:
|
||||
return None
|
||||
|
||||
if dollar_delta > 0:
|
||||
return min(dollar_delta, cash)
|
||||
return -min(abs(dollar_delta), spy_market_value)
|
||||
|
||||
|
||||
def _build_spy_market_order_request(symbol: str, notional: float) -> OrderRequest:
|
||||
return OrderRequest(
|
||||
symbol=symbol,
|
||||
notional=abs(notional),
|
||||
side=OrderSide.BUY if notional > 0 else OrderSide.SELL,
|
||||
type=OrderType.MARKET,
|
||||
time_in_force=TimeInForce.DAY,
|
||||
)
|
||||
|
||||
|
||||
def cancel_open_alpaca_orders(client: TradingClient) -> None:
|
||||
client.cancel_orders()
|
||||
|
||||
|
||||
def rebalance_alpaca_portfolio(
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
paper: bool = True,
|
||||
symbol: str = DEFAULT_SYMBOL,
|
||||
model_path: str = DEFAULT_MODEL_PATH,
|
||||
metadata_path: str = DEFAULT_METADATA_PATH,
|
||||
raw_data_dir: str | Path = DEFAULT_DATA_DIR,
|
||||
min_order_dollars: float = DEFAULT_MIN_ORDER_DOLLARS,
|
||||
sensitivity: float = DEFAULT_SENSITIVITY,
|
||||
max_age_days: int = 1,
|
||||
fetch_recent_data: bool = True,
|
||||
) -> tuple[AlpacaPortfolioSummary, Order | dict[str, Any] | None]:
|
||||
client = create_alpaca_trading_client(api_key=api_key, secret_key=secret_key, paper=paper)
|
||||
summary = fetch_alpaca_portfolio_summary(
|
||||
client=client,
|
||||
symbol=symbol,
|
||||
)
|
||||
|
||||
prediction = predict_latest_probability(
|
||||
model_path=model_path,
|
||||
metadata_path=metadata_path,
|
||||
raw_data_dir=raw_data_dir,
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
fetch_recent_data=fetch_recent_data,
|
||||
max_age_days=max_age_days,
|
||||
)
|
||||
|
||||
target_exposure = get_target_exposure(
|
||||
prediction.probability,
|
||||
prediction.base_probability,
|
||||
sensitivity=sensitivity,
|
||||
)
|
||||
current_exposure = get_current_spy_exposure(summary.cash, summary.spy_market_value) or 0.0
|
||||
|
||||
order_notional = calculate_spy_order_notional(
|
||||
summary.cash,
|
||||
summary.spy_market_value,
|
||||
target_exposure,
|
||||
min_order_dollars,
|
||||
)
|
||||
|
||||
if order_notional is None:
|
||||
print(
|
||||
f"Target exposure {target_exposure:.4f} is close to current exposure {current_exposure:.4f}; "
|
||||
f"skipping trades below ${min_order_dollars:.2f}."
|
||||
)
|
||||
return summary, None
|
||||
|
||||
cancel_open_alpaca_orders(client)
|
||||
order_request = _build_spy_market_order_request(symbol, round(order_notional, 2))
|
||||
order = client.submit_order(order_request)
|
||||
|
||||
print(
|
||||
f"Placed {'buy' if order_notional > 0 else 'sell'} market order for ${abs(order_notional):,.2f} of {symbol}."
|
||||
)
|
||||
print(
|
||||
f"Current exposure: {current_exposure:.4f}, target exposure: {target_exposure:.4f}."
|
||||
)
|
||||
return summary, order
|
||||
|
||||
|
||||
def summarize_alpaca_portfolio(
|
||||
account: TradeAccount,
|
||||
positions: list[Position],
|
||||
open_orders: list[Order],
|
||||
symbol: str = DEFAULT_SYMBOL,
|
||||
) -> AlpacaPortfolioSummary:
|
||||
cash = _string_to_float(account.cash or "0.0")
|
||||
spy_quantity, spy_market_value, spy_avg_entry_price = _find_spy_position(
|
||||
positions, symbol=symbol
|
||||
)
|
||||
current_ratio = _safe_ratio(cash, spy_market_value)
|
||||
|
||||
spy_open_orders = [
|
||||
_build_order_summary(order)
|
||||
for order in open_orders
|
||||
if _build_order_summary(order).symbol == symbol.upper()
|
||||
]
|
||||
|
||||
net_open_spy_qty = 0.0
|
||||
open_order_price = None
|
||||
for order_summary in spy_open_orders:
|
||||
if order_summary.side == "sell":
|
||||
net_open_spy_qty -= order_summary.unfilled_qty
|
||||
else:
|
||||
net_open_spy_qty += order_summary.unfilled_qty
|
||||
if open_order_price is None and order_summary.limit_price is not None:
|
||||
open_order_price = order_summary.limit_price
|
||||
|
||||
estimated_price = None
|
||||
if spy_quantity > 0 and spy_market_value > 0:
|
||||
estimated_price = spy_market_value / spy_quantity
|
||||
elif open_order_price is not None:
|
||||
estimated_price = open_order_price
|
||||
|
||||
projected_cash = None
|
||||
projected_spy_value = None
|
||||
projected_ratio = None
|
||||
if estimated_price is not None:
|
||||
projected_cash = cash - (net_open_spy_qty * estimated_price)
|
||||
projected_spy_value = spy_market_value + (net_open_spy_qty * estimated_price)
|
||||
projected_ratio = _safe_ratio(projected_cash, projected_spy_value)
|
||||
|
||||
return AlpacaPortfolioSummary(
|
||||
cash=cash,
|
||||
spy_quantity=spy_quantity,
|
||||
spy_market_value=spy_market_value,
|
||||
spy_avg_entry_price=spy_avg_entry_price,
|
||||
current_cash_to_spy_ratio=current_ratio,
|
||||
projected_cash_to_spy_ratio=projected_ratio,
|
||||
open_spy_order_count=len(spy_open_orders),
|
||||
estimated_spy_price=estimated_price,
|
||||
)
|
||||
|
||||
|
||||
def fetch_alpaca_portfolio_summary(
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
paper: bool = True,
|
||||
symbol: str = DEFAULT_SYMBOL,
|
||||
client: TradingClient | None = None,
|
||||
) -> AlpacaPortfolioSummary:
|
||||
if client is None:
|
||||
client = create_alpaca_trading_client(
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
paper=paper,
|
||||
)
|
||||
account = client.get_account()
|
||||
if not isinstance(account, TradeAccount):
|
||||
raise TypeError(f"Expected account to be a TradeAccount but got: {type(account)}")
|
||||
positions = client.get_all_positions()
|
||||
if not isinstance(positions, list):
|
||||
raise TypeError(f"Expected positions to be a list but got: {type(positions)}")
|
||||
orders = client.get_orders(GetOrdersRequest(status=QueryOrderStatus.OPEN))
|
||||
if not isinstance(orders, list):
|
||||
raise TypeError(f"Expected orders to be a list but got: {type(orders)}")
|
||||
return summarize_alpaca_portfolio(account, positions, orders, symbol=symbol)
|
||||
|
||||
|
||||
def print_alpaca_portfolio_summary(summary: AlpacaPortfolioSummary) -> None:
|
||||
print("Alpaca portfolio summary")
|
||||
print(f" cash: ${summary.cash:,.2f}")
|
||||
print(f" SPY quantity: {summary.spy_quantity:.4f}")
|
||||
print(f" SPY market value: ${summary.spy_market_value:,.2f}")
|
||||
if summary.spy_avg_entry_price is not None:
|
||||
print(f" SPY avg entry price: ${summary.spy_avg_entry_price:,.4f}")
|
||||
if summary.estimated_spy_price is not None:
|
||||
print(f" estimated SPY price: ${summary.estimated_spy_price:,.4f}")
|
||||
print(
|
||||
f" current cash / SPY value ratio: "
|
||||
+ (
|
||||
f"{summary.current_cash_to_spy_ratio:.4f}"
|
||||
if summary.current_cash_to_spy_ratio is not None
|
||||
else "n/a"
|
||||
)
|
||||
)
|
||||
print(f" open SPY order count: {summary.open_spy_order_count}")
|
||||
print(
|
||||
f" projected cash / SPY ratio after open SPY orders: "
|
||||
+ (
|
||||
f"{summary.projected_cash_to_spy_ratio:.4f}"
|
||||
if summary.projected_cash_to_spy_ratio is not None
|
||||
else "n/a"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Rebalance Alpaca SPY exposure using model predictions and open market orders."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--api-key",
|
||||
default=os.getenv("ALPACA_API_KEY"),
|
||||
help="Alpaca API Key ID (defaults to ALPACA_API_KEY env var).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--secret-key",
|
||||
default=os.getenv("ALPACA_SECRET_KEY"),
|
||||
help="Alpaca Secret Key (defaults to ALPACA_SECRET_KEY env var).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--symbol",
|
||||
default=DEFAULT_SYMBOL,
|
||||
help="Symbol to inspect for SPY exposure. Defaults to SPY.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model-path",
|
||||
default=Path(DEFAULT_MODEL_PATH),
|
||||
type=Path,
|
||||
help="Path to the XGBoost model artifact.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--metadata-path",
|
||||
default=Path(DEFAULT_METADATA_PATH),
|
||||
type=Path,
|
||||
help="Path to the model metadata JSON file.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--data-dir",
|
||||
default=DEFAULT_DATA_DIR,
|
||||
type=Path,
|
||||
help="Directory containing raw Parquet market data files.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--fetch-recent-data",
|
||||
action="store_true",
|
||||
help="Refresh recent market data before running the prediction.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--sensitivity",
|
||||
type=float,
|
||||
default=DEFAULT_SENSITIVITY,
|
||||
help="Exposure sensitivity multiplier used by the prediction model.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--min-order-dollar",
|
||||
type=float,
|
||||
default=DEFAULT_MIN_ORDER_DOLLARS,
|
||||
help="Minimum dollar amount for a trade to execute.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-age-days",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Allowable age of the latest market data row in days.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--paper",
|
||||
action="store_true",
|
||||
help="Force Alpaca paper trading mode. Defaults to paper if no base URL is configured.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
summary, order = rebalance_alpaca_portfolio(
|
||||
api_key=args.api_key,
|
||||
secret_key=args.secret_key,
|
||||
paper=args.paper if args.paper else True,
|
||||
symbol=args.symbol,
|
||||
model_path=str(args.model_path),
|
||||
metadata_path=str(args.metadata_path),
|
||||
raw_data_dir=args.data_dir,
|
||||
min_order_dollars=args.min_order_dollar,
|
||||
sensitivity=args.sensitivity,
|
||||
max_age_days=args.max_age_days,
|
||||
fetch_recent_data=args.fetch_recent_data,
|
||||
)
|
||||
print_alpaca_portfolio_summary(summary)
|
||||
if order is None:
|
||||
print("No new order was placed.")
|
||||
else:
|
||||
print("Rebalance complete.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
"""Read-only web UI for inspecting trading bot status."""
|
||||
@@ -0,0 +1,891 @@
|
||||
"""Read-only Alpaca performance dashboard served by a small Python backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
from html import escape
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from alpaca.trading.client import TradingClient
|
||||
from alpaca.trading.enums import QueryOrderStatus
|
||||
from alpaca.trading.models import Order, Position, TradeAccount
|
||||
from alpaca.trading.requests import GetOrdersRequest, GetPortfolioHistoryRequest
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import (
|
||||
DailyCandle,
|
||||
EASTERN_TZ,
|
||||
default_end_date,
|
||||
fetch_daily_candles,
|
||||
)
|
||||
|
||||
load_dotenv()
|
||||
|
||||
DEFAULT_DB_PATH = Path("data/ui/performance.sqlite")
|
||||
DEFAULT_HOST = "127.0.0.1"
|
||||
DEFAULT_PORT = 8000
|
||||
DEFAULT_REFRESH_HOURS = 12
|
||||
DEFAULT_PERIOD_DAYS = 92
|
||||
BENCHMARK_SYMBOL = "SPY"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PerformancePoint:
|
||||
trading_day: date
|
||||
account_value: float
|
||||
benchmark_value: float | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PortfolioRow:
|
||||
symbol: str
|
||||
asset_type: str
|
||||
quantity: float | None
|
||||
market_value: float
|
||||
average_entry_price: float | None
|
||||
current_price: float | None
|
||||
unrealized_pl: float | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TradeRow:
|
||||
filled_at: datetime | None
|
||||
symbol: str
|
||||
side: str
|
||||
quantity: float
|
||||
filled_average_price: float | None
|
||||
notional: float | None
|
||||
status: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DashboardSnapshot:
|
||||
refreshed_at: datetime
|
||||
performance: list[PerformancePoint]
|
||||
portfolio: list[PortfolioRow]
|
||||
trades: list[TradeRow]
|
||||
warning: str | None = None
|
||||
|
||||
|
||||
def _to_float(value: Any, default: float = 0.0) -> float:
|
||||
if value is None:
|
||||
return default
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def _to_optional_float(value: Any) -> float | None:
|
||||
if value is None or value == "":
|
||||
return None
|
||||
return _to_float(value)
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def _from_unix_timestamp(value: int | float) -> date:
|
||||
return datetime.fromtimestamp(value, tz=UTC).astimezone(EASTERN_TZ).date()
|
||||
|
||||
|
||||
def _benchmark_end_date(account_history_last_day: date) -> date:
|
||||
return min(account_history_last_day, default_end_date())
|
||||
|
||||
|
||||
def init_database(db_path: Path) -> None:
|
||||
db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
connection.executescript(
|
||||
"""
|
||||
create table if not exists metadata (
|
||||
key text primary key,
|
||||
value text not null
|
||||
);
|
||||
create table if not exists performance_points (
|
||||
trading_day text primary key,
|
||||
account_value real not null,
|
||||
benchmark_value real
|
||||
);
|
||||
create table if not exists portfolio_rows (
|
||||
symbol text primary key,
|
||||
asset_type text not null,
|
||||
quantity real,
|
||||
market_value real not null,
|
||||
average_entry_price real,
|
||||
current_price real,
|
||||
unrealized_pl real
|
||||
);
|
||||
create table if not exists trade_rows (
|
||||
id integer primary key autoincrement,
|
||||
filled_at text,
|
||||
symbol text not null,
|
||||
side text not null,
|
||||
quantity real not null,
|
||||
filled_average_price real,
|
||||
notional real,
|
||||
status text not null
|
||||
);
|
||||
"""
|
||||
)
|
||||
|
||||
|
||||
def get_last_refresh(db_path: Path) -> datetime | None:
|
||||
if not db_path.exists():
|
||||
return None
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
row = connection.execute(
|
||||
"select value from metadata where key = 'refreshed_at'"
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return datetime.fromisoformat(row[0])
|
||||
|
||||
|
||||
def is_cache_stale(db_path: Path, refresh_interval: timedelta, now: datetime | None = None) -> bool:
|
||||
refreshed_at = get_last_refresh(db_path)
|
||||
if refreshed_at is None:
|
||||
return True
|
||||
return (now or _utc_now()) - refreshed_at >= refresh_interval
|
||||
|
||||
|
||||
def create_trading_client(
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
paper: bool,
|
||||
) -> TradingClient:
|
||||
if api_key is None or secret_key is None:
|
||||
raise ValueError("ALPACA_API_KEY and ALPACA_SECRET_KEY are required.")
|
||||
return TradingClient(api_key, secret_key, paper=paper)
|
||||
|
||||
|
||||
def fetch_performance_points(
|
||||
client: TradingClient,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
) -> list[PerformancePoint]:
|
||||
history = client.get_portfolio_history(
|
||||
GetPortfolioHistoryRequest(period="3M", timeframe="1D")
|
||||
)
|
||||
account_values_by_day = {
|
||||
_from_unix_timestamp(timestamp): float(equity)
|
||||
for timestamp, equity in zip(history.timestamp, history.equity, strict=False)
|
||||
if equity is not None
|
||||
}
|
||||
if not account_values_by_day:
|
||||
return []
|
||||
|
||||
first_day = min(account_values_by_day)
|
||||
last_day = max(account_values_by_day)
|
||||
benchmark = fetch_daily_candles(
|
||||
BENCHMARK_SYMBOL,
|
||||
end_date=_benchmark_end_date(last_day),
|
||||
duration="3 M",
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
)
|
||||
benchmark_values_by_day = _normalize_benchmark(
|
||||
benchmark,
|
||||
start_day=first_day,
|
||||
initial_account_value=account_values_by_day[first_day],
|
||||
)
|
||||
|
||||
return [
|
||||
PerformancePoint(
|
||||
trading_day=trading_day,
|
||||
account_value=account_value,
|
||||
benchmark_value=benchmark_values_by_day.get(trading_day),
|
||||
)
|
||||
for trading_day, account_value in sorted(account_values_by_day.items())
|
||||
]
|
||||
|
||||
|
||||
def _normalize_benchmark(
|
||||
candles: list[DailyCandle],
|
||||
start_day: date,
|
||||
initial_account_value: float,
|
||||
) -> dict[date, float]:
|
||||
closes = {
|
||||
candle.trading_day: candle.close
|
||||
for candle in candles
|
||||
if candle.trading_day >= start_day and candle.close > 0
|
||||
}
|
||||
if not closes:
|
||||
return {}
|
||||
base_day = min(closes)
|
||||
base_close = closes[base_day]
|
||||
return {
|
||||
trading_day: initial_account_value * (close / base_close)
|
||||
for trading_day, close in closes.items()
|
||||
}
|
||||
|
||||
|
||||
def fetch_portfolio_rows(client: TradingClient) -> list[PortfolioRow]:
|
||||
account = client.get_account()
|
||||
if not isinstance(account, TradeAccount):
|
||||
raise TypeError(f"Expected TradeAccount, got {type(account)}")
|
||||
|
||||
rows = [
|
||||
PortfolioRow(
|
||||
symbol="CASH",
|
||||
asset_type="cash",
|
||||
quantity=None,
|
||||
market_value=_to_float(account.cash),
|
||||
average_entry_price=None,
|
||||
current_price=None,
|
||||
unrealized_pl=None,
|
||||
)
|
||||
]
|
||||
|
||||
positions = client.get_all_positions()
|
||||
if not isinstance(positions, list):
|
||||
raise TypeError(f"Expected list of positions, got {type(positions)}")
|
||||
for position in positions:
|
||||
if not isinstance(position, Position):
|
||||
raise TypeError(f"Expected Position, got {type(position)}")
|
||||
rows.append(
|
||||
PortfolioRow(
|
||||
symbol=str(position.symbol).upper(),
|
||||
asset_type=str(position.asset_class or "equity"),
|
||||
quantity=_to_optional_float(position.qty),
|
||||
market_value=_to_float(position.market_value),
|
||||
average_entry_price=_to_optional_float(position.avg_entry_price),
|
||||
current_price=_to_optional_float(position.current_price),
|
||||
unrealized_pl=_to_optional_float(position.unrealized_pl),
|
||||
)
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def fetch_trade_rows(client: TradingClient) -> list[TradeRow]:
|
||||
orders = client.get_orders(
|
||||
GetOrdersRequest(status=QueryOrderStatus.CLOSED, limit=100)
|
||||
)
|
||||
if not isinstance(orders, list):
|
||||
raise TypeError(f"Expected list of orders, got {type(orders)}")
|
||||
|
||||
trades: list[TradeRow] = []
|
||||
for order in orders:
|
||||
if not isinstance(order, Order):
|
||||
raise TypeError(f"Expected Order, got {type(order)}")
|
||||
quantity = _to_float(order.filled_qty)
|
||||
if quantity <= 0:
|
||||
continue
|
||||
trades.append(
|
||||
TradeRow(
|
||||
filled_at=order.filled_at,
|
||||
symbol=str(order.symbol).upper(),
|
||||
side=str(order.side).lower(),
|
||||
quantity=quantity,
|
||||
filled_average_price=_to_optional_float(order.filled_avg_price),
|
||||
notional=_to_optional_float(order.notional),
|
||||
status=str(order.status).lower(),
|
||||
)
|
||||
)
|
||||
return trades
|
||||
|
||||
|
||||
def fetch_alpaca_snapshot(
|
||||
client: TradingClient,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
) -> DashboardSnapshot:
|
||||
return DashboardSnapshot(
|
||||
refreshed_at=_utc_now(),
|
||||
performance=fetch_performance_points(client, api_key, secret_key),
|
||||
portfolio=fetch_portfolio_rows(client),
|
||||
trades=fetch_trade_rows(client),
|
||||
)
|
||||
|
||||
|
||||
def write_snapshot(db_path: Path, snapshot: DashboardSnapshot) -> None:
|
||||
init_database(db_path)
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
connection.execute("delete from performance_points")
|
||||
connection.execute("delete from portfolio_rows")
|
||||
connection.execute("delete from trade_rows")
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into performance_points (
|
||||
trading_day, account_value, benchmark_value
|
||||
) values (?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
point.trading_day.isoformat(),
|
||||
point.account_value,
|
||||
point.benchmark_value,
|
||||
)
|
||||
for point in snapshot.performance
|
||||
],
|
||||
)
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into portfolio_rows (
|
||||
symbol, asset_type, quantity, market_value,
|
||||
average_entry_price, current_price, unrealized_pl
|
||||
) values (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
row.symbol,
|
||||
row.asset_type,
|
||||
row.quantity,
|
||||
row.market_value,
|
||||
row.average_entry_price,
|
||||
row.current_price,
|
||||
row.unrealized_pl,
|
||||
)
|
||||
for row in snapshot.portfolio
|
||||
],
|
||||
)
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into trade_rows (
|
||||
filled_at, symbol, side, quantity,
|
||||
filled_average_price, notional, status
|
||||
) values (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
row.filled_at.isoformat() if row.filled_at else None,
|
||||
row.symbol,
|
||||
row.side,
|
||||
row.quantity,
|
||||
row.filled_average_price,
|
||||
row.notional,
|
||||
row.status,
|
||||
)
|
||||
for row in snapshot.trades
|
||||
],
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
insert into metadata (key, value)
|
||||
values ('refreshed_at', ?)
|
||||
on conflict(key) do update set value = excluded.value
|
||||
""",
|
||||
(snapshot.refreshed_at.isoformat(),),
|
||||
)
|
||||
|
||||
|
||||
def read_snapshot(db_path: Path, warning: str | None = None) -> DashboardSnapshot:
|
||||
init_database(db_path)
|
||||
refreshed_at = get_last_refresh(db_path) or _utc_now()
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
performance = [
|
||||
PerformancePoint(
|
||||
trading_day=date.fromisoformat(row[0]),
|
||||
account_value=row[1],
|
||||
benchmark_value=row[2],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select trading_day, account_value, benchmark_value
|
||||
from performance_points
|
||||
order by trading_day
|
||||
"""
|
||||
)
|
||||
]
|
||||
portfolio = [
|
||||
PortfolioRow(
|
||||
symbol=row[0],
|
||||
asset_type=row[1],
|
||||
quantity=row[2],
|
||||
market_value=row[3],
|
||||
average_entry_price=row[4],
|
||||
current_price=row[5],
|
||||
unrealized_pl=row[6],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select symbol, asset_type, quantity, market_value,
|
||||
average_entry_price, current_price, unrealized_pl
|
||||
from portfolio_rows
|
||||
order by case when symbol = 'CASH' then 0 else 1 end, symbol
|
||||
"""
|
||||
)
|
||||
]
|
||||
trades = [
|
||||
TradeRow(
|
||||
filled_at=datetime.fromisoformat(row[0]) if row[0] else None,
|
||||
symbol=row[1],
|
||||
side=row[2],
|
||||
quantity=row[3],
|
||||
filled_average_price=row[4],
|
||||
notional=row[5],
|
||||
status=row[6],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select filled_at, symbol, side, quantity,
|
||||
filled_average_price, notional, status
|
||||
from trade_rows
|
||||
order by filled_at desc
|
||||
"""
|
||||
)
|
||||
]
|
||||
return DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=performance,
|
||||
portfolio=portfolio,
|
||||
trades=trades,
|
||||
warning=warning,
|
||||
)
|
||||
|
||||
|
||||
def load_or_refresh_snapshot(
|
||||
db_path: Path,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
paper: bool,
|
||||
refresh_interval: timedelta,
|
||||
) -> DashboardSnapshot:
|
||||
if not is_cache_stale(db_path, refresh_interval):
|
||||
return read_snapshot(db_path)
|
||||
|
||||
try:
|
||||
client = create_trading_client(api_key=api_key, secret_key=secret_key, paper=paper)
|
||||
snapshot = fetch_alpaca_snapshot(client, api_key, secret_key)
|
||||
write_snapshot(db_path, snapshot)
|
||||
return snapshot
|
||||
except Exception as exc:
|
||||
if db_path.exists() and get_last_refresh(db_path) is not None:
|
||||
return read_snapshot(db_path, warning=f"Refresh failed; showing cached data. {exc}")
|
||||
raise
|
||||
|
||||
|
||||
def _format_currency(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
sign = "-" if value < 0 else ""
|
||||
return f"{sign}${abs(value):,.2f}"
|
||||
|
||||
|
||||
def _format_number(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return f"{value:,.4f}"
|
||||
|
||||
|
||||
def _format_percent(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return f"{value:.2%}"
|
||||
|
||||
|
||||
def _clean_enum_text(value: Any) -> str:
|
||||
text = str(value or "").strip()
|
||||
if "." in text:
|
||||
text = text.rsplit(".", maxsplit=1)[-1]
|
||||
return text.lower()
|
||||
|
||||
|
||||
def _pill_class(value: str) -> str:
|
||||
normalized = _clean_enum_text(value)
|
||||
if normalized in {"buy", "filled"}:
|
||||
return "pill pill-positive"
|
||||
if normalized in {"sell", "canceled", "cancelled", "failed", "expired", "rejected"}:
|
||||
return "pill pill-negative"
|
||||
return "pill"
|
||||
|
||||
|
||||
def _pill_html(value: str) -> str:
|
||||
label = _clean_enum_text(value).upper()
|
||||
return f"<span class=\"{_pill_class(label)}\">{escape(label)}</span>"
|
||||
|
||||
|
||||
def _performance_text(points: list[PerformancePoint]) -> str:
|
||||
if len(points) < 2:
|
||||
return "Performance unavailable"
|
||||
first = points[0]
|
||||
last = points[-1]
|
||||
change = last.account_value - first.account_value
|
||||
percent = change / first.account_value if first.account_value else 0.0
|
||||
days = (last.trading_day - first.trading_day).days
|
||||
sign = "+" if change >= 0 else ""
|
||||
return (
|
||||
f"{sign}{_format_currency(change)} ({sign}{percent:.2%}) "
|
||||
f"over {days} days"
|
||||
)
|
||||
|
||||
|
||||
def render_dashboard(snapshot: DashboardSnapshot) -> str:
|
||||
chart_data = [
|
||||
{
|
||||
"date": point.trading_day.isoformat(),
|
||||
"account": point.account_value,
|
||||
"benchmark": point.benchmark_value,
|
||||
}
|
||||
for point in snapshot.performance
|
||||
]
|
||||
chart_json = json.dumps(chart_data)
|
||||
warning = (
|
||||
f"<p class=\"warning\">{escape(snapshot.warning)}</p>"
|
||||
if snapshot.warning
|
||||
else ""
|
||||
)
|
||||
current_portfolio_value = sum(row.market_value for row in snapshot.portfolio)
|
||||
portfolio_rows = "\n".join(
|
||||
"<tr>"
|
||||
f"<td>{escape(row.symbol)}</td>"
|
||||
f"<td>{_format_number(row.quantity)}</td>"
|
||||
f"<td>{_format_currency(row.market_value)}</td>"
|
||||
f"<td>{_format_percent(row.market_value / current_portfolio_value if current_portfolio_value else None)}</td>"
|
||||
f"<td>{_format_currency(row.average_entry_price)}</td>"
|
||||
f"<td>{_format_currency(row.current_price)}</td>"
|
||||
f"<td>{_format_currency(row.unrealized_pl)}</td>"
|
||||
"</tr>"
|
||||
for row in snapshot.portfolio
|
||||
)
|
||||
trade_rows = "\n".join(
|
||||
"<tr>"
|
||||
f"<td>{escape(row.filled_at.isoformat(sep=' ', timespec='minutes') if row.filled_at else '')}</td>"
|
||||
f"<td>{escape(row.symbol)}</td>"
|
||||
f"<td>{_pill_html(row.side)}</td>"
|
||||
f"<td>{_format_number(row.quantity)}</td>"
|
||||
f"<td>{_format_currency(row.filled_average_price)}</td>"
|
||||
f"<td>{_format_currency(row.notional)}</td>"
|
||||
f"<td>{_pill_html(row.status)}</td>"
|
||||
"</tr>"
|
||||
for row in snapshot.trades
|
||||
)
|
||||
portfolio_rows = portfolio_rows or "<tr><td colspan=\"7\">No portfolio rows cached.</td></tr>"
|
||||
trade_rows = trade_rows or "<tr><td colspan=\"7\">No completed trades cached.</td></tr>"
|
||||
|
||||
return f"""<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Trading Bot Performance</title>
|
||||
<style>
|
||||
:root {{
|
||||
color-scheme: light;
|
||||
--bg: #f6f7f9;
|
||||
--panel: #ffffff;
|
||||
--text: #17202a;
|
||||
--muted: #5f6b7a;
|
||||
--line: #d8dee8;
|
||||
--account: #116466;
|
||||
--benchmark: #c75000;
|
||||
--warning: #7a4100;
|
||||
}}
|
||||
* {{ box-sizing: border-box; }}
|
||||
body {{
|
||||
margin: 0;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
line-height: 1.45;
|
||||
}}
|
||||
main {{
|
||||
width: min(1120px, calc(100% - 32px));
|
||||
margin: 0 auto;
|
||||
padding: 32px 0 48px;
|
||||
}}
|
||||
header {{
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
margin-bottom: 24px;
|
||||
}}
|
||||
h1, h2 {{ margin: 0; letter-spacing: 0; }}
|
||||
h1 {{ font-size: clamp(2rem, 4vw, 3.5rem); }}
|
||||
h2 {{ font-size: 1.15rem; margin-bottom: 12px; }}
|
||||
.summary {{
|
||||
font-size: 1.25rem;
|
||||
font-weight: 650;
|
||||
}}
|
||||
.meta, .legend, .warning {{
|
||||
color: var(--muted);
|
||||
margin: 0;
|
||||
}}
|
||||
.warning {{ color: var(--warning); }}
|
||||
section {{
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
padding: 18px;
|
||||
margin-top: 18px;
|
||||
}}
|
||||
.chart-wrap {{
|
||||
min-height: 420px;
|
||||
}}
|
||||
canvas {{
|
||||
width: 100%;
|
||||
height: 360px;
|
||||
display: block;
|
||||
}}
|
||||
.legend {{
|
||||
display: flex;
|
||||
gap: 18px;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 10px;
|
||||
font-size: 0.95rem;
|
||||
}}
|
||||
.legend span::before {{
|
||||
content: "";
|
||||
display: inline-block;
|
||||
width: 20px;
|
||||
height: 3px;
|
||||
margin-right: 8px;
|
||||
vertical-align: middle;
|
||||
background: var(--account);
|
||||
}}
|
||||
.legend span:last-child::before {{ background: var(--benchmark); }}
|
||||
.table-wrap {{ overflow-x: auto; }}
|
||||
table {{
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
min-width: 760px;
|
||||
}}
|
||||
th, td {{
|
||||
padding: 10px 12px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
text-align: right;
|
||||
white-space: nowrap;
|
||||
}}
|
||||
th:first-child, td:first-child,
|
||||
.trades th:nth-child(3), .trades td:nth-child(3),
|
||||
.trades th:nth-child(7), .trades td:nth-child(7) {{
|
||||
text-align: left;
|
||||
}}
|
||||
th {{
|
||||
color: var(--muted);
|
||||
font-size: 0.82rem;
|
||||
font-weight: 700;
|
||||
text-transform: uppercase;
|
||||
}}
|
||||
.pill {{
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-width: 58px;
|
||||
padding: 3px 9px;
|
||||
border-radius: 999px;
|
||||
background: #edf1f5;
|
||||
color: #3d4854;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 800;
|
||||
letter-spacing: 0;
|
||||
}}
|
||||
.pill-positive {{
|
||||
background: #dff3e8;
|
||||
color: #17643a;
|
||||
}}
|
||||
.pill-negative {{
|
||||
background: #ffe1df;
|
||||
color: #9a2219;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<header>
|
||||
<h1>Trading Bot Performance</h1>
|
||||
<div class="summary">{escape(_performance_text(snapshot.performance))}</div>
|
||||
<p class="meta">Last refreshed {escape(snapshot.refreshed_at.astimezone().isoformat(sep=' ', timespec='minutes'))}</p>
|
||||
{warning}
|
||||
</header>
|
||||
|
||||
<section class="chart-wrap">
|
||||
<h2>Account Value vs S&P 500 Proxy</h2>
|
||||
<canvas id="performanceChart" width="1040" height="360"></canvas>
|
||||
<p class="legend"><span>Account value</span><span>SPY normalized to account start value</span></p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Portfolio</h2>
|
||||
<div class="table-wrap">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Symbol</th><th>Quantity</th><th>Market Value</th><th>Account %</th>
|
||||
<th>Avg Entry</th><th>Current Price</th><th>Unrealized P/L</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>{portfolio_rows}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Completed Trades</h2>
|
||||
<div class="table-wrap">
|
||||
<table class="trades">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Filled At</th><th>Symbol</th><th>Side</th><th>Quantity</th>
|
||||
<th>Avg Fill</th><th>Notional</th><th>Status</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>{trade_rows}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const points = {chart_json};
|
||||
const canvas = document.getElementById("performanceChart");
|
||||
const ctx = canvas.getContext("2d");
|
||||
const css = getComputedStyle(document.documentElement);
|
||||
const accountColor = css.getPropertyValue("--account").trim();
|
||||
const benchmarkColor = css.getPropertyValue("--benchmark").trim();
|
||||
const gridColor = css.getPropertyValue("--line").trim();
|
||||
const textColor = css.getPropertyValue("--muted").trim();
|
||||
|
||||
function currency(value) {{
|
||||
return "$" + value.toLocaleString(undefined, {{ maximumFractionDigits: 0 }});
|
||||
}}
|
||||
|
||||
function drawLine(values, color, minValue, maxValue, left, top, width, height) {{
|
||||
const valid = values.filter((point) => point.value !== null && point.value !== undefined);
|
||||
if (valid.length < 2) return;
|
||||
ctx.beginPath();
|
||||
valid.forEach((point, index) => {{
|
||||
const x = left + (point.index / Math.max(points.length - 1, 1)) * width;
|
||||
const y = top + (1 - ((point.value - minValue) / Math.max(maxValue - minValue, 1))) * height;
|
||||
if (index === 0) ctx.moveTo(x, y);
|
||||
else ctx.lineTo(x, y);
|
||||
}});
|
||||
ctx.strokeStyle = color;
|
||||
ctx.lineWidth = 3;
|
||||
ctx.stroke();
|
||||
}}
|
||||
|
||||
function drawChart() {{
|
||||
const ratio = window.devicePixelRatio || 1;
|
||||
const rect = canvas.getBoundingClientRect();
|
||||
canvas.width = Math.max(Math.floor(rect.width * ratio), 320);
|
||||
canvas.height = Math.floor(360 * ratio);
|
||||
ctx.setTransform(ratio, 0, 0, ratio, 0, 0);
|
||||
ctx.clearRect(0, 0, rect.width, 360);
|
||||
|
||||
const left = 72, right = 18, top = 16, bottom = 46;
|
||||
const width = rect.width - left - right;
|
||||
const height = 360 - top - bottom;
|
||||
const series = points.flatMap((point) => [point.account, point.benchmark]).filter((value) => value !== null);
|
||||
if (series.length === 0) {{
|
||||
ctx.fillStyle = textColor;
|
||||
ctx.fillText("No performance history cached.", left, top + 24);
|
||||
return;
|
||||
}}
|
||||
const minValue = Math.min(...series) * 0.995;
|
||||
const maxValue = Math.max(...series) * 1.005;
|
||||
|
||||
ctx.font = "12px system-ui, sans-serif";
|
||||
ctx.strokeStyle = gridColor;
|
||||
ctx.fillStyle = textColor;
|
||||
for (let i = 0; i <= 4; i++) {{
|
||||
const y = top + (i / 4) * height;
|
||||
const value = maxValue - (i / 4) * (maxValue - minValue);
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(left, y);
|
||||
ctx.lineTo(left + width, y);
|
||||
ctx.stroke();
|
||||
ctx.fillText(currency(value), 6, y + 4);
|
||||
}}
|
||||
|
||||
drawLine(points.map((point, index) => ({{ index, value: point.account }})), accountColor, minValue, maxValue, left, top, width, height);
|
||||
drawLine(points.map((point, index) => ({{ index, value: point.benchmark }})), benchmarkColor, minValue, maxValue, left, top, width, height);
|
||||
|
||||
if (points.length > 0) {{
|
||||
ctx.fillStyle = textColor;
|
||||
ctx.fillText(points[0].date, left, 342);
|
||||
ctx.textAlign = "right";
|
||||
ctx.fillText(points[points.length - 1].date, left + width, 342);
|
||||
ctx.textAlign = "left";
|
||||
}}
|
||||
}}
|
||||
|
||||
drawChart();
|
||||
window.addEventListener("resize", drawChart);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
class DashboardRequestHandler(BaseHTTPRequestHandler):
|
||||
db_path: Path
|
||||
api_key: str | None
|
||||
secret_key: str | None
|
||||
paper: bool
|
||||
refresh_interval: timedelta
|
||||
|
||||
def do_GET(self) -> None:
|
||||
if self.path not in {"/", "/index.html"}:
|
||||
self.send_error(404)
|
||||
return
|
||||
try:
|
||||
snapshot = load_or_refresh_snapshot(
|
||||
db_path=self.db_path,
|
||||
api_key=self.api_key,
|
||||
secret_key=self.secret_key,
|
||||
paper=self.paper,
|
||||
refresh_interval=self.refresh_interval,
|
||||
)
|
||||
body = render_dashboard(snapshot).encode("utf-8")
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
except Exception as exc:
|
||||
body = f"Dashboard unavailable: {escape(str(exc))}".encode("utf-8")
|
||||
self.send_response(500)
|
||||
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, format: str, *args: Any) -> None:
|
||||
print(f"{self.address_string()} - {format % args}")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="Serve the read-only trading bot dashboard.")
|
||||
parser.add_argument("--host", default=DEFAULT_HOST, help=f"Host to bind. Defaults to {DEFAULT_HOST}.")
|
||||
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"Port to bind. Defaults to {DEFAULT_PORT}.")
|
||||
parser.add_argument("--db-path", type=Path, default=DEFAULT_DB_PATH, help=f"SQLite cache path. Defaults to {DEFAULT_DB_PATH}.")
|
||||
parser.add_argument("--refresh-hours", type=float, default=DEFAULT_REFRESH_HOURS, help="Refresh Alpaca data after this many hours. Defaults to 12.")
|
||||
parser.add_argument("--api-key", default=os.getenv("ALPACA_API_KEY"), help="Alpaca API Key ID.")
|
||||
parser.add_argument("--secret-key", default=os.getenv("ALPACA_SECRET_KEY"), help="Alpaca Secret Key.")
|
||||
parser.add_argument("--paper", action="store_true", help="Use Alpaca paper trading mode. This is the default.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
handler = type(
|
||||
"ConfiguredDashboardRequestHandler",
|
||||
(DashboardRequestHandler,),
|
||||
{
|
||||
"db_path": args.db_path,
|
||||
"api_key": args.api_key,
|
||||
"secret_key": args.secret_key,
|
||||
"paper": True if args.paper else True,
|
||||
"refresh_interval": timedelta(hours=args.refresh_hours),
|
||||
},
|
||||
)
|
||||
server = ThreadingHTTPServer((args.host, args.port), handler)
|
||||
print(f"Serving read-only dashboard at http://{args.host}:{args.port}")
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopping dashboard server.")
|
||||
finally:
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,160 @@
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import DailyCandle
|
||||
from trading_bot.ui.dashboard import (
|
||||
DashboardSnapshot,
|
||||
PerformancePoint,
|
||||
PortfolioRow,
|
||||
TradeRow,
|
||||
_benchmark_end_date,
|
||||
_from_unix_timestamp,
|
||||
_normalize_benchmark,
|
||||
is_cache_stale,
|
||||
read_snapshot,
|
||||
render_dashboard,
|
||||
write_snapshot,
|
||||
)
|
||||
|
||||
|
||||
def test_normalize_benchmark_scales_spy_to_account_start_value() -> None:
|
||||
candles = [
|
||||
DailyCandle(date(2026, 5, 1), 100.0, 100.0, 100.0, 100.0, 1_000),
|
||||
DailyCandle(date(2026, 5, 2), 110.0, 110.0, 110.0, 110.0, 1_000),
|
||||
]
|
||||
|
||||
values = _normalize_benchmark(
|
||||
candles,
|
||||
start_day=date(2026, 5, 1),
|
||||
initial_account_value=1_000.0,
|
||||
)
|
||||
|
||||
assert values[date(2026, 5, 1)] == 1_000.0
|
||||
assert values[date(2026, 5, 2)] == 1_100.0
|
||||
|
||||
|
||||
def test_portfolio_history_timestamps_are_interpreted_as_eastern_dates() -> None:
|
||||
timestamp = datetime(2026, 8, 11, 2, 30, tzinfo=UTC).timestamp()
|
||||
|
||||
assert _from_unix_timestamp(timestamp) == date(2026, 8, 10)
|
||||
|
||||
|
||||
def test_benchmark_end_date_does_not_go_past_completed_market_day(monkeypatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.ui.dashboard.default_end_date",
|
||||
lambda: date(2026, 8, 10),
|
||||
)
|
||||
|
||||
assert _benchmark_end_date(date(2026, 8, 11)) == date(2026, 8, 10)
|
||||
assert _benchmark_end_date(date(2026, 8, 8)) == date(2026, 8, 8)
|
||||
|
||||
|
||||
def test_sqlite_snapshot_round_trip(tmp_path) -> None:
|
||||
db_path = tmp_path / "dashboard.sqlite"
|
||||
refreshed_at = datetime(2026, 8, 11, 10, 0, tzinfo=UTC)
|
||||
snapshot = DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 10), 10_000.0, 9_900.0),
|
||||
PerformancePoint(date(2026, 8, 11), 10_100.0, 10_000.0),
|
||||
],
|
||||
portfolio=[
|
||||
PortfolioRow("CASH", "cash", None, 500.0, None, None, None),
|
||||
PortfolioRow("SPY", "us_equity", 10.0, 5_000.0, 490.0, 500.0, 100.0),
|
||||
],
|
||||
trades=[
|
||||
TradeRow(
|
||||
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
|
||||
"SPY",
|
||||
"buy",
|
||||
1.0,
|
||||
500.0,
|
||||
500.0,
|
||||
"filled",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
write_snapshot(db_path, snapshot)
|
||||
loaded = read_snapshot(db_path)
|
||||
|
||||
assert loaded.refreshed_at == refreshed_at
|
||||
assert loaded.performance == snapshot.performance
|
||||
assert loaded.portfolio == snapshot.portfolio
|
||||
assert loaded.trades == snapshot.trades
|
||||
|
||||
|
||||
def test_cache_stale_after_refresh_interval(tmp_path) -> None:
|
||||
db_path = tmp_path / "dashboard.sqlite"
|
||||
refreshed_at = datetime(2026, 8, 11, 0, 0, tzinfo=UTC)
|
||||
snapshot = DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=[],
|
||||
portfolio=[],
|
||||
trades=[],
|
||||
)
|
||||
write_snapshot(db_path, snapshot)
|
||||
|
||||
assert not is_cache_stale(
|
||||
db_path,
|
||||
timedelta(hours=12),
|
||||
now=refreshed_at + timedelta(hours=11, minutes=59),
|
||||
)
|
||||
assert is_cache_stale(
|
||||
db_path,
|
||||
timedelta(hours=12),
|
||||
now=refreshed_at + timedelta(hours=12),
|
||||
)
|
||||
|
||||
|
||||
def test_render_dashboard_contains_requested_sections() -> None:
|
||||
html = render_dashboard(
|
||||
DashboardSnapshot(
|
||||
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 10), 10_000.0, 10_000.0),
|
||||
PerformancePoint(date(2026, 8, 11), 10_250.0, 10_100.0),
|
||||
],
|
||||
portfolio=[PortfolioRow("CASH", "cash", None, 250.0, None, None, None)],
|
||||
trades=[
|
||||
TradeRow(
|
||||
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
|
||||
"SPY",
|
||||
"orderside.sell",
|
||||
1.0,
|
||||
500.0,
|
||||
500.0,
|
||||
"orderstatus.filled",
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
assert "Account Value vs S&P 500 Proxy" in html
|
||||
assert "Portfolio" in html
|
||||
assert "Completed Trades" in html
|
||||
assert "+$250.00 (+2.50%)" in html
|
||||
assert "<th>Type</th>" not in html
|
||||
assert "<th>Account %</th>" in html
|
||||
assert "SELL</span>" in html
|
||||
assert "FILLED</span>" in html
|
||||
assert "orderside.sell" not in html
|
||||
assert "orderstatus.filled" not in html
|
||||
|
||||
|
||||
def test_portfolio_percentage_uses_current_portfolio_total() -> None:
|
||||
html = render_dashboard(
|
||||
DashboardSnapshot(
|
||||
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 11), 10_000.0, 10_000.0),
|
||||
],
|
||||
portfolio=[
|
||||
PortfolioRow("CASH", "cash", None, 100.0, None, None, None),
|
||||
PortfolioRow("SPY", "us_equity", 10.0, 10_000.0, 900.0, 1_000.0, 1_000.0),
|
||||
],
|
||||
trades=[],
|
||||
)
|
||||
)
|
||||
|
||||
assert "99.01%" in html
|
||||
assert "100.00%" not in html
|
||||
@@ -0,0 +1,79 @@
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from trading_bot.models.prediction import FEATURE_COLUMNS, predict_latest_probability
|
||||
|
||||
|
||||
class FakeModel:
|
||||
def predict_proba(self, model_input):
|
||||
return np.array([[0.1, 0.9]])
|
||||
|
||||
|
||||
def test_predict_latest_probability_refreshes_recent_market_data(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
refresh_calls: dict[str, object] = {}
|
||||
|
||||
def fake_refresh_recent_data(
|
||||
output_dir: Path,
|
||||
symbols: dict[str, str] | None = None,
|
||||
api_key: str | None = None,
|
||||
secret_key: str | None = None,
|
||||
) -> None:
|
||||
refresh_calls["symbols"] = list((symbols or {}).keys())
|
||||
refresh_calls["output_dir"] = output_dir
|
||||
refresh_calls["api_key"] = api_key
|
||||
refresh_calls["secret_key"] = secret_key
|
||||
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.prediction.refresh_recent_market_data",
|
||||
fake_refresh_recent_data,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.prediction.load_model",
|
||||
lambda model_path: FakeModel(),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.prediction.load_feature_metadata",
|
||||
lambda metadata_path: {"feature_cols": FEATURE_COLUMNS, "p_base": 0.5},
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.prediction.get_latest_inference_features",
|
||||
lambda raw_data_dir, symbols=None, max_age_days=1: (
|
||||
pd.Timestamp("2026-08-03"),
|
||||
pd.DataFrame(
|
||||
[[0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08]],
|
||||
columns=FEATURE_COLUMNS,
|
||||
index=[pd.Timestamp("2026-08-03")],
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
result = predict_latest_probability(
|
||||
model_path=Path("notebooks/models/spy_xgb_v1.json"),
|
||||
metadata_path=Path("notebooks/models/spy_xgb_v1_meta.json"),
|
||||
raw_data_dir=Path("data/alpaca/daily"),
|
||||
api_key="test-key",
|
||||
secret_key="test-secret",
|
||||
fetch_recent_data=True,
|
||||
)
|
||||
|
||||
assert refresh_calls["output_dir"] == Path("data/alpaca/daily")
|
||||
assert refresh_calls["symbols"] == ["SPY", "VIXY", "TLT", "USO"]
|
||||
assert refresh_calls["api_key"] == "test-key"
|
||||
assert refresh_calls["secret_key"] == "test-secret"
|
||||
assert result.probability == 0.9
|
||||
|
||||
|
||||
def test_predict_latest_probability_returns_probability_between_zero_and_one() -> None:
|
||||
result = predict_latest_probability(
|
||||
model_path=Path("notebooks/models/spy_xgb_v1.json"),
|
||||
metadata_path=Path("notebooks/models/spy_xgb_v1_meta.json"),
|
||||
raw_data_dir=Path("data/alpaca/daily"),
|
||||
)
|
||||
|
||||
assert set(result.feature_columns).issubset(FEATURE_COLUMNS)
|
||||
assert 0.0 <= result.probability <= 1.0
|
||||
assert result.probability > 0.0
|
||||
@@ -0,0 +1,186 @@
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
from trading_bot.models.prediction import PredictionResult
|
||||
from trading_bot.models.trade import (
|
||||
rebalance_alpaca_portfolio,
|
||||
summarize_alpaca_portfolio,
|
||||
)
|
||||
|
||||
|
||||
def test_calculate_cash_to_spy_ratio() -> None:
|
||||
account = {"cash": "1000.00"}
|
||||
positions = [{"symbol": "SPY", "qty": "10", "market_value": "5000", "avg_entry_price": "500"}]
|
||||
orders = []
|
||||
|
||||
summary = summarize_alpaca_portfolio(account, positions, orders)
|
||||
|
||||
assert summary.cash == 1000.0
|
||||
assert summary.spy_quantity == 10.0
|
||||
assert summary.spy_market_value == 5000.0
|
||||
assert summary.current_cash_to_spy_ratio == 0.2
|
||||
assert summary.projected_cash_to_spy_ratio == 0.2
|
||||
assert summary.open_spy_order_count == 0
|
||||
|
||||
|
||||
def test_projected_ratio_includes_open_spy_buy_order() -> None:
|
||||
account = {"cash": "1000.00"}
|
||||
positions = [{"symbol": "SPY", "qty": "10", "market_value": "5000", "avg_entry_price": "500"}]
|
||||
orders = [
|
||||
{
|
||||
"symbol": "SPY",
|
||||
"side": "buy",
|
||||
"qty": "2",
|
||||
"filled_qty": "0",
|
||||
"limit_price": "500",
|
||||
"status": "open",
|
||||
"type": "limit",
|
||||
}
|
||||
]
|
||||
|
||||
summary = summarize_alpaca_portfolio(account, positions, orders)
|
||||
|
||||
assert summary.open_spy_order_count == 1
|
||||
assert summary.estimated_spy_price == 500.0
|
||||
assert summary.projected_cash_to_spy_ratio == 0.0
|
||||
|
||||
|
||||
def test_projected_ratio_handles_no_spy_position_but_open_order() -> None:
|
||||
account = {"cash": "1500.00"}
|
||||
positions = []
|
||||
orders = [
|
||||
{
|
||||
"symbol": "SPY",
|
||||
"side": "buy",
|
||||
"qty": "3",
|
||||
"filled_qty": "0",
|
||||
"limit_price": "500",
|
||||
"status": "open",
|
||||
"type": "limit",
|
||||
}
|
||||
]
|
||||
|
||||
summary = summarize_alpaca_portfolio(account, positions, orders)
|
||||
|
||||
assert summary.spy_quantity == 0.0
|
||||
assert summary.spy_market_value == 0.0
|
||||
assert summary.current_cash_to_spy_ratio is None
|
||||
assert summary.projected_cash_to_spy_ratio == 0.0
|
||||
assert summary.open_spy_order_count == 1
|
||||
|
||||
|
||||
class FakeAlpacaClient:
|
||||
def __init__(self) -> None:
|
||||
self.cancelled = False
|
||||
self.order_request = None
|
||||
|
||||
def get_account(self) -> dict[str, str]:
|
||||
return {"cash": "1000.00"}
|
||||
|
||||
def get_all_positions(self) -> list[dict[str, str]]:
|
||||
return [
|
||||
{
|
||||
"symbol": "SPY",
|
||||
"qty": "10",
|
||||
"market_value": "5000",
|
||||
"avg_entry_price": "500",
|
||||
}
|
||||
]
|
||||
|
||||
def get_orders(self, filter=None) -> list[dict[str, Any]]:
|
||||
return []
|
||||
|
||||
def cancel_orders(self) -> list[dict[str, Any]]:
|
||||
self.cancelled = True
|
||||
return []
|
||||
|
||||
def submit_order(self, order_request: Any) -> dict[str, Any]:
|
||||
self.order_request = order_request
|
||||
return {
|
||||
"id": "fake-order",
|
||||
"status": "new",
|
||||
"symbol": getattr(order_request, "symbol", "SPY"),
|
||||
"side": getattr(order_request, "side", "buy"),
|
||||
"notional": getattr(order_request, "notional", 0.0),
|
||||
}
|
||||
|
||||
|
||||
def test_rebalance_submits_buy_order_when_target_exposure_is_higher(monkeypatch) -> None:
|
||||
fake_client = FakeAlpacaClient()
|
||||
|
||||
def fake_create_client(api_key=None, secret_key=None, paper=True):
|
||||
return fake_client
|
||||
|
||||
def fake_predict_latest_probability(**kwargs):
|
||||
return PredictionResult(
|
||||
prediction_date=pd.Timestamp("2026-08-03"),
|
||||
probability=0.9,
|
||||
base_probability=0.5,
|
||||
feature_columns=["SPY_ret_5"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.trade.create_alpaca_trading_client",
|
||||
fake_create_client,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.trade.predict_latest_probability",
|
||||
fake_predict_latest_probability,
|
||||
)
|
||||
|
||||
summary, order = rebalance_alpaca_portfolio(
|
||||
api_key="test-key",
|
||||
secret_key="test-secret",
|
||||
paper=True,
|
||||
symbol="SPY",
|
||||
min_order_dollars=100.0,
|
||||
sensitivity=5.0,
|
||||
)
|
||||
|
||||
assert fake_client.cancelled is True
|
||||
assert order is not None
|
||||
assert order["side"] == "buy"
|
||||
assert order["notional"] == pytest.approx(1000.0)
|
||||
assert summary.cash == 1000.0
|
||||
assert summary.spy_market_value == 5000.0
|
||||
|
||||
|
||||
def test_rebalance_skips_small_orders_below_minimum(monkeypatch) -> None:
|
||||
fake_client = FakeAlpacaClient()
|
||||
|
||||
def fake_create_client(api_key=None, secret_key=None, paper=True):
|
||||
return fake_client
|
||||
|
||||
def fake_predict_latest_probability(**kwargs):
|
||||
return PredictionResult(
|
||||
prediction_date=pd.Timestamp("2026-08-03"),
|
||||
probability=0.569,
|
||||
base_probability=0.5,
|
||||
feature_columns=["SPY_ret_5"],
|
||||
)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.trade.create_alpaca_trading_client",
|
||||
fake_create_client,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.models.trade.predict_latest_probability",
|
||||
fake_predict_latest_probability,
|
||||
)
|
||||
|
||||
summary, order = rebalance_alpaca_portfolio(
|
||||
api_key="test-key",
|
||||
secret_key="test-secret",
|
||||
paper=True,
|
||||
symbol="SPY",
|
||||
min_order_dollars=100.0,
|
||||
sensitivity=5.0,
|
||||
)
|
||||
|
||||
assert fake_client.cancelled is False
|
||||
assert order is None
|
||||
assert summary.cash == 1000.0
|
||||
assert summary.spy_market_value == 5000.0
|
||||
Reference in New Issue
Block a user