Compare commits
4
Commits
2165c4269d
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03a724f7ea | ||
|
|
1faca2cdda | ||
|
|
c1c02e2b0d | ||
|
|
900b70d6df |
@@ -2,7 +2,7 @@
|
||||
|
||||
## Project Purpose
|
||||
|
||||
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot will use the IBKR API to facilitate trades, starting with paper trading and allowing a carefully controlled path to real trading later.
|
||||
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot uses the Alpaca Markets API for market data and paper trading, with any future live trading behind a carefully controlled path.
|
||||
|
||||
## Current Direction
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Trading Bot
|
||||
|
||||
Python trading bot project using machine learning to decide which assets should be held. The bot is intended to trade through the IBKR API, beginning with paper trading and potentially supporting real trades later behind explicit safeguards.
|
||||
Python trading bot project using machine learning to decide which assets should be held. The bot currently uses the Alpaca Markets API for market data and paper trading, with any future live trading kept behind explicit safeguards.
|
||||
|
||||
## Project Status
|
||||
|
||||
This repository is at the planning and scaffolding stage. The machine learning model, trading parameters, asset universe, and risk rules will be designed later.
|
||||
This repository now has a working prototype flow for Alpaca market data, dataset generation, XGBoost training, prediction, and paper-trading rebalancing. The strategy is still intentionally narrow: it currently focuses on `SPY` exposure using a small market-regime feature set.
|
||||
|
||||
## Main Parts
|
||||
|
||||
@@ -29,22 +29,62 @@ The current local Python version is pinned in `.mise.toml`.
|
||||
|
||||
Runtime dependencies are declared in `pyproject.toml`.
|
||||
|
||||
- `ib-insync` for the IBKR API connection.
|
||||
- `alpaca-py` for Alpaca market data and trading clients.
|
||||
- `pandas` for tabular candle data handling.
|
||||
- `pyarrow` for Parquet file support.
|
||||
- `xgboost` and `scikit-learn` for model training and evaluation.
|
||||
- `python-dotenv` for loading local Alpaca credentials from `.env`.
|
||||
|
||||
## Trading Safety
|
||||
|
||||
The default target is paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
|
||||
The default target is Alpaca paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
|
||||
|
||||
Do not commit secrets such as IBKR credentials, account identifiers, API tokens, or private configuration.
|
||||
Do not commit secrets such as Alpaca API keys, account identifiers, API tokens, or private configuration.
|
||||
|
||||
## Current Commands
|
||||
|
||||
Fetch Alpaca daily candles:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
Build the training dataset and train the current model:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
|
||||
```
|
||||
|
||||
Run the current Alpaca paper-trading rebalance flow:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/models/trade.py \
|
||||
--paper \
|
||||
--fetch-recent-data \
|
||||
--model-path models/spy_xgb_v1.json \
|
||||
--metadata-path models/spy_xgb_v1_meta.json
|
||||
```
|
||||
|
||||
The main Python entry points are:
|
||||
|
||||
- `src/trading_bot/data/train_pipeline.py` for dataset generation and model training.
|
||||
- `src/trading_bot/models/trade.py` for Alpaca account inspection and SPY rebalancing.
|
||||
- `src/trading_bot/ui/dashboard.py` for the read-only performance dashboard.
|
||||
|
||||
Serve the read-only dashboard:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
See [docs/README.md](docs/README.md) for the initial architecture notes and decision log.
|
||||
|
||||
The first data collection design note is [docs/data-fetcher.md](docs/data-fetcher.md), and the first training dataset contract is [docs/training-dataset.md](docs/training-dataset.md).
|
||||
The current data collection note is [docs/data-fetcher.md](docs/data-fetcher.md), and the training dataset contract is [docs/training-dataset.md](docs/training-dataset.md).
|
||||
|
||||
The read-only dashboard design is documented in [docs/read-only-ui.md](docs/read-only-ui.md).
|
||||
|
||||
The first dataset generation notebook is [notebooks/spy_direction_dataset.ipynb](notebooks/spy_direction_dataset.ipynb).
|
||||
|
||||
Manual test instructions for the IBKR fetcher skeleton are in [docs/manual-test/README.md](docs/manual-test/README.md).
|
||||
Manual test instructions for the Alpaca fetcher are in [docs/manual-test/README.md](docs/manual-test/README.md).
|
||||
|
||||
+23
-17
@@ -2,7 +2,7 @@
|
||||
|
||||
## Objective
|
||||
|
||||
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses the IBKR API to facilitate trades. The first supported trading mode should be paper trading.
|
||||
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses Alpaca Markets for market data and broker execution. The first supported trading mode is paper trading.
|
||||
|
||||
## Planned Modules
|
||||
|
||||
@@ -10,9 +10,9 @@ Build a Python-based trading bot that uses machine learning to determine which a
|
||||
|
||||
Responsible for acquiring and storing market, asset, and any future feature data needed for model training and evaluation.
|
||||
|
||||
The first planned tool is an IBKR daily candle fetcher. It should fetch open, high, low, close, and volume data for a ticker and date range, then eventually persist that data to a ticker-named Parquet file. See [data-fetcher.md](data-fetcher.md).
|
||||
The current data tool is an Alpaca daily candle fetcher. It fetches open, high, low, close, and volume data for a ticker and date range, then persists that data to a ticker-named Parquet file. See [data-fetcher.md](data-fetcher.md).
|
||||
|
||||
The initial supervised training dataset is documented in [training-dataset.md](training-dataset.md). It derives market-regime features from `SPY`, `VIX`, `TLT`, and `USO`, then labels whether `SPY` closes higher five trading days later.
|
||||
The initial supervised training dataset is documented in [training-dataset.md](training-dataset.md). It derives market-regime features from `SPY`, a volatility proxy, `TLT`, and `USO`, then labels whether `SPY` closes higher five trading days later. The checked-in configuration currently maps the volatility input to `VIXY` for Alpaca data availability.
|
||||
|
||||
Parquet files are partitioned by ticker, not by date.
|
||||
|
||||
@@ -29,6 +29,8 @@ Open decisions:
|
||||
|
||||
Responsible for building datasets, training models, evaluating candidates, and writing versioned model artifacts.
|
||||
|
||||
The main entry point is `src/trading_bot/data/train_pipeline.py`. It reads raw Alpaca Parquet files from `data/alpaca/daily`, builds `data/training/spy_direction_5d.parquet`, trains an XGBoost classifier, and writes model artifacts to `models/`.
|
||||
|
||||
Open decisions:
|
||||
|
||||
- prediction target;
|
||||
@@ -41,11 +43,13 @@ Open decisions:
|
||||
|
||||
### Trading Bot
|
||||
|
||||
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the IBKR API to place or simulate orders.
|
||||
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the Alpaca trading API to place paper-trading orders.
|
||||
|
||||
The main entry point is `src/trading_bot/models/trade.py`. It loads a model and metadata, optionally refreshes recent Alpaca market data, estimates a target `SPY` exposure from the model probability, cancels open Alpaca orders, and submits a day market order when the desired rebalance exceeds the configured minimum notional amount. Pass `--model-path models/spy_xgb_v1.json --metadata-path models/spy_xgb_v1_meta.json` to trade with artifacts produced by the current training pipeline.
|
||||
|
||||
Initial expectations:
|
||||
|
||||
- paper trading first;
|
||||
- Alpaca paper trading first;
|
||||
- real trading later only behind explicit configuration;
|
||||
- clear logging of model version, signals, target holdings, generated orders, and broker responses;
|
||||
- separation between signal generation, portfolio construction, and broker execution.
|
||||
@@ -56,32 +60,34 @@ Open decisions:
|
||||
- position sizing;
|
||||
- risk limits;
|
||||
- cash handling;
|
||||
- order types;
|
||||
- order types and time-in-force choices;
|
||||
- failed order handling;
|
||||
- market hours behavior;
|
||||
- manual override behavior.
|
||||
|
||||
### Read-Only Web UI
|
||||
|
||||
Optional future module for checking status without controlling trading behavior.
|
||||
Optional module for checking status without controlling trading behavior.
|
||||
|
||||
Possible scope:
|
||||
The initial implementation is documented in [read-only-ui.md](read-only-ui.md).
|
||||
It serves one backend-rendered page, reads Alpaca account state with `alpaca-py`,
|
||||
and caches snapshots in SQLite so normal page loads do not repeatedly call Alpaca.
|
||||
|
||||
- current holdings;
|
||||
- latest signals;
|
||||
- recent orders;
|
||||
- account summary;
|
||||
- model version;
|
||||
- bot health and logs.
|
||||
Current scope:
|
||||
|
||||
- three-month account performance summary;
|
||||
- account value chart with a `SPY` S&P 500 proxy comparison;
|
||||
- cash and equity portfolio table;
|
||||
- completed trades table.
|
||||
|
||||
## Near-Term Priorities
|
||||
|
||||
1. Decide the initial project package structure.
|
||||
2. Keep Python packaging and dependency management current with `uv`.
|
||||
3. Add a minimal configuration system.
|
||||
4. Define interfaces for data collection, model artifacts, and broker execution.
|
||||
5. Add tests for the core trading decision boundaries before connecting real broker behavior.
|
||||
4. Harden interfaces for data collection, model artifacts, and Alpaca broker execution.
|
||||
5. Expand tests around trading decision boundaries, portfolio sizing, stale data handling, and broker API boundaries.
|
||||
|
||||
## Decisions Deferred
|
||||
|
||||
The model, features, labels, asset selection rules, risk management rules, and trading cadence are intentionally deferred for later discussion.
|
||||
Broader model design, asset selection rules, risk management rules, live-trading gates, and trading cadence are intentionally deferred for later discussion.
|
||||
|
||||
+11
-12
@@ -2,7 +2,7 @@
|
||||
|
||||
## Initial Goal
|
||||
|
||||
Create a Python module/tool that fetches daily candlestick data from the IBKR API for a specific ticker and date range.
|
||||
Create a Python module/tool that fetches daily candlestick data from the Alpaca Market Data API for a specific ticker and date range.
|
||||
|
||||
The intended example workflow is:
|
||||
|
||||
@@ -14,41 +14,40 @@ The intended example workflow is:
|
||||
|
||||
## Current Implementation
|
||||
|
||||
The first implementation is intentionally small:
|
||||
The current implementation is intentionally small:
|
||||
|
||||
- ticker passed as a required command line argument;
|
||||
- end date passed with `--end-date YYYYMMDD`, defaulting to yesterday;
|
||||
- end date can also be derived with `--end-date-from-parquet`;
|
||||
- duration passed with `--duration`, defaulting to `1 W`;
|
||||
- hard-coded IBKR Gateway target: `127.0.0.1:4002`;
|
||||
- uses the IBKR API through `ib_insync`;
|
||||
- prints fetched candles as CSV-like rows;
|
||||
- uses the Alpaca Market Data API through `alpaca-py`;
|
||||
- reads credentials from `--api-key` / `--secret-key`, `ALPACA_API_KEY` / `ALPACA_SECRET_KEY`, or `.env`;
|
||||
- writes candles to a symbol-named Parquet file;
|
||||
- appends to an existing symbol file and keeps one row per date.
|
||||
|
||||
Run it with:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
To override the requested range:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
```
|
||||
|
||||
To fetch backward from the oldest date already stored in the symbol file:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
|
||||
```
|
||||
|
||||
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
|
||||
|
||||
This expects a local IBKR Gateway session to be running and accepting API connections on `127.0.0.1:4002`.
|
||||
This expects Alpaca API credentials to be available via CLI arguments, environment variables, or `.env`.
|
||||
|
||||
By default, output is written to `data/ibkr/daily/SPY.parquet`. Use `--output-dir` to choose another directory.
|
||||
By default, output is written to `data/alpaca/daily/SPY.parquet`. Use `--output-dir` to choose another directory.
|
||||
|
||||
Manual test instructions are in [manual-test/README.md](manual-test/README.md).
|
||||
|
||||
@@ -78,5 +77,5 @@ Candidate output schema:
|
||||
Open decisions:
|
||||
|
||||
- how to handle adjusted versus unadjusted prices;
|
||||
- how to handle missing sessions and IBKR pacing limits;
|
||||
- whether to use `ib_insync` long term or a lower-level IBKR client wrapper.
|
||||
- how to handle missing sessions, market holidays, and Alpaca API rate limits;
|
||||
- whether to add explicit feed selection, adjustment settings, or data entitlement checks.
|
||||
|
||||
+23
-30
@@ -1,30 +1,24 @@
|
||||
# Manual Test Instructions
|
||||
|
||||
## IBKR Daily Fetcher
|
||||
## Alpaca Daily Fetcher
|
||||
|
||||
This test checks the IBKR data fetcher and confirms it writes a symbol-named Parquet file.
|
||||
This test checks the Alpaca data fetcher and confirms it writes a symbol-named Parquet file.
|
||||
|
||||
The fetcher currently requests:
|
||||
|
||||
- gateway: `127.0.0.1:4002`;
|
||||
- client id: `101`;
|
||||
- symbol: provided as a command line argument;
|
||||
- end date: provided with `--end-date YYYYMMDD`, defaulting to yesterday;
|
||||
- end date can also be derived with `--end-date-from-parquet`;
|
||||
- duration: provided with `--duration`, defaulting to `1 W`;
|
||||
- bar size: `1 day`;
|
||||
- data type: `TRADES`;
|
||||
- regular trading hours only;
|
||||
- output: printed CSV-like rows and a Parquet file in `data/ibkr/daily/`.
|
||||
- data source: Alpaca stock bars through `alpaca-py`;
|
||||
- output: a Parquet file in `data/alpaca/daily/`.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Start IBKR Gateway.
|
||||
2. Log in to the paper trading account.
|
||||
3. Confirm API access is enabled in IBKR Gateway.
|
||||
4. Confirm the API socket port is `4002`.
|
||||
5. Confirm no other API client is already using client id `101`.
|
||||
6. Sync Python dependencies:
|
||||
1. Create or confirm Alpaca API credentials.
|
||||
2. Provide credentials through `ALPACA_API_KEY` and `ALPACA_SECRET_KEY`, a local `.env`, or CLI arguments.
|
||||
3. Sync Python dependencies:
|
||||
|
||||
```sh
|
||||
mise exec -- uv sync
|
||||
@@ -35,52 +29,51 @@ mise exec -- uv sync
|
||||
From the repository root, run:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
|
||||
```
|
||||
|
||||
To fetch a specific IBKR range, pass an end date and duration:
|
||||
To fetch a specific Alpaca range, pass an end date and duration:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
|
||||
```
|
||||
|
||||
To fetch backward from the oldest date already stored in `data/ibkr/daily/SPY.parquet`:
|
||||
To fetch backward from the oldest date already stored in `data/alpaca/daily/SPY.parquet`:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
|
||||
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
|
||||
```
|
||||
|
||||
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
|
||||
|
||||
## Expected Output
|
||||
|
||||
The tool should first print the request range and connection target:
|
||||
The tool should first print the request range:
|
||||
|
||||
```text
|
||||
Fetching SPY daily candles ending 2025-06-05 for duration 1 M
|
||||
Connecting to IBKR Gateway at 127.0.0.1:4002 with client id 101
|
||||
Fetching SPY daily candles ending 2025-06-05 for duration 1 M via Alpaca API
|
||||
```
|
||||
|
||||
If the request succeeds, it should then print a header and one row per returned trading day:
|
||||
If the request succeeds, it should print how many candles were fetched and where the merged Parquet file was written:
|
||||
|
||||
```text
|
||||
date,symbol,open,high,low,close,volume
|
||||
2025-06-05,SPY,...
|
||||
Fetched 21 daily candles for SPY.
|
||||
Wrote 21 total daily rows to data/alpaca/daily/SPY.parquet
|
||||
```
|
||||
|
||||
Exact prices and volume depend on what IBKR returns.
|
||||
Exact row counts depend on the requested date range and market calendar.
|
||||
|
||||
The tool should then write or update:
|
||||
|
||||
```text
|
||||
data/ibkr/daily/SPY.parquet
|
||||
data/alpaca/daily/SPY.parquet
|
||||
```
|
||||
|
||||
If the Parquet file already exists, rows from the latest fetch are merged into it. The trading date is used as the row key, so a symbol file keeps only one row for each date.
|
||||
|
||||
## Common Issues
|
||||
|
||||
- Connection refused: IBKR Gateway is not running, the port is not `4002`, or API access is disabled.
|
||||
- Client id already in use: change `IBKR_CLIENT_ID` in the fetcher or disconnect the other client.
|
||||
- No historical bars: confirm the account has market data permissions and that IBKR accepts the requested historical data range.
|
||||
- Pacing or permission errors: note the IBKR error message before changing the request.
|
||||
- Missing credentials: set `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` or pass them with CLI flags.
|
||||
- No historical bars: confirm the symbol, requested range, and Alpaca market data permissions.
|
||||
- Authentication or entitlement errors: confirm the keys belong to the intended Alpaca account and data plan.
|
||||
- Rate-limit errors: wait before retrying or reduce repeated requests.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Read-Only Performance UI
|
||||
|
||||
## Goal
|
||||
|
||||
Provide a single read-only page for checking Alpaca paper account performance and recent trading activity without exposing any trading controls.
|
||||
|
||||
## Requested Feature List
|
||||
|
||||
- Show the last three months of account performance in the header as text, or a shorter range when less data is available.
|
||||
- Show account value over time as the main body chart.
|
||||
- Include an S&P 500 comparison line in the chart. The initial implementation uses `SPY` daily candles as the S&P 500 proxy because Alpaca stock market data exposes it through the same API path.
|
||||
- Show a portfolio table below the chart with cash plus equity positions.
|
||||
- Show a completed trades table at the end of the page.
|
||||
- Use Python and `alpaca-py` in the backend.
|
||||
- Serve the page directly from the backend as a single page with no user options.
|
||||
- Cache fetched data in SQLite and refresh periodically, initially every 12 hours.
|
||||
|
||||
## Design Notes
|
||||
|
||||
The UI is intentionally separate from the trading and training modules. It reads account state from Alpaca, stores normalized snapshots in SQLite, and renders HTML from the backend. It does not place, replace, or cancel orders.
|
||||
|
||||
The first cache path is `data/ui/performance.sqlite`. On each page request, the server refreshes data only when the cache is older than the configured refresh interval. If Alpaca refresh fails but cached data exists, the page can still render stale data with a warning.
|
||||
|
||||
## Backend Data
|
||||
|
||||
The dashboard stores:
|
||||
|
||||
- account equity history for up to three months;
|
||||
- normalized S&P 500 proxy values from `SPY`;
|
||||
- the latest cash and equity holdings snapshot;
|
||||
- recently closed orders that have a filled quantity.
|
||||
|
||||
## Run Command
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
|
||||
```
|
||||
|
||||
The server defaults to `127.0.0.1:8000` and reads `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` from the environment or local `.env`.
|
||||
@@ -2,16 +2,16 @@
|
||||
|
||||
## Initial Goal
|
||||
|
||||
Build a supervised learning dataset from stored IBKR daily candle data. The first dataset predicts whether `SPY` closes higher five trading days after the observation date.
|
||||
Build a supervised learning dataset from stored Alpaca daily candle data. The first dataset predicts whether `SPY` closes higher five trading days after the observation date.
|
||||
|
||||
The first implementation is [../notebooks/spy_direction_dataset.ipynb](../notebooks/spy_direction_dataset.ipynb), a Python notebook using pandas. This keeps the feature calculations inspectable while the dataset design is still changing. Once the feature contract settles, reusable loading and feature-building code can move into `src/trading_bot/models` or `src/trading_bot/data`.
|
||||
The current implementation is `src/trading_bot/data/train_pipeline.py`, which combines the original dataset notebook and XGBoost training notebook into one runnable script. The earlier notebook, [../notebooks/spy_direction_dataset.ipynb](../notebooks/spy_direction_dataset.ipynb), remains useful background for inspecting the initial feature design.
|
||||
|
||||
## Raw Data
|
||||
|
||||
Raw daily candles are stored under:
|
||||
|
||||
```text
|
||||
data/ibkr/daily
|
||||
data/alpaca/daily
|
||||
```
|
||||
|
||||
Storage expectations:
|
||||
@@ -21,10 +21,10 @@ Storage expectations:
|
||||
- each row contains one daily candle for that symbol;
|
||||
- expected columns are `date`, `symbol`, `open`, `high`, `low`, `close`, and `volume`.
|
||||
|
||||
The initial dataset requires at least these symbols:
|
||||
The initial dataset requires at least these logical inputs:
|
||||
|
||||
- `SPY`;
|
||||
- `VIX`;
|
||||
- volatility proxy, currently `VIXY` in `config/train_config.json`;
|
||||
- `TLT`;
|
||||
- `USO`.
|
||||
|
||||
@@ -57,7 +57,7 @@ The target column is a binary indicator of `SPY` forward return over the next fi
|
||||
| --- | --- |
|
||||
| `spy_up_5d` | `1.0` when `SPY` closes above today's close five trading days later; `0.0` when `SPY` closes below today's close five trading days later; `0.5` when the future close equals today's close. |
|
||||
|
||||
Using `0.5` for unchanged prices preserves the row while making the target explicitly neutral.
|
||||
The original dataset design used `0.5` for unchanged prices as an explicit neutral target. The current training pipeline drops unchanged `0.5` rows before training so the XGBoost model remains a binary classifier.
|
||||
|
||||
## Data Alignment
|
||||
|
||||
@@ -109,9 +109,24 @@ Training, validation, and test splits should be chronological:
|
||||
|
||||
A split indicator column such as `split` is useful in the dataset artifact for auditability and reproducibility. It should be treated as metadata, not as a model input feature. The model training code should build `X` from the explicit feature column list and exclude metadata columns such as `date`, `split`, raw close prices, and the target.
|
||||
|
||||
## Training Pipeline
|
||||
|
||||
Run the current pipeline from the repository root:
|
||||
|
||||
```sh
|
||||
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
|
||||
```
|
||||
|
||||
The script reads configuration from `config/train_config.json`, writes the dataset to `data/training/spy_direction_5d.parquet`, and saves model artifacts to:
|
||||
|
||||
- `models/spy_xgb_v1.json`;
|
||||
- `models/spy_xgb_v1_meta.json`.
|
||||
|
||||
The metadata file stores the training base probability, feature column list, last trained date, and model configuration.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
- exact adjusted versus unadjusted close handling;
|
||||
- whether same-day `VIX`, `TLT`, and `USO` values are acceptable for the intended trading decision timing;
|
||||
- whether same-day volatility proxy, `TLT`, and `USO` values are acceptable for the intended trading decision timing;
|
||||
- exact train, validation, and test date boundaries or split percentages;
|
||||
- output dataset file location, schema metadata, and versioning.
|
||||
- output dataset schema metadata and versioning.
|
||||
|
||||
@@ -1,228 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cd849b8e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# IBKR scratch notebook\n",
|
||||
"\n",
|
||||
"This notebook is for quick manual testing of the IBKR gateway connection, portfolio lookup, and a simple SPY order flow.\n",
|
||||
"\n",
|
||||
"> Use this only with a paper-trading or test setup unless you explicitly intend to submit a live order.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "543426a2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Connecting to IBKR Gateway at 127.0.0.1:4002 with client id 101...\n",
|
||||
"Failed to connect to IBKR Gateway: This event loop is already running\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Error 200, reqId 9: No security definition has been found for the request, contract: Stock(symbol='VUAA', exchange='SMART', currency='USD')\n",
|
||||
"Error 200, reqId 10: No security definition has been found for the request\n",
|
||||
"Canceled order: Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0), TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 743208, tzinfo=datetime.timezone.utc), status='Cancelled', message='Error 200, reqId 10: No security definition has been found for the request', errorCode=200)], advancedError='')\n",
|
||||
"Peer closed connection.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"from pathlib import Path\n",
|
||||
"\n",
|
||||
"from ib_insync import IB, Stock\n",
|
||||
"\n",
|
||||
"# Allow the notebook to import local source code from the repository.\n",
|
||||
"repo_root = Path.cwd().resolve()\n",
|
||||
"if (repo_root / \"src\").exists():\n",
|
||||
" repo_root = repo_root\n",
|
||||
"else:\n",
|
||||
" repo_root = repo_root.parent\n",
|
||||
"\n",
|
||||
"src_path = repo_root / \"src\"\n",
|
||||
"if str(src_path) not in sys.path:\n",
|
||||
" sys.path.insert(0, str(src_path))\n",
|
||||
"\n",
|
||||
"IBKR_HOST = \"127.0.0.1\"\n",
|
||||
"IBKR_PORT = 4002\n",
|
||||
"IBKR_CLIENT_ID = 101\n",
|
||||
"IBKR_CONNECT_TIMEOUT_SECONDS = 10\n",
|
||||
"\n",
|
||||
"ib = IB()\n",
|
||||
"print(f\"Connecting to IBKR Gateway at {IBKR_HOST}:{IBKR_PORT} with client id {IBKR_CLIENT_ID}...\")\n",
|
||||
"try:\n",
|
||||
" ib.connect(IBKR_HOST, IBKR_PORT, clientId=IBKR_CLIENT_ID, timeout=IBKR_CONNECT_TIMEOUT_SECONDS)\n",
|
||||
" print(\"Connection established.\")\n",
|
||||
"except Exception as e:\n",
|
||||
" print(f\"Failed to connect to IBKR Gateway: {e}\")\n",
|
||||
" ib.disconnect()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "ecdc721f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Connected: True\n",
|
||||
"Client ID: 101\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(f\"Connected: {ib.isConnected()}\")\n",
|
||||
"print(f\"Client ID: {ib.client.clientId}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"id": "10ebc240",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No open portfolio positions were returned.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"portfolio = ib.portfolio()\n",
|
||||
"if not portfolio:\n",
|
||||
" print(\"No open portfolio positions were returned.\")\n",
|
||||
"else:\n",
|
||||
" for item in portfolio:\n",
|
||||
" print(\n",
|
||||
" f\"{item.contract.symbol}: position={item.position}, \"\n",
|
||||
" f\"market_value={item.marketValue}, unrealized_pnl={item.unrealizedPNL}\"\n",
|
||||
" )\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 23,
|
||||
"id": "fed654ec",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Unknown contract: Stock(symbol='VUAA', exchange='SMART', currency='USD')\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Submitted order for VUAA: Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='PendingSubmit', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0)], advancedError='')\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from ib_insync import MarketOrder\n",
|
||||
"\n",
|
||||
"contract = Stock(\"VUAA\", \"SMART\", \"USD\")\n",
|
||||
"await ib.qualifyContractsAsync(contract)\n",
|
||||
"\n",
|
||||
"# Adjust the quantity as needed before running this cell.\n",
|
||||
"order = MarketOrder(\"BUY\", 0.70)\n",
|
||||
"trade = ib.placeOrder(contract, order)\n",
|
||||
"\n",
|
||||
"print(f\"Submitted order for {contract.symbol}: {trade}\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"id": "06881101",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"trds = ib.trades()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"id": "08663602",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['Cancelled', 'Cancelled', 'Cancelled']"
|
||||
]
|
||||
},
|
||||
"execution_count": 42,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"list(map(lambda x: x.orderStatus.status, trds))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"id": "2c7abf23",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[Trade(contract=Stock(conId=756733, symbol='SPY', right='?', exchange='SMART', currency='USD', localSymbol='SPY', tradingClass='SPY'), order=Order(permId=1578393268, action='BUY', totalQuantity=1.0, orderType='MKT', lmtPrice=0.0, auxPrice=0.0, tif='DAY', ocaType=3, displaySize=2147483647, rule80A='0', openClose='', volatilityType=0, deltaNeutralOrderType='None', referencePriceType=0, account='DUR281921', clearingIntent='IB', cashQty=0.0, dontUseAutoPriceForHedge=True, filledQuantity=0.0, refFuturesConId=2147483647, shareholder='Not an insider or substantial shareholder'), orderStatus=OrderStatus(orderId=0, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[], advancedError=''),\n",
|
||||
" Trade(contract=Stock(conId=756733, symbol='SPY', right='?', exchange='SMART', currency='USD', localSymbol='SPY', tradingClass='SPY'), order=Order(permId=24474350, action='BUY', totalQuantity=0.1345, orderType='MKT', lmtPrice=0.0, auxPrice=0.0, tif='DAY', ocaType=3, displaySize=2147483647, rule80A='0', openClose='', volatilityType=0, deltaNeutralOrderType='None', referencePriceType=0, account='DUR281921', clearingIntent='IB', cashQty=0.0, dontUseAutoPriceForHedge=True, filledQuantity=0.0, refFuturesConId=2147483647, shareholder='Not an insider or substantial shareholder'), orderStatus=OrderStatus(orderId=0, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[], advancedError=''),\n",
|
||||
" Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0), TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 743208, tzinfo=datetime.timezone.utc), status='Cancelled', message='Error 200, reqId 10: No security definition has been found for the request', errorCode=200)], advancedError='')]"
|
||||
]
|
||||
},
|
||||
"execution_count": 43,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"trds"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.15"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -87,16 +87,16 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<class 'pandas.DataFrame'>\n",
|
||||
"DatetimeIndex: 1255 entries, 2021-08-02 to 2026-07-31\n",
|
||||
"DatetimeIndex: 1258 entries, 2021-08-02 to 2026-08-05\n",
|
||||
"Data columns (total 4 columns):\n",
|
||||
" # Column Non-Null Count Dtype \n",
|
||||
"--- ------ -------------- ----- \n",
|
||||
" 0 SPY_close 1255 non-null float64\n",
|
||||
" 1 VIX_close 1255 non-null float64\n",
|
||||
" 2 TLT_close 1255 non-null float64\n",
|
||||
" 3 USO_close 1255 non-null float64\n",
|
||||
" 0 SPY_close 1258 non-null float64\n",
|
||||
" 1 VIX_close 1258 non-null float64\n",
|
||||
" 2 TLT_close 1258 non-null float64\n",
|
||||
" 3 USO_close 1258 non-null float64\n",
|
||||
"dtypes: float64(4)\n",
|
||||
"memory usage: 49.0 KB\n"
|
||||
"memory usage: 49.1 KB\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -234,7 +234,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -364,7 +364,7 @@
|
||||
"2021-10-15 -0.002202 0.030287 -0.003823 1.0 "
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -418,7 +418,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -458,24 +458,24 @@
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>train</th>\n",
|
||||
" <td>840</td>\n",
|
||||
" <td>842</td>\n",
|
||||
" <td>2021-10-11</td>\n",
|
||||
" <td>2025-02-13</td>\n",
|
||||
" <td>0.583333</td>\n",
|
||||
" <td>2025-02-18</td>\n",
|
||||
" <td>0.581948</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>validation</th>\n",
|
||||
" <td>180</td>\n",
|
||||
" <td>2025-02-14</td>\n",
|
||||
" <td>2025-10-31</td>\n",
|
||||
" <td>0.633333</td>\n",
|
||||
" <td>2025-02-19</td>\n",
|
||||
" <td>2025-11-04</td>\n",
|
||||
" <td>0.638889</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>test</th>\n",
|
||||
" <td>181</td>\n",
|
||||
" <td>2025-11-03</td>\n",
|
||||
" <td>2026-07-24</td>\n",
|
||||
" <td>0.569061</td>\n",
|
||||
" <td>182</td>\n",
|
||||
" <td>2025-11-05</td>\n",
|
||||
" <td>2026-07-29</td>\n",
|
||||
" <td>0.576923</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
@@ -484,12 +484,12 @@
|
||||
"text/plain": [
|
||||
" rows start_date end_date target_mean\n",
|
||||
"split \n",
|
||||
"train 840 2021-10-11 2025-02-13 0.583333\n",
|
||||
"validation 180 2025-02-14 2025-10-31 0.633333\n",
|
||||
"test 181 2025-11-03 2026-07-24 0.569061"
|
||||
"train 842 2021-10-11 2025-02-18 0.581948\n",
|
||||
"validation 180 2025-02-19 2025-11-04 0.638889\n",
|
||||
"test 182 2025-11-05 2026-07-29 0.576923"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -543,17 +543,17 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Rows: 1,201\n",
|
||||
"Rows: 1,204\n",
|
||||
"Feature columns: ['SPY_ret_5', 'SPY_ret_20', 'SPY_dist_sma50', 'VIX_change_5', 'VIX_rank_20', 'TLT_ret_10', 'USO_ret_5', 'SPY_TLT_ratio_ret']\n",
|
||||
"Target column: spy_up_5d\n",
|
||||
"Training matrix shape: (840, 8)\n"
|
||||
"Training matrix shape: (842, 8)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -592,16 +592,16 @@
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>count</th>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201.000000</td>\n",
|
||||
" <td>1201</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204.000000</td>\n",
|
||||
" <td>1204</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>unique</th>\n",
|
||||
@@ -640,32 +640,32 @@
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" <td>840</td>\n",
|
||||
" <td>842</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>mean</th>\n",
|
||||
" <td>0.002502</td>\n",
|
||||
" <td>0.009848</td>\n",
|
||||
" <td>0.011028</td>\n",
|
||||
" <td>0.017763</td>\n",
|
||||
" <td>0.403476</td>\n",
|
||||
" <td>-0.004099</td>\n",
|
||||
" <td>0.005024</td>\n",
|
||||
" <td>0.004880</td>\n",
|
||||
" <td>0.588676</td>\n",
|
||||
" <td>0.002464</td>\n",
|
||||
" <td>0.009816</td>\n",
|
||||
" <td>0.010973</td>\n",
|
||||
" <td>0.017820</td>\n",
|
||||
" <td>0.404506</td>\n",
|
||||
" <td>-0.004103</td>\n",
|
||||
" <td>0.004938</td>\n",
|
||||
" <td>0.004838</td>\n",
|
||||
" <td>0.589701</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>std</th>\n",
|
||||
" <td>0.023057</td>\n",
|
||||
" <td>0.043558</td>\n",
|
||||
" <td>0.037315</td>\n",
|
||||
" <td>0.299417</td>\n",
|
||||
" <td>0.337501</td>\n",
|
||||
" <td>0.028502</td>\n",
|
||||
" <td>0.052628</td>\n",
|
||||
" <td>0.027606</td>\n",
|
||||
" <td>0.492279</td>\n",
|
||||
" <td>0.023044</td>\n",
|
||||
" <td>0.043515</td>\n",
|
||||
" <td>0.037287</td>\n",
|
||||
" <td>0.299050</td>\n",
|
||||
" <td>0.337773</td>\n",
|
||||
" <td>0.028469</td>\n",
|
||||
" <td>0.052606</td>\n",
|
||||
" <td>0.027586</td>\n",
|
||||
" <td>0.492092</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
@@ -683,40 +683,40 @@
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>25%</th>\n",
|
||||
" <td>-0.009601</td>\n",
|
||||
" <td>-0.016509</td>\n",
|
||||
" <td>-0.008656</td>\n",
|
||||
" <td>-0.056615</td>\n",
|
||||
" <td>-0.009749</td>\n",
|
||||
" <td>-0.016557</td>\n",
|
||||
" <td>-0.008704</td>\n",
|
||||
" <td>-0.056606</td>\n",
|
||||
" <td>0.100000</td>\n",
|
||||
" <td>-0.023304</td>\n",
|
||||
" <td>-0.025866</td>\n",
|
||||
" <td>-0.010013</td>\n",
|
||||
" <td>-0.023139</td>\n",
|
||||
" <td>-0.025950</td>\n",
|
||||
" <td>-0.010029</td>\n",
|
||||
" <td>0.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>50%</th>\n",
|
||||
" <td>0.003798</td>\n",
|
||||
" <td>0.015725</td>\n",
|
||||
" <td>0.017802</td>\n",
|
||||
" <td>-0.018570</td>\n",
|
||||
" <td>0.003749</td>\n",
|
||||
" <td>0.015639</td>\n",
|
||||
" <td>0.017755</td>\n",
|
||||
" <td>-0.018430</td>\n",
|
||||
" <td>0.300000</td>\n",
|
||||
" <td>-0.004435</td>\n",
|
||||
" <td>0.004950</td>\n",
|
||||
" <td>0.005537</td>\n",
|
||||
" <td>-0.004429</td>\n",
|
||||
" <td>0.004869</td>\n",
|
||||
" <td>0.005508</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>75%</th>\n",
|
||||
" <td>0.015955</td>\n",
|
||||
" <td>0.038179</td>\n",
|
||||
" <td>0.038282</td>\n",
|
||||
" <td>0.030814</td>\n",
|
||||
" <td>0.015950</td>\n",
|
||||
" <td>0.038167</td>\n",
|
||||
" <td>0.038124</td>\n",
|
||||
" <td>0.030815</td>\n",
|
||||
" <td>0.750000</td>\n",
|
||||
" <td>0.014227</td>\n",
|
||||
" <td>0.032486</td>\n",
|
||||
" <td>0.021390</td>\n",
|
||||
" <td>0.014188</td>\n",
|
||||
" <td>0.032295</td>\n",
|
||||
" <td>0.021358</td>\n",
|
||||
" <td>1.000000</td>\n",
|
||||
" <td>NaN</td>\n",
|
||||
" </tr>\n",
|
||||
@@ -739,33 +739,33 @@
|
||||
],
|
||||
"text/plain": [
|
||||
" SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 VIX_rank_20 \\\n",
|
||||
"count 1201.000000 1201.000000 1201.000000 1201.000000 1201.000000 \n",
|
||||
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204.000000 \n",
|
||||
"unique NaN NaN NaN NaN NaN \n",
|
||||
"top NaN NaN NaN NaN NaN \n",
|
||||
"freq NaN NaN NaN NaN NaN \n",
|
||||
"mean 0.002502 0.009848 0.011028 0.017763 0.403476 \n",
|
||||
"std 0.023057 0.043558 0.037315 0.299417 0.337501 \n",
|
||||
"mean 0.002464 0.009816 0.010973 0.017820 0.404506 \n",
|
||||
"std 0.023044 0.043515 0.037287 0.299050 0.337773 \n",
|
||||
"min -0.114962 -0.123975 -0.141995 -0.387709 0.050000 \n",
|
||||
"25% -0.009601 -0.016509 -0.008656 -0.056615 0.100000 \n",
|
||||
"50% 0.003798 0.015725 0.017802 -0.018570 0.300000 \n",
|
||||
"75% 0.015955 0.038179 0.038282 0.030814 0.750000 \n",
|
||||
"25% -0.009749 -0.016557 -0.008704 -0.056606 0.100000 \n",
|
||||
"50% 0.003749 0.015639 0.017755 -0.018430 0.300000 \n",
|
||||
"75% 0.015950 0.038167 0.038124 0.030815 0.750000 \n",
|
||||
"max 0.082843 0.157566 0.088105 3.846154 1.000000 \n",
|
||||
"\n",
|
||||
" TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret spy_up_5d split \n",
|
||||
"count 1201.000000 1201.000000 1201.000000 1201.000000 1201 \n",
|
||||
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204 \n",
|
||||
"unique NaN NaN NaN NaN 3 \n",
|
||||
"top NaN NaN NaN NaN train \n",
|
||||
"freq NaN NaN NaN NaN 840 \n",
|
||||
"mean -0.004099 0.005024 0.004880 0.588676 NaN \n",
|
||||
"std 0.028502 0.052628 0.027606 0.492279 NaN \n",
|
||||
"freq NaN NaN NaN NaN 842 \n",
|
||||
"mean -0.004103 0.004938 0.004838 0.589701 NaN \n",
|
||||
"std 0.028469 0.052606 0.027586 0.492092 NaN \n",
|
||||
"min -0.092880 -0.196652 -0.117208 0.000000 NaN \n",
|
||||
"25% -0.023304 -0.025866 -0.010013 0.000000 NaN \n",
|
||||
"50% -0.004435 0.004950 0.005537 1.000000 NaN \n",
|
||||
"75% 0.014227 0.032486 0.021390 1.000000 NaN \n",
|
||||
"25% -0.023139 -0.025950 -0.010029 0.000000 NaN \n",
|
||||
"50% -0.004429 0.004869 0.005508 1.000000 NaN \n",
|
||||
"75% 0.014188 0.032295 0.021358 1.000000 NaN \n",
|
||||
"max 0.091322 0.327273 0.129204 1.000000 NaN "
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -787,14 +787,14 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(PosixPath('/home/jarno/repos/trading-bot/data/training/spy_direction_5d.parquet'),\n",
|
||||
" (1201, 11),\n",
|
||||
" (1204, 11),\n",
|
||||
" date SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 \\\n",
|
||||
" 0 2021-10-11 0.014114 -0.026625 -0.018145 -0.090869 \n",
|
||||
" 1 2021-10-12 0.001201 -0.023752 -0.020386 -0.074091 \n",
|
||||
@@ -810,7 +810,7 @@
|
||||
" 4 0.05 -0.002202 0.030287 -0.003823 1.0 train )"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
|
||||
+1
-2
@@ -1,12 +1,11 @@
|
||||
[project]
|
||||
name = "trading-bot"
|
||||
version = "0.1.0"
|
||||
description = "Python trading bot using machine learning and the IBKR API."
|
||||
description = "Python trading bot using machine learning and the Alpaca Markets API."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.12"
|
||||
dependencies = [
|
||||
"alpaca-py>=0.43.5",
|
||||
"ib-insync>=0.9.86",
|
||||
"matplotlib>=3.11.1",
|
||||
"pandas>=2.3.0",
|
||||
"pyarrow>=20.0.0",
|
||||
|
||||
@@ -98,9 +98,10 @@ def main() -> None:
|
||||
api_key=args.api_key,
|
||||
secret_key=args.secret_key,
|
||||
)
|
||||
print(f"Fetched {len(candles)} daily candles for {symbol}.")
|
||||
stored = write_candles(output_path, symbol, candles)
|
||||
print(f"Wrote {len(stored)} total daily rows to {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
|
||||
@@ -1,293 +0,0 @@
|
||||
"""IBKR daily candle fetcher."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pandas as pd
|
||||
|
||||
DEFAULT_OUTPUT_DIR = Path("data/ibkr/daily")
|
||||
DEFAULT_DURATION = "1 W"
|
||||
EASTERN_TZ = ZoneInfo("America/New_York")
|
||||
DURATION_PATTERN = re.compile(r"^\d+\s+[SDWMY]$")
|
||||
|
||||
IBKR_HOST = "127.0.0.1"
|
||||
IBKR_PORT = 4002
|
||||
IBKR_CLIENT_ID = 101
|
||||
IBKR_CONNECT_TIMEOUT_SECONDS = 10
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DailyCandle:
|
||||
"""Daily OHLCV market data for one trading session."""
|
||||
|
||||
trading_day: date
|
||||
open: float
|
||||
high: float
|
||||
low: float
|
||||
close: float
|
||||
volume: int
|
||||
|
||||
|
||||
def default_end_date() -> date:
|
||||
"""Return yesterday's date in the US/Eastern market timezone."""
|
||||
|
||||
return datetime.now(EASTERN_TZ).date() - timedelta(days=1)
|
||||
|
||||
|
||||
def current_market_date() -> date:
|
||||
"""Return today's date in the US/Eastern market timezone."""
|
||||
|
||||
return datetime.now(EASTERN_TZ).date()
|
||||
|
||||
|
||||
def parse_end_date(value: str) -> date:
|
||||
"""Parse an end date in IBKR-friendly YYYYMMDD form."""
|
||||
|
||||
try:
|
||||
return datetime.strptime(value, "%Y%m%d").date()
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError(
|
||||
"end date must use YYYYMMDD format, such as 20250605"
|
||||
) from exc
|
||||
|
||||
|
||||
def parse_duration(value: str) -> str:
|
||||
"""Normalize and validate an IBKR duration string like '1 W' or '1 M'."""
|
||||
|
||||
normalized = " ".join(value.upper().split())
|
||||
if not DURATION_PATTERN.fullmatch(normalized):
|
||||
raise argparse.ArgumentTypeError(
|
||||
"duration must look like an IBKR value, such as '1 W' or '1 M'"
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def format_ibkr_end_datetime(end_date: date) -> str:
|
||||
"""Convert a date to the US/Eastern end datetime string IBKR expects."""
|
||||
|
||||
return f"{end_date:%Y%m%d} 23:59:59 US/Eastern"
|
||||
|
||||
|
||||
def normalize_bar_date(value: date | datetime | str) -> date:
|
||||
"""Normalize an IBKR historical bar date value."""
|
||||
|
||||
if isinstance(value, datetime):
|
||||
return value.date()
|
||||
if isinstance(value, date):
|
||||
return value
|
||||
return parse_end_date(value)
|
||||
|
||||
|
||||
def fetch_daily_candles(
|
||||
symbol: str, end_date: date, duration: str
|
||||
) -> list[DailyCandle]:
|
||||
"""Fetch daily candles for a symbol from IBKR."""
|
||||
|
||||
try:
|
||||
from ib_insync import IB, Stock # type: ignore[import-not-found]
|
||||
except ImportError as exc:
|
||||
raise SystemExit(
|
||||
"Missing dependency: ib_insync. Install it before running the "
|
||||
"IBKR fetcher."
|
||||
) from exc
|
||||
|
||||
ib = IB()
|
||||
try:
|
||||
print(
|
||||
f"Connecting to IBKR Gateway at {IBKR_HOST}:{IBKR_PORT} "
|
||||
f"with client id {IBKR_CLIENT_ID}"
|
||||
)
|
||||
ib.connect(
|
||||
IBKR_HOST,
|
||||
IBKR_PORT,
|
||||
clientId=IBKR_CLIENT_ID,
|
||||
timeout=IBKR_CONNECT_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
contract = Stock(symbol, "SMART", "USD")
|
||||
ib.qualifyContracts(contract)
|
||||
bars = ib.reqHistoricalData(
|
||||
contract,
|
||||
endDateTime=format_ibkr_end_datetime(end_date),
|
||||
durationStr=duration,
|
||||
barSizeSetting="1 day",
|
||||
whatToShow="TRADES",
|
||||
useRTH=True,
|
||||
formatDate=1,
|
||||
)
|
||||
|
||||
if not bars:
|
||||
print(f"IBKR returned no historical bars for {symbol}.")
|
||||
|
||||
candles: list[DailyCandle] = []
|
||||
for bar in bars:
|
||||
trading_day = normalize_bar_date(bar.date)
|
||||
candles.append(
|
||||
DailyCandle(
|
||||
trading_day=trading_day,
|
||||
open=float(bar.open),
|
||||
high=float(bar.high),
|
||||
low=float(bar.low),
|
||||
close=float(bar.close),
|
||||
volume=int(bar.volume),
|
||||
)
|
||||
)
|
||||
|
||||
return candles
|
||||
finally:
|
||||
if ib.isConnected():
|
||||
ib.disconnect()
|
||||
|
||||
|
||||
def candles_to_frame(symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
|
||||
"""Convert candles to a date-indexed dataframe ready for Parquet storage."""
|
||||
|
||||
rows = [
|
||||
{
|
||||
"date": candle.trading_day,
|
||||
"symbol": symbol,
|
||||
"open": candle.open,
|
||||
"high": candle.high,
|
||||
"low": candle.low,
|
||||
"close": candle.close,
|
||||
"volume": candle.volume,
|
||||
}
|
||||
for candle in candles
|
||||
]
|
||||
frame = pd.DataFrame.from_records(rows)
|
||||
if frame.empty:
|
||||
return pd.DataFrame(
|
||||
columns=["symbol", "open", "high", "low", "close", "volume"],
|
||||
index=pd.Index([], name="date"),
|
||||
)
|
||||
|
||||
frame["date"] = pd.to_datetime(frame["date"]).dt.date
|
||||
return frame.set_index("date")
|
||||
|
||||
|
||||
def read_existing_candles(path: Path) -> pd.DataFrame:
|
||||
"""Read an existing candle Parquet file as a date-indexed dataframe."""
|
||||
|
||||
if not path.exists():
|
||||
return pd.DataFrame(
|
||||
columns=["symbol", "open", "high", "low", "close", "volume"],
|
||||
index=pd.Index([], name="date"),
|
||||
)
|
||||
|
||||
frame = pd.read_parquet(path)
|
||||
if "date" in frame.columns:
|
||||
frame["date"] = pd.to_datetime(frame["date"]).dt.date
|
||||
frame = frame.set_index("date")
|
||||
|
||||
frame.index = pd.to_datetime(frame.index).date
|
||||
frame.index.name = "date"
|
||||
return frame
|
||||
|
||||
|
||||
def oldest_stored_date_or_today(path: Path) -> date:
|
||||
"""Return the oldest stored candle date, or today if no data exists yet."""
|
||||
|
||||
existing = read_existing_candles(path)
|
||||
if existing.empty:
|
||||
return current_market_date()
|
||||
return min(existing.index)
|
||||
|
||||
|
||||
def write_candles(path: Path, symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
|
||||
"""Append candles to a ticker Parquet file, keeping one row per date."""
|
||||
|
||||
existing = read_existing_candles(path)
|
||||
fetched = candles_to_frame(symbol, candles)
|
||||
combined = pd.concat([existing, fetched])
|
||||
if not combined.empty:
|
||||
combined = combined[~combined.index.duplicated(keep="last")]
|
||||
combined = combined.sort_index()
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
combined.to_parquet(path, index=True)
|
||||
return combined
|
||||
|
||||
|
||||
def print_candles(symbol: str, candles: list[DailyCandle]) -> None:
|
||||
"""Print candles in a compact table."""
|
||||
|
||||
print("date,symbol,open,high,low,close,volume")
|
||||
for candle in candles:
|
||||
print(
|
||||
f"{candle.trading_day.isoformat()},"
|
||||
f"{symbol},"
|
||||
f"{candle.open:.2f},"
|
||||
f"{candle.high:.2f},"
|
||||
f"{candle.low:.2f},"
|
||||
f"{candle.close:.2f},"
|
||||
f"{candle.volume}"
|
||||
)
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""Parse command line arguments."""
|
||||
|
||||
parser = argparse.ArgumentParser(description="Fetch daily IBKR candles.")
|
||||
parser.add_argument("symbol", help="Ticker symbol to fetch, such as SPY.")
|
||||
end_date_group = parser.add_mutually_exclusive_group()
|
||||
end_date_group.add_argument(
|
||||
"--end-date",
|
||||
type=parse_end_date,
|
||||
help="Request end date in YYYYMMDD format. Defaults to yesterday.",
|
||||
)
|
||||
end_date_group.add_argument(
|
||||
"--end-date-from-parquet",
|
||||
action="store_true",
|
||||
help=(
|
||||
"Use the oldest date from the symbol Parquet file as the request "
|
||||
"end date. Defaults to today if the file is missing or empty."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--duration",
|
||||
type=parse_duration,
|
||||
default=DEFAULT_DURATION,
|
||||
help=(
|
||||
"IBKR duration string, such as '1 W' or '1 M'. "
|
||||
f"Defaults to {DEFAULT_DURATION}."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
default=DEFAULT_OUTPUT_DIR,
|
||||
help=f"Directory for Parquet files. Defaults to {DEFAULT_OUTPUT_DIR}.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Run the daily candle fetcher."""
|
||||
|
||||
args = parse_args()
|
||||
symbol = args.symbol.upper()
|
||||
output_path = args.output_dir / f"{symbol}.parquet"
|
||||
end_date = (
|
||||
oldest_stored_date_or_today(output_path)
|
||||
if args.end_date_from_parquet
|
||||
else args.end_date or default_end_date()
|
||||
)
|
||||
|
||||
print(
|
||||
f"Fetching {symbol} daily candles ending {end_date:%Y-%m-%d} "
|
||||
f"for duration {args.duration}"
|
||||
)
|
||||
candles = fetch_daily_candles(symbol, end_date, args.duration)
|
||||
print_candles(symbol, candles)
|
||||
stored = write_candles(output_path, symbol, candles)
|
||||
print(f"Wrote {len(stored)} total daily rows to {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
"""Read-only web UI for inspecting trading bot status."""
|
||||
@@ -0,0 +1,891 @@
|
||||
"""Read-only Alpaca performance dashboard served by a small Python backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
from html import escape
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from alpaca.trading.client import TradingClient
|
||||
from alpaca.trading.enums import QueryOrderStatus
|
||||
from alpaca.trading.models import Order, Position, TradeAccount
|
||||
from alpaca.trading.requests import GetOrdersRequest, GetPortfolioHistoryRequest
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import (
|
||||
DailyCandle,
|
||||
EASTERN_TZ,
|
||||
default_end_date,
|
||||
fetch_daily_candles,
|
||||
)
|
||||
|
||||
load_dotenv()
|
||||
|
||||
DEFAULT_DB_PATH = Path("data/ui/performance.sqlite")
|
||||
DEFAULT_HOST = "127.0.0.1"
|
||||
DEFAULT_PORT = 8000
|
||||
DEFAULT_REFRESH_HOURS = 12
|
||||
DEFAULT_PERIOD_DAYS = 92
|
||||
BENCHMARK_SYMBOL = "SPY"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PerformancePoint:
|
||||
trading_day: date
|
||||
account_value: float
|
||||
benchmark_value: float | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PortfolioRow:
|
||||
symbol: str
|
||||
asset_type: str
|
||||
quantity: float | None
|
||||
market_value: float
|
||||
average_entry_price: float | None
|
||||
current_price: float | None
|
||||
unrealized_pl: float | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TradeRow:
|
||||
filled_at: datetime | None
|
||||
symbol: str
|
||||
side: str
|
||||
quantity: float
|
||||
filled_average_price: float | None
|
||||
notional: float | None
|
||||
status: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DashboardSnapshot:
|
||||
refreshed_at: datetime
|
||||
performance: list[PerformancePoint]
|
||||
portfolio: list[PortfolioRow]
|
||||
trades: list[TradeRow]
|
||||
warning: str | None = None
|
||||
|
||||
|
||||
def _to_float(value: Any, default: float = 0.0) -> float:
|
||||
if value is None:
|
||||
return default
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def _to_optional_float(value: Any) -> float | None:
|
||||
if value is None or value == "":
|
||||
return None
|
||||
return _to_float(value)
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def _from_unix_timestamp(value: int | float) -> date:
|
||||
return datetime.fromtimestamp(value, tz=UTC).astimezone(EASTERN_TZ).date()
|
||||
|
||||
|
||||
def _benchmark_end_date(account_history_last_day: date) -> date:
|
||||
return min(account_history_last_day, default_end_date())
|
||||
|
||||
|
||||
def init_database(db_path: Path) -> None:
|
||||
db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
connection.executescript(
|
||||
"""
|
||||
create table if not exists metadata (
|
||||
key text primary key,
|
||||
value text not null
|
||||
);
|
||||
create table if not exists performance_points (
|
||||
trading_day text primary key,
|
||||
account_value real not null,
|
||||
benchmark_value real
|
||||
);
|
||||
create table if not exists portfolio_rows (
|
||||
symbol text primary key,
|
||||
asset_type text not null,
|
||||
quantity real,
|
||||
market_value real not null,
|
||||
average_entry_price real,
|
||||
current_price real,
|
||||
unrealized_pl real
|
||||
);
|
||||
create table if not exists trade_rows (
|
||||
id integer primary key autoincrement,
|
||||
filled_at text,
|
||||
symbol text not null,
|
||||
side text not null,
|
||||
quantity real not null,
|
||||
filled_average_price real,
|
||||
notional real,
|
||||
status text not null
|
||||
);
|
||||
"""
|
||||
)
|
||||
|
||||
|
||||
def get_last_refresh(db_path: Path) -> datetime | None:
|
||||
if not db_path.exists():
|
||||
return None
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
row = connection.execute(
|
||||
"select value from metadata where key = 'refreshed_at'"
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return datetime.fromisoformat(row[0])
|
||||
|
||||
|
||||
def is_cache_stale(db_path: Path, refresh_interval: timedelta, now: datetime | None = None) -> bool:
|
||||
refreshed_at = get_last_refresh(db_path)
|
||||
if refreshed_at is None:
|
||||
return True
|
||||
return (now or _utc_now()) - refreshed_at >= refresh_interval
|
||||
|
||||
|
||||
def create_trading_client(
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
paper: bool,
|
||||
) -> TradingClient:
|
||||
if api_key is None or secret_key is None:
|
||||
raise ValueError("ALPACA_API_KEY and ALPACA_SECRET_KEY are required.")
|
||||
return TradingClient(api_key, secret_key, paper=paper)
|
||||
|
||||
|
||||
def fetch_performance_points(
|
||||
client: TradingClient,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
) -> list[PerformancePoint]:
|
||||
history = client.get_portfolio_history(
|
||||
GetPortfolioHistoryRequest(period="3M", timeframe="1D")
|
||||
)
|
||||
account_values_by_day = {
|
||||
_from_unix_timestamp(timestamp): float(equity)
|
||||
for timestamp, equity in zip(history.timestamp, history.equity, strict=False)
|
||||
if equity is not None
|
||||
}
|
||||
if not account_values_by_day:
|
||||
return []
|
||||
|
||||
first_day = min(account_values_by_day)
|
||||
last_day = max(account_values_by_day)
|
||||
benchmark = fetch_daily_candles(
|
||||
BENCHMARK_SYMBOL,
|
||||
end_date=_benchmark_end_date(last_day),
|
||||
duration="3 M",
|
||||
api_key=api_key,
|
||||
secret_key=secret_key,
|
||||
)
|
||||
benchmark_values_by_day = _normalize_benchmark(
|
||||
benchmark,
|
||||
start_day=first_day,
|
||||
initial_account_value=account_values_by_day[first_day],
|
||||
)
|
||||
|
||||
return [
|
||||
PerformancePoint(
|
||||
trading_day=trading_day,
|
||||
account_value=account_value,
|
||||
benchmark_value=benchmark_values_by_day.get(trading_day),
|
||||
)
|
||||
for trading_day, account_value in sorted(account_values_by_day.items())
|
||||
]
|
||||
|
||||
|
||||
def _normalize_benchmark(
|
||||
candles: list[DailyCandle],
|
||||
start_day: date,
|
||||
initial_account_value: float,
|
||||
) -> dict[date, float]:
|
||||
closes = {
|
||||
candle.trading_day: candle.close
|
||||
for candle in candles
|
||||
if candle.trading_day >= start_day and candle.close > 0
|
||||
}
|
||||
if not closes:
|
||||
return {}
|
||||
base_day = min(closes)
|
||||
base_close = closes[base_day]
|
||||
return {
|
||||
trading_day: initial_account_value * (close / base_close)
|
||||
for trading_day, close in closes.items()
|
||||
}
|
||||
|
||||
|
||||
def fetch_portfolio_rows(client: TradingClient) -> list[PortfolioRow]:
|
||||
account = client.get_account()
|
||||
if not isinstance(account, TradeAccount):
|
||||
raise TypeError(f"Expected TradeAccount, got {type(account)}")
|
||||
|
||||
rows = [
|
||||
PortfolioRow(
|
||||
symbol="CASH",
|
||||
asset_type="cash",
|
||||
quantity=None,
|
||||
market_value=_to_float(account.cash),
|
||||
average_entry_price=None,
|
||||
current_price=None,
|
||||
unrealized_pl=None,
|
||||
)
|
||||
]
|
||||
|
||||
positions = client.get_all_positions()
|
||||
if not isinstance(positions, list):
|
||||
raise TypeError(f"Expected list of positions, got {type(positions)}")
|
||||
for position in positions:
|
||||
if not isinstance(position, Position):
|
||||
raise TypeError(f"Expected Position, got {type(position)}")
|
||||
rows.append(
|
||||
PortfolioRow(
|
||||
symbol=str(position.symbol).upper(),
|
||||
asset_type=str(position.asset_class or "equity"),
|
||||
quantity=_to_optional_float(position.qty),
|
||||
market_value=_to_float(position.market_value),
|
||||
average_entry_price=_to_optional_float(position.avg_entry_price),
|
||||
current_price=_to_optional_float(position.current_price),
|
||||
unrealized_pl=_to_optional_float(position.unrealized_pl),
|
||||
)
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def fetch_trade_rows(client: TradingClient) -> list[TradeRow]:
|
||||
orders = client.get_orders(
|
||||
GetOrdersRequest(status=QueryOrderStatus.CLOSED, limit=100)
|
||||
)
|
||||
if not isinstance(orders, list):
|
||||
raise TypeError(f"Expected list of orders, got {type(orders)}")
|
||||
|
||||
trades: list[TradeRow] = []
|
||||
for order in orders:
|
||||
if not isinstance(order, Order):
|
||||
raise TypeError(f"Expected Order, got {type(order)}")
|
||||
quantity = _to_float(order.filled_qty)
|
||||
if quantity <= 0:
|
||||
continue
|
||||
trades.append(
|
||||
TradeRow(
|
||||
filled_at=order.filled_at,
|
||||
symbol=str(order.symbol).upper(),
|
||||
side=str(order.side).lower(),
|
||||
quantity=quantity,
|
||||
filled_average_price=_to_optional_float(order.filled_avg_price),
|
||||
notional=_to_optional_float(order.notional),
|
||||
status=str(order.status).lower(),
|
||||
)
|
||||
)
|
||||
return trades
|
||||
|
||||
|
||||
def fetch_alpaca_snapshot(
|
||||
client: TradingClient,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
) -> DashboardSnapshot:
|
||||
return DashboardSnapshot(
|
||||
refreshed_at=_utc_now(),
|
||||
performance=fetch_performance_points(client, api_key, secret_key),
|
||||
portfolio=fetch_portfolio_rows(client),
|
||||
trades=fetch_trade_rows(client),
|
||||
)
|
||||
|
||||
|
||||
def write_snapshot(db_path: Path, snapshot: DashboardSnapshot) -> None:
|
||||
init_database(db_path)
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
connection.execute("delete from performance_points")
|
||||
connection.execute("delete from portfolio_rows")
|
||||
connection.execute("delete from trade_rows")
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into performance_points (
|
||||
trading_day, account_value, benchmark_value
|
||||
) values (?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
point.trading_day.isoformat(),
|
||||
point.account_value,
|
||||
point.benchmark_value,
|
||||
)
|
||||
for point in snapshot.performance
|
||||
],
|
||||
)
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into portfolio_rows (
|
||||
symbol, asset_type, quantity, market_value,
|
||||
average_entry_price, current_price, unrealized_pl
|
||||
) values (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
row.symbol,
|
||||
row.asset_type,
|
||||
row.quantity,
|
||||
row.market_value,
|
||||
row.average_entry_price,
|
||||
row.current_price,
|
||||
row.unrealized_pl,
|
||||
)
|
||||
for row in snapshot.portfolio
|
||||
],
|
||||
)
|
||||
connection.executemany(
|
||||
"""
|
||||
insert into trade_rows (
|
||||
filled_at, symbol, side, quantity,
|
||||
filled_average_price, notional, status
|
||||
) values (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
row.filled_at.isoformat() if row.filled_at else None,
|
||||
row.symbol,
|
||||
row.side,
|
||||
row.quantity,
|
||||
row.filled_average_price,
|
||||
row.notional,
|
||||
row.status,
|
||||
)
|
||||
for row in snapshot.trades
|
||||
],
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
insert into metadata (key, value)
|
||||
values ('refreshed_at', ?)
|
||||
on conflict(key) do update set value = excluded.value
|
||||
""",
|
||||
(snapshot.refreshed_at.isoformat(),),
|
||||
)
|
||||
|
||||
|
||||
def read_snapshot(db_path: Path, warning: str | None = None) -> DashboardSnapshot:
|
||||
init_database(db_path)
|
||||
refreshed_at = get_last_refresh(db_path) or _utc_now()
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
performance = [
|
||||
PerformancePoint(
|
||||
trading_day=date.fromisoformat(row[0]),
|
||||
account_value=row[1],
|
||||
benchmark_value=row[2],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select trading_day, account_value, benchmark_value
|
||||
from performance_points
|
||||
order by trading_day
|
||||
"""
|
||||
)
|
||||
]
|
||||
portfolio = [
|
||||
PortfolioRow(
|
||||
symbol=row[0],
|
||||
asset_type=row[1],
|
||||
quantity=row[2],
|
||||
market_value=row[3],
|
||||
average_entry_price=row[4],
|
||||
current_price=row[5],
|
||||
unrealized_pl=row[6],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select symbol, asset_type, quantity, market_value,
|
||||
average_entry_price, current_price, unrealized_pl
|
||||
from portfolio_rows
|
||||
order by case when symbol = 'CASH' then 0 else 1 end, symbol
|
||||
"""
|
||||
)
|
||||
]
|
||||
trades = [
|
||||
TradeRow(
|
||||
filled_at=datetime.fromisoformat(row[0]) if row[0] else None,
|
||||
symbol=row[1],
|
||||
side=row[2],
|
||||
quantity=row[3],
|
||||
filled_average_price=row[4],
|
||||
notional=row[5],
|
||||
status=row[6],
|
||||
)
|
||||
for row in connection.execute(
|
||||
"""
|
||||
select filled_at, symbol, side, quantity,
|
||||
filled_average_price, notional, status
|
||||
from trade_rows
|
||||
order by filled_at desc
|
||||
"""
|
||||
)
|
||||
]
|
||||
return DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=performance,
|
||||
portfolio=portfolio,
|
||||
trades=trades,
|
||||
warning=warning,
|
||||
)
|
||||
|
||||
|
||||
def load_or_refresh_snapshot(
|
||||
db_path: Path,
|
||||
api_key: str | None,
|
||||
secret_key: str | None,
|
||||
paper: bool,
|
||||
refresh_interval: timedelta,
|
||||
) -> DashboardSnapshot:
|
||||
if not is_cache_stale(db_path, refresh_interval):
|
||||
return read_snapshot(db_path)
|
||||
|
||||
try:
|
||||
client = create_trading_client(api_key=api_key, secret_key=secret_key, paper=paper)
|
||||
snapshot = fetch_alpaca_snapshot(client, api_key, secret_key)
|
||||
write_snapshot(db_path, snapshot)
|
||||
return snapshot
|
||||
except Exception as exc:
|
||||
if db_path.exists() and get_last_refresh(db_path) is not None:
|
||||
return read_snapshot(db_path, warning=f"Refresh failed; showing cached data. {exc}")
|
||||
raise
|
||||
|
||||
|
||||
def _format_currency(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
sign = "-" if value < 0 else ""
|
||||
return f"{sign}${abs(value):,.2f}"
|
||||
|
||||
|
||||
def _format_number(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return f"{value:,.4f}"
|
||||
|
||||
|
||||
def _format_percent(value: float | None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return f"{value:.2%}"
|
||||
|
||||
|
||||
def _clean_enum_text(value: Any) -> str:
|
||||
text = str(value or "").strip()
|
||||
if "." in text:
|
||||
text = text.rsplit(".", maxsplit=1)[-1]
|
||||
return text.lower()
|
||||
|
||||
|
||||
def _pill_class(value: str) -> str:
|
||||
normalized = _clean_enum_text(value)
|
||||
if normalized in {"buy", "filled"}:
|
||||
return "pill pill-positive"
|
||||
if normalized in {"sell", "canceled", "cancelled", "failed", "expired", "rejected"}:
|
||||
return "pill pill-negative"
|
||||
return "pill"
|
||||
|
||||
|
||||
def _pill_html(value: str) -> str:
|
||||
label = _clean_enum_text(value).upper()
|
||||
return f"<span class=\"{_pill_class(label)}\">{escape(label)}</span>"
|
||||
|
||||
|
||||
def _performance_text(points: list[PerformancePoint]) -> str:
|
||||
if len(points) < 2:
|
||||
return "Performance unavailable"
|
||||
first = points[0]
|
||||
last = points[-1]
|
||||
change = last.account_value - first.account_value
|
||||
percent = change / first.account_value if first.account_value else 0.0
|
||||
days = (last.trading_day - first.trading_day).days
|
||||
sign = "+" if change >= 0 else ""
|
||||
return (
|
||||
f"{sign}{_format_currency(change)} ({sign}{percent:.2%}) "
|
||||
f"over {days} days"
|
||||
)
|
||||
|
||||
|
||||
def render_dashboard(snapshot: DashboardSnapshot) -> str:
|
||||
chart_data = [
|
||||
{
|
||||
"date": point.trading_day.isoformat(),
|
||||
"account": point.account_value,
|
||||
"benchmark": point.benchmark_value,
|
||||
}
|
||||
for point in snapshot.performance
|
||||
]
|
||||
chart_json = json.dumps(chart_data)
|
||||
warning = (
|
||||
f"<p class=\"warning\">{escape(snapshot.warning)}</p>"
|
||||
if snapshot.warning
|
||||
else ""
|
||||
)
|
||||
current_portfolio_value = sum(row.market_value for row in snapshot.portfolio)
|
||||
portfolio_rows = "\n".join(
|
||||
"<tr>"
|
||||
f"<td>{escape(row.symbol)}</td>"
|
||||
f"<td>{_format_number(row.quantity)}</td>"
|
||||
f"<td>{_format_currency(row.market_value)}</td>"
|
||||
f"<td>{_format_percent(row.market_value / current_portfolio_value if current_portfolio_value else None)}</td>"
|
||||
f"<td>{_format_currency(row.average_entry_price)}</td>"
|
||||
f"<td>{_format_currency(row.current_price)}</td>"
|
||||
f"<td>{_format_currency(row.unrealized_pl)}</td>"
|
||||
"</tr>"
|
||||
for row in snapshot.portfolio
|
||||
)
|
||||
trade_rows = "\n".join(
|
||||
"<tr>"
|
||||
f"<td>{escape(row.filled_at.isoformat(sep=' ', timespec='minutes') if row.filled_at else '')}</td>"
|
||||
f"<td>{escape(row.symbol)}</td>"
|
||||
f"<td>{_pill_html(row.side)}</td>"
|
||||
f"<td>{_format_number(row.quantity)}</td>"
|
||||
f"<td>{_format_currency(row.filled_average_price)}</td>"
|
||||
f"<td>{_format_currency(row.notional)}</td>"
|
||||
f"<td>{_pill_html(row.status)}</td>"
|
||||
"</tr>"
|
||||
for row in snapshot.trades
|
||||
)
|
||||
portfolio_rows = portfolio_rows or "<tr><td colspan=\"7\">No portfolio rows cached.</td></tr>"
|
||||
trade_rows = trade_rows or "<tr><td colspan=\"7\">No completed trades cached.</td></tr>"
|
||||
|
||||
return f"""<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Trading Bot Performance</title>
|
||||
<style>
|
||||
:root {{
|
||||
color-scheme: light;
|
||||
--bg: #f6f7f9;
|
||||
--panel: #ffffff;
|
||||
--text: #17202a;
|
||||
--muted: #5f6b7a;
|
||||
--line: #d8dee8;
|
||||
--account: #116466;
|
||||
--benchmark: #c75000;
|
||||
--warning: #7a4100;
|
||||
}}
|
||||
* {{ box-sizing: border-box; }}
|
||||
body {{
|
||||
margin: 0;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
line-height: 1.45;
|
||||
}}
|
||||
main {{
|
||||
width: min(1120px, calc(100% - 32px));
|
||||
margin: 0 auto;
|
||||
padding: 32px 0 48px;
|
||||
}}
|
||||
header {{
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
margin-bottom: 24px;
|
||||
}}
|
||||
h1, h2 {{ margin: 0; letter-spacing: 0; }}
|
||||
h1 {{ font-size: clamp(2rem, 4vw, 3.5rem); }}
|
||||
h2 {{ font-size: 1.15rem; margin-bottom: 12px; }}
|
||||
.summary {{
|
||||
font-size: 1.25rem;
|
||||
font-weight: 650;
|
||||
}}
|
||||
.meta, .legend, .warning {{
|
||||
color: var(--muted);
|
||||
margin: 0;
|
||||
}}
|
||||
.warning {{ color: var(--warning); }}
|
||||
section {{
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
padding: 18px;
|
||||
margin-top: 18px;
|
||||
}}
|
||||
.chart-wrap {{
|
||||
min-height: 420px;
|
||||
}}
|
||||
canvas {{
|
||||
width: 100%;
|
||||
height: 360px;
|
||||
display: block;
|
||||
}}
|
||||
.legend {{
|
||||
display: flex;
|
||||
gap: 18px;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 10px;
|
||||
font-size: 0.95rem;
|
||||
}}
|
||||
.legend span::before {{
|
||||
content: "";
|
||||
display: inline-block;
|
||||
width: 20px;
|
||||
height: 3px;
|
||||
margin-right: 8px;
|
||||
vertical-align: middle;
|
||||
background: var(--account);
|
||||
}}
|
||||
.legend span:last-child::before {{ background: var(--benchmark); }}
|
||||
.table-wrap {{ overflow-x: auto; }}
|
||||
table {{
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
min-width: 760px;
|
||||
}}
|
||||
th, td {{
|
||||
padding: 10px 12px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
text-align: right;
|
||||
white-space: nowrap;
|
||||
}}
|
||||
th:first-child, td:first-child,
|
||||
.trades th:nth-child(3), .trades td:nth-child(3),
|
||||
.trades th:nth-child(7), .trades td:nth-child(7) {{
|
||||
text-align: left;
|
||||
}}
|
||||
th {{
|
||||
color: var(--muted);
|
||||
font-size: 0.82rem;
|
||||
font-weight: 700;
|
||||
text-transform: uppercase;
|
||||
}}
|
||||
.pill {{
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-width: 58px;
|
||||
padding: 3px 9px;
|
||||
border-radius: 999px;
|
||||
background: #edf1f5;
|
||||
color: #3d4854;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 800;
|
||||
letter-spacing: 0;
|
||||
}}
|
||||
.pill-positive {{
|
||||
background: #dff3e8;
|
||||
color: #17643a;
|
||||
}}
|
||||
.pill-negative {{
|
||||
background: #ffe1df;
|
||||
color: #9a2219;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<header>
|
||||
<h1>Trading Bot Performance</h1>
|
||||
<div class="summary">{escape(_performance_text(snapshot.performance))}</div>
|
||||
<p class="meta">Last refreshed {escape(snapshot.refreshed_at.astimezone().isoformat(sep=' ', timespec='minutes'))}</p>
|
||||
{warning}
|
||||
</header>
|
||||
|
||||
<section class="chart-wrap">
|
||||
<h2>Account Value vs S&P 500 Proxy</h2>
|
||||
<canvas id="performanceChart" width="1040" height="360"></canvas>
|
||||
<p class="legend"><span>Account value</span><span>SPY normalized to account start value</span></p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Portfolio</h2>
|
||||
<div class="table-wrap">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Symbol</th><th>Quantity</th><th>Market Value</th><th>Account %</th>
|
||||
<th>Avg Entry</th><th>Current Price</th><th>Unrealized P/L</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>{portfolio_rows}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Completed Trades</h2>
|
||||
<div class="table-wrap">
|
||||
<table class="trades">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Filled At</th><th>Symbol</th><th>Side</th><th>Quantity</th>
|
||||
<th>Avg Fill</th><th>Notional</th><th>Status</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>{trade_rows}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const points = {chart_json};
|
||||
const canvas = document.getElementById("performanceChart");
|
||||
const ctx = canvas.getContext("2d");
|
||||
const css = getComputedStyle(document.documentElement);
|
||||
const accountColor = css.getPropertyValue("--account").trim();
|
||||
const benchmarkColor = css.getPropertyValue("--benchmark").trim();
|
||||
const gridColor = css.getPropertyValue("--line").trim();
|
||||
const textColor = css.getPropertyValue("--muted").trim();
|
||||
|
||||
function currency(value) {{
|
||||
return "$" + value.toLocaleString(undefined, {{ maximumFractionDigits: 0 }});
|
||||
}}
|
||||
|
||||
function drawLine(values, color, minValue, maxValue, left, top, width, height) {{
|
||||
const valid = values.filter((point) => point.value !== null && point.value !== undefined);
|
||||
if (valid.length < 2) return;
|
||||
ctx.beginPath();
|
||||
valid.forEach((point, index) => {{
|
||||
const x = left + (point.index / Math.max(points.length - 1, 1)) * width;
|
||||
const y = top + (1 - ((point.value - minValue) / Math.max(maxValue - minValue, 1))) * height;
|
||||
if (index === 0) ctx.moveTo(x, y);
|
||||
else ctx.lineTo(x, y);
|
||||
}});
|
||||
ctx.strokeStyle = color;
|
||||
ctx.lineWidth = 3;
|
||||
ctx.stroke();
|
||||
}}
|
||||
|
||||
function drawChart() {{
|
||||
const ratio = window.devicePixelRatio || 1;
|
||||
const rect = canvas.getBoundingClientRect();
|
||||
canvas.width = Math.max(Math.floor(rect.width * ratio), 320);
|
||||
canvas.height = Math.floor(360 * ratio);
|
||||
ctx.setTransform(ratio, 0, 0, ratio, 0, 0);
|
||||
ctx.clearRect(0, 0, rect.width, 360);
|
||||
|
||||
const left = 72, right = 18, top = 16, bottom = 46;
|
||||
const width = rect.width - left - right;
|
||||
const height = 360 - top - bottom;
|
||||
const series = points.flatMap((point) => [point.account, point.benchmark]).filter((value) => value !== null);
|
||||
if (series.length === 0) {{
|
||||
ctx.fillStyle = textColor;
|
||||
ctx.fillText("No performance history cached.", left, top + 24);
|
||||
return;
|
||||
}}
|
||||
const minValue = Math.min(...series) * 0.995;
|
||||
const maxValue = Math.max(...series) * 1.005;
|
||||
|
||||
ctx.font = "12px system-ui, sans-serif";
|
||||
ctx.strokeStyle = gridColor;
|
||||
ctx.fillStyle = textColor;
|
||||
for (let i = 0; i <= 4; i++) {{
|
||||
const y = top + (i / 4) * height;
|
||||
const value = maxValue - (i / 4) * (maxValue - minValue);
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(left, y);
|
||||
ctx.lineTo(left + width, y);
|
||||
ctx.stroke();
|
||||
ctx.fillText(currency(value), 6, y + 4);
|
||||
}}
|
||||
|
||||
drawLine(points.map((point, index) => ({{ index, value: point.account }})), accountColor, minValue, maxValue, left, top, width, height);
|
||||
drawLine(points.map((point, index) => ({{ index, value: point.benchmark }})), benchmarkColor, minValue, maxValue, left, top, width, height);
|
||||
|
||||
if (points.length > 0) {{
|
||||
ctx.fillStyle = textColor;
|
||||
ctx.fillText(points[0].date, left, 342);
|
||||
ctx.textAlign = "right";
|
||||
ctx.fillText(points[points.length - 1].date, left + width, 342);
|
||||
ctx.textAlign = "left";
|
||||
}}
|
||||
}}
|
||||
|
||||
drawChart();
|
||||
window.addEventListener("resize", drawChart);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
class DashboardRequestHandler(BaseHTTPRequestHandler):
|
||||
db_path: Path
|
||||
api_key: str | None
|
||||
secret_key: str | None
|
||||
paper: bool
|
||||
refresh_interval: timedelta
|
||||
|
||||
def do_GET(self) -> None:
|
||||
if self.path not in {"/", "/index.html"}:
|
||||
self.send_error(404)
|
||||
return
|
||||
try:
|
||||
snapshot = load_or_refresh_snapshot(
|
||||
db_path=self.db_path,
|
||||
api_key=self.api_key,
|
||||
secret_key=self.secret_key,
|
||||
paper=self.paper,
|
||||
refresh_interval=self.refresh_interval,
|
||||
)
|
||||
body = render_dashboard(snapshot).encode("utf-8")
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
except Exception as exc:
|
||||
body = f"Dashboard unavailable: {escape(str(exc))}".encode("utf-8")
|
||||
self.send_response(500)
|
||||
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, format: str, *args: Any) -> None:
|
||||
print(f"{self.address_string()} - {format % args}")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="Serve the read-only trading bot dashboard.")
|
||||
parser.add_argument("--host", default=DEFAULT_HOST, help=f"Host to bind. Defaults to {DEFAULT_HOST}.")
|
||||
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"Port to bind. Defaults to {DEFAULT_PORT}.")
|
||||
parser.add_argument("--db-path", type=Path, default=DEFAULT_DB_PATH, help=f"SQLite cache path. Defaults to {DEFAULT_DB_PATH}.")
|
||||
parser.add_argument("--refresh-hours", type=float, default=DEFAULT_REFRESH_HOURS, help="Refresh Alpaca data after this many hours. Defaults to 12.")
|
||||
parser.add_argument("--api-key", default=os.getenv("ALPACA_API_KEY"), help="Alpaca API Key ID.")
|
||||
parser.add_argument("--secret-key", default=os.getenv("ALPACA_SECRET_KEY"), help="Alpaca Secret Key.")
|
||||
parser.add_argument("--paper", action="store_true", help="Use Alpaca paper trading mode. This is the default.")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
handler = type(
|
||||
"ConfiguredDashboardRequestHandler",
|
||||
(DashboardRequestHandler,),
|
||||
{
|
||||
"db_path": args.db_path,
|
||||
"api_key": args.api_key,
|
||||
"secret_key": args.secret_key,
|
||||
"paper": True if args.paper else True,
|
||||
"refresh_interval": timedelta(hours=args.refresh_hours),
|
||||
},
|
||||
)
|
||||
server = ThreadingHTTPServer((args.host, args.port), handler)
|
||||
print(f"Serving read-only dashboard at http://{args.host}:{args.port}")
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopping dashboard server.")
|
||||
finally:
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,160 @@
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
|
||||
from trading_bot.data.alpaca_daily_lib import DailyCandle
|
||||
from trading_bot.ui.dashboard import (
|
||||
DashboardSnapshot,
|
||||
PerformancePoint,
|
||||
PortfolioRow,
|
||||
TradeRow,
|
||||
_benchmark_end_date,
|
||||
_from_unix_timestamp,
|
||||
_normalize_benchmark,
|
||||
is_cache_stale,
|
||||
read_snapshot,
|
||||
render_dashboard,
|
||||
write_snapshot,
|
||||
)
|
||||
|
||||
|
||||
def test_normalize_benchmark_scales_spy_to_account_start_value() -> None:
|
||||
candles = [
|
||||
DailyCandle(date(2026, 5, 1), 100.0, 100.0, 100.0, 100.0, 1_000),
|
||||
DailyCandle(date(2026, 5, 2), 110.0, 110.0, 110.0, 110.0, 1_000),
|
||||
]
|
||||
|
||||
values = _normalize_benchmark(
|
||||
candles,
|
||||
start_day=date(2026, 5, 1),
|
||||
initial_account_value=1_000.0,
|
||||
)
|
||||
|
||||
assert values[date(2026, 5, 1)] == 1_000.0
|
||||
assert values[date(2026, 5, 2)] == 1_100.0
|
||||
|
||||
|
||||
def test_portfolio_history_timestamps_are_interpreted_as_eastern_dates() -> None:
|
||||
timestamp = datetime(2026, 8, 11, 2, 30, tzinfo=UTC).timestamp()
|
||||
|
||||
assert _from_unix_timestamp(timestamp) == date(2026, 8, 10)
|
||||
|
||||
|
||||
def test_benchmark_end_date_does_not_go_past_completed_market_day(monkeypatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
"trading_bot.ui.dashboard.default_end_date",
|
||||
lambda: date(2026, 8, 10),
|
||||
)
|
||||
|
||||
assert _benchmark_end_date(date(2026, 8, 11)) == date(2026, 8, 10)
|
||||
assert _benchmark_end_date(date(2026, 8, 8)) == date(2026, 8, 8)
|
||||
|
||||
|
||||
def test_sqlite_snapshot_round_trip(tmp_path) -> None:
|
||||
db_path = tmp_path / "dashboard.sqlite"
|
||||
refreshed_at = datetime(2026, 8, 11, 10, 0, tzinfo=UTC)
|
||||
snapshot = DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 10), 10_000.0, 9_900.0),
|
||||
PerformancePoint(date(2026, 8, 11), 10_100.0, 10_000.0),
|
||||
],
|
||||
portfolio=[
|
||||
PortfolioRow("CASH", "cash", None, 500.0, None, None, None),
|
||||
PortfolioRow("SPY", "us_equity", 10.0, 5_000.0, 490.0, 500.0, 100.0),
|
||||
],
|
||||
trades=[
|
||||
TradeRow(
|
||||
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
|
||||
"SPY",
|
||||
"buy",
|
||||
1.0,
|
||||
500.0,
|
||||
500.0,
|
||||
"filled",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
write_snapshot(db_path, snapshot)
|
||||
loaded = read_snapshot(db_path)
|
||||
|
||||
assert loaded.refreshed_at == refreshed_at
|
||||
assert loaded.performance == snapshot.performance
|
||||
assert loaded.portfolio == snapshot.portfolio
|
||||
assert loaded.trades == snapshot.trades
|
||||
|
||||
|
||||
def test_cache_stale_after_refresh_interval(tmp_path) -> None:
|
||||
db_path = tmp_path / "dashboard.sqlite"
|
||||
refreshed_at = datetime(2026, 8, 11, 0, 0, tzinfo=UTC)
|
||||
snapshot = DashboardSnapshot(
|
||||
refreshed_at=refreshed_at,
|
||||
performance=[],
|
||||
portfolio=[],
|
||||
trades=[],
|
||||
)
|
||||
write_snapshot(db_path, snapshot)
|
||||
|
||||
assert not is_cache_stale(
|
||||
db_path,
|
||||
timedelta(hours=12),
|
||||
now=refreshed_at + timedelta(hours=11, minutes=59),
|
||||
)
|
||||
assert is_cache_stale(
|
||||
db_path,
|
||||
timedelta(hours=12),
|
||||
now=refreshed_at + timedelta(hours=12),
|
||||
)
|
||||
|
||||
|
||||
def test_render_dashboard_contains_requested_sections() -> None:
|
||||
html = render_dashboard(
|
||||
DashboardSnapshot(
|
||||
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 10), 10_000.0, 10_000.0),
|
||||
PerformancePoint(date(2026, 8, 11), 10_250.0, 10_100.0),
|
||||
],
|
||||
portfolio=[PortfolioRow("CASH", "cash", None, 250.0, None, None, None)],
|
||||
trades=[
|
||||
TradeRow(
|
||||
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
|
||||
"SPY",
|
||||
"orderside.sell",
|
||||
1.0,
|
||||
500.0,
|
||||
500.0,
|
||||
"orderstatus.filled",
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
assert "Account Value vs S&P 500 Proxy" in html
|
||||
assert "Portfolio" in html
|
||||
assert "Completed Trades" in html
|
||||
assert "+$250.00 (+2.50%)" in html
|
||||
assert "<th>Type</th>" not in html
|
||||
assert "<th>Account %</th>" in html
|
||||
assert "SELL</span>" in html
|
||||
assert "FILLED</span>" in html
|
||||
assert "orderside.sell" not in html
|
||||
assert "orderstatus.filled" not in html
|
||||
|
||||
|
||||
def test_portfolio_percentage_uses_current_portfolio_total() -> None:
|
||||
html = render_dashboard(
|
||||
DashboardSnapshot(
|
||||
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
|
||||
performance=[
|
||||
PerformancePoint(date(2026, 8, 11), 10_000.0, 10_000.0),
|
||||
],
|
||||
portfolio=[
|
||||
PortfolioRow("CASH", "cash", None, 100.0, None, None, None),
|
||||
PortfolioRow("SPY", "us_equity", 10.0, 10_000.0, 900.0, 1_000.0, 1_000.0),
|
||||
],
|
||||
trades=[],
|
||||
)
|
||||
)
|
||||
|
||||
assert "99.01%" in html
|
||||
assert "100.00%" not in html
|
||||
@@ -183,18 +183,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/05/7f/798705f5296a58ca505d600456748d1be48078eac8a7050d8a98bc9edb89/decorator-5.3.1-py3-none-any.whl", hash = "sha256:f47fe6fdbd2edd623ecfe36875d37aba411624e2670dd395dddae1358689bb3c", size = 10365, upload-time = "2026-05-18T06:03:26.517Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "eventkit"
|
||||
version = "1.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "numpy" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/16/1e/0fac4e45d71ace143a2673ec642701c3cd16f833a0e77a57fa6a40472696/eventkit-1.0.3.tar.gz", hash = "sha256:99497f6f3c638a50ff7616f2f8cd887b18bbff3765dc1bd8681554db1467c933", size = 28320, upload-time = "2023-12-11T11:41:35.339Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/93/d9/7497d650b69b420e1a913329a843e16c715dac883750679240ef00a921e2/eventkit-1.0.3-py3-none-any.whl", hash = "sha256:0e199527a89aff9d195b9671ad45d2cc9f79ecda0900de8ecfb4c864d67ad6a2", size = 31837, upload-time = "2023-12-11T11:41:33.358Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "executing"
|
||||
version = "2.2.1"
|
||||
@@ -221,19 +209,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/47/c99d5268f354002ce80f8d029cd9d7d872969da1de8b93d32de4dc56d6f4/fonttools-4.63.0-py3-none-any.whl", hash = "sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d", size = 1164562, upload-time = "2026-05-14T12:04:29.092Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ib-insync"
|
||||
version = "0.9.86"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "eventkit" },
|
||||
{ name = "nest-asyncio" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/55/bb/733d5c81c8c2f54e90898afc7ff3a99f4d53619e6917c848833f9cc1ab56/ib_insync-0.9.86.tar.gz", hash = "sha256:73af602ca2463f260999970c5bd937b1c4325e383686eff301743a4de08d381e", size = 69859, upload-time = "2023-07-02T12:43:31.968Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/f3/28ea87be30570f4d6b8fd24380d12fa74e59467ee003755e76aeb29082b8/ib_insync-0.9.86-py3-none-any.whl", hash = "sha256:a61fbe56ff405d93d211dad8238d7300de76dd6399eafc04c320470edec9a4a4", size = 72980, upload-time = "2023-07-02T12:43:29.928Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.18"
|
||||
@@ -459,15 +434,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/85/a5bfaebfd305ac18b57b0854d74e37e586809061a91fda62f0bd50c8518e/narwhals-2.24.0-py3-none-any.whl", hash = "sha256:42fdedf44e5b2ca7505630d45b4ac3058f38d8485cba9fe1652ca23152df7489", size = 461030, upload-time = "2026-07-13T10:49:17.571Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nest-asyncio"
|
||||
version = "1.6.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/83/f8/51569ac65d696c8ecbee95938f89d4abf00f47d58d48f6fbabfe8f0baefe/nest_asyncio-1.6.0.tar.gz", hash = "sha256:6f172d5449aca15afd6c646851f4e31e02c598d553a667e38cafa997cfec55fe", size = 7418, upload-time = "2024-01-21T14:25:19.227Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nest-asyncio2"
|
||||
version = "1.7.2"
|
||||
@@ -946,7 +912,6 @@ version = "0.1.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "alpaca-py" },
|
||||
{ name = "ib-insync" },
|
||||
{ name = "matplotlib" },
|
||||
{ name = "pandas" },
|
||||
{ name = "pyarrow" },
|
||||
@@ -964,7 +929,6 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "alpaca-py", specifier = ">=0.43.5" },
|
||||
{ name = "ib-insync", specifier = ">=0.9.86" },
|
||||
{ name = "matplotlib", specifier = ">=3.11.1" },
|
||||
{ name = "pandas", specifier = ">=2.3.0" },
|
||||
{ name = "pyarrow", specifier = ">=20.0.0" },
|
||||
|
||||
Reference in New Issue
Block a user