Compare commits

..
4 Commits
Author SHA1 Message Date
jarno 03a724f7ea Fixed a UI bug calculating equity percentage incorrectly. 2026-08-11 21:22:21 +03:00
jarno 1faca2cdda Added a simple ui 2026-08-11 21:20:07 +03:00
jarno c1c02e2b0d Documentation update 2026-08-11 20:23:59 +03:00
jarno 900b70d6df Removed IBKR related code 2026-08-11 20:13:41 +03:00
16 changed files with 1315 additions and 728 deletions
+1 -1
View File
@@ -2,7 +2,7 @@
## Project Purpose
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot will use the IBKR API to facilitate trades, starting with paper trading and allowing a carefully controlled path to real trading later.
This repository is for a Python trading bot that uses machine learning to decide which assets should be held. The bot uses the Alpaca Markets API for market data and paper trading, with any future live trading behind a carefully controlled path.
## Current Direction
+47 -7
View File
@@ -1,10 +1,10 @@
# Trading Bot
Python trading bot project using machine learning to decide which assets should be held. The bot is intended to trade through the IBKR API, beginning with paper trading and potentially supporting real trades later behind explicit safeguards.
Python trading bot project using machine learning to decide which assets should be held. The bot currently uses the Alpaca Markets API for market data and paper trading, with any future live trading kept behind explicit safeguards.
## Project Status
This repository is at the planning and scaffolding stage. The machine learning model, trading parameters, asset universe, and risk rules will be designed later.
This repository now has a working prototype flow for Alpaca market data, dataset generation, XGBoost training, prediction, and paper-trading rebalancing. The strategy is still intentionally narrow: it currently focuses on `SPY` exposure using a small market-regime feature set.
## Main Parts
@@ -29,22 +29,62 @@ The current local Python version is pinned in `.mise.toml`.
Runtime dependencies are declared in `pyproject.toml`.
- `ib-insync` for the IBKR API connection.
- `alpaca-py` for Alpaca market data and trading clients.
- `pandas` for tabular candle data handling.
- `pyarrow` for Parquet file support.
- `xgboost` and `scikit-learn` for model training and evaluation.
- `python-dotenv` for loading local Alpaca credentials from `.env`.
## Trading Safety
The default target is paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
The default target is Alpaca paper trading. Real trading should only be added later with explicit configuration, clear documentation, and tests around order generation and broker integration.
Do not commit secrets such as IBKR credentials, account identifiers, API tokens, or private configuration.
Do not commit secrets such as Alpaca API keys, account identifiers, API tokens, or private configuration.
## Current Commands
Fetch Alpaca daily candles:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
```
Build the training dataset and train the current model:
```sh
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
```
Run the current Alpaca paper-trading rebalance flow:
```sh
mise exec -- uv run python src/trading_bot/models/trade.py \
--paper \
--fetch-recent-data \
--model-path models/spy_xgb_v1.json \
--metadata-path models/spy_xgb_v1_meta.json
```
The main Python entry points are:
- `src/trading_bot/data/train_pipeline.py` for dataset generation and model training.
- `src/trading_bot/models/trade.py` for Alpaca account inspection and SPY rebalancing.
- `src/trading_bot/ui/dashboard.py` for the read-only performance dashboard.
Serve the read-only dashboard:
```sh
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
```
## Documentation
See [docs/README.md](docs/README.md) for the initial architecture notes and decision log.
The first data collection design note is [docs/data-fetcher.md](docs/data-fetcher.md), and the first training dataset contract is [docs/training-dataset.md](docs/training-dataset.md).
The current data collection note is [docs/data-fetcher.md](docs/data-fetcher.md), and the training dataset contract is [docs/training-dataset.md](docs/training-dataset.md).
The read-only dashboard design is documented in [docs/read-only-ui.md](docs/read-only-ui.md).
The first dataset generation notebook is [notebooks/spy_direction_dataset.ipynb](notebooks/spy_direction_dataset.ipynb).
Manual test instructions for the IBKR fetcher skeleton are in [docs/manual-test/README.md](docs/manual-test/README.md).
Manual test instructions for the Alpaca fetcher are in [docs/manual-test/README.md](docs/manual-test/README.md).
+23 -17
View File
@@ -2,7 +2,7 @@
## Objective
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses the IBKR API to facilitate trades. The first supported trading mode should be paper trading.
Build a Python-based trading bot that uses machine learning to determine which assets should be held, then uses Alpaca Markets for market data and broker execution. The first supported trading mode is paper trading.
## Planned Modules
@@ -10,9 +10,9 @@ Build a Python-based trading bot that uses machine learning to determine which a
Responsible for acquiring and storing market, asset, and any future feature data needed for model training and evaluation.
The first planned tool is an IBKR daily candle fetcher. It should fetch open, high, low, close, and volume data for a ticker and date range, then eventually persist that data to a ticker-named Parquet file. See [data-fetcher.md](data-fetcher.md).
The current data tool is an Alpaca daily candle fetcher. It fetches open, high, low, close, and volume data for a ticker and date range, then persists that data to a ticker-named Parquet file. See [data-fetcher.md](data-fetcher.md).
The initial supervised training dataset is documented in [training-dataset.md](training-dataset.md). It derives market-regime features from `SPY`, `VIX`, `TLT`, and `USO`, then labels whether `SPY` closes higher five trading days later.
The initial supervised training dataset is documented in [training-dataset.md](training-dataset.md). It derives market-regime features from `SPY`, a volatility proxy, `TLT`, and `USO`, then labels whether `SPY` closes higher five trading days later. The checked-in configuration currently maps the volatility input to `VIXY` for Alpaca data availability.
Parquet files are partitioned by ticker, not by date.
@@ -29,6 +29,8 @@ Open decisions:
Responsible for building datasets, training models, evaluating candidates, and writing versioned model artifacts.
The main entry point is `src/trading_bot/data/train_pipeline.py`. It reads raw Alpaca Parquet files from `data/alpaca/daily`, builds `data/training/spy_direction_5d.parquet`, trains an XGBoost classifier, and writes model artifacts to `models/`.
Open decisions:
- prediction target;
@@ -41,11 +43,13 @@ Open decisions:
### Trading Bot
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the IBKR API to place or simulate orders.
Responsible for loading a model, generating portfolio signals, deciding target holdings, and using the Alpaca trading API to place paper-trading orders.
The main entry point is `src/trading_bot/models/trade.py`. It loads a model and metadata, optionally refreshes recent Alpaca market data, estimates a target `SPY` exposure from the model probability, cancels open Alpaca orders, and submits a day market order when the desired rebalance exceeds the configured minimum notional amount. Pass `--model-path models/spy_xgb_v1.json --metadata-path models/spy_xgb_v1_meta.json` to trade with artifacts produced by the current training pipeline.
Initial expectations:
- paper trading first;
- Alpaca paper trading first;
- real trading later only behind explicit configuration;
- clear logging of model version, signals, target holdings, generated orders, and broker responses;
- separation between signal generation, portfolio construction, and broker execution.
@@ -56,32 +60,34 @@ Open decisions:
- position sizing;
- risk limits;
- cash handling;
- order types;
- order types and time-in-force choices;
- failed order handling;
- market hours behavior;
- manual override behavior.
### Read-Only Web UI
Optional future module for checking status without controlling trading behavior.
Optional module for checking status without controlling trading behavior.
Possible scope:
The initial implementation is documented in [read-only-ui.md](read-only-ui.md).
It serves one backend-rendered page, reads Alpaca account state with `alpaca-py`,
and caches snapshots in SQLite so normal page loads do not repeatedly call Alpaca.
- current holdings;
- latest signals;
- recent orders;
- account summary;
- model version;
- bot health and logs.
Current scope:
- three-month account performance summary;
- account value chart with a `SPY` S&P 500 proxy comparison;
- cash and equity portfolio table;
- completed trades table.
## Near-Term Priorities
1. Decide the initial project package structure.
2. Keep Python packaging and dependency management current with `uv`.
3. Add a minimal configuration system.
4. Define interfaces for data collection, model artifacts, and broker execution.
5. Add tests for the core trading decision boundaries before connecting real broker behavior.
4. Harden interfaces for data collection, model artifacts, and Alpaca broker execution.
5. Expand tests around trading decision boundaries, portfolio sizing, stale data handling, and broker API boundaries.
## Decisions Deferred
The model, features, labels, asset selection rules, risk management rules, and trading cadence are intentionally deferred for later discussion.
Broader model design, asset selection rules, risk management rules, live-trading gates, and trading cadence are intentionally deferred for later discussion.
+11 -12
View File
@@ -2,7 +2,7 @@
## Initial Goal
Create a Python module/tool that fetches daily candlestick data from the IBKR API for a specific ticker and date range.
Create a Python module/tool that fetches daily candlestick data from the Alpaca Market Data API for a specific ticker and date range.
The intended example workflow is:
@@ -14,41 +14,40 @@ The intended example workflow is:
## Current Implementation
The first implementation is intentionally small:
The current implementation is intentionally small:
- ticker passed as a required command line argument;
- end date passed with `--end-date YYYYMMDD`, defaulting to yesterday;
- end date can also be derived with `--end-date-from-parquet`;
- duration passed with `--duration`, defaulting to `1 W`;
- hard-coded IBKR Gateway target: `127.0.0.1:4002`;
- uses the IBKR API through `ib_insync`;
- prints fetched candles as CSV-like rows;
- uses the Alpaca Market Data API through `alpaca-py`;
- reads credentials from `--api-key` / `--secret-key`, `ALPACA_API_KEY` / `ALPACA_SECRET_KEY`, or `.env`;
- writes candles to a symbol-named Parquet file;
- appends to an existing symbol file and keeps one row per date.
Run it with:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
```
To override the requested range:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
```
To fetch backward from the oldest date already stored in the symbol file:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
```
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
This expects a local IBKR Gateway session to be running and accepting API connections on `127.0.0.1:4002`.
This expects Alpaca API credentials to be available via CLI arguments, environment variables, or `.env`.
By default, output is written to `data/ibkr/daily/SPY.parquet`. Use `--output-dir` to choose another directory.
By default, output is written to `data/alpaca/daily/SPY.parquet`. Use `--output-dir` to choose another directory.
Manual test instructions are in [manual-test/README.md](manual-test/README.md).
@@ -78,5 +77,5 @@ Candidate output schema:
Open decisions:
- how to handle adjusted versus unadjusted prices;
- how to handle missing sessions and IBKR pacing limits;
- whether to use `ib_insync` long term or a lower-level IBKR client wrapper.
- how to handle missing sessions, market holidays, and Alpaca API rate limits;
- whether to add explicit feed selection, adjustment settings, or data entitlement checks.
+23 -30
View File
@@ -1,30 +1,24 @@
# Manual Test Instructions
## IBKR Daily Fetcher
## Alpaca Daily Fetcher
This test checks the IBKR data fetcher and confirms it writes a symbol-named Parquet file.
This test checks the Alpaca data fetcher and confirms it writes a symbol-named Parquet file.
The fetcher currently requests:
- gateway: `127.0.0.1:4002`;
- client id: `101`;
- symbol: provided as a command line argument;
- end date: provided with `--end-date YYYYMMDD`, defaulting to yesterday;
- end date can also be derived with `--end-date-from-parquet`;
- duration: provided with `--duration`, defaulting to `1 W`;
- bar size: `1 day`;
- data type: `TRADES`;
- regular trading hours only;
- output: printed CSV-like rows and a Parquet file in `data/ibkr/daily/`.
- data source: Alpaca stock bars through `alpaca-py`;
- output: a Parquet file in `data/alpaca/daily/`.
## Prerequisites
1. Start IBKR Gateway.
2. Log in to the paper trading account.
3. Confirm API access is enabled in IBKR Gateway.
4. Confirm the API socket port is `4002`.
5. Confirm no other API client is already using client id `101`.
6. Sync Python dependencies:
1. Create or confirm Alpaca API credentials.
2. Provide credentials through `ALPACA_API_KEY` and `ALPACA_SECRET_KEY`, a local `.env`, or CLI arguments.
3. Sync Python dependencies:
```sh
mise exec -- uv sync
@@ -35,52 +29,51 @@ mise exec -- uv sync
From the repository root, run:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY
```
To fetch a specific IBKR range, pass an end date and duration:
To fetch a specific Alpaca range, pass an end date and duration:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date 20250605 --duration "1 M"
```
To fetch backward from the oldest date already stored in `data/ibkr/daily/SPY.parquet`:
To fetch backward from the oldest date already stored in `data/alpaca/daily/SPY.parquet`:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
mise exec -- uv run python src/trading_bot/data/fetch_alpaca_daily.py SPY --end-date-from-parquet
```
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
## Expected Output
The tool should first print the request range and connection target:
The tool should first print the request range:
```text
Fetching SPY daily candles ending 2025-06-05 for duration 1 M
Connecting to IBKR Gateway at 127.0.0.1:4002 with client id 101
Fetching SPY daily candles ending 2025-06-05 for duration 1 M via Alpaca API
```
If the request succeeds, it should then print a header and one row per returned trading day:
If the request succeeds, it should print how many candles were fetched and where the merged Parquet file was written:
```text
date,symbol,open,high,low,close,volume
2025-06-05,SPY,...
Fetched 21 daily candles for SPY.
Wrote 21 total daily rows to data/alpaca/daily/SPY.parquet
```
Exact prices and volume depend on what IBKR returns.
Exact row counts depend on the requested date range and market calendar.
The tool should then write or update:
```text
data/ibkr/daily/SPY.parquet
data/alpaca/daily/SPY.parquet
```
If the Parquet file already exists, rows from the latest fetch are merged into it. The trading date is used as the row key, so a symbol file keeps only one row for each date.
## Common Issues
- Connection refused: IBKR Gateway is not running, the port is not `4002`, or API access is disabled.
- Client id already in use: change `IBKR_CLIENT_ID` in the fetcher or disconnect the other client.
- No historical bars: confirm the account has market data permissions and that IBKR accepts the requested historical data range.
- Pacing or permission errors: note the IBKR error message before changing the request.
- Missing credentials: set `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` or pass them with CLI flags.
- No historical bars: confirm the symbol, requested range, and Alpaca market data permissions.
- Authentication or entitlement errors: confirm the keys belong to the intended Alpaca account and data plan.
- Rate-limit errors: wait before retrying or reduce repeated requests.
+39
View File
@@ -0,0 +1,39 @@
# Read-Only Performance UI
## Goal
Provide a single read-only page for checking Alpaca paper account performance and recent trading activity without exposing any trading controls.
## Requested Feature List
- Show the last three months of account performance in the header as text, or a shorter range when less data is available.
- Show account value over time as the main body chart.
- Include an S&P 500 comparison line in the chart. The initial implementation uses `SPY` daily candles as the S&P 500 proxy because Alpaca stock market data exposes it through the same API path.
- Show a portfolio table below the chart with cash plus equity positions.
- Show a completed trades table at the end of the page.
- Use Python and `alpaca-py` in the backend.
- Serve the page directly from the backend as a single page with no user options.
- Cache fetched data in SQLite and refresh periodically, initially every 12 hours.
## Design Notes
The UI is intentionally separate from the trading and training modules. It reads account state from Alpaca, stores normalized snapshots in SQLite, and renders HTML from the backend. It does not place, replace, or cancel orders.
The first cache path is `data/ui/performance.sqlite`. On each page request, the server refreshes data only when the cache is older than the configured refresh interval. If Alpaca refresh fails but cached data exists, the page can still render stale data with a warning.
## Backend Data
The dashboard stores:
- account equity history for up to three months;
- normalized S&P 500 proxy values from `SPY`;
- the latest cash and equity holdings snapshot;
- recently closed orders that have a filled quantity.
## Run Command
```sh
mise exec -- uv run python -m trading_bot.ui.dashboard --paper
```
The server defaults to `127.0.0.1:8000` and reads `ALPACA_API_KEY` and `ALPACA_SECRET_KEY` from the environment or local `.env`.
+23 -8
View File
@@ -2,16 +2,16 @@
## Initial Goal
Build a supervised learning dataset from stored IBKR daily candle data. The first dataset predicts whether `SPY` closes higher five trading days after the observation date.
Build a supervised learning dataset from stored Alpaca daily candle data. The first dataset predicts whether `SPY` closes higher five trading days after the observation date.
The first implementation is [../notebooks/spy_direction_dataset.ipynb](../notebooks/spy_direction_dataset.ipynb), a Python notebook using pandas. This keeps the feature calculations inspectable while the dataset design is still changing. Once the feature contract settles, reusable loading and feature-building code can move into `src/trading_bot/models` or `src/trading_bot/data`.
The current implementation is `src/trading_bot/data/train_pipeline.py`, which combines the original dataset notebook and XGBoost training notebook into one runnable script. The earlier notebook, [../notebooks/spy_direction_dataset.ipynb](../notebooks/spy_direction_dataset.ipynb), remains useful background for inspecting the initial feature design.
## Raw Data
Raw daily candles are stored under:
```text
data/ibkr/daily
data/alpaca/daily
```
Storage expectations:
@@ -21,10 +21,10 @@ Storage expectations:
- each row contains one daily candle for that symbol;
- expected columns are `date`, `symbol`, `open`, `high`, `low`, `close`, and `volume`.
The initial dataset requires at least these symbols:
The initial dataset requires at least these logical inputs:
- `SPY`;
- `VIX`;
- volatility proxy, currently `VIXY` in `config/train_config.json`;
- `TLT`;
- `USO`.
@@ -57,7 +57,7 @@ The target column is a binary indicator of `SPY` forward return over the next fi
| --- | --- |
| `spy_up_5d` | `1.0` when `SPY` closes above today's close five trading days later; `0.0` when `SPY` closes below today's close five trading days later; `0.5` when the future close equals today's close. |
Using `0.5` for unchanged prices preserves the row while making the target explicitly neutral.
The original dataset design used `0.5` for unchanged prices as an explicit neutral target. The current training pipeline drops unchanged `0.5` rows before training so the XGBoost model remains a binary classifier.
## Data Alignment
@@ -109,9 +109,24 @@ Training, validation, and test splits should be chronological:
A split indicator column such as `split` is useful in the dataset artifact for auditability and reproducibility. It should be treated as metadata, not as a model input feature. The model training code should build `X` from the explicit feature column list and exclude metadata columns such as `date`, `split`, raw close prices, and the target.
## Training Pipeline
Run the current pipeline from the repository root:
```sh
mise exec -- uv run python src/trading_bot/data/train_pipeline.py
```
The script reads configuration from `config/train_config.json`, writes the dataset to `data/training/spy_direction_5d.parquet`, and saves model artifacts to:
- `models/spy_xgb_v1.json`;
- `models/spy_xgb_v1_meta.json`.
The metadata file stores the training base probability, feature column list, last trained date, and model configuration.
## Open Decisions
- exact adjusted versus unadjusted close handling;
- whether same-day `VIX`, `TLT`, and `USO` values are acceptable for the intended trading decision timing;
- whether same-day volatility proxy, `TLT`, and `USO` values are acceptable for the intended trading decision timing;
- exact train, validation, and test date boundaries or split percentages;
- output dataset file location, schema metadata, and versioning.
- output dataset schema metadata and versioning.
-228
View File
@@ -1,228 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "cd849b8e",
"metadata": {},
"source": [
"# IBKR scratch notebook\n",
"\n",
"This notebook is for quick manual testing of the IBKR gateway connection, portfolio lookup, and a simple SPY order flow.\n",
"\n",
"> Use this only with a paper-trading or test setup unless you explicitly intend to submit a live order.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "543426a2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Connecting to IBKR Gateway at 127.0.0.1:4002 with client id 101...\n",
"Failed to connect to IBKR Gateway: This event loop is already running\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Error 200, reqId 9: No security definition has been found for the request, contract: Stock(symbol='VUAA', exchange='SMART', currency='USD')\n",
"Error 200, reqId 10: No security definition has been found for the request\n",
"Canceled order: Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0), TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 743208, tzinfo=datetime.timezone.utc), status='Cancelled', message='Error 200, reqId 10: No security definition has been found for the request', errorCode=200)], advancedError='')\n",
"Peer closed connection.\n"
]
}
],
"source": [
"import sys\n",
"from pathlib import Path\n",
"\n",
"from ib_insync import IB, Stock\n",
"\n",
"# Allow the notebook to import local source code from the repository.\n",
"repo_root = Path.cwd().resolve()\n",
"if (repo_root / \"src\").exists():\n",
" repo_root = repo_root\n",
"else:\n",
" repo_root = repo_root.parent\n",
"\n",
"src_path = repo_root / \"src\"\n",
"if str(src_path) not in sys.path:\n",
" sys.path.insert(0, str(src_path))\n",
"\n",
"IBKR_HOST = \"127.0.0.1\"\n",
"IBKR_PORT = 4002\n",
"IBKR_CLIENT_ID = 101\n",
"IBKR_CONNECT_TIMEOUT_SECONDS = 10\n",
"\n",
"ib = IB()\n",
"print(f\"Connecting to IBKR Gateway at {IBKR_HOST}:{IBKR_PORT} with client id {IBKR_CLIENT_ID}...\")\n",
"try:\n",
" ib.connect(IBKR_HOST, IBKR_PORT, clientId=IBKR_CLIENT_ID, timeout=IBKR_CONNECT_TIMEOUT_SECONDS)\n",
" print(\"Connection established.\")\n",
"except Exception as e:\n",
" print(f\"Failed to connect to IBKR Gateway: {e}\")\n",
" ib.disconnect()\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "ecdc721f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Connected: True\n",
"Client ID: 101\n"
]
}
],
"source": [
"print(f\"Connected: {ib.isConnected()}\")\n",
"print(f\"Client ID: {ib.client.clientId}\")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "10ebc240",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"No open portfolio positions were returned.\n"
]
}
],
"source": [
"portfolio = ib.portfolio()\n",
"if not portfolio:\n",
" print(\"No open portfolio positions were returned.\")\n",
"else:\n",
" for item in portfolio:\n",
" print(\n",
" f\"{item.contract.symbol}: position={item.position}, \"\n",
" f\"market_value={item.marketValue}, unrealized_pnl={item.unrealizedPNL}\"\n",
" )\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "fed654ec",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Unknown contract: Stock(symbol='VUAA', exchange='SMART', currency='USD')\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Submitted order for VUAA: Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='PendingSubmit', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0)], advancedError='')\n"
]
}
],
"source": [
"from ib_insync import MarketOrder\n",
"\n",
"contract = Stock(\"VUAA\", \"SMART\", \"USD\")\n",
"await ib.qualifyContractsAsync(contract)\n",
"\n",
"# Adjust the quantity as needed before running this cell.\n",
"order = MarketOrder(\"BUY\", 0.70)\n",
"trade = ib.placeOrder(contract, order)\n",
"\n",
"print(f\"Submitted order for {contract.symbol}: {trade}\")\n"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "06881101",
"metadata": {},
"outputs": [],
"source": [
"trds = ib.trades()"
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "08663602",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['Cancelled', 'Cancelled', 'Cancelled']"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(map(lambda x: x.orderStatus.status, trds))"
]
},
{
"cell_type": "code",
"execution_count": 43,
"id": "2c7abf23",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Trade(contract=Stock(conId=756733, symbol='SPY', right='?', exchange='SMART', currency='USD', localSymbol='SPY', tradingClass='SPY'), order=Order(permId=1578393268, action='BUY', totalQuantity=1.0, orderType='MKT', lmtPrice=0.0, auxPrice=0.0, tif='DAY', ocaType=3, displaySize=2147483647, rule80A='0', openClose='', volatilityType=0, deltaNeutralOrderType='None', referencePriceType=0, account='DUR281921', clearingIntent='IB', cashQty=0.0, dontUseAutoPriceForHedge=True, filledQuantity=0.0, refFuturesConId=2147483647, shareholder='Not an insider or substantial shareholder'), orderStatus=OrderStatus(orderId=0, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[], advancedError=''),\n",
" Trade(contract=Stock(conId=756733, symbol='SPY', right='?', exchange='SMART', currency='USD', localSymbol='SPY', tradingClass='SPY'), order=Order(permId=24474350, action='BUY', totalQuantity=0.1345, orderType='MKT', lmtPrice=0.0, auxPrice=0.0, tif='DAY', ocaType=3, displaySize=2147483647, rule80A='0', openClose='', volatilityType=0, deltaNeutralOrderType='None', referencePriceType=0, account='DUR281921', clearingIntent='IB', cashQty=0.0, dontUseAutoPriceForHedge=True, filledQuantity=0.0, refFuturesConId=2147483647, shareholder='Not an insider or substantial shareholder'), orderStatus=OrderStatus(orderId=0, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[], advancedError=''),\n",
" Trade(contract=Stock(symbol='VUAA', exchange='SMART', currency='USD'), order=MarketOrder(orderId=10, clientId=101, action='BUY', totalQuantity=0.7), orderStatus=OrderStatus(orderId=10, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 539380, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0), TradeLogEntry(time=datetime.datetime(2026, 7, 30, 19, 36, 57, 743208, tzinfo=datetime.timezone.utc), status='Cancelled', message='Error 200, reqId 10: No security definition has been found for the request', errorCode=200)], advancedError='')]"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trds"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.15"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
+93 -93
View File
@@ -87,16 +87,16 @@
"output_type": "stream",
"text": [
"<class 'pandas.DataFrame'>\n",
"DatetimeIndex: 1255 entries, 2021-08-02 to 2026-07-31\n",
"DatetimeIndex: 1258 entries, 2021-08-02 to 2026-08-05\n",
"Data columns (total 4 columns):\n",
" # Column Non-Null Count Dtype \n",
"--- ------ -------------- ----- \n",
" 0 SPY_close 1255 non-null float64\n",
" 1 VIX_close 1255 non-null float64\n",
" 2 TLT_close 1255 non-null float64\n",
" 3 USO_close 1255 non-null float64\n",
" 0 SPY_close 1258 non-null float64\n",
" 1 VIX_close 1258 non-null float64\n",
" 2 TLT_close 1258 non-null float64\n",
" 3 USO_close 1258 non-null float64\n",
"dtypes: float64(4)\n",
"memory usage: 49.0 KB\n"
"memory usage: 49.1 KB\n"
]
},
{
@@ -234,7 +234,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 3,
"metadata": {},
"outputs": [
{
@@ -364,7 +364,7 @@
"2021-10-15 -0.002202 0.030287 -0.003823 1.0 "
]
},
"execution_count": 4,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
@@ -418,7 +418,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -458,24 +458,24 @@
" <tbody>\n",
" <tr>\n",
" <th>train</th>\n",
" <td>840</td>\n",
" <td>842</td>\n",
" <td>2021-10-11</td>\n",
" <td>2025-02-13</td>\n",
" <td>0.583333</td>\n",
" <td>2025-02-18</td>\n",
" <td>0.581948</td>\n",
" </tr>\n",
" <tr>\n",
" <th>validation</th>\n",
" <td>180</td>\n",
" <td>2025-02-14</td>\n",
" <td>2025-10-31</td>\n",
" <td>0.633333</td>\n",
" <td>2025-02-19</td>\n",
" <td>2025-11-04</td>\n",
" <td>0.638889</td>\n",
" </tr>\n",
" <tr>\n",
" <th>test</th>\n",
" <td>181</td>\n",
" <td>2025-11-03</td>\n",
" <td>2026-07-24</td>\n",
" <td>0.569061</td>\n",
" <td>182</td>\n",
" <td>2025-11-05</td>\n",
" <td>2026-07-29</td>\n",
" <td>0.576923</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
@@ -484,12 +484,12 @@
"text/plain": [
" rows start_date end_date target_mean\n",
"split \n",
"train 840 2021-10-11 2025-02-13 0.583333\n",
"validation 180 2025-02-14 2025-10-31 0.633333\n",
"test 181 2025-11-03 2026-07-24 0.569061"
"train 842 2021-10-11 2025-02-18 0.581948\n",
"validation 180 2025-02-19 2025-11-04 0.638889\n",
"test 182 2025-11-05 2026-07-29 0.576923"
]
},
"execution_count": 5,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@@ -543,17 +543,17 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Rows: 1,201\n",
"Rows: 1,204\n",
"Feature columns: ['SPY_ret_5', 'SPY_ret_20', 'SPY_dist_sma50', 'VIX_change_5', 'VIX_rank_20', 'TLT_ret_10', 'USO_ret_5', 'SPY_TLT_ratio_ret']\n",
"Target column: spy_up_5d\n",
"Training matrix shape: (840, 8)\n"
"Training matrix shape: (842, 8)\n"
]
},
{
@@ -592,16 +592,16 @@
" <tbody>\n",
" <tr>\n",
" <th>count</th>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201.000000</td>\n",
" <td>1201</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204.000000</td>\n",
" <td>1204</td>\n",
" </tr>\n",
" <tr>\n",
" <th>unique</th>\n",
@@ -640,32 +640,32 @@
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>840</td>\n",
" <td>842</td>\n",
" </tr>\n",
" <tr>\n",
" <th>mean</th>\n",
" <td>0.002502</td>\n",
" <td>0.009848</td>\n",
" <td>0.011028</td>\n",
" <td>0.017763</td>\n",
" <td>0.403476</td>\n",
" <td>-0.004099</td>\n",
" <td>0.005024</td>\n",
" <td>0.004880</td>\n",
" <td>0.588676</td>\n",
" <td>0.002464</td>\n",
" <td>0.009816</td>\n",
" <td>0.010973</td>\n",
" <td>0.017820</td>\n",
" <td>0.404506</td>\n",
" <td>-0.004103</td>\n",
" <td>0.004938</td>\n",
" <td>0.004838</td>\n",
" <td>0.589701</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>std</th>\n",
" <td>0.023057</td>\n",
" <td>0.043558</td>\n",
" <td>0.037315</td>\n",
" <td>0.299417</td>\n",
" <td>0.337501</td>\n",
" <td>0.028502</td>\n",
" <td>0.052628</td>\n",
" <td>0.027606</td>\n",
" <td>0.492279</td>\n",
" <td>0.023044</td>\n",
" <td>0.043515</td>\n",
" <td>0.037287</td>\n",
" <td>0.299050</td>\n",
" <td>0.337773</td>\n",
" <td>0.028469</td>\n",
" <td>0.052606</td>\n",
" <td>0.027586</td>\n",
" <td>0.492092</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
@@ -683,40 +683,40 @@
" </tr>\n",
" <tr>\n",
" <th>25%</th>\n",
" <td>-0.009601</td>\n",
" <td>-0.016509</td>\n",
" <td>-0.008656</td>\n",
" <td>-0.056615</td>\n",
" <td>-0.009749</td>\n",
" <td>-0.016557</td>\n",
" <td>-0.008704</td>\n",
" <td>-0.056606</td>\n",
" <td>0.100000</td>\n",
" <td>-0.023304</td>\n",
" <td>-0.025866</td>\n",
" <td>-0.010013</td>\n",
" <td>-0.023139</td>\n",
" <td>-0.025950</td>\n",
" <td>-0.010029</td>\n",
" <td>0.000000</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>50%</th>\n",
" <td>0.003798</td>\n",
" <td>0.015725</td>\n",
" <td>0.017802</td>\n",
" <td>-0.018570</td>\n",
" <td>0.003749</td>\n",
" <td>0.015639</td>\n",
" <td>0.017755</td>\n",
" <td>-0.018430</td>\n",
" <td>0.300000</td>\n",
" <td>-0.004435</td>\n",
" <td>0.004950</td>\n",
" <td>0.005537</td>\n",
" <td>-0.004429</td>\n",
" <td>0.004869</td>\n",
" <td>0.005508</td>\n",
" <td>1.000000</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75%</th>\n",
" <td>0.015955</td>\n",
" <td>0.038179</td>\n",
" <td>0.038282</td>\n",
" <td>0.030814</td>\n",
" <td>0.015950</td>\n",
" <td>0.038167</td>\n",
" <td>0.038124</td>\n",
" <td>0.030815</td>\n",
" <td>0.750000</td>\n",
" <td>0.014227</td>\n",
" <td>0.032486</td>\n",
" <td>0.021390</td>\n",
" <td>0.014188</td>\n",
" <td>0.032295</td>\n",
" <td>0.021358</td>\n",
" <td>1.000000</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
@@ -739,33 +739,33 @@
],
"text/plain": [
" SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 VIX_rank_20 \\\n",
"count 1201.000000 1201.000000 1201.000000 1201.000000 1201.000000 \n",
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204.000000 \n",
"unique NaN NaN NaN NaN NaN \n",
"top NaN NaN NaN NaN NaN \n",
"freq NaN NaN NaN NaN NaN \n",
"mean 0.002502 0.009848 0.011028 0.017763 0.403476 \n",
"std 0.023057 0.043558 0.037315 0.299417 0.337501 \n",
"mean 0.002464 0.009816 0.010973 0.017820 0.404506 \n",
"std 0.023044 0.043515 0.037287 0.299050 0.337773 \n",
"min -0.114962 -0.123975 -0.141995 -0.387709 0.050000 \n",
"25% -0.009601 -0.016509 -0.008656 -0.056615 0.100000 \n",
"50% 0.003798 0.015725 0.017802 -0.018570 0.300000 \n",
"75% 0.015955 0.038179 0.038282 0.030814 0.750000 \n",
"25% -0.009749 -0.016557 -0.008704 -0.056606 0.100000 \n",
"50% 0.003749 0.015639 0.017755 -0.018430 0.300000 \n",
"75% 0.015950 0.038167 0.038124 0.030815 0.750000 \n",
"max 0.082843 0.157566 0.088105 3.846154 1.000000 \n",
"\n",
" TLT_ret_10 USO_ret_5 SPY_TLT_ratio_ret spy_up_5d split \n",
"count 1201.000000 1201.000000 1201.000000 1201.000000 1201 \n",
"count 1204.000000 1204.000000 1204.000000 1204.000000 1204 \n",
"unique NaN NaN NaN NaN 3 \n",
"top NaN NaN NaN NaN train \n",
"freq NaN NaN NaN NaN 840 \n",
"mean -0.004099 0.005024 0.004880 0.588676 NaN \n",
"std 0.028502 0.052628 0.027606 0.492279 NaN \n",
"freq NaN NaN NaN NaN 842 \n",
"mean -0.004103 0.004938 0.004838 0.589701 NaN \n",
"std 0.028469 0.052606 0.027586 0.492092 NaN \n",
"min -0.092880 -0.196652 -0.117208 0.000000 NaN \n",
"25% -0.023304 -0.025866 -0.010013 0.000000 NaN \n",
"50% -0.004435 0.004950 0.005537 1.000000 NaN \n",
"75% 0.014227 0.032486 0.021390 1.000000 NaN \n",
"25% -0.023139 -0.025950 -0.010029 0.000000 NaN \n",
"50% -0.004429 0.004869 0.005508 1.000000 NaN \n",
"75% 0.014188 0.032295 0.021358 1.000000 NaN \n",
"max 0.091322 0.327273 0.129204 1.000000 NaN "
]
},
"execution_count": 6,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
@@ -787,14 +787,14 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(PosixPath('/home/jarno/repos/trading-bot/data/training/spy_direction_5d.parquet'),\n",
" (1201, 11),\n",
" (1204, 11),\n",
" date SPY_ret_5 SPY_ret_20 SPY_dist_sma50 VIX_change_5 \\\n",
" 0 2021-10-11 0.014114 -0.026625 -0.018145 -0.090869 \n",
" 1 2021-10-12 0.001201 -0.023752 -0.020386 -0.074091 \n",
@@ -810,7 +810,7 @@
" 4 0.05 -0.002202 0.030287 -0.003823 1.0 train )"
]
},
"execution_count": 7,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
+1 -2
View File
@@ -1,12 +1,11 @@
[project]
name = "trading-bot"
version = "0.1.0"
description = "Python trading bot using machine learning and the IBKR API."
description = "Python trading bot using machine learning and the Alpaca Markets API."
readme = "README.md"
requires-python = ">=3.11,<3.12"
dependencies = [
"alpaca-py>=0.43.5",
"ib-insync>=0.9.86",
"matplotlib>=3.11.1",
"pandas>=2.3.0",
"pyarrow>=20.0.0",
+2 -1
View File
@@ -98,9 +98,10 @@ def main() -> None:
api_key=args.api_key,
secret_key=args.secret_key,
)
print(f"Fetched {len(candles)} daily candles for {symbol}.")
stored = write_candles(output_path, symbol, candles)
print(f"Wrote {len(stored)} total daily rows to {output_path}")
if __name__ == "__main__":
main()
main()
-293
View File
@@ -1,293 +0,0 @@
"""IBKR daily candle fetcher."""
from __future__ import annotations
import argparse
import re
from dataclasses import dataclass
from datetime import date, datetime, timedelta
from pathlib import Path
from zoneinfo import ZoneInfo
import pandas as pd
DEFAULT_OUTPUT_DIR = Path("data/ibkr/daily")
DEFAULT_DURATION = "1 W"
EASTERN_TZ = ZoneInfo("America/New_York")
DURATION_PATTERN = re.compile(r"^\d+\s+[SDWMY]$")
IBKR_HOST = "127.0.0.1"
IBKR_PORT = 4002
IBKR_CLIENT_ID = 101
IBKR_CONNECT_TIMEOUT_SECONDS = 10
@dataclass(frozen=True)
class DailyCandle:
"""Daily OHLCV market data for one trading session."""
trading_day: date
open: float
high: float
low: float
close: float
volume: int
def default_end_date() -> date:
"""Return yesterday's date in the US/Eastern market timezone."""
return datetime.now(EASTERN_TZ).date() - timedelta(days=1)
def current_market_date() -> date:
"""Return today's date in the US/Eastern market timezone."""
return datetime.now(EASTERN_TZ).date()
def parse_end_date(value: str) -> date:
"""Parse an end date in IBKR-friendly YYYYMMDD form."""
try:
return datetime.strptime(value, "%Y%m%d").date()
except ValueError as exc:
raise argparse.ArgumentTypeError(
"end date must use YYYYMMDD format, such as 20250605"
) from exc
def parse_duration(value: str) -> str:
"""Normalize and validate an IBKR duration string like '1 W' or '1 M'."""
normalized = " ".join(value.upper().split())
if not DURATION_PATTERN.fullmatch(normalized):
raise argparse.ArgumentTypeError(
"duration must look like an IBKR value, such as '1 W' or '1 M'"
)
return normalized
def format_ibkr_end_datetime(end_date: date) -> str:
"""Convert a date to the US/Eastern end datetime string IBKR expects."""
return f"{end_date:%Y%m%d} 23:59:59 US/Eastern"
def normalize_bar_date(value: date | datetime | str) -> date:
"""Normalize an IBKR historical bar date value."""
if isinstance(value, datetime):
return value.date()
if isinstance(value, date):
return value
return parse_end_date(value)
def fetch_daily_candles(
symbol: str, end_date: date, duration: str
) -> list[DailyCandle]:
"""Fetch daily candles for a symbol from IBKR."""
try:
from ib_insync import IB, Stock # type: ignore[import-not-found]
except ImportError as exc:
raise SystemExit(
"Missing dependency: ib_insync. Install it before running the "
"IBKR fetcher."
) from exc
ib = IB()
try:
print(
f"Connecting to IBKR Gateway at {IBKR_HOST}:{IBKR_PORT} "
f"with client id {IBKR_CLIENT_ID}"
)
ib.connect(
IBKR_HOST,
IBKR_PORT,
clientId=IBKR_CLIENT_ID,
timeout=IBKR_CONNECT_TIMEOUT_SECONDS,
)
contract = Stock(symbol, "SMART", "USD")
ib.qualifyContracts(contract)
bars = ib.reqHistoricalData(
contract,
endDateTime=format_ibkr_end_datetime(end_date),
durationStr=duration,
barSizeSetting="1 day",
whatToShow="TRADES",
useRTH=True,
formatDate=1,
)
if not bars:
print(f"IBKR returned no historical bars for {symbol}.")
candles: list[DailyCandle] = []
for bar in bars:
trading_day = normalize_bar_date(bar.date)
candles.append(
DailyCandle(
trading_day=trading_day,
open=float(bar.open),
high=float(bar.high),
low=float(bar.low),
close=float(bar.close),
volume=int(bar.volume),
)
)
return candles
finally:
if ib.isConnected():
ib.disconnect()
def candles_to_frame(symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
"""Convert candles to a date-indexed dataframe ready for Parquet storage."""
rows = [
{
"date": candle.trading_day,
"symbol": symbol,
"open": candle.open,
"high": candle.high,
"low": candle.low,
"close": candle.close,
"volume": candle.volume,
}
for candle in candles
]
frame = pd.DataFrame.from_records(rows)
if frame.empty:
return pd.DataFrame(
columns=["symbol", "open", "high", "low", "close", "volume"],
index=pd.Index([], name="date"),
)
frame["date"] = pd.to_datetime(frame["date"]).dt.date
return frame.set_index("date")
def read_existing_candles(path: Path) -> pd.DataFrame:
"""Read an existing candle Parquet file as a date-indexed dataframe."""
if not path.exists():
return pd.DataFrame(
columns=["symbol", "open", "high", "low", "close", "volume"],
index=pd.Index([], name="date"),
)
frame = pd.read_parquet(path)
if "date" in frame.columns:
frame["date"] = pd.to_datetime(frame["date"]).dt.date
frame = frame.set_index("date")
frame.index = pd.to_datetime(frame.index).date
frame.index.name = "date"
return frame
def oldest_stored_date_or_today(path: Path) -> date:
"""Return the oldest stored candle date, or today if no data exists yet."""
existing = read_existing_candles(path)
if existing.empty:
return current_market_date()
return min(existing.index)
def write_candles(path: Path, symbol: str, candles: list[DailyCandle]) -> pd.DataFrame:
"""Append candles to a ticker Parquet file, keeping one row per date."""
existing = read_existing_candles(path)
fetched = candles_to_frame(symbol, candles)
combined = pd.concat([existing, fetched])
if not combined.empty:
combined = combined[~combined.index.duplicated(keep="last")]
combined = combined.sort_index()
path.parent.mkdir(parents=True, exist_ok=True)
combined.to_parquet(path, index=True)
return combined
def print_candles(symbol: str, candles: list[DailyCandle]) -> None:
"""Print candles in a compact table."""
print("date,symbol,open,high,low,close,volume")
for candle in candles:
print(
f"{candle.trading_day.isoformat()},"
f"{symbol},"
f"{candle.open:.2f},"
f"{candle.high:.2f},"
f"{candle.low:.2f},"
f"{candle.close:.2f},"
f"{candle.volume}"
)
def parse_args() -> argparse.Namespace:
"""Parse command line arguments."""
parser = argparse.ArgumentParser(description="Fetch daily IBKR candles.")
parser.add_argument("symbol", help="Ticker symbol to fetch, such as SPY.")
end_date_group = parser.add_mutually_exclusive_group()
end_date_group.add_argument(
"--end-date",
type=parse_end_date,
help="Request end date in YYYYMMDD format. Defaults to yesterday.",
)
end_date_group.add_argument(
"--end-date-from-parquet",
action="store_true",
help=(
"Use the oldest date from the symbol Parquet file as the request "
"end date. Defaults to today if the file is missing or empty."
),
)
parser.add_argument(
"--duration",
type=parse_duration,
default=DEFAULT_DURATION,
help=(
"IBKR duration string, such as '1 W' or '1 M'. "
f"Defaults to {DEFAULT_DURATION}."
),
)
parser.add_argument(
"--output-dir",
type=Path,
default=DEFAULT_OUTPUT_DIR,
help=f"Directory for Parquet files. Defaults to {DEFAULT_OUTPUT_DIR}.",
)
return parser.parse_args()
def main() -> None:
"""Run the daily candle fetcher."""
args = parse_args()
symbol = args.symbol.upper()
output_path = args.output_dir / f"{symbol}.parquet"
end_date = (
oldest_stored_date_or_today(output_path)
if args.end_date_from_parquet
else args.end_date or default_end_date()
)
print(
f"Fetching {symbol} daily candles ending {end_date:%Y-%m-%d} "
f"for duration {args.duration}"
)
candles = fetch_daily_candles(symbol, end_date, args.duration)
print_candles(symbol, candles)
stored = write_candles(output_path, symbol, candles)
print(f"Wrote {len(stored)} total daily rows to {output_path}")
if __name__ == "__main__":
main()
+1
View File
@@ -0,0 +1 @@
"""Read-only web UI for inspecting trading bot status."""
+891
View File
@@ -0,0 +1,891 @@
"""Read-only Alpaca performance dashboard served by a small Python backend."""
from __future__ import annotations
import argparse
import json
import os
import sqlite3
from dataclasses import dataclass
from datetime import UTC, date, datetime, timedelta
from html import escape
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
from alpaca.trading.client import TradingClient
from alpaca.trading.enums import QueryOrderStatus
from alpaca.trading.models import Order, Position, TradeAccount
from alpaca.trading.requests import GetOrdersRequest, GetPortfolioHistoryRequest
from dotenv import load_dotenv
from trading_bot.data.alpaca_daily_lib import (
DailyCandle,
EASTERN_TZ,
default_end_date,
fetch_daily_candles,
)
load_dotenv()
DEFAULT_DB_PATH = Path("data/ui/performance.sqlite")
DEFAULT_HOST = "127.0.0.1"
DEFAULT_PORT = 8000
DEFAULT_REFRESH_HOURS = 12
DEFAULT_PERIOD_DAYS = 92
BENCHMARK_SYMBOL = "SPY"
@dataclass(frozen=True)
class PerformancePoint:
trading_day: date
account_value: float
benchmark_value: float | None
@dataclass(frozen=True)
class PortfolioRow:
symbol: str
asset_type: str
quantity: float | None
market_value: float
average_entry_price: float | None
current_price: float | None
unrealized_pl: float | None
@dataclass(frozen=True)
class TradeRow:
filled_at: datetime | None
symbol: str
side: str
quantity: float
filled_average_price: float | None
notional: float | None
status: str
@dataclass(frozen=True)
class DashboardSnapshot:
refreshed_at: datetime
performance: list[PerformancePoint]
portfolio: list[PortfolioRow]
trades: list[TradeRow]
warning: str | None = None
def _to_float(value: Any, default: float = 0.0) -> float:
if value is None:
return default
try:
return float(value)
except (TypeError, ValueError):
return default
def _to_optional_float(value: Any) -> float | None:
if value is None or value == "":
return None
return _to_float(value)
def _utc_now() -> datetime:
return datetime.now(UTC)
def _from_unix_timestamp(value: int | float) -> date:
return datetime.fromtimestamp(value, tz=UTC).astimezone(EASTERN_TZ).date()
def _benchmark_end_date(account_history_last_day: date) -> date:
return min(account_history_last_day, default_end_date())
def init_database(db_path: Path) -> None:
db_path.parent.mkdir(parents=True, exist_ok=True)
with sqlite3.connect(db_path) as connection:
connection.executescript(
"""
create table if not exists metadata (
key text primary key,
value text not null
);
create table if not exists performance_points (
trading_day text primary key,
account_value real not null,
benchmark_value real
);
create table if not exists portfolio_rows (
symbol text primary key,
asset_type text not null,
quantity real,
market_value real not null,
average_entry_price real,
current_price real,
unrealized_pl real
);
create table if not exists trade_rows (
id integer primary key autoincrement,
filled_at text,
symbol text not null,
side text not null,
quantity real not null,
filled_average_price real,
notional real,
status text not null
);
"""
)
def get_last_refresh(db_path: Path) -> datetime | None:
if not db_path.exists():
return None
with sqlite3.connect(db_path) as connection:
row = connection.execute(
"select value from metadata where key = 'refreshed_at'"
).fetchone()
if row is None:
return None
return datetime.fromisoformat(row[0])
def is_cache_stale(db_path: Path, refresh_interval: timedelta, now: datetime | None = None) -> bool:
refreshed_at = get_last_refresh(db_path)
if refreshed_at is None:
return True
return (now or _utc_now()) - refreshed_at >= refresh_interval
def create_trading_client(
api_key: str | None,
secret_key: str | None,
paper: bool,
) -> TradingClient:
if api_key is None or secret_key is None:
raise ValueError("ALPACA_API_KEY and ALPACA_SECRET_KEY are required.")
return TradingClient(api_key, secret_key, paper=paper)
def fetch_performance_points(
client: TradingClient,
api_key: str | None,
secret_key: str | None,
) -> list[PerformancePoint]:
history = client.get_portfolio_history(
GetPortfolioHistoryRequest(period="3M", timeframe="1D")
)
account_values_by_day = {
_from_unix_timestamp(timestamp): float(equity)
for timestamp, equity in zip(history.timestamp, history.equity, strict=False)
if equity is not None
}
if not account_values_by_day:
return []
first_day = min(account_values_by_day)
last_day = max(account_values_by_day)
benchmark = fetch_daily_candles(
BENCHMARK_SYMBOL,
end_date=_benchmark_end_date(last_day),
duration="3 M",
api_key=api_key,
secret_key=secret_key,
)
benchmark_values_by_day = _normalize_benchmark(
benchmark,
start_day=first_day,
initial_account_value=account_values_by_day[first_day],
)
return [
PerformancePoint(
trading_day=trading_day,
account_value=account_value,
benchmark_value=benchmark_values_by_day.get(trading_day),
)
for trading_day, account_value in sorted(account_values_by_day.items())
]
def _normalize_benchmark(
candles: list[DailyCandle],
start_day: date,
initial_account_value: float,
) -> dict[date, float]:
closes = {
candle.trading_day: candle.close
for candle in candles
if candle.trading_day >= start_day and candle.close > 0
}
if not closes:
return {}
base_day = min(closes)
base_close = closes[base_day]
return {
trading_day: initial_account_value * (close / base_close)
for trading_day, close in closes.items()
}
def fetch_portfolio_rows(client: TradingClient) -> list[PortfolioRow]:
account = client.get_account()
if not isinstance(account, TradeAccount):
raise TypeError(f"Expected TradeAccount, got {type(account)}")
rows = [
PortfolioRow(
symbol="CASH",
asset_type="cash",
quantity=None,
market_value=_to_float(account.cash),
average_entry_price=None,
current_price=None,
unrealized_pl=None,
)
]
positions = client.get_all_positions()
if not isinstance(positions, list):
raise TypeError(f"Expected list of positions, got {type(positions)}")
for position in positions:
if not isinstance(position, Position):
raise TypeError(f"Expected Position, got {type(position)}")
rows.append(
PortfolioRow(
symbol=str(position.symbol).upper(),
asset_type=str(position.asset_class or "equity"),
quantity=_to_optional_float(position.qty),
market_value=_to_float(position.market_value),
average_entry_price=_to_optional_float(position.avg_entry_price),
current_price=_to_optional_float(position.current_price),
unrealized_pl=_to_optional_float(position.unrealized_pl),
)
)
return rows
def fetch_trade_rows(client: TradingClient) -> list[TradeRow]:
orders = client.get_orders(
GetOrdersRequest(status=QueryOrderStatus.CLOSED, limit=100)
)
if not isinstance(orders, list):
raise TypeError(f"Expected list of orders, got {type(orders)}")
trades: list[TradeRow] = []
for order in orders:
if not isinstance(order, Order):
raise TypeError(f"Expected Order, got {type(order)}")
quantity = _to_float(order.filled_qty)
if quantity <= 0:
continue
trades.append(
TradeRow(
filled_at=order.filled_at,
symbol=str(order.symbol).upper(),
side=str(order.side).lower(),
quantity=quantity,
filled_average_price=_to_optional_float(order.filled_avg_price),
notional=_to_optional_float(order.notional),
status=str(order.status).lower(),
)
)
return trades
def fetch_alpaca_snapshot(
client: TradingClient,
api_key: str | None,
secret_key: str | None,
) -> DashboardSnapshot:
return DashboardSnapshot(
refreshed_at=_utc_now(),
performance=fetch_performance_points(client, api_key, secret_key),
portfolio=fetch_portfolio_rows(client),
trades=fetch_trade_rows(client),
)
def write_snapshot(db_path: Path, snapshot: DashboardSnapshot) -> None:
init_database(db_path)
with sqlite3.connect(db_path) as connection:
connection.execute("delete from performance_points")
connection.execute("delete from portfolio_rows")
connection.execute("delete from trade_rows")
connection.executemany(
"""
insert into performance_points (
trading_day, account_value, benchmark_value
) values (?, ?, ?)
""",
[
(
point.trading_day.isoformat(),
point.account_value,
point.benchmark_value,
)
for point in snapshot.performance
],
)
connection.executemany(
"""
insert into portfolio_rows (
symbol, asset_type, quantity, market_value,
average_entry_price, current_price, unrealized_pl
) values (?, ?, ?, ?, ?, ?, ?)
""",
[
(
row.symbol,
row.asset_type,
row.quantity,
row.market_value,
row.average_entry_price,
row.current_price,
row.unrealized_pl,
)
for row in snapshot.portfolio
],
)
connection.executemany(
"""
insert into trade_rows (
filled_at, symbol, side, quantity,
filled_average_price, notional, status
) values (?, ?, ?, ?, ?, ?, ?)
""",
[
(
row.filled_at.isoformat() if row.filled_at else None,
row.symbol,
row.side,
row.quantity,
row.filled_average_price,
row.notional,
row.status,
)
for row in snapshot.trades
],
)
connection.execute(
"""
insert into metadata (key, value)
values ('refreshed_at', ?)
on conflict(key) do update set value = excluded.value
""",
(snapshot.refreshed_at.isoformat(),),
)
def read_snapshot(db_path: Path, warning: str | None = None) -> DashboardSnapshot:
init_database(db_path)
refreshed_at = get_last_refresh(db_path) or _utc_now()
with sqlite3.connect(db_path) as connection:
performance = [
PerformancePoint(
trading_day=date.fromisoformat(row[0]),
account_value=row[1],
benchmark_value=row[2],
)
for row in connection.execute(
"""
select trading_day, account_value, benchmark_value
from performance_points
order by trading_day
"""
)
]
portfolio = [
PortfolioRow(
symbol=row[0],
asset_type=row[1],
quantity=row[2],
market_value=row[3],
average_entry_price=row[4],
current_price=row[5],
unrealized_pl=row[6],
)
for row in connection.execute(
"""
select symbol, asset_type, quantity, market_value,
average_entry_price, current_price, unrealized_pl
from portfolio_rows
order by case when symbol = 'CASH' then 0 else 1 end, symbol
"""
)
]
trades = [
TradeRow(
filled_at=datetime.fromisoformat(row[0]) if row[0] else None,
symbol=row[1],
side=row[2],
quantity=row[3],
filled_average_price=row[4],
notional=row[5],
status=row[6],
)
for row in connection.execute(
"""
select filled_at, symbol, side, quantity,
filled_average_price, notional, status
from trade_rows
order by filled_at desc
"""
)
]
return DashboardSnapshot(
refreshed_at=refreshed_at,
performance=performance,
portfolio=portfolio,
trades=trades,
warning=warning,
)
def load_or_refresh_snapshot(
db_path: Path,
api_key: str | None,
secret_key: str | None,
paper: bool,
refresh_interval: timedelta,
) -> DashboardSnapshot:
if not is_cache_stale(db_path, refresh_interval):
return read_snapshot(db_path)
try:
client = create_trading_client(api_key=api_key, secret_key=secret_key, paper=paper)
snapshot = fetch_alpaca_snapshot(client, api_key, secret_key)
write_snapshot(db_path, snapshot)
return snapshot
except Exception as exc:
if db_path.exists() and get_last_refresh(db_path) is not None:
return read_snapshot(db_path, warning=f"Refresh failed; showing cached data. {exc}")
raise
def _format_currency(value: float | None) -> str:
if value is None:
return ""
sign = "-" if value < 0 else ""
return f"{sign}${abs(value):,.2f}"
def _format_number(value: float | None) -> str:
if value is None:
return ""
return f"{value:,.4f}"
def _format_percent(value: float | None) -> str:
if value is None:
return ""
return f"{value:.2%}"
def _clean_enum_text(value: Any) -> str:
text = str(value or "").strip()
if "." in text:
text = text.rsplit(".", maxsplit=1)[-1]
return text.lower()
def _pill_class(value: str) -> str:
normalized = _clean_enum_text(value)
if normalized in {"buy", "filled"}:
return "pill pill-positive"
if normalized in {"sell", "canceled", "cancelled", "failed", "expired", "rejected"}:
return "pill pill-negative"
return "pill"
def _pill_html(value: str) -> str:
label = _clean_enum_text(value).upper()
return f"<span class=\"{_pill_class(label)}\">{escape(label)}</span>"
def _performance_text(points: list[PerformancePoint]) -> str:
if len(points) < 2:
return "Performance unavailable"
first = points[0]
last = points[-1]
change = last.account_value - first.account_value
percent = change / first.account_value if first.account_value else 0.0
days = (last.trading_day - first.trading_day).days
sign = "+" if change >= 0 else ""
return (
f"{sign}{_format_currency(change)} ({sign}{percent:.2%}) "
f"over {days} days"
)
def render_dashboard(snapshot: DashboardSnapshot) -> str:
chart_data = [
{
"date": point.trading_day.isoformat(),
"account": point.account_value,
"benchmark": point.benchmark_value,
}
for point in snapshot.performance
]
chart_json = json.dumps(chart_data)
warning = (
f"<p class=\"warning\">{escape(snapshot.warning)}</p>"
if snapshot.warning
else ""
)
current_portfolio_value = sum(row.market_value for row in snapshot.portfolio)
portfolio_rows = "\n".join(
"<tr>"
f"<td>{escape(row.symbol)}</td>"
f"<td>{_format_number(row.quantity)}</td>"
f"<td>{_format_currency(row.market_value)}</td>"
f"<td>{_format_percent(row.market_value / current_portfolio_value if current_portfolio_value else None)}</td>"
f"<td>{_format_currency(row.average_entry_price)}</td>"
f"<td>{_format_currency(row.current_price)}</td>"
f"<td>{_format_currency(row.unrealized_pl)}</td>"
"</tr>"
for row in snapshot.portfolio
)
trade_rows = "\n".join(
"<tr>"
f"<td>{escape(row.filled_at.isoformat(sep=' ', timespec='minutes') if row.filled_at else '')}</td>"
f"<td>{escape(row.symbol)}</td>"
f"<td>{_pill_html(row.side)}</td>"
f"<td>{_format_number(row.quantity)}</td>"
f"<td>{_format_currency(row.filled_average_price)}</td>"
f"<td>{_format_currency(row.notional)}</td>"
f"<td>{_pill_html(row.status)}</td>"
"</tr>"
for row in snapshot.trades
)
portfolio_rows = portfolio_rows or "<tr><td colspan=\"7\">No portfolio rows cached.</td></tr>"
trade_rows = trade_rows or "<tr><td colspan=\"7\">No completed trades cached.</td></tr>"
return f"""<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Trading Bot Performance</title>
<style>
:root {{
color-scheme: light;
--bg: #f6f7f9;
--panel: #ffffff;
--text: #17202a;
--muted: #5f6b7a;
--line: #d8dee8;
--account: #116466;
--benchmark: #c75000;
--warning: #7a4100;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0;
background: var(--bg);
color: var(--text);
font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
line-height: 1.45;
}}
main {{
width: min(1120px, calc(100% - 32px));
margin: 0 auto;
padding: 32px 0 48px;
}}
header {{
display: grid;
gap: 8px;
margin-bottom: 24px;
}}
h1, h2 {{ margin: 0; letter-spacing: 0; }}
h1 {{ font-size: clamp(2rem, 4vw, 3.5rem); }}
h2 {{ font-size: 1.15rem; margin-bottom: 12px; }}
.summary {{
font-size: 1.25rem;
font-weight: 650;
}}
.meta, .legend, .warning {{
color: var(--muted);
margin: 0;
}}
.warning {{ color: var(--warning); }}
section {{
background: var(--panel);
border: 1px solid var(--line);
border-radius: 8px;
padding: 18px;
margin-top: 18px;
}}
.chart-wrap {{
min-height: 420px;
}}
canvas {{
width: 100%;
height: 360px;
display: block;
}}
.legend {{
display: flex;
gap: 18px;
flex-wrap: wrap;
margin-top: 10px;
font-size: 0.95rem;
}}
.legend span::before {{
content: "";
display: inline-block;
width: 20px;
height: 3px;
margin-right: 8px;
vertical-align: middle;
background: var(--account);
}}
.legend span:last-child::before {{ background: var(--benchmark); }}
.table-wrap {{ overflow-x: auto; }}
table {{
width: 100%;
border-collapse: collapse;
min-width: 760px;
}}
th, td {{
padding: 10px 12px;
border-bottom: 1px solid var(--line);
text-align: right;
white-space: nowrap;
}}
th:first-child, td:first-child,
.trades th:nth-child(3), .trades td:nth-child(3),
.trades th:nth-child(7), .trades td:nth-child(7) {{
text-align: left;
}}
th {{
color: var(--muted);
font-size: 0.82rem;
font-weight: 700;
text-transform: uppercase;
}}
.pill {{
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 58px;
padding: 3px 9px;
border-radius: 999px;
background: #edf1f5;
color: #3d4854;
font-size: 0.75rem;
font-weight: 800;
letter-spacing: 0;
}}
.pill-positive {{
background: #dff3e8;
color: #17643a;
}}
.pill-negative {{
background: #ffe1df;
color: #9a2219;
}}
</style>
</head>
<body>
<main>
<header>
<h1>Trading Bot Performance</h1>
<div class="summary">{escape(_performance_text(snapshot.performance))}</div>
<p class="meta">Last refreshed {escape(snapshot.refreshed_at.astimezone().isoformat(sep=' ', timespec='minutes'))}</p>
{warning}
</header>
<section class="chart-wrap">
<h2>Account Value vs S&amp;P 500 Proxy</h2>
<canvas id="performanceChart" width="1040" height="360"></canvas>
<p class="legend"><span>Account value</span><span>SPY normalized to account start value</span></p>
</section>
<section>
<h2>Portfolio</h2>
<div class="table-wrap">
<table>
<thead>
<tr>
<th>Symbol</th><th>Quantity</th><th>Market Value</th><th>Account %</th>
<th>Avg Entry</th><th>Current Price</th><th>Unrealized P/L</th>
</tr>
</thead>
<tbody>{portfolio_rows}</tbody>
</table>
</div>
</section>
<section>
<h2>Completed Trades</h2>
<div class="table-wrap">
<table class="trades">
<thead>
<tr>
<th>Filled At</th><th>Symbol</th><th>Side</th><th>Quantity</th>
<th>Avg Fill</th><th>Notional</th><th>Status</th>
</tr>
</thead>
<tbody>{trade_rows}</tbody>
</table>
</div>
</section>
</main>
<script>
const points = {chart_json};
const canvas = document.getElementById("performanceChart");
const ctx = canvas.getContext("2d");
const css = getComputedStyle(document.documentElement);
const accountColor = css.getPropertyValue("--account").trim();
const benchmarkColor = css.getPropertyValue("--benchmark").trim();
const gridColor = css.getPropertyValue("--line").trim();
const textColor = css.getPropertyValue("--muted").trim();
function currency(value) {{
return "$" + value.toLocaleString(undefined, {{ maximumFractionDigits: 0 }});
}}
function drawLine(values, color, minValue, maxValue, left, top, width, height) {{
const valid = values.filter((point) => point.value !== null && point.value !== undefined);
if (valid.length < 2) return;
ctx.beginPath();
valid.forEach((point, index) => {{
const x = left + (point.index / Math.max(points.length - 1, 1)) * width;
const y = top + (1 - ((point.value - minValue) / Math.max(maxValue - minValue, 1))) * height;
if (index === 0) ctx.moveTo(x, y);
else ctx.lineTo(x, y);
}});
ctx.strokeStyle = color;
ctx.lineWidth = 3;
ctx.stroke();
}}
function drawChart() {{
const ratio = window.devicePixelRatio || 1;
const rect = canvas.getBoundingClientRect();
canvas.width = Math.max(Math.floor(rect.width * ratio), 320);
canvas.height = Math.floor(360 * ratio);
ctx.setTransform(ratio, 0, 0, ratio, 0, 0);
ctx.clearRect(0, 0, rect.width, 360);
const left = 72, right = 18, top = 16, bottom = 46;
const width = rect.width - left - right;
const height = 360 - top - bottom;
const series = points.flatMap((point) => [point.account, point.benchmark]).filter((value) => value !== null);
if (series.length === 0) {{
ctx.fillStyle = textColor;
ctx.fillText("No performance history cached.", left, top + 24);
return;
}}
const minValue = Math.min(...series) * 0.995;
const maxValue = Math.max(...series) * 1.005;
ctx.font = "12px system-ui, sans-serif";
ctx.strokeStyle = gridColor;
ctx.fillStyle = textColor;
for (let i = 0; i <= 4; i++) {{
const y = top + (i / 4) * height;
const value = maxValue - (i / 4) * (maxValue - minValue);
ctx.beginPath();
ctx.moveTo(left, y);
ctx.lineTo(left + width, y);
ctx.stroke();
ctx.fillText(currency(value), 6, y + 4);
}}
drawLine(points.map((point, index) => ({{ index, value: point.account }})), accountColor, minValue, maxValue, left, top, width, height);
drawLine(points.map((point, index) => ({{ index, value: point.benchmark }})), benchmarkColor, minValue, maxValue, left, top, width, height);
if (points.length > 0) {{
ctx.fillStyle = textColor;
ctx.fillText(points[0].date, left, 342);
ctx.textAlign = "right";
ctx.fillText(points[points.length - 1].date, left + width, 342);
ctx.textAlign = "left";
}}
}}
drawChart();
window.addEventListener("resize", drawChart);
</script>
</body>
</html>
"""
class DashboardRequestHandler(BaseHTTPRequestHandler):
db_path: Path
api_key: str | None
secret_key: str | None
paper: bool
refresh_interval: timedelta
def do_GET(self) -> None:
if self.path not in {"/", "/index.html"}:
self.send_error(404)
return
try:
snapshot = load_or_refresh_snapshot(
db_path=self.db_path,
api_key=self.api_key,
secret_key=self.secret_key,
paper=self.paper,
refresh_interval=self.refresh_interval,
)
body = render_dashboard(snapshot).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
except Exception as exc:
body = f"Dashboard unavailable: {escape(str(exc))}".encode("utf-8")
self.send_response(500)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, format: str, *args: Any) -> None:
print(f"{self.address_string()} - {format % args}")
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Serve the read-only trading bot dashboard.")
parser.add_argument("--host", default=DEFAULT_HOST, help=f"Host to bind. Defaults to {DEFAULT_HOST}.")
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"Port to bind. Defaults to {DEFAULT_PORT}.")
parser.add_argument("--db-path", type=Path, default=DEFAULT_DB_PATH, help=f"SQLite cache path. Defaults to {DEFAULT_DB_PATH}.")
parser.add_argument("--refresh-hours", type=float, default=DEFAULT_REFRESH_HOURS, help="Refresh Alpaca data after this many hours. Defaults to 12.")
parser.add_argument("--api-key", default=os.getenv("ALPACA_API_KEY"), help="Alpaca API Key ID.")
parser.add_argument("--secret-key", default=os.getenv("ALPACA_SECRET_KEY"), help="Alpaca Secret Key.")
parser.add_argument("--paper", action="store_true", help="Use Alpaca paper trading mode. This is the default.")
return parser.parse_args()
def main() -> None:
args = parse_args()
handler = type(
"ConfiguredDashboardRequestHandler",
(DashboardRequestHandler,),
{
"db_path": args.db_path,
"api_key": args.api_key,
"secret_key": args.secret_key,
"paper": True if args.paper else True,
"refresh_interval": timedelta(hours=args.refresh_hours),
},
)
server = ThreadingHTTPServer((args.host, args.port), handler)
print(f"Serving read-only dashboard at http://{args.host}:{args.port}")
try:
server.serve_forever()
except KeyboardInterrupt:
print("\nStopping dashboard server.")
finally:
server.server_close()
if __name__ == "__main__":
main()
+160
View File
@@ -0,0 +1,160 @@
from datetime import UTC, date, datetime, timedelta
from trading_bot.data.alpaca_daily_lib import DailyCandle
from trading_bot.ui.dashboard import (
DashboardSnapshot,
PerformancePoint,
PortfolioRow,
TradeRow,
_benchmark_end_date,
_from_unix_timestamp,
_normalize_benchmark,
is_cache_stale,
read_snapshot,
render_dashboard,
write_snapshot,
)
def test_normalize_benchmark_scales_spy_to_account_start_value() -> None:
candles = [
DailyCandle(date(2026, 5, 1), 100.0, 100.0, 100.0, 100.0, 1_000),
DailyCandle(date(2026, 5, 2), 110.0, 110.0, 110.0, 110.0, 1_000),
]
values = _normalize_benchmark(
candles,
start_day=date(2026, 5, 1),
initial_account_value=1_000.0,
)
assert values[date(2026, 5, 1)] == 1_000.0
assert values[date(2026, 5, 2)] == 1_100.0
def test_portfolio_history_timestamps_are_interpreted_as_eastern_dates() -> None:
timestamp = datetime(2026, 8, 11, 2, 30, tzinfo=UTC).timestamp()
assert _from_unix_timestamp(timestamp) == date(2026, 8, 10)
def test_benchmark_end_date_does_not_go_past_completed_market_day(monkeypatch) -> None:
monkeypatch.setattr(
"trading_bot.ui.dashboard.default_end_date",
lambda: date(2026, 8, 10),
)
assert _benchmark_end_date(date(2026, 8, 11)) == date(2026, 8, 10)
assert _benchmark_end_date(date(2026, 8, 8)) == date(2026, 8, 8)
def test_sqlite_snapshot_round_trip(tmp_path) -> None:
db_path = tmp_path / "dashboard.sqlite"
refreshed_at = datetime(2026, 8, 11, 10, 0, tzinfo=UTC)
snapshot = DashboardSnapshot(
refreshed_at=refreshed_at,
performance=[
PerformancePoint(date(2026, 8, 10), 10_000.0, 9_900.0),
PerformancePoint(date(2026, 8, 11), 10_100.0, 10_000.0),
],
portfolio=[
PortfolioRow("CASH", "cash", None, 500.0, None, None, None),
PortfolioRow("SPY", "us_equity", 10.0, 5_000.0, 490.0, 500.0, 100.0),
],
trades=[
TradeRow(
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
"SPY",
"buy",
1.0,
500.0,
500.0,
"filled",
)
],
)
write_snapshot(db_path, snapshot)
loaded = read_snapshot(db_path)
assert loaded.refreshed_at == refreshed_at
assert loaded.performance == snapshot.performance
assert loaded.portfolio == snapshot.portfolio
assert loaded.trades == snapshot.trades
def test_cache_stale_after_refresh_interval(tmp_path) -> None:
db_path = tmp_path / "dashboard.sqlite"
refreshed_at = datetime(2026, 8, 11, 0, 0, tzinfo=UTC)
snapshot = DashboardSnapshot(
refreshed_at=refreshed_at,
performance=[],
portfolio=[],
trades=[],
)
write_snapshot(db_path, snapshot)
assert not is_cache_stale(
db_path,
timedelta(hours=12),
now=refreshed_at + timedelta(hours=11, minutes=59),
)
assert is_cache_stale(
db_path,
timedelta(hours=12),
now=refreshed_at + timedelta(hours=12),
)
def test_render_dashboard_contains_requested_sections() -> None:
html = render_dashboard(
DashboardSnapshot(
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
performance=[
PerformancePoint(date(2026, 8, 10), 10_000.0, 10_000.0),
PerformancePoint(date(2026, 8, 11), 10_250.0, 10_100.0),
],
portfolio=[PortfolioRow("CASH", "cash", None, 250.0, None, None, None)],
trades=[
TradeRow(
datetime(2026, 8, 11, 9, 30, tzinfo=UTC),
"SPY",
"orderside.sell",
1.0,
500.0,
500.0,
"orderstatus.filled",
)
],
)
)
assert "Account Value vs S&amp;P 500 Proxy" in html
assert "Portfolio" in html
assert "Completed Trades" in html
assert "+$250.00 (+2.50%)" in html
assert "<th>Type</th>" not in html
assert "<th>Account %</th>" in html
assert "SELL</span>" in html
assert "FILLED</span>" in html
assert "orderside.sell" not in html
assert "orderstatus.filled" not in html
def test_portfolio_percentage_uses_current_portfolio_total() -> None:
html = render_dashboard(
DashboardSnapshot(
refreshed_at=datetime(2026, 8, 11, 10, 0, tzinfo=UTC),
performance=[
PerformancePoint(date(2026, 8, 11), 10_000.0, 10_000.0),
],
portfolio=[
PortfolioRow("CASH", "cash", None, 100.0, None, None, None),
PortfolioRow("SPY", "us_equity", 10.0, 10_000.0, 900.0, 1_000.0, 1_000.0),
],
trades=[],
)
)
assert "99.01%" in html
assert "100.00%" not in html
Generated
-36
View File
@@ -183,18 +183,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/05/7f/798705f5296a58ca505d600456748d1be48078eac8a7050d8a98bc9edb89/decorator-5.3.1-py3-none-any.whl", hash = "sha256:f47fe6fdbd2edd623ecfe36875d37aba411624e2670dd395dddae1358689bb3c", size = 10365, upload-time = "2026-05-18T06:03:26.517Z" },
]
[[package]]
name = "eventkit"
version = "1.0.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy" },
]
sdist = { url = "https://files.pythonhosted.org/packages/16/1e/0fac4e45d71ace143a2673ec642701c3cd16f833a0e77a57fa6a40472696/eventkit-1.0.3.tar.gz", hash = "sha256:99497f6f3c638a50ff7616f2f8cd887b18bbff3765dc1bd8681554db1467c933", size = 28320, upload-time = "2023-12-11T11:41:35.339Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/93/d9/7497d650b69b420e1a913329a843e16c715dac883750679240ef00a921e2/eventkit-1.0.3-py3-none-any.whl", hash = "sha256:0e199527a89aff9d195b9671ad45d2cc9f79ecda0900de8ecfb4c864d67ad6a2", size = 31837, upload-time = "2023-12-11T11:41:33.358Z" },
]
[[package]]
name = "executing"
version = "2.2.1"
@@ -221,19 +209,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/2c/47/c99d5268f354002ce80f8d029cd9d7d872969da1de8b93d32de4dc56d6f4/fonttools-4.63.0-py3-none-any.whl", hash = "sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d", size = 1164562, upload-time = "2026-05-14T12:04:29.092Z" },
]
[[package]]
name = "ib-insync"
version = "0.9.86"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "eventkit" },
{ name = "nest-asyncio" },
]
sdist = { url = "https://files.pythonhosted.org/packages/55/bb/733d5c81c8c2f54e90898afc7ff3a99f4d53619e6917c848833f9cc1ab56/ib_insync-0.9.86.tar.gz", hash = "sha256:73af602ca2463f260999970c5bd937b1c4325e383686eff301743a4de08d381e", size = 69859, upload-time = "2023-07-02T12:43:31.968Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/8f/f3/28ea87be30570f4d6b8fd24380d12fa74e59467ee003755e76aeb29082b8/ib_insync-0.9.86-py3-none-any.whl", hash = "sha256:a61fbe56ff405d93d211dad8238d7300de76dd6399eafc04c320470edec9a4a4", size = 72980, upload-time = "2023-07-02T12:43:29.928Z" },
]
[[package]]
name = "idna"
version = "3.18"
@@ -459,15 +434,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/85/a5bfaebfd305ac18b57b0854d74e37e586809061a91fda62f0bd50c8518e/narwhals-2.24.0-py3-none-any.whl", hash = "sha256:42fdedf44e5b2ca7505630d45b4ac3058f38d8485cba9fe1652ca23152df7489", size = 461030, upload-time = "2026-07-13T10:49:17.571Z" },
]
[[package]]
name = "nest-asyncio"
version = "1.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/83/f8/51569ac65d696c8ecbee95938f89d4abf00f47d58d48f6fbabfe8f0baefe/nest_asyncio-1.6.0.tar.gz", hash = "sha256:6f172d5449aca15afd6c646851f4e31e02c598d553a667e38cafa997cfec55fe", size = 7418, upload-time = "2024-01-21T14:25:19.227Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" },
]
[[package]]
name = "nest-asyncio2"
version = "1.7.2"
@@ -946,7 +912,6 @@ version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "alpaca-py" },
{ name = "ib-insync" },
{ name = "matplotlib" },
{ name = "pandas" },
{ name = "pyarrow" },
@@ -964,7 +929,6 @@ dev = [
[package.metadata]
requires-dist = [
{ name = "alpaca-py", specifier = ">=0.43.5" },
{ name = "ib-insync", specifier = ">=0.9.86" },
{ name = "matplotlib", specifier = ">=3.11.1" },
{ name = "pandas", specifier = ">=2.3.0" },
{ name = "pyarrow", specifier = ">=20.0.0" },