jarno/trading-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Data fetcher implementation

Browse Source
This commit is contained in:
jarno
2026-07-26 14:18:07 +03:00
parent 8d3e663e71
commit e2eb5dd574
4 changed files with 273 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/manual-test/README.md
+34 -12
View File
@@ -1,19 +1,21 @@
# Manual Test Instructions
## IBKR Daily Fetcher Skeleton
## IBKR Daily Fetcher
This test checks the first hard-coded IBKR data fetcher without writing any data files.
This test checks the IBKR data fetcher and confirms it writes a symbol-named Parquet file.
The fetcher currently requests:
- gateway: `127.0.0.1:4002`;
- client id: `101`;
- ticker: `SPY`;
- date range: `2026-06-01` to `2026-06-05`;
- symbol: provided as a command line argument;
- end date: provided with `--end-date YYYYMMDD`, defaulting to yesterday;
- end date can also be derived with `--end-date-from-parquet`;
- duration: provided with `--duration`, defaulting to `1 W`;
- bar size: `1 day`;
- data type: `TRADES`;
- regular trading hours only;
- output: printed CSV-like rows.
- output: printed CSV-like rows and a Parquet file in `data/ibkr/daily/`.
## Prerequisites
@@ -33,32 +35,52 @@ mise exec -- uv sync
From the repository root, run:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
```
To fetch a specific IBKR range, pass an end date and duration:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
```
To fetch backward from the oldest date already stored in `data/ibkr/daily/SPY.parquet`:
```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
```
`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.
## Expected Output
The tool should first print the hard-coded request range and connection target:
The tool should first print the request range and connection target:
```text
Fetching SPY daily candles from 2026-06-01 to 2026-06-05
Fetching SPY daily candles ending 2025-06-05 for duration 1 M
Connecting to IBKR Gateway at 127.0.0.1:4002 with client id 101
```
If the request succeeds, it should then print a header and one row per returned trading day:
```text
date,ticker,open,high,low,close,volume
2026-06-01,SPY,...
date,symbol,open,high,low,close,volume
2025-06-05,SPY,...
```
Exact prices and volume depend on what IBKR returns.
The tool should then write or update:
```text
data/ibkr/daily/SPY.parquet
```
If the Parquet file already exists, rows from the latest fetch are merged into it. The trading date is used as the row key, so a symbol file keeps only one row for each date.
## Common Issues
- Connection refused: IBKR Gateway is not running, the port is not `4002`, or API access is disabled.
- Client id already in use: change `IBKR_CLIENT_ID` in the fetcher or disconnect the other client.
- No historical bars: confirm the account has market data permissions and that IBKR accepts the requested historical data range.
- Pacing or permission errors: note the IBKR error message before changing the request.
This skeleton does not write Parquet files yet.