# Data Fetcher Design

## Initial Goal

Create a Python module/tool that fetches daily candlestick data from the IBKR API for a specific ticker and date range.

The intended example workflow is:

- ticker: `SPY`;
- date range: `2026-06-01` to `2026-06-30`;
- bar size: one trading day;
- fields: open, high, low, close, volume;
- output: one Parquet file named for the ticker, such as `SPY.parquet`.

## Current Implementation

The first implementation is intentionally small:

- ticker passed as a required command line argument;
- end date passed with `--end-date YYYYMMDD`, defaulting to yesterday;
- end date can also be derived with `--end-date-from-parquet`;
- duration passed with `--duration`, defaulting to `1 W`;
- hard-coded IBKR Gateway target: `127.0.0.1:4002`;
- uses the IBKR API through `ib_insync`;
- prints fetched candles as CSV-like rows;
- writes candles to a symbol-named Parquet file;
- appends to an existing symbol file and keeps one row per date.

Run it with:

```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY
```

To override the requested range:

```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date 20250605 --duration "1 M"
```

To fetch backward from the oldest date already stored in the symbol file:

```sh
mise exec -- uv run python src/trading_bot/data/fetch_ibkr_daily.py SPY --end-date-from-parquet
```

`--end-date` and `--end-date-from-parquet` cannot be used together. If the symbol Parquet file does not exist or has no rows, `--end-date-from-parquet` uses today's US/Eastern date.

This expects a local IBKR Gateway session to be running and accepting API connections on `127.0.0.1:4002`.

By default, output is written to `data/ibkr/daily/SPY.parquet`. Use `--output-dir` to choose another directory.

Manual test instructions are in [manual-test/README.md](manual-test/README.md).

## Intended Future Behavior

Later, this tool should broaden configuration around data source and normalization choices.

Decided storage behavior:

- Parquet files are partitioned by ticker, not by date.
- Each ticker should have its own Parquet file, such as `SPY.parquet`.
- The trading date is used as the row key for merges.
- Re-fetching a date replaces the existing row for that date in that symbol's file.

Candidate output schema:

| Column | Type | Description |
| --- | --- | --- |
| `date` | date index | Trading session date |
| `symbol` | string | Asset ticker |
| `open` | float | Daily open price |
| `high` | float | Daily high price |
| `low` | float | Daily low price |
| `close` | float | Daily close price |
| `volume` | integer | Daily traded volume |

Open decisions:

- how to handle adjusted versus unadjusted prices;
- how to handle missing sessions and IBKR pacing limits;
- whether to use `ib_insync` long term or a lower-level IBKR client wrapper.