Initial commit — TSLA grid trading bot

- Grid strategy with survival-gated spacing and depth
- Full 60% drop simulation for all survival checks
- Calibration report with auto-updating survival threshold
- Transaction history sync from Capital.com
- Dip mode with bottom-two TP rules

.gitignore

@@ -0,0 +1,22 @@
+# Credentials — never commit these
+.env
+
+# Database — large binary, regenerated by fetch_history.py
+data/
+
+# Logs — runtime output, not source code
+logs/
+
+# Python cache
+__pycache__/
+*.pyc
+*.pyo
+
+# OS files
+.DS_Store
+Thumbs.db
+
+# Editor files
+.vscode/
+.idea/
+*.swp

CONTEXT.md

@@ -0,0 +1,240 @@
+# MAXBOT — Full Context Prompt
+
+Paste this at the start of any new conversation to restore full context.
+
+---
+
+## What this is
+
+A live grid trading bot running on Ubuntu server (portainer.local), trading
+TSLA CFDs on Capital.com. Written in Python, running as a systemd service.
+Account owner: George, UK retail account, GBP, 5:1 leverage (20% margin).
+
+---
+
+## Project structure
+
+```
+~/maxbot/
+  bot.py          — main entry point, loop, startup audit, CLI
+  config.py       — ALL settings (edit this for day-to-day changes)
+  client.py       — Capital.com REST API calls
+  state.py        — account state snapshot, field access helpers
+  calculator.py   — grid maths, survival check, tier lookups
+  grid.py         — order management logic (gap-filling, TP rules, dip mode)
+  actions.py      — dry/confirm/live action handler with visible logging
+  calibration.py  — startup calibration report, history sync
+  fetch_history.py — fetches full Capital.com transaction history to SQLite
+  data/history.db — SQLite: all trades, swaps, deposits from Jan 2021
+  logs/
+    bot.log        — full detail every loop
+    actions.log    — actions only (orders placed/cancelled/TP changes)
+    calibration.log — startup calibration reports
+    fetch_history.log
+```
+
+---
+
+## The trading strategy
+
+**Instrument:** TSLA CFD on Capital.com (UK retail, GBP account)
+**Type:** Grid / mean-reversion dip buying
+
+1. Bot places limit buy orders BELOW current price at regular intervals
+2. When TSLA dips, orders fill automatically
+3. Each position has a take profit set to return exactly £1.00 (0.2 shares)
+   or £0.50 (0.1 shares)
+4. Take profit is set as an ABSOLUTE PRICE LEVEL (not amount/distance)
+   Formula: tp_pct = (profit_gbp * gbpusd / size) / entry_price
+            tp_price = entry_price * (1 + tp_pct)
+5. Positions close automatically at take profit
+6. Strategy works as long as TSLA doesn't drop catastrophically and
+   long-term price grows
+
+**Position sizes:**
+- 0.2 shares: below SIZE_SMALL_THRESHOLD (highest_open * 0.95)
+- 0.1 shares: within top 5% of highest open price (expensive overnight)
+
+**Profit targets (configurable in config.py):**
+- TP_PROFIT_LARGE = 1.00 (£1.00 for 0.2 share positions)
+- TP_PROFIT_SMALL = 0.50 (£0.50 for 0.1 share positions)
+
+---
+
+## Grid logic
+
+**Anchor:** Grid starts just below the LOWEST OPEN POSITION (not current
+price). This prevents churn from minute-to-minute price movements. Grid
+only shifts when a position actually fills.
+
+**Normal mode** (price within 25% of highest open):
+- Anchor = lowest open position
+- Grid fills below anchor, queue_depth orders maintained
+
+**Dip mode** (price dropped 25%+ from highest open):
+- Lowest open position has TP DISABLED (manual close only)
+- Anchor = second lowest open position (lowest with TP)
+- Grid fills below anchor
+- Also fills gaps between no-TP position and anchor if spacing fits
+- No orders placed at or above anchor or no-TP position
+
+**Grid floor:** Always fixed at 60% drop from highest open position.
+Floor = highest_open * 0.40. Never changes regardless of other settings.
+
+**Queue depth:** Always maintain exactly QUEUE_DEPTH pending orders
+in the grid window. Scan for gaps anywhere (not just bottom) and fill them.
+Cancel orders outside the window.
+
+---
+
+## Bottom-two position rule
+
+Only activates when price drops 25%+ from highest open:
+- Lowest open position → TP DISABLED (manual close, holds for max recovery)
+- Second lowest → TP ENABLED at standard level
+- When new lower position opens → re-evaluate bottom two
+
+---
+
+## Survival system (CRITICAL — everything depends on this)
+
+**The survival check simulates a FULL 60% drop from highest open position.**
+
+It does NOT just check current orders. It simulates:
+1. All existing open positions
+2. ALL grid levels from lowest existing down to floor (no depth cap)
+   — because in a real crash, every level fills on the way down
+
+Formula:
+- Floor = highest_open * 0.40
+- For each level: margin = price * size * 0.20 / gbpusd
+- For each level: unrealised_loss = (floor - price) * size / gbpusd
+- equity_at_floor = current_equity + sum(unrealised_losses)
+- margin_level_at_floor = equity_at_floor / total_margin * 100
+- SAFE if margin_level_at_floor >= 50% (Capital.com closeout threshold)
+
+**Capital.com margin stages:**
+- >100%: normal
+- ≤100%: warning 1 — bot stops placing orders
+- ≤75%:  warning 2 — urgent
+- ≤50%:  auto closeout — Capital.com closes positions
+
+**SURVIVAL_DROP_PCT in config.py:** This is the bot's operational setting.
+It auto-steps (30→35→40→45→50→55→60%) as equity grows. It does NOT
+control grid depth — the grid always goes to 60% floor regardless.
+SURVIVAL_DROP_PCT is used to gate order placement in the main loop.
+
+**Auto-update:** Calibration checks max safe % each startup. If account
+can survive a higher milestone, config.py is auto-updated.
+
+---
+
+## Grid tier upgrades (survival-gated)
+
+**ORDER OF OPERATIONS — strictly enforced:**
+1. Survival check first (always, against full 60% simulation)
+2. Spacing tightens second (only when 60% survival passes at tighter spacing)
+3. Depth increases third (only after spacing tightened AND 60% survival passes)
+
+Spacing tiers (config.py SPACING_TIERS):
+- £600: 1.5% (base)
+- £800: 1.2% (only if 60% survival passes at 1.2%)
+- £1100: 1.0%
+- £1500: 0.8%
+
+Queue depth tiers (config.py QUEUE_TIERS):
+- (£600, 10 orders, requires 1.5% spacing)
+- (£900, 12 orders, requires 1.2% spacing first)
+- (£1200, 15 orders, requires 1.0% spacing first)
+- (£1500, 18 orders, requires 0.8% spacing first)
+
+Depth NEVER increases until spacing has already tightened to required level.
+Spacing NEVER tightens until full 60% drop simulation passes.
+
+---
+
+## Calibration report (runs every startup)
+
+Sections in logic order:
+1. ACCOUNT — equity, margin, positions, deposits
+2. SURVIVAL — max safe %, progress to 60%, auto-update status
+3. GRID — current spacing/depth, next upgrade status (survival-gated)
+4. TSLA PERFORMANCE — trades, profit, fees, monthly trend (from Jan 2024)
+5. MILESTONES — £1k, £10k, £100k, £1M with ETA based on real profit rate
+
+Deposit detection: auto-detects weekly average from Capital.com history
+(last 12 weeks, needs 3+ deposits). Falls back to WEEKLY_DEPOSIT_GBP
+in config.py.
+
+History sync: fetch_history.py runs incrementally on every startup,
+fetching only new days since last run. Uses /history/transactions endpoint.
+3 second delay to avoid Capital.com session rate limit (429).
+
+---
+
+## Operating modes
+
+```
+python3 bot.py --mode dryrun    # reads everything, prints what it would do, no actions
+python3 bot.py --mode confirm   # asks y/n before every action
+python3 bot.py --mode live      # fully autonomous
+```
+
+Service: systemd, auto-restart, credentials in ~/maxbot/.env
+
+---
+
+## Capital.com API notes
+
+- Field names (verified against live API):
+  Orders: workingOrderData.orderLevel, orderSize, dealId
+  Positions: position.level, position.size, position.profitLevel, position.dealId
+  Account: accounts[].balance.balance (equity), .deposit (funds), .available
+  Price: snapshot.bid, snapshot.offer
+- Session tokens expire after 10 minutes, refresh at 9 minutes
+- Rate limit: 1 request per 0.1s for order operations
+- POST /session: 1 request/second limit
+- TP is sent as profitLevel (absolute price), displays as "Price level" in UI
+- No custom order tags/references available in API
+
+---
+
+## Current state (as of May 27 2026)
+
+- Equity: ~£664
+- Highest open position: $450.02
+- Max safe survival: 40% drop
+- SURVIVAL_DROP_PCT: 40%
+- Grid: 1.5% spacing, 10 orders, FROZEN until 60% survival met
+- Account open: Jan 28 2021
+- Current strategy start: Jan 2024
+- Trades (Jan 2024+): 407, profit £484, fees -£74, net £409
+- Blended daily rate: £2.67/day
+- Next milestone: £1,000 equity (~126 days at current rate)
+- Weekly deposits: none detected yet (last deposit May 26 2026)
+
+---
+
+## Planned future features (not yet built)
+
+1. Telegram alerts — fills, closures, margin warnings
+2. Auto survival threshold stepping already done
+3. Claude AI integration — consult Claude on edge cases
+4. Web dashboard — browser view of positions, grid, P&L
+5. ProQuant-style backtester — test strategies against TSLA history
+   (data already in history.db, S&P500/NASDAQ planned as context)
+6. US100 and S&P500 as secondary instruments (future)
+
+---
+
+## Key design decisions made
+
+- Grid floor FIXED at 60% — never changes, decoupled from SURVIVAL_DROP_PCT
+- TP as price level % of entry (not fixed dollar, not fixed % of current price)
+- Grid anchored to lowest OPEN POSITION (not current price) — prevents churn
+- Survival always simulates FULL grid to floor (no depth cap) — honest
+- Spacing frozen until 60% survival passes — safety before profit frequency
+- Depth requires spacing first — correct order of operations
+- History uses /history/transactions not /history/activity (richer data)
+- Deposits filtered from Jan 1 2024 onwards (new strategy era)
+- SURVIVAL_DROP_PCT auto-steps in config.py via calibration — no manual edits

README.md

@@ -0,0 +1,203 @@
+# TSLA Grid Trading Bot — Capital.com
+
+A grid/mean-reversion trading bot for Tesla (TSLA) CFDs on Capital.com.
+Runs on Ubuntu server. Written in Python 3.
+
+---
+
+## How the strategy works
+
+The bot places limit buy orders at regular intervals **below the current price**.
+When TSLA dips, orders fill automatically. Each position has a take profit set
+to return a fixed GBP amount. Positions close automatically at take profit.
+
+- Many small £1.00 profits compound over time
+- Deeper dips = more positions open = more profit potential on recovery
+- Grid self-adjusts as equity grows (tighter spacing = more frequent fills)
+- Designed to survive a 60% drop from highest open position (builds toward this)
+
+---
+
+## Project structure
+
+```
+maxbot/
+  bot.py         Main entry point — run this
+  config.py      ALL settings — edit this for day-to-day changes
+  client.py      Capital.com API calls
+  state.py       Account state snapshot + field access helpers
+  calculator.py  Grid maths, survival check, gap detection
+  grid.py        Order management (gap-filling, TP rules)
+  actions.py     dry/confirm/live action handler
+  bot.log        Activity log (auto-created on first run)
+  README.md      This file
+```
+
+**When making changes:** each file has one job. Bug in order placement → `grid.py`.
+Change a setting → `config.py`. API field name changed → `client.py` and `state.py`.
+
+---
+
+## Setup
+
+### 1. Requirements
+
+Python 3.8+ and requests:
+```bash
+pip install requests --break-system-packages
+```
+
+### 2. Capital.com API key
+
+1. Log into Capital.com
+2. Go to **Settings → API integrations**
+3. Enable 2FA if not already done (required for API key)
+4. Generate API key
+
+### 3. Set credentials as environment variables
+
+```bash
+# Add to ~/.bashrc so they persist across reboots
+echo 'export CAPITAL_API_KEY="your_key_here"' >> ~/.bashrc
+echo 'export CAPITAL_IDENTIFIER="your@email.com"' >> ~/.bashrc
+echo 'export CAPITAL_PASSWORD="your_password_here"' >> ~/.bashrc
+source ~/.bashrc
+```
+
+### 4. Verify credentials work
+
+```bash
+cd ~/maxbot
+python3 bot.py --mode dryrun
+```
+
+Should connect, read your positions and orders, print the audit. No actions taken.
+
+---
+
+## Running the bot
+
+### Step 1 — Dry run (safe, always start here)
+```bash
+python3 bot.py --mode dryrun
+```
+Reads everything, prints what it would do. Zero actions. Run this first.
+
+### Step 2 — Confirm mode (approve each action)
+```bash
+python3 bot.py --mode confirm
+```
+Asks `y/n` before every order placement, cancellation, or TP change.
+Run this for several days until you trust the behaviour completely.
+
+### Step 3 — Live mode (autonomous)
+```bash
+python3 bot.py --mode live
+```
+Fully autonomous. Only switch here after thorough testing.
+
+---
+
+## Running as a system service (auto-start on boot)
+
+### Create the service file
+
+```bash
+sudo nano /etc/systemd/system/maxbot.service
+```
+
+Paste this (adjust paths/credentials):
+
+```ini
+[Unit]
+Description=TSLA Grid Trading Bot
+After=network.target
+StartLimitIntervalSec=0
+
+[Service]
+Type=simple
+User=george
+WorkingDirectory=/home/george/maxbot
+Environment=CAPITAL_API_KEY=your_key_here
+Environment=CAPITAL_IDENTIFIER=your@email.com
+Environment=CAPITAL_PASSWORD=your_password_here
+ExecStart=/usr/bin/python3 /home/george/maxbot/bot.py --mode live
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=multi-user.target
+```
+
+### Enable and start
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable maxbot
+sudo systemctl start maxbot
+```
+
+### Check status and logs
+
+```bash
+sudo systemctl status maxbot          # current status
+sudo journalctl -u maxbot -f          # live systemd logs
+tail -f ~/maxbot/bot.log              # bot's own log file
+```
+
+### Stop / restart
+
+```bash
+sudo systemctl stop maxbot
+sudo systemctl restart maxbot
+```
+
+---
+
+## Day-to-day configuration
+
+**All settings are in `config.py`.** Common changes:
+
+### Increase survival threshold as equity grows
+
+```python
+# In config.py — increase this as equity grows:
+SURVIVAL_DROP_PCT = 30.0   # current (£640 equity)
+# → 35.0  at £900
+# → 40.0  at £1200
+# → 50.0  at £1500
+# → 60.0  at £1800 (target)
+```
+
+### Adjust queue depth
+
+```python
+QUEUE_DEPTH = 10  # default
+```
+
+### Toggle demo mode for testing
+
+```python
+USE_DEMO = True   # points to demo.capital.com
+```
+
+---
+
+## Margin call stages (Capital.com UK retail)
+
+| Margin level | What happens |
+|---|---|
+| > 100% | Normal — bot operates |
+| ≤ 100% | Warning 1 — bot stops placing new orders |
+| ≤ 75%  | Warning 2 — urgent alert in logs |
+| ≤ 50%  | Auto closeout — Capital.com closes positions |
+
+---
+
+## Planned future enhancements
+
+- [ ] Telegram alerts — ping on fills, closures, margin warnings
+- [ ] Profit tracker — log every closed position, running total
+- [ ] Automatic survival threshold stepping (reads equity, updates config)
+- [ ] Claude AI integration — consult Claude on edge cases
+- [ ] Web dashboard — view positions and grid on homelab browser

actions.py

@@ -0,0 +1,119 @@
+"""
+actions.py — Action handler for dry/confirm/live modes.
+========================================================
+Every action the bot takes (place order, cancel order, update TP)
+goes through this handler. The mode controls what actually happens.
+
+MODES:
+------
+dryrun  → Logs what it would do. Returns None. No API calls made.
+confirm → Prints the action, waits for y/n input. Only proceeds on 'y'.
+live    → Executes immediately, no prompts.
+
+All three modes log the action clearly so you can audit what happened.
+
+WHY THIS MATTERS:
+-----------------
+Before trusting the bot with live orders, run in dryrun and confirm modes.
+Watch the logs for several days. Only switch to live when you're confident
+the logic is correct and the orders it wants to place match your expectations.
+"""
+
+import logging
+from typing import Optional
+from client import CapitalClient
+import config
+
+log = logging.getLogger("maxbot.actions")
+
+
+class ActionHandler:
+
+    def __init__(self, mode: str, client: CapitalClient):
+        self.mode   = mode
+        self.client = client
+        self.actions_taken   = 0
+        self.orders_placed   = 0
+        self.orders_cancelled = 0
+        self.tp_updates      = 0
+
+    def _log_action(self, symbol: str, description: str):
+        """Print a highly visible action log line."""
+        log.warning(f"")
+        log.warning(f"{'━' * 50}")
+        log.warning(f"  {symbol}  {description}")
+        log.warning(f"{'━' * 50}")
+
+    def _confirm(self, description: str) -> bool:
+        if self.mode == "dryrun":
+            log.info(f"[DRY RUN] Would: {description}")
+            return False
+        if self.mode == "confirm":
+            print(f"\n  ACTION: {description}")
+            try:
+                answer = input("  Proceed? (y/n): ").strip().lower()
+            except (EOFError, KeyboardInterrupt):
+                return False
+            return answer == "y"
+        return True  # live
+
+    # ─────────────────────────────────────────
+    #  ACTIONS
+    # ─────────────────────────────────────────
+
+    def place_limit_order(self, size: float, limit_price: float,
+                          take_profit: float) -> Optional[dict]:
+        desc = (
+            f"PLACE ORDER  entry ${limit_price:.2f}  "
+            f"size {size}  TP ${take_profit:.2f}"
+        )
+        if self._confirm(desc):
+            result = self.client.place_limit_order(size, limit_price, take_profit)
+            self._log_action("✚", desc)
+            self.actions_taken += 1
+            self.orders_placed += 1
+            return result
+        return None
+
+    def cancel_order(self, deal_id: str, price: float) -> Optional[dict]:
+        desc = f"CANCEL ORDER  ${price:.2f}"
+        if self._confirm(desc):
+            result = self.client.cancel_order(deal_id)
+            self._log_action("✖", desc)
+            self.actions_taken += 1
+            self.orders_cancelled += 1
+            return result
+        return None
+
+    def enable_tp(self, deal_id: str, entry_price: float,
+                  take_profit: float) -> Optional[dict]:
+        desc = (
+            f"ENABLE TP  position ${entry_price:.2f} "
+            f"→ TP ${take_profit:.2f}"
+        )
+        if self._confirm(desc):
+            result = self.client.update_position_tp(deal_id, take_profit)
+            self._log_action("✔", desc)
+            self.actions_taken += 1
+            self.tp_updates += 1
+            return result
+        return None
+
+    def disable_tp(self, deal_id: str, entry_price: float) -> Optional[dict]:
+        desc = f"DISABLE TP  position ${entry_price:.2f} (manual close)"
+        if self._confirm(desc):
+            result = self.client.update_position_tp(deal_id, None)
+            self._log_action("⊘", desc)
+            self.actions_taken += 1
+            self.tp_updates += 1
+            return result
+        return None
+
+    def summary(self) -> str:
+        return (
+            f"Session summary: "
+            f"{self.actions_taken} total actions | "
+            f"{self.orders_placed} placed | "
+            f"{self.orders_cancelled} cancelled | "
+            f"{self.tp_updates} TP updates"
+        )

bot.py

@@ -0,0 +1,490 @@
+"""
+bot.py — Main entry point for the TSLA grid trading bot.
+=========================================================
+Run this file to start the bot.
+
+USAGE:
+------
+  python3 bot.py --mode dryrun    # default — safe, no actions
+  python3 bot.py --mode confirm   # ask before every action
+  python3 bot.py --mode live      # fully autonomous
+
+FIRST TIME SETUP:
+-----------------
+1. Get API key from Capital.com → Settings → API integrations
+2. Set environment variables:
+     export CAPITAL_API_KEY="your_key"
+     export CAPITAL_IDENTIFIER="your@email.com"
+     export CAPITAL_PASSWORD="your_password"
+3. Run dryrun first and verify the output looks correct.
+4. Run confirm mode for several days, approve each action manually.
+5. Only switch to live mode when fully confident.
+
+TO RUN AS A SYSTEM SERVICE (auto-start on boot):
+-------------------------------------------------
+See README.md for systemd setup instructions.
+
+PROJECT STRUCTURE:
+------------------
+  bot.py         ← YOU ARE HERE — main loop, startup, CLI
+  config.py      ← ALL settings — edit this for day-to-day changes
+  client.py      ← Capital.com API calls
+  state.py       ← Account state snapshot, field access helpers
+  calculator.py  ← Grid maths, survival check, gap detection
+  grid.py        ← Order management logic (gap-filling, TP rules)
+  actions.py     ← dry/confirm/live action handler
+  bot.log        ← All activity logged here (auto-created)
+
+ENHANCING THE BOT:
+------------------
+The code is intentionally split so future additions are isolated:
+  - Telegram alerts  → new file telegram.py, called from run_loop()
+  - Profit tracking  → new file tracker.py, log closed positions
+  - Claude AI        → new file ai_advisor.py, called on edge cases
+  - Web dashboard    → new file dashboard.py, reads state and logs
+  - Auto threshold   → add equity check in run_loop(), update config
+Each addition touches one file without breaking anything else.
+
+WHAT THE BOT DOES EACH LOOP (every 60 seconds):
+------------------------------------------------
+1. Refresh API session if tokens approaching expiry
+2. Fetch: positions, orders, account balance, TSLA price, GBP/USD rate
+3. Check for abnormal price gap (> 20%) — pause if detected
+4. Check margin health — stop placing orders if level < 100%
+5. Manage bottom two positions (TP rules)
+6. Manage order grid (cancel out-of-window, fill gaps)
+7. Sleep 60 seconds, repeat
+"""
+
+import argparse
+import logging
+import os
+import sys
+import time
+from datetime import datetime
+
+import config
+from client import CapitalClient
+from state import BotState, pos_level, pos_size, pos_tp, ord_level, ord_size
+from calculator import GridCalculator
+from grid import GridManager
+from actions import ActionHandler
+from calibration import Calibrator, setup_calibration_log
+
+log = logging.getLogger("maxbot")
+
+
+# ─────────────────────────────────────────────
+#  LOGGING SETUP
+# ─────────────────────────────────────────────
+
+def setup_logging():
+    """
+    Two log files in the logs/ subfolder:
+      logs/bot.log         — full detail, every loop, for debugging
+      logs/actions.log     — only actions taken (orders placed/cancelled, TP changes)
+      logs/calibration.log — startup calibration reports, appended each run
+
+    Monitoring tip:
+      tail -f ~/maxbot/logs/actions.log      ← clean, just what the bot did
+      tail -f ~/maxbot/logs/bot.log          ← full detail if something goes wrong
+      cat ~/maxbot/logs/calibration.log      ← progress reports over time
+    """
+    base     = os.path.dirname(os.path.abspath(__file__))
+    logs_dir = os.path.join(base, "logs")
+    os.makedirs(logs_dir, exist_ok=True)
+    fmt  = "%(asctime)s [%(levelname)s] %(message)s"
+
+    full_handler = logging.FileHandler(os.path.join(logs_dir, "bot.log"))
+    full_handler.setFormatter(logging.Formatter(fmt))
+
+    console_handler = logging.StreamHandler(sys.stdout)
+    console_handler.setFormatter(logging.Formatter(fmt))
+
+    root = logging.getLogger()
+    root.setLevel(logging.INFO)
+    root.addHandler(full_handler)
+    root.addHandler(console_handler)
+
+    actions_handler = logging.FileHandler(os.path.join(logs_dir, "actions.log"))
+    actions_handler.setFormatter(logging.Formatter(fmt))
+    actions_handler.setLevel(logging.WARNING)
+    logging.getLogger("maxbot.actions").addHandler(actions_handler)
+
+    setup_calibration_log()
+
+
+# ─────────────────────────────────────────────
+#  MAIN BOT CLASS
+# ─────────────────────────────────────────────
+
+class TeslaGridBot:
+
+    def __init__(self, mode: str):
+        self.mode        = mode
+        self.client      = CapitalClient()
+        self.state       = BotState()
+        self.calculator  = GridCalculator()
+        self.actions     = ActionHandler(mode, self.client)
+        self.grid        = GridManager(self.state, self.calculator, self.actions)
+        self.calibrator  = Calibrator(self.state, self.calculator)
+        self.loop_count  = 0
+        self.gap_pause   = False
+        self._prev_position_count = None
+        self._prev_order_count    = None
+        self._prev_equity         = None
+
+    # ─────────────────────────────────────────
+    #  STARTUP
+    # ─────────────────────────────────────────
+
+    def banner(self):
+        labels = {
+            "dryrun":  "DRY RUN — no actions will be taken",
+            "confirm": "CONFIRM MODE — will ask before every action",
+            "live":    "LIVE MODE — fully autonomous",
+        }
+        print("\n" + "=" * 60)
+        print("  TSLA Grid Trading Bot — Capital.com")
+        print(f"  Mode: {labels.get(self.mode, self.mode)}")
+        print(f"  Started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+        print("=" * 60 + "\n")
+
+    def fetch_state(self):
+        """Refresh all state from Capital.com API."""
+        self.client.refresh_session_if_needed()
+        positions  = self.client.get_positions()
+        orders     = self.client.get_working_orders()
+        account    = self.client.get_account_details()
+        price_data = self.client.get_price(config.EPIC)
+        gbpusd     = self.client.get_gbpusd()
+        self.state.update(positions, orders, account, price_data, gbpusd)
+
+    def print_state(self):
+        """Print current account and grid status to log."""
+        s = self.state
+        log.info("─" * 50)
+        log.info(f"TSLA price:       ${s.current_price:.2f}")
+        log.info(f"Equity:           £{s.equity:.2f}")
+        log.info(f"Available:        £{s.available:.2f}")
+        log.info(f"Margin in use:    £{s.total_margin_gbp:.2f}")
+        log.info(f"Margin level:     {s.margin_level_pct:.1f}%")
+        log.info(f"Open positions:   {len(s.positions)}")
+        log.info(f"Pending orders:   {len(s.orders)}")
+        log.info(f"Highest open:     ${s.highest_open_price:.2f}")
+        log.info(f"Lowest level:     ${s.lowest_level_price:.2f}")
+        log.info(f"GBP/USD:          {s.gbpusd:.4f}")
+        calc     = self.calculator
+        spacing  = calc.get_spacing_pct(s.equity)
+        depth    = calc.get_queue_depth(s.equity)
+        full_sim = calc._simulate_grid(
+            s.highest_open_price, spacing, s.gbpusd, s.all_levels()
+        )
+        survival = calc.survival_check(
+            s.equity, s.highest_open_price, full_sim, s.gbpusd,
+            override_pct=60.0
+        )
+        # Find max safe % for clear display
+        max_safe = 0.0
+        for pct in range(10, 81, 5):
+            sim = calc._simulate_grid(
+                s.highest_open_price, spacing, s.gbpusd, s.all_levels()
+            )
+            r = calc.survival_check(
+                s.equity, s.highest_open_price, sim, s.gbpusd,
+                override_pct=float(pct)
+            )
+            if r["safe"]:
+                max_safe = float(pct)
+            else:
+                break
+
+        log.info(f"Grid spacing:     {spacing}%")
+        log.info(f"Queue depth:      {depth}")
+        log.info(
+            f"Survival:         max safe {max_safe:.0f}% | "
+            f"60% sim {'✓' if survival['safe'] else '✗'} "
+            f"(margin at floor: {survival['margin_level_at_floor_pct']:.1f}%)"
+        )
+        log.info("─" * 50)
+
+    def startup_audit(self):
+        """
+        Full account audit on first start.
+        Reads and displays everything before taking any action.
+        """
+        log.info("=" * 50)
+        log.info("STARTUP AUDIT")
+        log.info("=" * 50)
+
+        self.fetch_state()
+        self.print_state()
+
+        s = self.state
+
+        # Print all open positions
+        sorted_pos = s.positions_sorted_asc()
+        log.info(f"\nOpen positions ({len(sorted_pos)}) — lowest to highest:")
+        for p in sorted_pos:
+            tp    = pos_tp(p)
+            tp_str = f"${tp:.2f}" if tp else "NONE (manual close)"
+            upl   = float(p["position"].get("upl", 0))
+            log.info(
+                f"  ${pos_level(p):.2f} | "
+                f"size {p['position']['size']} | "
+                f"TP {tp_str} | "
+                f"P&L £{upl / self.state.gbpusd:.2f}"
+            )
+
+        # Print all pending orders
+        sorted_ord = s.orders_sorted_asc()
+        log.info(f"\nPending orders ({len(sorted_ord)}) — lowest to highest:")
+        for o in sorted_ord:
+            log.info(f"  ${ord_level(o):.2f} | size {ord_size(o)}")
+
+        # Full survival check
+        log.info("")
+        self._log_survival_check()
+
+        log.info("\nStartup audit complete.")
+        log.info("=" * 50)
+
+    # ─────────────────────────────────────────
+    #  CHECKS
+    # ─────────────────────────────────────────
+
+    def _log_survival_check(self):
+        """Run and log full survival check — always simulates full 60% grid."""
+        s    = self.state
+        calc = self.calculator
+        # Always simulate full grid at 60% drop — honest worst case
+        full_sim = calc._simulate_grid(
+            s.highest_open_price, calc.get_spacing_pct(s.equity),
+            s.gbpusd, s.all_levels()
+        )
+        result = calc.survival_check(
+            s.equity, s.highest_open_price,
+            full_sim, s.gbpusd,
+            override_pct=60.0
+        )
+        status = "✓ SAFE" if result["safe"] else "✗ UNSAFE"
+        log.info(f"Survival check (full 60% grid simulation): {status}")
+        log.info(f"  Floor price:        ${result['floor_price']:.2f}")
+        log.info(f"  Total margin (GBP): £{result['total_margin_gbp']:.2f}")
+        log.info(f"  Equity at floor:    £{result['equity_at_floor_gbp']:.2f}")
+        log.info(f"  Margin level there: {result['margin_level_at_floor_pct']:.1f}%")
+        log.info(f"  Unrealised P&L:     £{result['unrealised_loss_gbp']:.2f}")
+        if not result["safe"]:
+            log.warning(
+                "SURVIVAL CHECK FAILED — "
+                "account may be margin called at 60% drop."
+            )
+        return result
+
+    def check_price_gap(self) -> bool:
+        """
+        Detect abnormal price gaps between loops.
+        A gap > GAP_ALERT_PCT triggers a pause on order placement.
+        This protects against Monday gap-downs, stock splits, news events.
+        The bot will resume normal operation next loop after alerting.
+        To resume immediately, restart the bot.
+        """
+        s = self.state
+        if s.last_price and s.last_price > 0:
+            change_pct = abs(s.current_price - s.last_price) / s.last_price * 100
+            if change_pct >= config.GAP_ALERT_PCT:
+                log.warning("!" * 50)
+                log.warning(
+                    f"PRICE GAP DETECTED: "
+                    f"${s.last_price:.2f} → ${s.current_price:.2f} "
+                    f"({change_pct:.1f}%)"
+                )
+                log.warning(
+                    "This may be a gap-down open, stock split, or major news event."
+                )
+                log.warning(
+                    "Bot is PAUSING order placement this cycle. "
+                    "Review Capital.com manually."
+                )
+                log.warning("!" * 50)
+                self.gap_pause = True
+                return True
+        s.last_price = s.current_price
+        self.gap_pause = False
+        return False
+
+    def check_margin_health(self) -> str:
+        """
+        Check current margin level against Capital.com thresholds.
+        Returns: 'ok', 'warning1', 'warning2', or 'closeout'
+
+        Capital.com UK retail margin call stages:
+          > 100% → Normal
+          = 100% → Warning 1: cannot open new positions
+          = 75%  → Warning 2: urgent
+          = 50%  → Auto closeout begins
+        """
+        ml = self.state.margin_level_pct
+        if ml <= 50:
+            log.warning(
+                f"MARGIN CLOSEOUT LEVEL ({ml:.1f}% ≤ 50%). "
+                "Capital.com may be closing positions!"
+            )
+            return "closeout"
+        if ml <= 75:
+            log.warning(
+                f"MARGIN WARNING 2 ({ml:.1f}% ≤ 75%). "
+                "Cannot place new orders."
+            )
+            return "warning2"
+        if ml <= 100:
+            log.warning(
+                f"MARGIN WARNING 1 ({ml:.1f}% ≤ 100%). "
+                "Cannot place new orders."
+            )
+            return "warning1"
+        return "ok"
+
+    # ─────────────────────────────────────────
+    #  MAIN LOOP
+    # ─────────────────────────────────────────
+
+    def run_loop(self):
+        """Single iteration of the main loop."""
+        self.loop_count += 1
+
+        self.fetch_state()
+        s = self.state
+
+        # Detect meaningful changes since last loop
+        pos_count    = len(s.positions)
+        ord_count    = len(s.orders)
+        equity_delta = abs((s.equity - self._prev_equity) if self._prev_equity else 0)
+        state_changed = (
+            pos_count  != self._prev_position_count or
+            ord_count  != self._prev_order_count    or
+            equity_delta > 0.50  # equity moved more than 50p
+        )
+
+        if state_changed:
+            # Full state print when something meaningful changed
+            log.info(f"\n{'=' * 50}")
+            log.info(f"Loop #{self.loop_count} — {datetime.now().strftime('%H:%M:%S')} — STATE CHANGED")
+            self.print_state()
+        else:
+            # Quiet heartbeat — one line only
+            log.info(
+                f"Loop #{self.loop_count} — {datetime.now().strftime('%H:%M:%S')} — "
+                f"no changes  "
+                f"(price ${s.current_price:.2f}  "
+                f"equity £{s.equity:.2f}  "
+                f"pos {pos_count}  orders {ord_count})"
+            )
+
+        # Update previous state
+        self._prev_position_count = pos_count
+        self._prev_order_count    = ord_count
+        self._prev_equity         = s.equity
+
+        # Gap detection
+        if self.check_price_gap():
+            log.warning("Gap detected — skipping order management this cycle.")
+            return
+
+        # Margin health
+        margin_status = self.check_margin_health()
+        if margin_status != "ok":
+            log.warning(
+                f"Margin [{margin_status}] — managing positions only, "
+                "no new orders."
+            )
+            self.grid.manage_bottom_two_positions()
+            return
+
+        # Normal operation
+        self.grid.manage_bottom_two_positions()
+        self.grid.manage_orders()
+
+    def run(self):
+        """Start the bot."""
+        self.banner()
+
+        log.info("Connecting to Capital.com...")
+        try:
+            self.client.create_session()
+        except Exception as e:
+            log.error(f"Connection failed: {e}")
+            log.error(
+                "Check credentials: "
+                "CAPITAL_API_KEY, CAPITAL_IDENTIFIER, CAPITAL_PASSWORD"
+            )
+            sys.exit(1)
+
+        self.startup_audit()
+        try:
+            self.calibrator.run()
+        except Exception as e:
+            log.error(f"Calibration failed: {e}", exc_info=True)
+
+        if self.mode == "dryrun":
+            log.info("\nDRY RUN complete — no actions taken.")
+            log.info("Review output above, then run with --mode confirm")
+            return
+
+        log.info(
+            f"\nEntering main loop "
+            f"(every {config.LOOP_INTERVAL_SECS}s)..."
+        )
+        log.info("Press Ctrl+C to stop.\n")
+
+        try:
+            while True:
+                try:
+                    self.run_loop()
+                except KeyboardInterrupt:
+                    raise
+                except Exception as e:
+                    log.error(f"Loop error: {e}", exc_info=True)
+                    log.info("Waiting 30s before retry...")
+                    time.sleep(30)
+                    continue
+                time.sleep(config.LOOP_INTERVAL_SECS)
+
+        except KeyboardInterrupt:
+            log.info(f"\nBot stopped by user.")
+            log.info(self.actions.summary())
+
+
+# ─────────────────────────────────────────────
+#  CLI
+# ─────────────────────────────────────────────
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="TSLA Grid Trading Bot — Capital.com",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Modes:
+  dryrun   Connect and read everything. Print what bot would do. No actions.
+  confirm  Ask y/n before every action. Run this for days before going live.
+  live     Fully autonomous. Only use after thorough testing.
+
+Examples:
+  python3 bot.py                      # dryrun (default)
+  python3 bot.py --mode confirm       # confirm mode
+  python3 bot.py --mode live          # live mode
+        """
+    )
+    parser.add_argument(
+        "--mode",
+        choices=["dryrun", "confirm", "live"],
+        default="dryrun",
+        help="Operation mode (default: dryrun)"
+    )
+    args = parser.parse_args()
+    setup_logging()
+    TeslaGridBot(mode=args.mode).run()
+
+
+if __name__ == "__main__":
+    main()

calculator.py

@@ -0,0 +1,325 @@
+"""
+calculator.py — Grid maths and survival check.
+===============================================
+All calculations live here. No API calls, no side effects.
+Pure functions that take numbers and return numbers.
+
+KEY CALCULATIONS:
+-----------------
+
+1. GRID SPACING
+   Percentage-based so it self-adjusts as price changes.
+   At $420 with 1.5% spacing: $420 → $413.70 → $407.49 → $401.38 etc.
+   Tighter at lower prices naturally (1.5% of $200 = $3, vs 1.5% of $400 = $6).
+   Also tightens as equity grows via SPACING_TIERS in config.
+
+2. TAKE PROFIT PRICE
+   Target is fixed GBP profit per position (£1.00 or £0.50).
+   Formula: tp_distance_usd = profit_gbp * gbpusd / size
+   tp_price = entry_price + tp_distance_usd
+   Both 0.1 and 0.2 share positions travel the same price distance.
+   0.2 shares earns £1.00, 0.1 shares earns £0.50 (same distance, half the size).
+
+3. SURVIVAL CHECK
+   Simulates a X% drop from highest_open_price.
+   Calculates total unrealised loss at the floor price.
+   Checks if equity_at_floor > 50% of total_margin (Capital.com closeout level).
+   If not safe, bot stops placing new orders.
+
+4. GRID GAP DETECTION (KEY FIX)
+   Instead of just adding orders at the bottom, the bot:
+   a) Calculates ALL expected grid levels from just below lowest position
+      down to the survival floor.
+   b) Checks which levels already have an order within 0.5% tolerance.
+   c) Places orders for any missing levels (gaps), anywhere in the window.
+   This means if the top order fills or is cancelled, the gap is filled next loop.
+"""
+
+import logging
+import config
+
+log = logging.getLogger("maxbot.calculator")
+
+
+class GridCalculator:
+
+    # ─────────────────────────────────────────
+    #  SURVIVAL-GATED TIER LOOKUPS
+    # ─────────────────────────────────────────
+
+    def get_spacing_pct(self, equity: float,
+                        highest_open: float = None,
+                        all_levels: list = None,
+                        gbpusd: float = None) -> float:
+        """
+        Return current grid spacing %.
+        If survival parameters provided, returns the tightest spacing
+        that still passes survival check. Otherwise returns equity-based value.
+
+        Order of operations:
+        1. Start from current spacing
+        2. Check if next tighter tier is affordable (equity threshold met)
+        3. Simulate full grid at tighter spacing → survival check
+        4. Only apply if survival passes
+        """
+        # Simple equity-based lookup when no survival check needed
+        if highest_open is None or all_levels is None or gbpusd is None:
+            result = config.BASE_SPACING_PCT
+            for threshold, spacing in sorted(config.SPACING_TIERS):
+                if equity >= threshold:
+                    result = spacing
+            return result
+
+        # Survival-gated: find tightest spacing that still survives 60% drop
+        best_spacing = config.BASE_SPACING_PCT
+        for threshold, spacing in sorted(config.SPACING_TIERS):
+            if equity < threshold:
+                break
+            # Simulate full grid at this spacing
+            simulated = self._simulate_grid(
+                highest_open, spacing, gbpusd, all_levels
+            )
+            # Always check against 60% — spacing only tightens when
+            # account can genuinely survive the full target drop
+            result = self.survival_check(
+                equity, highest_open, simulated, gbpusd,
+                override_pct=60.0
+            )
+            if result["safe"]:
+                best_spacing = spacing
+            else:
+                break  # can't go tighter — stop here
+        return best_spacing
+
+    def get_queue_depth(self, equity: float,
+                        current_spacing: float = None,
+                        highest_open: float = None,
+                        all_levels: list = None,
+                        gbpusd: float = None) -> int:
+        """
+        Return current queue depth.
+        Depth tier requires:
+          a) Equity threshold met
+          b) Spacing already at required level
+          c) Survival check passes at new depth
+
+        Order of operations:
+        1. Spacing must tighten BEFORE depth can increase
+        2. Only increase depth if survival passes at new depth
+        """
+        # Simple lookup when no survival check needed
+        if current_spacing is None or highest_open is None:
+            result = config.QUEUE_TIERS[0][1]
+            for threshold, depth, req_spacing in sorted(config.QUEUE_TIERS):
+                if equity >= threshold:
+                    result = depth
+            return result
+
+        # Survival-gated: find deepest queue that still survives 60% drop
+        best_depth = config.QUEUE_TIERS[0][1]
+        for threshold, depth, req_spacing in sorted(config.QUEUE_TIERS):
+            if equity < threshold:
+                break
+            # Check spacing requirement — depth only unlocks after spacing tightened
+            if current_spacing > req_spacing:
+                break  # spacing hasn't tightened enough yet
+            # Simulate grid at this depth
+            simulated = self._simulate_grid(
+                highest_open, current_spacing, gbpusd,
+                all_levels, depth=depth
+            )
+            # Always check against 60% — depth only increases when
+            # account can genuinely survive the full target drop
+            result = self.survival_check(
+                equity, highest_open, simulated, gbpusd,
+                override_pct=60.0
+            )
+            if result["safe"]:
+                best_depth = depth
+            else:
+                break
+        return best_depth
+
+    def _simulate_grid(self, highest_open: float, spacing_pct: float,
+                       gbpusd: float, existing_levels: list,
+                       depth: int = None) -> list:
+        """
+        Simulate the COMPLETE set of positions/orders that would exist
+        in a worst-case 60% crash scenario.
+
+        This is used for survival checks — NOT for grid placement.
+
+        In a real crash:
+        - All existing open positions stay open (accumulating unrealised loss)
+        - ALL pending orders between current price and floor fill
+        - Bot keeps placing new orders on the way down (all fill too)
+
+        So we simulate:
+        1. All existing open positions (real entries, real sizes)
+        2. All grid levels from lowest existing position down to floor
+           (NO depth cap — every level fills in a crash)
+
+        existing_levels = s.all_levels() — list of {price, size} dicts
+        depth param ignored — full grid always simulated for honest result
+        """
+        GRID_FLOOR_PCT = 60.0
+        floor   = highest_open * (1.0 - GRID_FLOOR_PCT / 100.0)
+        spacing = spacing_pct / 100.0
+
+        # Include ALL existing levels (both positions and orders)
+        # These are already real entries with real sizes
+        simulated = [{"price": lv["price"], "size": lv["size"]}
+                     for lv in existing_levels]
+
+        # Find lowest existing level as starting point for simulation
+        if simulated:
+            start = min(lv["price"] for lv in simulated)
+        else:
+            start = highest_open * 0.95
+
+        # Add ALL simulated grid levels from just below start down to floor
+        # No depth cap — in a crash every level fills
+        level = start * (1.0 - spacing)
+        while level >= floor:
+            size = self.get_size(level, highest_open)
+            simulated.append({"price": level, "size": size})
+            level = level * (1.0 - spacing)
+
+        return simulated
+
+    # ─────────────────────────────────────────
+    #  SIZE AND TP
+    # ─────────────────────────────────────────
+
+    def get_size(self, price: float, highest_open: float) -> float:
+        """
+        Return position size for a given price level.
+        0.1 shares: only within top 5% of highest open (expensive to hold overnight).
+        0.2 shares: everything else (faster recovery, more profit per fill).
+        Example: highest_open=$450 → threshold=$427.50
+          Above $427.50 → 0.1 shares
+          Below $427.50 → 0.2 shares (almost all orders will be here)
+        """
+        threshold = highest_open * config.SIZE_SMALL_THRESHOLD_PCT
+        return 0.1 if price >= threshold else 0.2
+
+    def get_tp_price(self, entry_price: float, size: float,
+                     gbpusd: float, reference_price: float = None) -> float:
+        """
+        Calculate take profit as an absolute price level.
+
+        Target profit is configurable in config.py:
+          TP_PROFIT_LARGE = 1.00  (£1.00 for 0.2 share positions)
+          TP_PROFIT_SMALL = 0.50  (£0.50 for 0.1 share positions)
+        Change these values to adjust profit target (e.g. 1.50 for £1.50).
+
+        Formula:
+          tp_distance = profit_gbp * gbpusd / size   (converts £ to $ distance)
+          tp_pct      = tp_distance / entry_price     (as % of entry)
+          tp_price    = entry_price * (1 + tp_pct)    (absolute price level)
+
+        Result is always sent as an absolute price level to Capital.com.
+        Capital.com UI displays it correctly as "Price level".
+
+        reference_price parameter kept for compatibility but not used.
+        """
+        profit_gbp  = config.TP_PROFIT_LARGE if size >= 0.2 else config.TP_PROFIT_SMALL
+        profit_usd  = profit_gbp * gbpusd
+        tp_distance = profit_usd / size
+        tp_pct      = tp_distance / entry_price
+        return entry_price * (1.0 + tp_pct)
+
+    # ─────────────────────────────────────────
+    #  GRID GENERATION
+    # ─────────────────────────────────────────
+
+    def generate_grid(self, top_price: float, floor_price: float,
+                      equity: float, spacing_pct: float = None) -> list:
+        """
+        Generate all expected grid levels from just below top_price
+        down to floor_price. Uses provided spacing_pct or calculates
+        from equity tiers if not provided.
+        """
+        spacing = (spacing_pct if spacing_pct is not None
+                   else self.get_spacing_pct(equity)) / 100.0
+        levels  = []
+        price   = top_price * (1.0 - spacing)  # start just below top
+
+        while price >= floor_price:
+            levels.append(round(price, 2))
+            price = price * (1.0 - spacing)
+
+        return levels  # descending order
+
+    def find_missing_levels(self, expected_levels: list,
+                             existing_order_prices: set,
+                             queue_depth: int) -> list:
+        """
+        Find which expected grid levels are missing an order.
+        Uses 0.5% tolerance when matching levels to existing orders.
+
+        ALWAYS scans the full grid for gaps regardless of how many
+        orders already exist. This is critical for correct behaviour
+        when positions close and the window shifts upward — gaps appear
+        at the top and must be filled even if the queue is already full.
+
+        The manage_orders function handles cancelling excess bottom orders
+        before calling this, so the net result is always queue_depth orders
+        in the right positions.
+
+        Returns gaps in descending order (highest first = closest to price).
+        """
+        missing = []
+
+        for level in expected_levels:
+            already_covered = any(
+                abs(existing - level) / level < 0.005
+                for existing in existing_order_prices
+            )
+            if not already_covered:
+                missing.append(level)
+
+        # Highest gaps first (descending already from generate_grid)
+        # Limit to queue_depth so we don't flood with orders
+        return missing[:queue_depth]
+
+    # ─────────────────────────────────────────
+    #  SURVIVAL CHECK
+    # ─────────────────────────────────────────
+
+    def survival_check(self, equity: float, highest_open: float,
+                        all_levels: list, gbpusd: float,
+                        override_pct: float = None) -> dict:
+        """
+        Simulate a drop from highest_open_price.
+        Uses SURVIVAL_DROP_PCT from config by default.
+        Pass override_pct to test a different percentage (used by calibration).
+        """
+        drop_pct    = (override_pct if override_pct is not None else config.SURVIVAL_DROP_PCT) / 100.0
+        floor_price = highest_open * (1.0 - drop_pct)
+
+        total_margin_gbp    = 0.0
+        total_unrealised_gbp = 0.0
+
+        for lv in all_levels:
+            price = lv["price"]
+            size  = lv["size"]
+            margin_gbp     = price * size * config.MARGIN_RATE / gbpusd
+            unrealised_gbp = (floor_price - price) * size / gbpusd
+            total_margin_gbp     += margin_gbp
+            total_unrealised_gbp += unrealised_gbp
+
+        equity_at_floor  = equity + total_unrealised_gbp
+        margin_level_pct = (
+            (equity_at_floor / total_margin_gbp * 100)
+            if total_margin_gbp > 0 else 999.0
+        )
+
+        return {
+            "safe":                      margin_level_pct >= 50.0,
+            "floor_price":               floor_price,
+            "total_margin_gbp":          total_margin_gbp,
+            "equity_at_floor_gbp":       equity_at_floor,
+            "margin_level_at_floor_pct": margin_level_pct,
+            "unrealised_loss_gbp":       total_unrealised_gbp,
+        }

calibration.py

@@ -0,0 +1,687 @@
+"""
+calibration.py — Startup calibration report and analytics.
+============================================================
+Runs once after every bot startup, after the initial state fetch.
+Also triggered by fetch_history.py after each history sync.
+
+Writes to logs/calibration.log — each run appends a dated report
+so you can track progress over time.
+
+WHAT IT DOES:
+-------------
+1. Fetches latest transaction history (incremental — only new days)
+2. Analyses TSLA-only performance from real historical data
+3. Shows survival analysis with progress toward 60% target
+4. Tracks all milestones with real estimated days to reach each
+5. Recommends next config change (survival threshold increase)
+
+MILESTONE TARGETS:
+------------------
+  Priority 1: 60% survival (account resilience target)
+  Priority 2: £1,000 equity
+  Priority 3: £10,000 equity
+  Priority 4: £100,000 equity
+  Priority 5: £1,000,000 equity (the dream)
+
+DATA SOURCE:
+------------
+All financial projections use real TSLA trade history from history.db.
+Monthly profit trend used for projections (recent months weighted more).
+Falls back to estimates if no history available.
+"""
+
+import logging
+import os
+import sqlite3
+from datetime import datetime, date, timedelta
+from state import BotState, pos_level, pos_size, ord_level, ord_size
+from calculator import GridCalculator
+import config
+
+log = logging.getLogger("maxbot.calibration")
+
+DB_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "data", "history.db"
+)
+
+
+def setup_calibration_log():
+    """Add calibration log handler. Called from bot.py setup_logging."""
+    base     = os.path.dirname(os.path.abspath(__file__))
+    logs_dir = os.path.join(base, "logs")
+    os.makedirs(logs_dir, exist_ok=True)
+    handler = logging.FileHandler(os.path.join(logs_dir, "calibration.log"))
+    handler.setFormatter(logging.Formatter("%(asctime)s %(message)s"))
+    handler.setLevel(logging.INFO)
+    logging.getLogger("maxbot.calibration").addHandler(handler)
+    logging.getLogger("maxbot.calibration").propagate = True
+
+
+# ─────────────────────────────────────────────
+#  HISTORY READER
+# ─────────────────────────────────────────────
+
+class HistoryReader:
+    """Read TSLA performance data from history.db."""
+
+    def __init__(self):
+        self.available = os.path.exists(DB_PATH)
+        if self.available:
+            try:
+                self.conn = sqlite3.connect(DB_PATH)
+                self.conn.row_factory = sqlite3.Row
+            except Exception as e:
+                log.warning(f"Could not open history DB: {e}")
+                self.available = False
+
+    def get_deposit_stats(self) -> dict:
+        """
+        Analyse deposit history to detect recurring deposit pattern.
+        Returns weekly average based on recent deposits if a pattern exists.
+        Falls back to config.WEEKLY_DEPOSIT_GBP if no pattern detected.
+        """
+        if not self.available:
+            return {
+                "total_deposited": 0,
+                "deposit_count":   0,
+                "last_deposit":    None,
+                "weekly_avg":      getattr(config, "WEEKLY_DEPOSIT_GBP", 0.0),
+                "pattern":         "config",
+            }
+        try:
+            # All deposits from 2024 onwards only
+            row = self.conn.execute("""
+                SELECT COUNT(*) as c, SUM(size_gbp) as total,
+                       MAX(date_utc) as last
+                FROM transactions
+                WHERE transaction_type='DEPOSIT'
+                AND date_utc >= '2024-01-01'
+            """).fetchone()
+
+            # Recent deposits — last 12 weeks (from 2024 onwards)
+            recent = self.conn.execute("""
+                SELECT COUNT(*) as c, SUM(size_gbp) as total,
+                       MIN(date_utc) as first, MAX(date_utc) as last
+                FROM transactions
+                WHERE transaction_type='DEPOSIT'
+                AND date_utc >= date('now', '-84 days')
+                AND date_utc >= '2024-01-01'
+            """).fetchone()
+
+            # Detect pattern: 3+ deposits in last 12 weeks = regular
+            weekly_avg   = 0.0
+            pattern      = "none"
+            config_weekly = getattr(config, "WEEKLY_DEPOSIT_GBP", 0.0)
+
+            if recent["c"] >= 3:
+                # Regular deposit pattern detected
+                weekly_avg = (recent["total"] or 0) / 12.0
+                pattern    = f"detected ({recent['c']} deposits in 12 weeks)"
+            elif config_weekly > 0:
+                # Use config value
+                weekly_avg = config_weekly
+                pattern    = "config"
+            else:
+                pattern = "none yet — will auto-detect when regular deposits start"
+
+            return {
+                "total_deposited": row["total"] or 0,
+                "deposit_count":   row["c"] or 0,
+                "last_deposit":    row["last"][:10] if row["last"] else None,
+                "weekly_avg":      weekly_avg,
+                "pattern":         pattern,
+            }
+        except Exception as e:
+            log.warning(f"Could not read deposit stats: {e}")
+            return {
+                "total_deposited": 0,
+                "deposit_count":   0,
+                "last_deposit":    None,
+                "weekly_avg":      getattr(config, "WEEKLY_DEPOSIT_GBP", 0.0),
+                "pattern":         "error",
+            }
+    def get_tsla_stats(self) -> dict:
+        """
+        Return TSLA performance stats from new strategy start date.
+        New strategy detected as starting July 2024 based on trade history —
+        that's when consistent £1/trade pattern began after previous strategies.
+        """
+        NEW_STRATEGY_DATE = "2024-01-01"
+
+        if not self.available:
+            return {}
+        try:
+            # New strategy period only
+            row = self.conn.execute("""
+                SELECT COUNT(*) as trades,
+                       SUM(size_gbp) as profit,
+                       MIN(date_utc) as first_trade,
+                       MAX(date_utc) as last_trade
+                FROM transactions
+                WHERE transaction_type='TRADE' AND instrument='TSLA'
+                AND date_utc >= ?
+            """, (NEW_STRATEGY_DATE,)).fetchone()
+
+            swap_row = self.conn.execute("""
+                SELECT SUM(size_gbp) as fees
+                FROM transactions
+                WHERE transaction_type='SWAP' AND instrument='TSLA'
+                AND date_utc >= ?
+            """, (NEW_STRATEGY_DATE,)).fetchone()
+
+            deposit_row = None  # deposits tracked separately
+
+            # Monthly profit for last 6 months
+            monthly = []
+            for m in self.conn.execute("""
+                SELECT SUBSTR(date_utc,1,7) as month,
+                       COUNT(*) as trades,
+                       SUM(size_gbp) as profit
+                FROM transactions
+                WHERE transaction_type='TRADE' AND instrument='TSLA'
+                AND date_utc >= date('now','-6 months')
+                AND date_utc >= '2024-01-01'
+                GROUP BY month ORDER BY month
+            """):
+                monthly.append({
+                    "month":  m["month"],
+                    "trades": m["trades"],
+                    "profit": m["profit"],
+                })
+
+            # Recent daily average (last 30 days with trades)
+            recent_row = self.conn.execute("""
+                SELECT AVG(daily) FROM (
+                    SELECT DATE(date_utc) as d, SUM(size_gbp) as daily
+                    FROM transactions
+                    WHERE transaction_type='TRADE' AND instrument='TSLA'
+                    AND date_utc >= date('now','-30 days')
+                    AND date_utc >= '2024-01-01'
+                    GROUP BY DATE(date_utc)
+                )
+            """).fetchone()
+
+            # Last 90 days daily average (more stable)
+            avg90_row = self.conn.execute("""
+                SELECT AVG(daily) FROM (
+                    SELECT DATE(date_utc) as d, SUM(size_gbp) as daily
+                    FROM transactions
+                    WHERE transaction_type='TRADE' AND instrument='TSLA'
+                    AND date_utc >= date('now','-90 days')
+                    AND date_utc >= '2024-01-01'
+                    GROUP BY DATE(date_utc)
+                )
+            """).fetchone()
+
+            swap_fees    = swap_row["fees"] or 0
+            trade_profit = row["profit"] or 0
+            recent_daily = recent_row[0] or 0
+            avg90_daily  = avg90_row[0] or 0
+
+            return {
+                "trades":           row["trades"] or 0,
+                "trade_profit":     trade_profit,
+                "swap_fees":        swap_fees,
+                "net":              trade_profit + swap_fees,
+                "first_trade":      row["first_trade"],
+                "last_trade":       row["last_trade"],
+                "monthly":          monthly,
+                "recent_daily":     recent_daily,
+                "avg90_daily":      avg90_daily,
+            }
+        except Exception as e:
+            log.warning(f"Could not read TSLA stats: {e}")
+            return {}
+
+    def sync_history(self):
+        """Run incremental history fetch to get latest data."""
+        try:
+            import subprocess, sys, time
+            script = os.path.join(
+                os.path.dirname(os.path.abspath(__file__)), "fetch_history.py"
+            )
+            if os.path.exists(script):
+                log.info("Syncing transaction history...")
+                # Wait 3 seconds — main bot already created a session moments ago
+                # Capital.com rate limits POST /session to 1 request/second
+                time.sleep(3)
+                result = subprocess.run(
+                    [sys.executable, script, "--mode", "live"],
+                    capture_output=True, text=True, timeout=300
+                )
+                if result.returncode == 0:
+                    log.info("History sync complete.")
+                else:
+                    output = result.stderr or result.stdout
+                    err_lines = [l for l in output.split("\n")
+                                 if "ERROR" in l or "error" in l.lower()]
+                    err_msg = err_lines[-1].strip() if err_lines else output[:200]
+                    log.warning(f"History sync warning: {err_msg}")
+        except Exception as e:
+            log.warning(f"History sync failed: {e}")
+
+    def close(self):
+        if self.available:
+            try:
+                self.conn.close()
+            except Exception:
+                pass
+
+
+# ─────────────────────────────────────────────
+#  CALIBRATOR
+# ─────────────────────────────────────────────
+
+class Calibrator:
+
+    MILESTONES = [
+        ("£1,000",     1_000),
+        ("£10,000",   10_000),
+        ("£100,000", 100_000),
+        ("£1,000,000", 1_000_000),
+    ]
+
+    SURVIVAL_MILESTONES = [30, 35, 40, 45, 50, 55, 60]
+
+    def __init__(self, state: BotState, calculator: GridCalculator):
+        self.state      = state
+        self.calculator = calculator
+
+    def run(self):
+        """Run full calibration and write report."""
+        s    = self.state
+        calc = self.calculator
+
+        # Sync latest history first
+        reader   = HistoryReader()
+        reader.sync_history()
+        tsla     = reader.get_tsla_stats()
+        deposits = reader.get_deposit_stats()
+        reader.close()
+
+        # Pre-calculate everything needed
+        all_levels = s.all_levels()
+
+        # Daily profit rate
+        if tsla and tsla.get("avg90_daily", 0) > 0:
+            base_daily   = tsla["avg90_daily"]
+            recent_daily = tsla.get("recent_daily", base_daily)
+            daily_rate   = (recent_daily * 0.6) + (base_daily * 0.4)
+        else:
+            daily_rate = calc.get_queue_depth(s.equity) * config.TP_PROFIT_LARGE * 0.3
+
+        weekly_deposit = deposits.get("weekly_avg", 0.0)
+        daily_deposit  = weekly_deposit / 7.0
+        total_daily    = daily_rate + daily_deposit
+
+        # Survival
+        max_safe      = self._find_max_safe_pct(s, all_levels)
+        next_survival = self._next_survival_milestone(max_safe)
+
+        # Grid (survival-gated)
+        spacing = calc.get_spacing_pct(
+            s.equity, s.highest_open_price, all_levels, s.gbpusd
+        )
+        depth = calc.get_queue_depth(
+            s.equity, spacing, s.highest_open_price, all_levels, s.gbpusd
+        )
+
+        # Full 60% simulation result (uses current spacing)
+        full_sim_60 = calc._simulate_grid(
+            s.highest_open_price, spacing, s.gbpusd, all_levels
+        )
+        survival_60 = calc.survival_check(
+            s.equity, s.highest_open_price, full_sim_60, s.gbpusd,
+            override_pct=60.0
+        )
+
+        lines = []
+        lines.append("")
+        lines.append("=" * 60)
+        lines.append(
+            f"  CALIBRATION REPORT — "
+            f"{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
+        )
+        lines.append("=" * 60)
+
+        # ══════════════════════════════════════
+        # 1. ACCOUNT SNAPSHOT
+        # ══════════════════════════════════════
+        lines.append("")
+        lines.append("  1. ACCOUNT")
+        lines.append(f"  Equity:            £{s.equity:.2f}")
+        lines.append(f"  Available:         £{s.available:.2f}")
+        lines.append(f"  Margin in use:     £{s.total_margin_gbp:.2f}")
+        lines.append(f"  Margin level:      {s.margin_level_pct:.1f}%")
+        lines.append(f"  TSLA price:        ${s.current_price:.2f}")
+        lines.append(f"  Open positions:    {len(s.positions)}")
+        lines.append(f"  Pending orders:    {len(s.orders)}")
+        if deposits.get("total_deposited"):
+            lines.append(f"  Deposited (2024+): £{deposits['total_deposited']:.2f}")
+        if deposits.get("last_deposit"):
+            lines.append(f"  Last deposit:      {deposits['last_deposit']}")
+
+        # ══════════════════════════════════════
+        # 2. SURVIVAL  (calculated first — everything else depends on it)
+        # ══════════════════════════════════════
+        progress   = min(max_safe / 60.0, 1.0)
+        bar_filled = int(progress * 20)
+        surv_bar   = "█" * bar_filled + "░" * (20 - bar_filled)
+
+        lines.append("")
+        lines.append("  2. SURVIVAL  (target: 60% drop resistance)")
+        lines.append(f"  Can survive:       {max_safe:.0f}% drop")
+        lines.append(f"  Bot setting:       {config.SURVIVAL_DROP_PCT:.0f}%")
+        lines.append(f"  Progress to 60%:   [{surv_bar}] {max_safe:.0f}/60%")
+
+        # Auto-update logic — all in one place, clear and unambiguous
+        old_pct = config.SURVIVAL_DROP_PCT
+        if max_safe >= 60:
+            lines.append(f"  Status:            ✓ TARGET REACHED")
+            if config.SURVIVAL_DROP_PCT < 60:
+                self._update_survival_pct(60.0, lines)
+        elif max_safe > config.SURVIVAL_DROP_PCT:
+            new_pct = self._next_survival_milestone(config.SURVIVAL_DROP_PCT)
+            if new_pct <= max_safe:
+                self._update_survival_pct(new_pct, lines)
+                lines.append(
+                    f"  Status:            ↑ Updated {old_pct:.0f}% → "
+                    f"{config.SURVIVAL_DROP_PCT:.0f}%  (auto)"
+                )
+            else:
+                lines.append(
+                    f"  Status:            → Holding at {config.SURVIVAL_DROP_PCT:.0f}%"
+                )
+        else:
+            equity_needed = self._equity_for_survival(s, all_levels, next_survival)
+            gap = max(equity_needed - s.equity, 0)
+            lines.append(
+                f"  Status:            → Holding at {config.SURVIVAL_DROP_PCT:.0f}%"
+            )
+            lines.append(
+                f"  Next milestone:    {next_survival:.0f}% needs "
+                f"~£{equity_needed:.0f} equity"
+            )
+            if gap > 0 and daily_rate > 0:
+                days = gap / daily_rate
+                lines.append(
+                    f"  Est. time:         ~{days:.0f} days "
+                    f"(£{gap:.0f} at £{daily_rate:.2f}/day)"
+                )
+
+        # ══════════════════════════════════════
+        # 3. GRID  (survival-gated — spacing first, then depth)
+        # ══════════════════════════════════════
+        # ══════════════════════════════════════
+        # 3. GRID  (frozen until 60% survival achieved)
+        # Spacing tightens only when full 60% drop simulation passes.
+        # Depth increases only after spacing tightens AND 60% passes.
+        # ══════════════════════════════════════
+        lines.append("")
+        lines.append("  3. GRID  (locked to 60% survival — spacing first, depth second)")
+        lines.append(f"  Spacing:           {spacing}%  {'← frozen, 60% survival not yet met' if not survival_60['safe'] else '← active'}")
+        lines.append(f"  Queue depth:       {depth} orders")
+
+        # Next spacing upgrade
+        for threshold, sp in sorted(config.SPACING_TIERS):
+            if sp < spacing:
+                gap  = max(threshold - s.equity, 0)
+                days = gap / daily_rate if daily_rate > 0 else 0
+                sim  = calc._simulate_grid(
+                    s.highest_open_price, sp, s.gbpusd, all_levels
+                )
+                sv = calc.survival_check(
+                    s.equity, s.highest_open_price, sim, s.gbpusd,
+                    override_pct=60.0
+                )
+                sv_status = "✓ survival ok" if sv["safe"] else "✗ needs more equity"
+                if gap > 0:
+                    lines.append(
+                        f"  Spacing → {sp}%:     £{threshold} needed  "
+                        f"(£{gap:.0f} away ~{days:.0f}d)  {sv_status}"
+                    )
+                else:
+                    lines.append(
+                        f"  Spacing → {sp}%:     equity met  {sv_status}"
+                    )
+                break
+
+        # Next depth upgrade
+        for threshold, d, req_sp in sorted(config.QUEUE_TIERS):
+            if d > depth:
+                gap  = max(threshold - s.equity, 0)
+                days = gap / daily_rate if daily_rate > 0 else 0
+                if spacing > req_sp:
+                    lines.append(
+                        f"  Depth → {d} orders: spacing {req_sp}% required first"
+                    )
+                elif gap > 0:
+                    lines.append(
+                        f"  Depth → {d} orders: £{threshold} needed  "
+                        f"(£{gap:.0f} away ~{days:.0f}d)"
+                    )
+                else:
+                    lines.append(
+                        f"  Depth → {d} orders: equity met — survival check active"
+                    )
+                break
+
+        # ══════════════════════════════════════
+        # 4. PERFORMANCE  (the engine driving everything)
+        # ══════════════════════════════════════
+        lines.append("")
+        lines.append("  4. TSLA PERFORMANCE  (current strategy — from Jan 2024)")
+        if tsla:
+            lines.append(f"  Trades:            {tsla['trades']}")
+            lines.append(f"  Trade profit:      £{tsla['trade_profit']:.2f}")
+            lines.append(f"  Overnight fees:    £{tsla['swap_fees']:.2f}")
+            lines.append(f"  Net:               £{tsla['net']:.2f}")
+            lines.append(f"  Last 30d avg:      £{tsla.get('recent_daily', 0):.2f}/day")
+            lines.append(f"  Last 90d avg:      £{tsla.get('avg90_daily', 0):.2f}/day")
+            lines.append(f"  Blended rate:      £{daily_rate:.2f}/day  (used for projections)")
+
+            if tsla.get("monthly"):
+                lines.append("")
+                lines.append("  Monthly (last 6 months):")
+                for m in tsla["monthly"]:
+                    bar_len = min(int(m["profit"] / 2), 25)
+                    bar     = "█" * max(bar_len, 0)
+                    lines.append(
+                        f"  {m['month']}  "
+                        f"{m['trades']:3} trades  "
+                        f"£{m['profit']:6.2f}  {bar}"
+                    )
+
+        # ══════════════════════════════════════
+        # 5. MILESTONES  (projections based on all the above)
+        # ══════════════════════════════════════
+        lines.append("")
+        lines.append("  5. MILESTONES  (target: £1,000,000)")
+        lines.append(f"  Current equity:    £{s.equity:.2f}")
+
+        if weekly_deposit > 0:
+            lines.append(
+                f"  Weekly deposits:   £{weekly_deposit:.2f}/week  "
+                f"({deposits['pattern']})"
+            )
+            lines.append(
+                f"  Total rate:        £{total_daily:.2f}/day  "
+                f"(£{daily_rate:.2f} trading + £{daily_deposit:.2f} deposits)"
+            )
+        else:
+            lines.append(
+                f"  Weekly deposits:   none yet — "
+                f"auto-detects when regular deposits start"
+            )
+        lines.append("")
+
+        for label, target in self.MILESTONES:
+            gap      = target - s.equity
+            progress = min(s.equity / target, 1.0)
+            filled   = int(progress * 20)
+            bar      = "█" * filled + "░" * (20 - filled)
+            pct      = progress * 100
+
+            if s.equity >= target:
+                lines.append(f"  {label:12} [{bar}] ✓ REACHED")
+            else:
+                lines.append(f"  {label:12} [{bar}] {pct:.1f}%")
+                if total_daily > 0:
+                    days  = gap / total_daily
+                    years = days / 365
+                    if days < 365:
+                        eta = f"~{days:.0f} days"
+                    elif years < 2:
+                        eta = f"~{years:.1f} years"
+                    else:
+                        eta = f"~{years:.0f} years"
+                    rate_str = (
+                        f"£{total_daily:.2f}/day"
+                        if weekly_deposit > 0
+                        else f"£{daily_rate:.2f}/day"
+                    )
+                    lines.append(
+                        f"  {'':12}   £{gap:,.0f} needed  →  {eta} at {rate_str}"
+                    )
+
+        lines.append("")
+        lines.append("=" * 60)
+        lines.append("")
+
+        for line in lines:
+            log.info(line)
+
+    # ─────────────────────────────────────────
+    #  HELPERS
+    # ─────────────────────────────────────────
+
+    def _update_survival_pct(self, new_pct: float, lines: list):
+        """
+        Automatically update SURVIVAL_DROP_PCT in config.py.
+        Does a safe in-place string replacement so all other
+        config values and comments are preserved.
+        """
+        try:
+            config_path = os.path.join(
+                os.path.dirname(os.path.abspath(__file__)), "config.py"
+            )
+            with open(config_path, "r") as f:
+                content = f.read()
+
+            old_line = f"SURVIVAL_DROP_PCT = {config.SURVIVAL_DROP_PCT:.1f}"
+            new_line = f"SURVIVAL_DROP_PCT = {new_pct:.1f}"
+
+            if old_line not in content:
+                # Try without decimal
+                old_line = f"SURVIVAL_DROP_PCT = {int(config.SURVIVAL_DROP_PCT)}"
+
+            if old_line in content:
+                new_content = content.replace(old_line, new_line, 1)
+                with open(config_path, "w") as f:
+                    f.write(new_content)
+                # Update in-memory config so rest of this run uses new value
+                config.SURVIVAL_DROP_PCT = new_pct
+                lines.append(
+                    f"  ✓ config.py updated: SURVIVAL_DROP_PCT = {new_pct:.1f}"
+                )
+                log.warning(
+                    f"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"
+                    f"  ↑  SURVIVAL THRESHOLD AUTO-UPDATED: {new_pct:.0f}%\n"
+                    f"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+                )
+            else:
+                lines.append(
+                    f"  ⚠ Could not auto-update config.py — "
+                    f"update manually: SURVIVAL_DROP_PCT = {new_pct:.1f}"
+                )
+        except Exception as e:
+            lines.append(f"  ⚠ Auto-update failed: {e}")
+            lines.append(
+                f"  Update manually in config.py: "
+                f"SURVIVAL_DROP_PCT = {new_pct:.1f}"
+            )
+
+    def _find_max_safe_pct(self, s: BotState, all_levels: list) -> float:
+        """
+        Find maximum drop % the account can currently survive.
+
+        IMPORTANT: Does NOT just use current orders — simulates the FULL grid
+        that would exist at each drop level. In a real crash, as price falls
+        the bot keeps placing orders on the way down. All of those fill.
+        So we simulate all grid levels from current price down to the floor,
+        not just the 10 orders currently queued.
+        """
+        if not s.highest_open_price:
+            return 0.0
+
+        max_safe = 0.0
+        for pct in range(10, 81, 5):
+            floor        = s.highest_open_price * (1.0 - pct / 100.0)
+            spacing      = self.calculator.get_spacing_pct(s.equity) / 100.0
+
+            # Build full simulated grid from current price down to floor
+            # These are all the orders that would fill in a crash of this size
+            simulated_levels = []
+
+            # Existing open positions always fill
+            for p in s.positions:
+                simulated_levels.append({
+                    "price": pos_level(p),
+                    "size":  pos_size(p)
+                })
+
+            # Simulate all grid levels that would be placed between
+            # current price and the floor
+            level = s.current_price * (1.0 - spacing)
+            while level >= floor:
+                size = self.calculator.get_size(level, s.highest_open_price)
+                simulated_levels.append({"price": level, "size": size})
+                level = level * (1.0 - spacing)
+
+            result = self.calculator.survival_check(
+                s.equity, s.highest_open_price,
+                simulated_levels, s.gbpusd,
+                override_pct=float(pct)
+            )
+            if result["safe"]:
+                max_safe = float(pct)
+            else:
+                return max_safe if pct > 10 else 0.0
+
+        return 80.0  # survived all tested levels
+
+    def _next_survival_milestone(self, current_pct: float) -> float:
+        for m in self.SURVIVAL_MILESTONES:
+            if m > current_pct:
+                return float(m)
+        return 60.0
+
+    def _equity_for_survival(self, s: BotState, all_levels: list,
+                              target_pct: float) -> float:
+        """
+        Estimate equity needed to survive target_pct% drop.
+        Uses full simulated grid — all levels that would fill in a crash.
+        Formula: equity_needed = 0.5 * total_margin - total_unrealised_loss
+        (equity must be above 50% of margin to avoid Capital.com closeout)
+        """
+        try:
+            spacing = self.calculator.get_spacing_pct(s.equity)
+            full_sim = self.calculator._simulate_grid(
+                s.highest_open_price, spacing, s.gbpusd, all_levels
+            )
+            drop  = target_pct / 100.0
+            floor = s.highest_open_price * (1.0 - drop)
+
+            total_margin = sum(
+                lv["price"] * lv["size"] * config.MARGIN_RATE / s.gbpusd
+                for lv in full_sim
+            )
+            total_loss = sum(
+                (floor - lv["price"]) * lv["size"] / s.gbpusd
+                for lv in full_sim
+            )
+            # equity_needed + total_loss >= 0.5 * total_margin
+            # equity_needed >= 0.5 * total_margin - total_loss
+            return max((0.5 * total_margin) - total_loss, 0)
+        except Exception:
+            return 0.0

client.py

@@ -0,0 +1,200 @@
+"""
+client.py — Capital.com REST API client.
+=========================================
+Handles all HTTP communication with Capital.com.
+Session management, authentication, and all API calls live here.
+
+API NOTES:
+----------
+- Authentication: POST /session with API key + credentials
+  Returns CST and X-SECURITY-TOKEN headers, valid for 10 minutes.
+- All subsequent requests need CST + X-SECURITY-TOKEN headers.
+- Rate limit: 1 request per 0.1s for order operations.
+- Working orders use field names: workingOrderData.orderLevel, orderSize, dealId
+- Positions use field names: position.level, position.size, position.profitLevel
+- Account balance: accounts[].balance.balance (equity), .deposit (funds), .available
+- Price snapshot: snapshot.bid, snapshot.offer (mid = average of both)
+
+FIELD NAME REFERENCE (verified against live API):
+--------------------------------------------------
+Working orders response:
+  o["workingOrderData"]["dealId"]       — order ID
+  o["workingOrderData"]["orderLevel"]   — entry price
+  o["workingOrderData"]["orderSize"]    — position size in shares
+  o["workingOrderData"]["profitDistance"] — TP distance in points
+
+Positions response:
+  p["position"]["dealId"]       — position ID
+  p["position"]["level"]        — entry price
+  p["position"]["size"]         — position size in shares
+  p["position"]["profitLevel"]  — take profit price (None if not set)
+  p["position"]["upl"]          — unrealised P&L in USD
+
+Accounts response:
+  acc["balance"]["balance"]     — current equity (GBP)
+  acc["balance"]["deposit"]     — deposited funds (GBP)
+  acc["balance"]["available"]   — available to trade (GBP)
+"""
+
+import time
+import logging
+from typing import Optional
+import requests
+import config
+
+log = logging.getLogger("maxbot.client")
+
+
+class CapitalClient:
+    """
+    Wraps all Capital.com API calls.
+    Handles session creation, token refresh, and error handling.
+    """
+
+    def __init__(self):
+        self.base = config.DEMO_URL if config.USE_DEMO else config.BASE_URL
+        self.cst            = None
+        self.security_token = None
+        self.session_started = None
+
+    # ─────────────────────────────────────────
+    #  SESSION MANAGEMENT
+    # ─────────────────────────────────────────
+
+    def create_session(self):
+        """Authenticate and store session tokens."""
+        url = f"{self.base}/api/v1/session"
+        headers = {
+            "Content-Type": "application/json",
+            "X-CAP-API-KEY": config.API_KEY,
+        }
+        body = {
+            "identifier": config.IDENTIFIER,
+            "password":   config.PASSWORD,
+        }
+        resp = requests.post(url, headers=headers, json=body, timeout=15)
+        resp.raise_for_status()
+        self.cst             = resp.headers.get("CST")
+        self.security_token  = resp.headers.get("X-SECURITY-TOKEN")
+        self.session_started = time.time()
+        log.info("Session created successfully.")
+
+    def refresh_session_if_needed(self):
+        """Refresh session if tokens are approaching expiry (10 min lifetime)."""
+        if self.session_started is None:
+            self.create_session()
+            return
+        age = time.time() - self.session_started
+        if age >= config.SESSION_REFRESH_SECS:
+            log.info("Refreshing session tokens...")
+            self.create_session()
+
+    # ─────────────────────────────────────────
+    #  HTTP HELPERS
+    # ─────────────────────────────────────────
+
+    def _headers(self) -> dict:
+        h = {
+            "Content-Type":    "application/json",
+            "X-CAP-API-KEY":   config.API_KEY,
+        }
+        if self.cst:
+            h["CST"] = self.cst
+        if self.security_token:
+            h["X-SECURITY-TOKEN"] = self.security_token
+        return h
+
+    def _request(self, method: str, path: str, body: dict = None) -> dict:
+        """Make an authenticated API request. Auto-retries once on 401."""
+        url = f"{self.base}/api/v1{path}"
+        try:
+            resp = requests.request(
+                method, url,
+                headers=self._headers(),
+                json=body,
+                timeout=15
+            )
+            if resp.status_code == 401:
+                log.warning("401 received — refreshing session and retrying...")
+                self.create_session()
+                resp = requests.request(
+                    method, url,
+                    headers=self._headers(),
+                    json=body,
+                    timeout=15
+                )
+            resp.raise_for_status()
+            return resp.json() if resp.text else {}
+        except requests.RequestException as e:
+            log.error(f"API error: {method} {path} — {e}")
+            raise
+
+    # ─────────────────────────────────────────
+    #  DATA FETCHERS
+    # ─────────────────────────────────────────
+
+    def get_positions(self) -> list:
+        """Return list of all open positions."""
+        return self._request("GET", "/positions").get("positions", [])
+
+    def get_working_orders(self) -> list:
+        """Return list of all pending working orders."""
+        return self._request("GET", "/workingorders").get("workingOrders", [])
+
+    def get_account_details(self) -> dict:
+        """Return account details including balance."""
+        return self._request("GET", "/accounts")
+
+    def get_price(self, epic: str) -> dict:
+        """Return market snapshot for an epic (bid, offer, status etc)."""
+        return self._request("GET", f"/markets/{epic}")
+
+    def get_gbpusd(self) -> float:
+        """Fetch live GBP/USD mid rate. Falls back to config value on failure."""
+        try:
+            data = self._request("GET", "/markets/GBPUSD")
+            snap = data.get("snapshot", {})
+            return (float(snap["bid"]) + float(snap["offer"])) / 2.0
+        except Exception:
+            log.warning(f"GBP/USD fetch failed — using fallback {config.GBPUSD_FALLBACK}")
+            return config.GBPUSD_FALLBACK
+
+    # ─────────────────────────────────────────
+    #  ORDER OPERATIONS
+    # ─────────────────────────────────────────
+
+    def place_limit_order(self, size: float, limit_price: float,
+                          take_profit: float) -> dict:
+        """
+        Place a limit buy order for TSLA.
+        Order fills automatically when price reaches limit_price.
+        take_profit is the absolute price level (not distance).
+        """
+        body = {
+            "epic":           config.EPIC,
+            "direction":      "BUY",
+            "size":           size,
+            "level":          round(limit_price, 2),
+            "type":           "LIMIT",
+            "guaranteedStop": False,
+            "profitLevel":    round(take_profit, 2),
+        }
+        return self._request("POST", "/workingorders", body)
+
+    def cancel_order(self, deal_id: str) -> dict:
+        """Cancel a pending working order by deal ID."""
+        return self._request("DELETE", f"/workingorders/{deal_id}")
+
+    def update_position_tp(self, deal_id: str,
+                           take_profit: Optional[float]) -> dict:
+        """
+        Update the take profit on an open position.
+        Pass take_profit=None to remove the TP (manual close only).
+        Used for the bottom-two position rule:
+          - Lowest open position: TP removed (held until manual close)
+          - Second lowest: TP set to standard level
+        """
+        body = {
+            "profitLevel": round(take_profit, 2) if take_profit is not None else None
+        }
+        return self._request("PUT", f"/positions/{deal_id}", body)

config.py

@@ -0,0 +1,183 @@
+"""
+config.py — All bot settings in one place.
+==========================================
+This is the ONLY file you need to edit for day-to-day configuration.
+All other files import from here.
+
+STRATEGY SUMMARY:
+-----------------
+This bot trades TSLA CFDs on Capital.com using a grid/mean-reversion strategy.
+It places limit buy orders at regular intervals BELOW the current price.
+Each order has a take profit set to generate a fixed GBP profit.
+As equity grows, the grid tightens (more orders = more frequent fills = more profit).
+
+SURVIVAL RULE:
+--------------
+The bot calculates whether the account can survive a X% drop from the highest
+open position without being margin called. If not safe, no new orders are placed.
+Start at 30% and increase as equity grows:
+  £640  → 30%  survival (current)
+  £900  → 35%
+  £1200 → 40%
+  £1500 → 50%
+  £1800 → 60%  (target)
+
+SIZE RULE:
+----------
+0.2 shares for all orders (more profit per fill, faster recovery at lower prices).
+0.1 shares only within top 5% of highest open price (expensive to hold overnight).
+threshold = highest_open * 0.95
+
+PROFIT TARGETS:
+---------------
+0.2 shares → £1.00 profit per position (same price travel as 0.1)
+0.1 shares → £0.50 profit per position (half size = half profit, same distance)
+Take profit distance = profit_gbp * gbpusd / size
+
+QUEUE MANAGEMENT:
+-----------------
+Bot maintains exactly QUEUE_DEPTH pending orders covering the grid window.
+Window = from just below lowest open position down to survival floor.
+On each loop, bot scans for gaps in the grid and fills them.
+It does NOT just add to the bottom — it fills gaps anywhere in the window.
+
+MARGIN STAGES (Capital.com UK retail):
+---------------------------------------
+> 100%  → Normal operation
+= 100%  → Warning 1: bot stops placing new orders
+= 75%   → Warning 2: urgent alert
+= 50%   → Auto closeout: Capital.com closes positions automatically
+
+MODES:
+------
+dryrun  → connects, reads everything, prints what it would do. ZERO actions.
+confirm → asks y/n before every single action. Start here.
+live    → fully autonomous. Only use after confirming behaviour in other modes.
+"""
+
+import os
+
+# ─────────────────────────────────────────────
+#  CREDENTIALS
+#  Set these as environment variables, not hardcoded.
+#  export CAPITAL_API_KEY="your_key"
+#  export CAPITAL_IDENTIFIER="your@email.com"
+#  export CAPITAL_PASSWORD="your_password"
+# ─────────────────────────────────────────────
+
+API_KEY    = os.environ.get("CAPITAL_API_KEY",    "YOUR_API_KEY_HERE")
+IDENTIFIER = os.environ.get("CAPITAL_IDENTIFIER", "your@email.com")
+PASSWORD   = os.environ.get("CAPITAL_PASSWORD",   "YOUR_PASSWORD_HERE")
+
+# ─────────────────────────────────────────────
+#  API ENDPOINTS
+# ─────────────────────────────────────────────
+
+BASE_URL = "https://api-capital.backend-capital.com"
+DEMO_URL = "https://demo-api-capital.backend-capital.com"
+USE_DEMO = False   # Set True to point at demo account for testing
+
+# ─────────────────────────────────────────────
+#  INSTRUMENT
+# ─────────────────────────────────────────────
+
+EPIC = "TSLA"   # Capital.com epic for Tesla Inc CFD
+
+# ─────────────────────────────────────────────
+#  GRID STRATEGY
+# ─────────────────────────────────────────────
+
+# % spacing between grid levels. Tightens automatically as equity grows.
+# At 1.5%: $414.99 → $408.77 → $402.63 → $396.59 etc.
+BASE_SPACING_PCT = 1.5
+
+# Number of pending orders to maintain in the grid window at all times.
+# Grows with equity — see QUEUE_TIERS below.
+QUEUE_DEPTH = 10
+
+# Bot will not place orders if simulated X% drop from highest open position
+# would result in margin closeout. Increase this as equity grows.
+# Current safe value for £640 equity = 30%. Target = 60%.
+SURVIVAL_DROP_PCT = 40.0
+
+# Capital.com retail margin rate for shares CFDs (5:1 leverage = 20% margin)
+MARGIN_RATE = 0.20
+
+# Stop placing NEW orders if current margin level drops below this %.
+# Capital.com warning stage 1 = 100%. We stop here before it gets worse.
+MARGIN_STOP_PCT = 100.0
+
+# Alert and pause if price moves more than this % between loops.
+# Protects against gap-down opens (e.g. Monday open after weekend news).
+GAP_ALERT_PCT = 20.0
+
+# Size threshold: orders within top 5% of highest open price use 0.1 shares.
+# Everything else uses 0.2 shares.
+# Example: highest open = $450 → threshold = $427.50 → above = 0.1, below = 0.2
+SIZE_SMALL_THRESHOLD_PCT = 0.95  # top 5% = 0.1 shares
+
+# ─────────────────────────────────────────────
+#  PROFIT TARGETS (GBP)
+# ─────────────────────────────────────────────
+
+TP_PROFIT_LARGE = 1.00   # £1.00 target for 0.2 share positions
+TP_PROFIT_SMALL = 0.50   # £0.50 target for 0.1 share positions
+# Both use the same price travel distance. 0.1 shares just earns half as much.
+
+# ─────────────────────────────────────────────
+#  GBP/USD
+# ─────────────────────────────────────────────
+
+# Bot fetches live GBP/USD rate each loop. Falls back to this if fetch fails.
+GBPUSD_FALLBACK = 1.27
+
+# ─────────────────────────────────────────────
+#  EQUITY GROWTH TIERS
+#  IMPORTANT — ORDER OF OPERATIONS:
+#  1. Survival check first (always)
+#  2. Spacing tightens second (more frequent fills)
+#  3. Queue depth increases third (only after spacing already tightened)
+#  Never both spacing and depth upgrade in the same calibration run.
+#
+#  Each tier upgrade is SURVIVAL-GATED:
+#  Before applying an upgrade, the full grid is simulated at the new
+#  settings. Only if survival check still passes does the upgrade apply.
+# ─────────────────────────────────────────────
+
+# Grid spacing tightens with equity.
+# Each tier is only applied if survival check passes at new spacing.
+# Spacing must tighten BEFORE queue depth can increase.
+SPACING_TIERS = [
+    (600,  1.5),   # base — current
+    (800,  1.2),   # tightens at £800 IF survival passes
+    (1100, 1.0),   # tightens at £1100 IF survival passes
+    (1500, 0.8),   # tightens at £1500 IF survival passes
+]
+
+# Queue depth increases with equity.
+# Each tier is only applied if:
+#   a) Spacing has already tightened to the corresponding level
+#   b) Survival check passes at new depth
+# Depth at tier N requires spacing to be at tier N-1 already.
+QUEUE_TIERS = [
+    (600,  10, 1.5),   # 10 orders at 1.5% spacing — base
+    (900,  12, 1.2),   # 12 orders — requires spacing already at 1.2%
+    (1200, 15, 1.0),   # 15 orders — requires spacing already at 1.0%
+    (1500, 18, 0.8),   # 18 orders — requires spacing already at 0.8%
+]
+# Queue tier format: (equity_threshold, depth, required_spacing)
+
+# ─────────────────────────────────────────────
+#  DEPOSITS
+#  Set your planned weekly deposit amount here.
+#  Used by calibration report for milestone projections.
+#  Set to 0 if not making regular deposits.
+# ─────────────────────────────────────────────
+
+WEEKLY_DEPOSIT_GBP = 0.0   # £ per week — update when you start depositing
+
+# Capital.com session tokens expire after 10 minutes. Refresh at 9 minutes.
+SESSION_REFRESH_SECS = 540
+
+# How often the main monitoring loop runs.
+LOOP_INTERVAL_SECS = 60

fetch_history.py

@@ -0,0 +1,491 @@
+#!/usr/bin/env python3
+"""
+fetch_history.py — Capital.com full transaction history fetcher.
+================================================================
+Fetches ALL account transactions from Capital.com and stores them
+in a local SQLite database at data/history.db.
+
+ENDPOINT USED:
+--------------
+GET /history/transactions
+Much richer than /history/activity — returns actual GBP amounts,
+deposits, trades, overnight fees, all in one place.
+
+MODES:
+------
+  python3 fetch_history.py --mode dryrun    # preview only, no changes
+  python3 fetch_history.py --mode confirm   # asks before starting bulk fetch
+  python3 fetch_history.py --mode live      # fully automatic
+
+FIRST RUN:
+----------
+Fetches from 28 Jan 2021 (account open date) to today.
+~1,900 API calls at 1/second = ~32 minutes.
+Progress shown every 10 days. Safe to interrupt — resumes from
+last successful day on next run.
+
+SUBSEQUENT RUNS:
+----------------
+Only fetches since last successful fetch date. Runs in seconds.
+Called automatically from calibration report on every bot startup.
+
+DATABASE: data/history.db
+--------------------------
+Table: transactions
+  id               TEXT PRIMARY KEY  (reference field from Capital.com)
+  date_utc         TEXT              (ISO datetime)
+  transaction_type TEXT              (TRADE, SWAP, DEPOSIT, WITHDRAWAL,
+                                      TRADE_SLIPPAGE_PROTECTION)
+  note             TEXT              (e.g. "Trade closed", "Overnight fee")
+  instrument       TEXT              (e.g. "TSLA", "US100", empty for deposits)
+  size_gbp         REAL              (GBP amount — positive=credit, negative=cost)
+  currency         TEXT              (always GBP for this account)
+  status           TEXT              (PROCESSED)
+  raw              TEXT              (full JSON for future use)
+
+Table: fetch_log
+  fetch_date       TEXT PRIMARY KEY  (YYYY-MM-DD)
+  fetched_at       TEXT              (when we fetched it)
+  record_count     INTEGER           (records found that day)
+  status           TEXT              (ok / error)
+
+FINANCIAL LOGIC:
+----------------
+  TRADE size       → always positive (GBP profit from closed position)
+  SWAP size        → always negative (overnight fee cost)
+  DEPOSIT size     → always positive (money added to account)
+  WITHDRAWAL size  → negative (money removed)
+  Net performance  = sum(TRADE) + sum(SWAP)
+  Net deposited    = sum(DEPOSIT) + sum(WITHDRAWAL)
+  Return on capital = net_performance / net_deposited * 100
+
+FUTURE USE:
+-----------
+  Calibration report  → real profit/day, real fees, real deposits
+  ProQuant backtester → full trade history with entry/exit prices
+  Web dashboard       → P&L charts, fee analysis, milestone tracking
+"""
+
+import argparse
+import json
+import logging
+import os
+import sqlite3
+import sys
+import time
+from datetime import datetime, timedelta, date, timezone
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import config
+from client import CapitalClient
+
+# ─────────────────────────────────────────────
+#  CONSTANTS
+# ─────────────────────────────────────────────
+
+ACCOUNT_OPEN_DATE = date(2021, 1, 28)
+DB_PATH           = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "data", "history.db"
+)
+LOG_PATH          = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "logs", "fetch_history.log"
+)
+FETCH_DELAY_SECS  = 1.1   # slightly over 1s to respect rate limits
+
+
+# ─────────────────────────────────────────────
+#  LOGGING
+# ─────────────────────────────────────────────
+
+def setup_logging():
+    os.makedirs(os.path.dirname(LOG_PATH), exist_ok=True)
+    fmt = "%(asctime)s [%(levelname)s] %(message)s"
+    logging.basicConfig(
+        level=logging.INFO,
+        format=fmt,
+        handlers=[
+            logging.FileHandler(LOG_PATH),
+            logging.StreamHandler(sys.stdout),
+        ]
+    )
+
+log = logging.getLogger("fetch_history")
+
+
+# ─────────────────────────────────────────────
+#  DATABASE
+# ─────────────────────────────────────────────
+
+class HistoryDB:
+
+    def __init__(self, db_path: str):
+        os.makedirs(os.path.dirname(db_path), exist_ok=True)
+        self.conn = sqlite3.connect(db_path)
+        self.conn.row_factory = sqlite3.Row
+        self._create_tables()
+
+    def _create_tables(self):
+        self.conn.executescript("""
+            CREATE TABLE IF NOT EXISTS transactions (
+                id               TEXT PRIMARY KEY,
+                date_utc         TEXT NOT NULL,
+                transaction_type TEXT,
+                note             TEXT,
+                instrument       TEXT,
+                size_gbp         REAL,
+                currency         TEXT,
+                status           TEXT,
+                raw              TEXT
+            );
+
+            CREATE INDEX IF NOT EXISTS idx_tx_date
+                ON transactions (date_utc);
+            CREATE INDEX IF NOT EXISTS idx_tx_type
+                ON transactions (transaction_type);
+            CREATE INDEX IF NOT EXISTS idx_tx_instrument
+                ON transactions (instrument);
+
+            CREATE TABLE IF NOT EXISTS fetch_log (
+                fetch_date   TEXT PRIMARY KEY,
+                fetched_at   TEXT NOT NULL,
+                record_count INTEGER DEFAULT 0,
+                status       TEXT DEFAULT 'ok'
+            );
+        """)
+        self.conn.commit()
+
+    def date_already_fetched(self, fetch_date: date) -> bool:
+        row = self.conn.execute(
+            "SELECT status FROM fetch_log WHERE fetch_date = ? AND status = 'ok'",
+            (fetch_date.isoformat(),)
+        ).fetchone()
+        return row is not None
+
+    def get_last_fetched_date(self):
+        row = self.conn.execute(
+            "SELECT MAX(fetch_date) as d FROM fetch_log WHERE status = 'ok'"
+        ).fetchone()
+        if row and row["d"]:
+            return date.fromisoformat(row["d"])
+        return None
+
+    def save_transactions(self, transactions: list) -> int:
+        """Insert transaction records. Returns number actually inserted."""
+        inserted = 0
+        for t in transactions:
+            try:
+                ref = t.get("reference", "")
+                # Build a unique ID from reference + date to avoid collisions
+                tx_id = f"{ref}_{t.get('dateUtc', t.get('date', ''))}"
+
+                self.conn.execute("""
+                    INSERT OR IGNORE INTO transactions
+                    (id, date_utc, transaction_type, note, instrument,
+                     size_gbp, currency, status, raw)
+                    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+                """, (
+                    tx_id,
+                    t.get("dateUtc", t.get("date", "")),
+                    t.get("transactionType", ""),
+                    t.get("note", ""),
+                    t.get("instrumentName", ""),
+                    float(t.get("size", 0)),
+                    t.get("currency", "GBP"),
+                    t.get("status", ""),
+                    json.dumps(t),
+                ))
+                if self.conn.execute("SELECT changes()").fetchone()[0] > 0:
+                    inserted += 1
+            except Exception as e:
+                log.warning(f"Could not insert transaction: {e} — {t}")
+
+        self.conn.commit()
+        return inserted
+
+    def log_fetch(self, fetch_date: date, record_count: int, status: str = "ok"):
+        self.conn.execute("""
+            INSERT OR REPLACE INTO fetch_log (fetch_date, fetched_at, record_count, status)
+            VALUES (?, ?, ?, ?)
+        """, (
+            fetch_date.isoformat(),
+            datetime.now(tz=timezone.utc).isoformat(),
+            record_count,
+            status,
+        ))
+        self.conn.commit()
+
+    def get_stats(self) -> dict:
+        """Return summary statistics from the database."""
+        def scalar(sql, default=0):
+            row = self.conn.execute(sql).fetchone()
+            return row[0] if row and row[0] is not None else default
+
+        total      = scalar("SELECT COUNT(*) FROM transactions")
+        trades     = scalar("SELECT COUNT(*) FROM transactions WHERE transaction_type='TRADE'")
+        swaps      = scalar("SELECT COUNT(*) FROM transactions WHERE transaction_type='SWAP'")
+        deposits   = scalar("SELECT COUNT(*) FROM transactions WHERE transaction_type='DEPOSIT'")
+        other      = total - trades - swaps - deposits
+        fetched_days = scalar("SELECT COUNT(*) FROM fetch_log WHERE status='ok'")
+        first_date = scalar("SELECT MIN(date_utc) FROM transactions", None)
+        last_date  = scalar("SELECT MAX(date_utc) FROM transactions", None)
+
+        trade_profit  = scalar("SELECT SUM(size_gbp) FROM transactions WHERE transaction_type='TRADE'")
+        swap_fees     = scalar("SELECT SUM(size_gbp) FROM transactions WHERE transaction_type='SWAP'")
+        total_deposits = scalar("SELECT SUM(size_gbp) FROM transactions WHERE transaction_type='DEPOSIT'")
+        slippage      = scalar("SELECT SUM(size_gbp) FROM transactions WHERE transaction_type='TRADE_SLIPPAGE_PROTECTION'")
+
+        # Per-instrument breakdown
+        instruments = {}
+        for row in self.conn.execute("""
+            SELECT instrument, COUNT(*) as c, SUM(size_gbp) as total
+            FROM transactions
+            WHERE transaction_type='TRADE' AND instrument != ''
+            GROUP BY instrument
+            ORDER BY total DESC
+        """):
+            instruments[row["instrument"]] = {
+                "trades": row["c"],
+                "profit": row["total"],
+            }
+
+        # Daily profit average (only days with trades)
+        daily_avg_row = self.conn.execute("""
+            SELECT AVG(daily_total) FROM (
+                SELECT DATE(date_utc) as d, SUM(size_gbp) as daily_total
+                FROM transactions
+                WHERE transaction_type='TRADE'
+                GROUP BY DATE(date_utc)
+                HAVING daily_total > 0
+            )
+        """).fetchone()
+        daily_avg = daily_avg_row[0] if daily_avg_row and daily_avg_row[0] else 0
+
+        return {
+            "total_records":    total,
+            "trades":           trades,
+            "swaps":            swaps,
+            "deposits":         deposits,
+            "other":            other,
+            "fetched_days":     fetched_days,
+            "first_date":       first_date,
+            "last_date":        last_date,
+            "trade_profit":     trade_profit,
+            "swap_fees":        swap_fees,
+            "total_deposits":   total_deposits,
+            "slippage_refunds": slippage,
+            "net_performance":  trade_profit + swap_fees + slippage,
+            "instruments":      instruments,
+            "daily_avg_profit": daily_avg,
+        }
+
+    def close(self):
+        self.conn.close()
+
+
+# ─────────────────────────────────────────────
+#  FETCHER
+# ─────────────────────────────────────────────
+
+class HistoryFetcher:
+
+    def __init__(self, mode: str):
+        self.mode   = mode
+        self.client = CapitalClient()
+        self.db     = HistoryDB(DB_PATH)
+
+    def fetch_day(self, fetch_date: date):
+        """
+        Fetch all transactions for a single day.
+        Returns list of records, or None on API error.
+        Capital.com max date range = 1 day.
+        """
+        from_dt = f"{fetch_date.isoformat()}T00:00:00"
+        to_dt   = f"{fetch_date.isoformat()}T23:59:59"
+        try:
+            data = self.client._request(
+                "GET",
+                f"/history/transactions?from={from_dt}&to={to_dt}"
+            )
+            return data.get("transactions", [])
+        except Exception as e:
+            log.error(f"Failed to fetch {fetch_date}: {e}")
+            return None
+
+    def print_stats(self):
+        """Print current database statistics."""
+        stats = self.db.get_stats()
+        log.info("")
+        log.info("  DATABASE SUMMARY")
+        log.info(f"  Total records:      {stats['total_records']}")
+        log.info(f"  Closed trades:      {stats['trades']}")
+        log.info(f"  Overnight fees:     {stats['swaps']}")
+        log.info(f"  Deposits:           {stats['deposits']}")
+        log.info(f"  Other:              {stats['other']}")
+        log.info(f"  Days fetched:       {stats['fetched_days']}")
+        if stats['first_date']:
+            log.info(f"  Date range:         {stats['first_date'][:10]} → {stats['last_date'][:10]}")
+        log.info("")
+        log.info("  FINANCIAL SUMMARY")
+        log.info(f"  Total deposited:    £{stats['total_deposits']:.2f}")
+        log.info(f"  Trade profit:       £{stats['trade_profit']:.2f}")
+        log.info(f"  Overnight fees:     £{stats['swap_fees']:.2f}")
+        log.info(f"  Slippage refunds:   £{stats['slippage_refunds']:.2f}")
+        log.info(f"  Net performance:    £{stats['net_performance']:.2f}")
+        if stats['total_deposits'] > 0:
+            roi = stats['net_performance'] / stats['total_deposits'] * 100
+            log.info(f"  Return on capital:  {roi:.1f}%")
+        if stats['daily_avg_profit'] > 0:
+            log.info(f"  Avg profit/day:     £{stats['daily_avg_profit']:.2f} (days with trades)")
+        log.info("")
+        log.info("  BY INSTRUMENT")
+        for instrument, data in stats['instruments'].items():
+            log.info(f"  {instrument:12} {data['trades']:4} trades  £{data['profit']:.2f}")
+
+    def run(self):
+        log.info("=" * 60)
+        log.info(f"  History Fetcher — {self.mode.upper()} MODE")
+        log.info(f"  Started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+        log.info("=" * 60)
+
+        log.info("Connecting to Capital.com...")
+        try:
+            self.client.create_session()
+        except Exception as e:
+            log.error(f"Connection failed: {e}")
+            sys.exit(1)
+        log.info("Connected.")
+
+        # Determine start date
+        today = date.today()
+        last  = self.db.get_last_fetched_date()
+        if last:
+            start = last + timedelta(days=1)
+            log.info(f"Resuming from {start} (last fetched: {last})")
+        else:
+            start = ACCOUNT_OPEN_DATE
+            log.info(f"First run — fetching from {start}")
+
+        total_days     = max((today - start).days + 1, 0)
+        estimated_mins = total_days * FETCH_DELAY_SECS / 60
+
+        # Show current DB state
+        existing_stats = self.db.get_stats()
+        if existing_stats["total_records"] > 0:
+            log.info(
+                f"Database has {existing_stats['total_records']} records "
+                f"({existing_stats['fetched_days']} days already fetched)"
+            )
+
+        log.info(f"Days to fetch:  {total_days}")
+        log.info(f"Est. time:      {estimated_mins:.0f} minutes")
+
+        if self.mode == "dryrun":
+            log.info("\n[DRY RUN] No data will be fetched or stored.")
+            self.print_stats()
+            self.db.close()
+            return
+
+        if total_days == 0:
+            log.info("Already up to date.")
+            self.print_stats()
+            self.db.close()
+            return
+
+        # Confirm in confirm mode
+        if self.mode == "confirm":
+            print(f"\n  Fetch {total_days} days from {start} to {today}")
+            print(f"  Estimated time: {estimated_mins:.0f} minutes")
+            answer = input("  Start? (y/n): ").strip().lower()
+            if answer != "y":
+                log.info("Aborted.")
+                self.db.close()
+                return
+
+        # ── Main fetch loop ──
+        current        = start
+        total_inserted = 0
+        days_done      = 0
+        days_empty     = 0
+
+        log.info("\nFetching...")
+
+        while current <= today:
+            # Skip already fetched
+            if self.db.date_already_fetched(current):
+                current += timedelta(days=1)
+                continue
+
+            # Progress every 10 days
+            if days_done % 10 == 0 and days_done > 0:
+                pct = (days_done / max(total_days, 1)) * 100
+                log.info(
+                    f"  {days_done}/{total_days} days ({pct:.0f}%) — "
+                    f"{total_inserted} records inserted"
+                )
+
+            transactions = self.fetch_day(current)
+
+            if transactions is None:
+                self.db.log_fetch(current, 0, "error")
+                current += timedelta(days=1)
+                days_done += 1
+                time.sleep(FETCH_DELAY_SECS)
+                continue
+
+            count = len(transactions)
+            if count > 0:
+                inserted = self.db.save_transactions(transactions)
+                total_inserted += inserted
+            else:
+                days_empty += 1
+
+            self.db.log_fetch(current, count)
+            days_done += 1
+            current   += timedelta(days=1)
+            time.sleep(FETCH_DELAY_SECS)
+
+        # ── Final summary ──
+        log.info("\n" + "=" * 60)
+        log.info("  FETCH COMPLETE")
+        log.info("=" * 60)
+        log.info(f"  Days fetched:   {days_done}")
+        log.info(f"  Days empty:     {days_empty}")
+        log.info(f"  Records added:  {total_inserted}")
+        self.print_stats()
+        self.db.close()
+
+
+# ─────────────────────────────────────────────
+#  ENTRY POINT
+# ─────────────────────────────────────────────
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Capital.com Transaction History Fetcher",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Modes:
+  dryrun   Show current DB state. No API calls beyond login. No writes.
+  confirm  Ask before starting bulk fetch. Use for first run.
+  live     Fully automatic. Use for incremental/scheduled runs.
+
+To start fresh (recommended after switching from old activity endpoint):
+  rm ~/maxbot/data/history.db
+  python3 fetch_history.py --mode confirm
+
+Database: data/history.db
+Log:      logs/fetch_history.log
+        """
+    )
+    parser.add_argument(
+        "--mode",
+        choices=["dryrun", "confirm", "live"],
+        default="dryrun",
+    )
+    args = parser.parse_args()
+    setup_logging()
+    HistoryFetcher(mode=args.mode).run()
+
+
+if __name__ == "__main__":
+    main()

grid.py

@@ -0,0 +1,334 @@
+"""
+grid.py — Core grid management logic.
+======================================
+This is the brain of the bot. Every loop it:
+  1. Manages the bottom two open positions (TP rules)
+  2. Audits the order grid for gaps and fills them
+  3. Cancels orders that fall outside the active window
+
+THE GAP-FILLING FIX (key improvement over v1):
+-----------------------------------------------
+Old behaviour: bot anchored from lowest existing order and added to bottom.
+Problem: if top order filled or was cancelled, gap was never filled at top.
+Result: grid drifted downward over time, missing fills near current price.
+
+New behaviour:
+  1. Calculate ALL expected grid levels from top (below lowest position)
+     down to bottom (survival floor).
+  2. Compare expected levels against ALL existing orders.
+  3. Place orders for any missing level, anywhere in the window.
+  4. Result: gaps are filled wherever they occur, not just at the bottom.
+
+BOTTOM TWO POSITION RULE:
+--------------------------
+Lowest open position  → TP DISABLED (hold until manual close)
+Second lowest         → TP ENABLED at standard level
+
+Rationale: the lowest position represents a deep dip buy.
+We don't know when price will recover to take profit from there.
+Holding it open means maximum profit on recovery.
+When a new even-lower position opens, the previous lowest gets its TP
+re-enabled because recovery to that level is now more likely.
+
+ORDER WINDOW:
+-------------
+Top of window    = just below lowest open position entry price
+Bottom of window = highest_open * (1 - survival_drop_pct / 100)
+Orders outside this window are cancelled.
+Orders above window_top are too close to open positions (confusing).
+Orders below floor are beyond our risk tolerance.
+"""
+
+import logging
+import time
+from state import BotState, pos_level, pos_size, pos_id, pos_tp, ord_level, ord_id
+from calculator import GridCalculator
+from actions import ActionHandler
+import config
+
+log = logging.getLogger("maxbot.grid")
+
+
+class GridManager:
+
+    def __init__(self, state: BotState, calculator: GridCalculator,
+                 actions: ActionHandler):
+        self.state      = state
+        self.calculator = calc = calculator
+        self.actions    = actions
+        # Track deal IDs we have already attempted to disable TP on.
+        # Capital.com may not support removing profitLevel via API,
+        # so we avoid spamming the same call every loop.
+        self._tp_disabled_ids: set = set()
+
+    # ─────────────────────────────────────────
+    #  BOTTOM TWO POSITIONS
+    # ─────────────────────────────────────────
+
+    def manage_bottom_two_positions(self):
+        """
+        Enforce TP rules on the two lowest open positions.
+        Only activates when price has dropped 25%+ from highest open position.
+
+        RATIONALE:
+        If price is near the top (within 25% of highest open), positions are
+        likely to recover quickly and the standard TP will close them normally.
+        The lowest/second-lowest TP rule is only meaningful when price has
+        dropped significantly and we want to hold the lowest position for
+        maximum recovery rather than a small TP.
+        """
+        s          = self.state
+        sorted_pos = s.positions_sorted_asc()
+
+        if not sorted_pos:
+            log.info("No open positions.")
+            return
+
+        # ── Check 25% drop threshold ──
+        threshold_price = s.highest_open_price * 0.75
+        if s.current_price > threshold_price:
+            log.info(
+                f"Price ${s.current_price:.2f} is within 25% of highest open "
+                f"${s.highest_open_price:.2f} (threshold ${threshold_price:.2f}) "
+                f"— TP rules not active."
+            )
+            return
+
+        # ── Lowest position: TP must be DISABLED ──
+        lowest  = sorted_pos[0]
+        l_id    = pos_id(lowest)
+        l_price = pos_level(lowest)
+        l_tp    = pos_tp(lowest)
+
+        if l_id in self._tp_disabled_ids:
+            log.info(f"Lowest position ${l_price:.2f} — TP disable already attempted. ✓")
+        elif l_tp is not None:
+            log.info(
+                f"Lowest position ${l_price:.2f} has TP ${l_tp:.2f} — disabling."
+            )
+            self.actions.disable_tp(l_id, l_price)
+            self._tp_disabled_ids.add(l_id)
+        else:
+            log.info(f"Lowest position ${l_price:.2f} — TP disabled. ✓")
+            self._tp_disabled_ids.add(l_id)
+
+        # Clear IDs that are no longer the lowest position
+        # (position closed or new lower opened)
+        current_ids = {pos_id(p) for p in sorted_pos}
+        self._tp_disabled_ids &= current_ids
+
+        # ── Second lowest position: TP must be ENABLED ──
+        if len(sorted_pos) < 2:
+            return
+
+        second  = sorted_pos[1]
+        s_id    = pos_id(second)
+        s_price = pos_level(second)
+        s_tp    = pos_tp(second)
+        s_size  = pos_size(second)
+
+        expected_tp = self.calculator.get_tp_price(s_price, s_size, s.gbpusd)
+
+        if s_tp is None:
+            log.info(
+                f"Second lowest ${s_price:.2f} has no TP — "
+                f"enabling at ${expected_tp:.2f}."
+            )
+            self.actions.enable_tp(s_id, s_price, expected_tp)
+        else:
+            log.info(
+                f"Second lowest ${s_price:.2f} — TP at ${s_tp:.2f}. ✓"
+            )
+
+    # ─────────────────────────────────────────
+    #  ORDER GRID MANAGEMENT
+    # ─────────────────────────────────────────
+
+    def manage_orders(self):
+        """
+        Main order management function. Called every loop.
+
+        NORMAL MODE (price within 25% of highest open):
+        - Anchor = lowest open position
+        - Grid fills below anchor down to floor
+        - depth orders maintained
+
+        DIP MODE (price dropped 25%+ from highest open):
+        - Lowest open position has TP removed (manual hold) — ignored as anchor
+        - Anchor = second lowest open position (lowest with TP)
+        - Grid fills below anchor down to floor
+        - ALSO fills gaps between no-TP position and anchor if spacing fits
+        - No orders placed at or above anchor
+        - No orders placed at or above the no-TP position level
+
+        Grid floor is always highest_open * 0.40 (60% drop), fixed.
+        """
+        s      = self.state
+        calc   = self.calculator
+        equity = s.equity
+
+        # Get survival-gated spacing and depth
+        # Spacing first, then depth (depth requires spacing already tightened)
+        spacing_pct = calc.get_spacing_pct(
+            equity, s.highest_open_price, s.all_levels(), s.gbpusd
+        )
+        depth = calc.get_queue_depth(
+            equity, spacing_pct, s.highest_open_price,
+            s.all_levels(), s.gbpusd
+        )
+
+        GRID_FLOOR_PCT = 60.0
+        floor = s.highest_open_price * (1.0 - GRID_FLOOR_PCT / 100.0)
+
+        sorted_pos  = s.positions_sorted_asc()
+        open_prices = [pos_level(p) for p in sorted_pos]
+
+        # ── Determine mode and anchor ──
+        dip_threshold  = s.highest_open_price * 0.75  # 25% drop
+        in_dip_mode    = s.current_price <= dip_threshold
+        no_tp_price    = None  # price of the no-TP (manual hold) position
+
+        if not open_prices:
+            anchor = s.current_price
+            log.info(f"No open positions — anchoring to current price ${anchor:.2f}")
+        elif in_dip_mode and len(sorted_pos) >= 2:
+            # Dip mode: anchor to second lowest (lowest with TP)
+            no_tp_price = pos_level(sorted_pos[0])   # lowest — no TP
+            anchor      = pos_level(sorted_pos[1])   # second lowest — has TP
+            log.info(
+                f"DIP MODE — anchor: ${anchor:.2f} (second lowest, has TP) | "
+                f"no-TP position: ${no_tp_price:.2f} (manual hold)"
+            )
+        else:
+            # Normal mode: anchor to lowest open position
+            anchor = min(open_prices)
+            log.info(f"NORMAL MODE — anchor: ${anchor:.2f} (lowest open position)")
+
+        log.info(
+            f"Floor: ${floor:.2f} | depth: {depth} | "
+            f"spacing: {spacing_pct}% (survival-gated)"
+        )
+
+        # ── Generate target slots ──
+        # Main grid: below anchor down to floor
+        main_levels = calc.generate_grid(anchor, floor, equity, spacing_pct)
+        if not main_levels:
+            log.info("No grid levels fit within current window.")
+            return
+
+        # In dip mode: also generate levels in the gap between
+        # no-TP position and anchor (if gap is large enough)
+        gap_levels = []
+        if in_dip_mode and no_tp_price is not None:
+            spacing = spacing_pct / 100.0
+            # Generate levels from just below anchor down to just above no-TP
+            level = anchor * (1.0 - spacing)
+            while level > no_tp_price * (1.0 + spacing):
+                gap_levels.append(round(level, 2))
+                level = level * (1.0 - spacing)
+            if gap_levels:
+                log.info(
+                    f"Gap levels between ${no_tp_price:.2f} and "
+                    f"${anchor:.2f}: {[f'${l:.2f}' for l in gap_levels]}"
+                )
+
+        # Combined target: gap levels + main grid, capped at depth
+        all_target_levels = (gap_levels + main_levels)
+        target_slots = all_target_levels[:depth]
+
+        if target_slots:
+            log.info(
+                f"Target slots: ${target_slots[-1]:.2f} → ${target_slots[0]:.2f}"
+            )
+
+        # ── Scan existing orders — cancel any not matching targets ──
+        current_orders  = {round(ord_level(o), 2): o for o in s.orders}
+        existing_prices = set(current_orders.keys())
+        orders_to_cancel = []
+
+        for price in sorted(existing_prices):
+            # Never keep orders at or above anchor
+            if price >= anchor:
+                orders_to_cancel.append(price)
+                continue
+            # In dip mode: never keep orders at or above no-TP position
+            if no_tp_price and price >= no_tp_price:
+                orders_to_cancel.append(price)
+                continue
+            # Cancel if not matching any target slot
+            matches = any(
+                abs(price - slot) / slot < 0.005
+                for slot in target_slots
+            )
+            if not matches:
+                orders_to_cancel.append(price)
+
+        if orders_to_cancel:
+            log.info(
+                f"Cancelling {len(orders_to_cancel)} order(s) "
+                f"not matching grid: "
+                f"{[f'${p:.2f}' for p in orders_to_cancel]}"
+            )
+            for price in orders_to_cancel:
+                oid = ord_id(current_orders[price])
+                self.actions.cancel_order(oid, price)
+                existing_prices.discard(price)
+                time.sleep(0.15)
+
+        # ── Find and fill gaps ──
+        missing = calc.find_missing_levels(
+            target_slots, existing_prices, depth
+        )
+
+        if not missing:
+            log.info(
+                f"Grid healthy — {len(existing_prices)}/{depth} "
+                f"orders in place."
+            )
+            return
+
+        log.info(
+            f"Filling {len(missing)} gap(s): "
+            f"{[f'${p:.2f}' for p in missing]}"
+        )
+
+        for target_price in missing:
+            # Safety: never place at or above current price
+            if target_price >= s.current_price:
+                log.info(f"Skipping ${target_price:.2f} — at/above current price.")
+                continue
+
+            # Safety: never place at or above anchor
+            if target_price >= anchor:
+                log.info(f"Skipping ${target_price:.2f} — at/above anchor ${anchor:.2f}.")
+                continue
+
+            # Safety: never place at or above no-TP position in dip mode
+            if no_tp_price and target_price >= no_tp_price:
+                log.info(f"Skipping ${target_price:.2f} — at/above no-TP position.")
+                continue
+
+            # Survival check — simulate FULL grid at 60% drop
+            # (not just current orders — honest worst case)
+            full_sim = calc._simulate_grid(
+                s.highest_open_price, spacing_pct, s.gbpusd,
+                s.all_levels()
+            )
+            survival = calc.survival_check(
+                s.equity, s.highest_open_price,
+                full_sim, s.gbpusd,
+                override_pct=60.0
+            )
+            if not survival["safe"]:
+                log.warning(
+                    f"Survival check failed — stopping. "
+                    f"Margin at floor: {survival['margin_level_at_floor_pct']:.1f}%"
+                )
+                break
+
+            size     = calc.get_size(target_price, s.highest_open_price)
+            tp_price = calc.get_tp_price(target_price, size, s.gbpusd)
+
+            self.actions.place_limit_order(size, target_price, tp_price)
+            existing_prices.add(target_price)
+            time.sleep(0.15)

state.py

@@ -0,0 +1,153 @@
+"""
+state.py — Account state and position/order field helpers.
+===========================================================
+Holds the current snapshot of the account, positions, and orders.
+Updated every loop from the API. All other modules read from here.
+
+FIELD ACCESS HELPERS:
+---------------------
+Capital.com uses nested dicts with non-obvious field names.
+Rather than repeating o["workingOrderData"]["orderLevel"] everywhere,
+use the helper functions at the bottom of this file:
+  pos_level(p)  → float entry price of a position
+  pos_size(p)   → float size in shares
+  pos_id(p)     → deal ID string
+  pos_tp(p)     → take profit price (float) or None if not set
+  pos_upl(p)    → unrealised P&L in USD (float)
+  ord_level(o)  → float entry price of a working order
+  ord_size(o)   → float size in shares
+  ord_id(o)     → deal ID string
+"""
+
+import logging
+import config
+
+log = logging.getLogger("maxbot.state")
+
+
+# ─────────────────────────────────────────────
+#  FIELD ACCESS HELPERS
+#  Use these everywhere instead of raw dict access.
+#  If Capital.com ever changes field names, fix it here only.
+# ─────────────────────────────────────────────
+
+def pos_level(p) -> float:
+    return float(p["position"]["level"])
+
+def pos_size(p) -> float:
+    return float(p["position"]["size"])
+
+def pos_id(p) -> str:
+    return p["position"]["dealId"]
+
+def pos_tp(p):
+    """Returns take profit price as float, or None if not set."""
+    tp = p["position"].get("profitLevel")
+    return float(tp) if tp is not None else None
+
+def pos_upl(p) -> float:
+    """Unrealised P&L in USD."""
+    return float(p["position"].get("upl", 0))
+
+def ord_level(o) -> float:
+    return float(o["workingOrderData"]["orderLevel"])
+
+def ord_size(o) -> float:
+    return float(o["workingOrderData"]["orderSize"])
+
+def ord_id(o) -> str:
+    return o["workingOrderData"]["dealId"]
+
+
+# ─────────────────────────────────────────────
+#  BOT STATE
+# ─────────────────────────────────────────────
+
+class BotState:
+    """
+    Snapshot of the account updated every loop.
+    All values are recalculated from scratch each time fetch() is called.
+    """
+
+    def __init__(self):
+        self.positions:    list  = []
+        self.orders:       list  = []
+        self.equity:       float = 0.0   # running equity in GBP
+        self.funds:        float = 0.0   # deposited funds in GBP
+        self.available:    float = 0.0   # available to trade in GBP
+        self.current_price: float = 0.0  # TSLA mid price in USD
+        self.gbpusd:       float = config.GBPUSD_FALLBACK
+        self.highest_open_price: float = 0.0  # highest open position entry
+        self.lowest_level_price: float = 0.0  # lowest of positions OR orders
+        self.total_margin_gbp:   float = 0.0  # estimated margin in use
+        self.margin_level_pct:   float = 999.0
+        self.last_price:   float = None  # previous loop price (gap detection)
+
+    def update(self, positions, orders, account, price_data, gbpusd):
+        """Recalculate all state from fresh API data."""
+        self.positions = positions
+        self.orders    = orders
+        self.gbpusd    = gbpusd
+
+        # Account balance
+        for acc in account.get("accounts", []):
+            if acc.get("preferred"):
+                bal = acc.get("balance", {})
+                self.equity    = float(bal.get("balance",   0))
+                self.funds     = float(bal.get("deposit",   0))
+                self.available = float(bal.get("available", 0))
+                break
+
+        # Current TSLA mid price
+        snap = price_data.get("snapshot", {})
+        self.current_price = (
+            float(snap.get("bid",   0)) +
+            float(snap.get("offer", 0))
+        ) / 2.0
+
+        # Highest open position entry price
+        open_prices = [pos_level(p) for p in positions]
+        self.highest_open_price = (
+            max(open_prices) if open_prices else self.current_price
+        )
+
+        # Lowest level across all positions and orders
+        order_prices = [ord_level(o) for o in orders]
+        all_prices   = open_prices + order_prices
+        self.lowest_level_price = (
+            min(all_prices) if all_prices else self.current_price
+        )
+
+        # Approximate total margin in use (GBP)
+        # Formula: entry_price * size * margin_rate / gbpusd
+        self.total_margin_gbp = sum(
+            pos_level(p) * pos_size(p) * config.MARGIN_RATE / gbpusd
+            for p in positions
+        )
+
+        # Margin level % = equity / margin * 100
+        self.margin_level_pct = (
+            (self.equity / self.total_margin_gbp * 100)
+            if self.total_margin_gbp > 0 else 999.0
+        )
+
+    def positions_sorted_asc(self) -> list:
+        """Open positions sorted by entry price, lowest first."""
+        return sorted(self.positions, key=pos_level)
+
+    def orders_sorted_asc(self) -> list:
+        """Pending orders sorted by level, lowest first."""
+        return sorted(self.orders, key=ord_level)
+
+    def all_levels(self) -> list:
+        """
+        Combined list of all open positions and pending orders
+        as dicts with 'price' and 'size' keys.
+        Used by the survival check calculator.
+        """
+        levels = []
+        for p in self.positions:
+            levels.append({"price": pos_level(p), "size": pos_size(p)})
+        for o in self.orders:
+            levels.append({"price": ord_level(o), "size": ord_size(o)})
+        return levels