SQLite in MQL5: Designing a Local Tick & Feature Ledger

Why SQLite inside an EA

Spreadsheet exports and ad-hoc CSV dumps do not scale once you ingest ticks, signals, engineered features, and execution slippage together. SQLite gives you ACID semantics in a portable file beneath the MetaTrader Data Folder — invaluable for journaling what the strategy actually saw, not merely what Strategy Tester rewrote retroactively.

Schema philosophy

Split layers:

• RAW immutable bars/ticks keyed by canonical timestamp and symbol surrogate id.


• FEATURES generated only from information available strictly before decision time \(t\) (no lookahead).


• EXECUTION records linking requested price, deviation, latency proxy, fill type.

Index composite keys thoughtfully; brute-force sequential scans inside OnTick negate the point.

Durability knobs

Understand journal vs WAL. Write-Ahead Logging can smooth writer contention but interacts with VPS disk quotas. Wrap bulk inserts inside explicit transactions (BEGIN IMMEDIATE pattern) — otherwise autocommit will thrash disk.

Failure modes & recovery

Assume power loss mid-transaction; design UPSERT uniqueness constraints that encode broker microstructure (microsecond collision keys if available). Periodic offline vacuum chores keep files lean on long-lived robots.

Finvestopia context

We treat observability as a first-class deliverable. A local SQLite discipline on the client mirrors how our stack retains append-only market logs for audit — you cannot improve what you cannot replay faithfully.