Engineering
Engineering · 2025
MastiDB
An on-disk OLAP database engine built from scratch in Python, with columnar storage, a query planner and execution engine, and vectorised scans.
Why I built it
I wanted to understand databases from the inside — not as a user of Postgres or Druid, but as someone who has to make the storage, planning, and execution choices himself. MastiDB is that learning engine: an on-disk OLAP system built from scratch so the abstractions stop being magical.
How it works
The path of a query is deliberately simple and readable:
- Storage — data lands in columnar, on-disk layouts chosen for analytical scans rather than row-oriented lookups.
- Planning — a query planner turns a request into an executable plan, choosing how to read and combine data.
- Execution — an execution engine walks that plan, pulling batches through operators.
- Scans — vectorised scans move through columns in chunks, keeping the hot path tight and inspectable.
Each stage is small enough to read, change, and reason about — which is the point of building it this way.