BisonDB (Document Database Engine in C++20)

Problem

Every backend engineer uses databases; almost none have built one. BisonDB exists to close that gap the hard way: a MongoDB-inspired document database written in C++20 with no third-party storage or networking libraries — no Asio, no RocksDB, not even std::map standing in for an index. Every layer that production databases hide behind an API is implemented by hand here: the BSON codec, the on-disk pages, the index structure, the query planner, the wire protocol, and the server loop. The full system is documented at the BisonDB docs site, with architecture deep-dives and downloads.

Architecture

The core (bisondb_core) provides a BSON value model with a hardened decoder/encoder, MongoDB Extended JSON v2 reading and writing, an append-only collection store, and a hand-written on-disk B+Tree. On top sit the query engine with index-aware planning, a portable socket layer written directly against Winsock2/POSIX, the bisond daemon, the bisonc CLI, and the bisonsh interactive shell. Prairie — a separate Tauri 2 + React desktop app in the style of MongoDB Compass — browses documents, runs filters with explain plans, manages indexes, imports/exports data, and can open local databases directly by bundling bisond as a sidecar process.

Tech decisions & trade-offs

Why zero third-party storage and networking libraries

The constraint is the curriculum. Pulling in RocksDB or Asio would make BisonDB a wrapper; refusing them forces the real problems to the surface — page layout, split/merge logic in the B+Tree, partial reads on sockets, malformed input on the wire. The decoder is hardened precisely because nothing else stands between untrusted bytes and the process. The trade-off is obvious and accepted: months of work reproducing what a dependency gives you in an afternoon, traded for actually understanding the layer underneath every database conversation.

Why an append-only store with B+Tree indexes on the side

Append-only collections make writes simple and crash behavior predictable — data is never overwritten in place, so a torn write can't corrupt existing documents. Point and range lookups are then the index's job: a hand-written on-disk B+Tree maps key → document location. The planner makes the division of labor measurable with explain plans: the same range query over the 29,470-document zips fixture examines all 29,470 documents as a scan, and exactly 1,015 after create-index pop — the planner switches to an index_range plan and re-checks the full filter on every fetched document, so an index can accelerate a query but never change its answer.

Why the wire protocol is a documented contract

bisond speaks one length-prefixed BSON document per message, and the protocol document specifies the framing, command set, and error codes completely enough to write a third-party client without reading the source. Limits are part of the contract too: find responses cap at 16 MiB, and larger result sets return truncated: true with a skipNext cursor-substitute that the bundled client follows automatically. Protocols rot when they live only in the implementation; writing it down forced every edge case to have a defined answer — and made the wire protocol versionable, which mattered the moment authentication arrived (below).

Why authentication and TLS arrived as a hardened, deliberate layer

The first release shipped with no auth and no transport security — an honest scope for a storage-engine exercise, and labelled as such. Making BisonDB network-safe was its own project, added in two tightly-scoped releases rather than bolted on.

Authentication (v1.1.0) is mandatory: every connection authenticates before any data command runs, gated by three roles — `read`, `readWrite`, and `admin` — through capability checks. Credentials live in a hidden `__auth.bsd` system file, with passwords Argon2id-hashed (memory-hard, via Monocypher) under per-user salts. A login issues a 256-bit session token, but only its BLAKE2b-256 hash is ever held in memory, so a leaked process dump yields nothing replayable. Bootstrapping sidesteps the chicken-and-egg with `--init-admin` (or an env var, or offline creation), and an anti-lockout guard refuses to drop the last admin. The wire protocol bumped to v2 to carry `authenticate`, `createUser`, `changePassword`, `listUsers`, and friends — exactly the versioned-contract payoff from writing the protocol down first.

TLS (v1.2.0) closed the gap the auth release deliberately left open: until then, credentials still crossed the wire in clear text, and the changelog says so plainly. Transport is now TLS 1.2 (ECDHE + AES-GCM) via a vendored Mbed-TLS 3.6 — `bisond --tls` with a cert/key pair or a self-signed option, and clients (`bisonsh`, `bisonc`) opt in with `--tls`, pin a certificate via `--tls-pin`, or trust a private CA with `--tls-ca`. Private keys are never logged, and a plaintext-vs-TLS mismatch fails fast instead of silently degrading. The encryption here is in transit — there is no at-rest document encryption, and the docs don't pretend otherwise. Honest scope, still.

Why bisonc round-trips byte-for-byte

The CLI converts BSON dumps (including mongodump output) to Extended JSON and back. In canonical mode the round-trip is lossless to the byte: to-json --canonical followed by to-bson reproduces the original file exactly. That property doubles as a continuous fuzz test of both the encoder and decoder — any asymmetry between them breaks a user-visible guarantee, not a hidden invariant.

Why Prairie is a Tauri 2 sidecar app

A database without a browser UI gets evaluated by its terminal output. Prairie gives BisonDB the Compass experience — visual document browsing, filter building with live explain plans, index management — and because Tauri bundles bisond as a sidecar binary, Prairie can open a local database directory with zero setup: no daemon to start, no port to configure. The web-tech UI (React) kept iteration fast, while the sidecar pattern keeps the client honest — Prairie talks to the same TCP protocol as every other client, with no private backdoor into the storage layer.

See it run

Prairie browsing the loaded zips collection — 29,473 documents, a live filter bar, and one-click Explain plans:

Play video