BisonDB (Document Database Engine in C++20)

Problem

Every backend engineer uses databases; almost none have built one. BisonDB exists to close that gap the hard way: a MongoDB-inspired document database written in C++20 with no third-party storage or networking libraries: no Asio, no RocksDB, not even std::map standing in for an index. Every layer that production databases hide behind an API is implemented by hand here: the BSON codec, the on-disk pages, the index structure, the query planner, the wire protocol, and the server loop. The full system is documented at the BisonDB docs site, with architecture deep-dives and downloads.

Architecture

The core (bisondb_core) provides a BSON value model with a hardened decoder/encoder, MongoDB Extended JSON v2 reading and writing, an append-only collection store, and a hand-written on-disk B+Tree. On top sit the query engine with index-aware planning, a portable socket layer written directly against Winsock2/POSIX, the bisond daemon, the bisonc CLI, and the bisonsh interactive shell. Prairie (a separate Tauri 2 + React desktop app in the style of MongoDB Compass) browses documents, runs filters with explain plans, manages indexes, imports/exports data, and can open local databases directly by bundling bisond as a sidecar process.

Tech decisions & trade-offs

Why zero third-party storage and networking libraries

The constraint is the curriculum. Pulling in RocksDB or Asio would make BisonDB a wrapper; refusing them forces the real problems to the surface: page layout, split/merge logic in the B+Tree, partial reads on sockets, malformed input on the wire. The decoder is hardened precisely because nothing else stands between untrusted bytes and the process. The trade-off is obvious and accepted: months of work reproducing what a dependency gives you in an afternoon, traded for actually understanding the layer underneath every database conversation.

Why an append-only store with B+Tree indexes on the side

Append-only collections make writes simple and crash behavior predictable: data is never overwritten in place, so a torn write can't corrupt existing documents. Point and range lookups are then the index's job: a hand-written on-disk B+Tree maps key → document location. The planner makes the division of labor measurable with explain plans: the same range query over the 29,470-document zips fixture examines all 29,470 documents as a scan, and exactly 1,015 after create-index pop. The planner switches to an index_range plan and re-checks the full filter on every fetched document, so an index can accelerate a query but never change its answer.

Why the wire protocol is a documented contract

bisond speaks one length-prefixed BSON document per message, and the protocol document specifies the framing, command set, and error codes completely enough to write a third-party client without reading the source. Limits are part of the contract too: find responses cap at 16 MiB, and larger result sets return truncated: true with a skipNext cursor-substitute that the bundled client follows automatically. Protocols rot when they live only in the implementation; writing it down forced every edge case to have a defined answer. It also made the wire protocol versionable, which mattered the moment authentication arrived (below).

Why authentication and TLS arrived as a hardened, deliberate layer

The first release shipped with no auth and no transport security: an honest scope for a storage-engine exercise, and labelled as such. Making BisonDB network-safe was its own project, added in two tightly-scoped releases rather than bolted on.

Authentication (v1.1.0) is mandatory: every connection authenticates before any data command runs, gated by three roles (`read`, `readWrite`, and `admin`) through capability checks. Credentials live in a hidden `__auth.bsd` system file, with passwords Argon2id-hashed (memory-hard, via Monocypher) under per-user salts. A login issues a 256-bit session token, but only its BLAKE2b-256 hash is ever held in memory, so a leaked process dump yields nothing replayable. Bootstrapping sidesteps the chicken-and-egg with `--init-admin` (or an env var, or offline creation), and an anti-lockout guard refuses to drop the last admin. The wire protocol bumped to v2 to carry `authenticate`, `createUser`, `changePassword`, `listUsers`, and friends, exactly the versioned-contract payoff from writing the protocol down first.

TLS (v1.2.0) closed the gap the auth release deliberately left open: until then, credentials still crossed the wire in clear text, and the changelog says so plainly. Transport is now TLS 1.2 (ECDHE + AES-GCM) via a vendored Mbed-TLS 3.6: `bisond --tls` with a cert/key pair or a self-signed option, and clients (`bisonsh`, `bisonc`) opt in with `--tls`, pin a certificate via `--tls-pin`, or trust a private CA with `--tls-ca`. Private keys are never logged, and a plaintext-vs-TLS mismatch fails fast instead of silently degrading. The encryption here is in transit: there is no at-rest document encryption, and the docs don't pretend otherwise. Honest scope, still.

Why bisonc round-trips byte-for-byte

The CLI converts BSON dumps (including mongodump output) to Extended JSON and back. In canonical mode the round-trip is lossless to the byte: to-json --canonical followed by to-bson reproduces the original file exactly. That property doubles as a continuous fuzz test of both the encoder and decoder: any asymmetry between them breaks a user-visible guarantee, not a hidden invariant.

Why Prairie is a Tauri 2 sidecar app

A database without a browser UI gets evaluated by its terminal output. Prairie gives BisonDB the Compass experience: visual document browsing, filter building with live explain plans, index management. Because Tauri bundles bisond as a sidecar binary, Prairie can open a local database directory with zero setup: no daemon to start, no port to configure. The web-tech UI (React) kept iteration fast, while the sidecar pattern keeps the client honest: Prairie talks to the same TCP protocol as every other client, with no private backdoor into the storage layer.

See it run

Prairie browsing the loaded zips collection, with 29,473 documents, a live filter bar, and one-click Explain plans:

Play video