v0.2.0 — 2026-08-17
nqbook stopped being a smoke test and became a service: it now takes a live
feed and persists everything it books.
- Three threads, two queues.
Pipeline.hdeclares the stage contracts — feed, book, writer — and joins each pair with onenlib::single_queue(SPSC), so the process runs lock-free end to end. A full queue makes the producer wait rather than drop, so backpressure from storage reaches ZMQ. - The feed is a ZMQ SUB socket. Framing is one tag byte followed by the
record in host layout; anything that matches no tag-plus-size is dropped.
order::prev/order::nextarrive as wire bytes and are discarded — the hooks are book-owned state. - Parquet persistence. The writer keeps one sink per record type, buffers
1024 rows into Arrow builders, and writes each batch as one zstd-compressed
row group. Files land in
data_out/, stamped with the writer start time, with the Arrow schema embedded so readers restore the fixed-size lists inbook. A storage error aborts the process; running on without persistence would lose data silently. - Snapshots are on the wall clock, every 3 s, whether or not the book saw an event.
- Shutdown drains. SIGINT or SIGTERM stops the stages upstream-first — feed, then book, then writer — and each drains its input before returning, so every received record reaches a file.
- Configuration is
nqbook [feed_endpoint] [out_dir], with the endpoint falling back toNQBOOK_FEED_ENDPOINTand then totcp://host.docker.internal:5555. - The build needs Arrow, Parquet, and cppzmq, which only the
devcontainer carries — the host can no longer configure this project. - nlib bumped to
84ee2cc:order::sideandtrade::sideare now declarednlib::side, without which this repository does not compile under GCC 13.
Comments
Sign in to join the discussion.
No comments yet. Be the first.