The first working stack: make up brings up storage, catalog, and both
clients, and make smoke writes and reads an Iceberg table end to end.
- The stack — MinIO for objects, Postgres 17 for catalog metadata,
Lakekeeper v0.13.1 as the Iceberg REST catalog, DuckDB v1.5.5 as the query
engine, wired by one compose file. Images are pinned in the parent repository
under
Dockerfiles/lakehouse-*. - Credential vending, not shared secrets. Lakekeeper hands DuckDB
short-lived MinIO STS credentials per table access, so no client holds an S3
secret.
minio-initcreates a dedicatedlakekeeperuser for this because MinIO refusesAssumeRolefor root credentials. - Idempotent bring-up.
make upre-runs provisioning and bootstrap on every invocation and converges, so it is safe to repeat against a live stack. - One command surface, two front ends.
scripts/console/nqlake.pyprints tables on a TTY and JSON when piped, so the same commands serve humans, scripts, and the Next.js console's API routes. loadinfers schemas. CSV/TSV/Parquet/JSON(L), optionally gzipped; the first load creates the namespace and table, later loads append by column name,--replacerebuilds.- Data lives in
images/as bind mounts and survivesmake down. Onlymake cleandeletes it.
Comments
Sign in to join the discussion.
No comments yet. Be the first.