Architecture · Diagram Design
Open data lake · End-to-end stack
Open data lake · End-to-end stack
Data lake architecture showing app servers and databases flowing through Apache NiFi into MinIO, then Trino, StarRocks, Superset, JupyterLab, and Python.
SOURCES
INGEST
DATA LAKE
QUERY
CONSUME
SQL
WRITE
EXT
App servers
Events · logs
EXT
Databases
CDC · exports
FLOW
Apache NiFi
Route · transform
ORCH
Airflow
DAG scheduling
STORE
MinIO
Object store · S3-API
QUERY
Trino
SQL · any format
OLAP
StarRocks
MPP · hot layer
BI
Superset
Dashboards
NB
JupyterLab
Exploration
PROC
Python
Batch · ML
LEGEND
MinIO data lake (focal)
Ingest · query tools
BI · notebooks · pipelines
Write-back path (batch)
Primary query path