← all writing

Data Platforms

A data platform is a promise you keep at 3am

You don’t choose a data stack. You make a promise, and then you choose the tools that let you keep it.

The promise is small to state and enormous to keep: when someone looks at a number, it means what they think it means. Not approximately. Not usually. At 3am, when the on-call phone goes off and a board deck is due at nine.

The unglamorous middle

Pipelines, contracts, schemas, lineage: the plumbing. Nobody asks to see the plumbing. They ask to see the dashboard, and the dashboard is only as honest as the pipe behind it.

select count(*) as rows, max(updated_at) as freshest
from analytics.fct_orders
where loaded_at >= current_date - 1;

If that query ever returns zero, you want to find out before the dashboard does.

This is a placeholder post. Swap in the real war stories.

Filed under Data Platforms