Anika Verma
Data Analyst | SQL, Power BI, Python | Turning messy data into decisions
Had 40 nearly identical measures (MTD, QTD, YTD × 13 metrics). Calculation groups replaced them with 13 measures + 1 group. Much easier to maintain.
Pipelines, warehouses, dbt, Spark and data quality
173 members 7 postsTechnology
Data engineers and analysts talking about Spark, Airflow, dbt, warehouses, streaming and data quality. Share your stack, your war stories and your SQL.
Anika Verma
Data Analyst | SQL, Power BI, Python | Turning messy data into decisions
Had 40 nearly identical measures (MTD, QTD, YTD × 13 metrics). Calculation groups replaced them with 13 measures + 1 group. Much easier to maintain.
Reyansh Iyer
Data Engineer | Spark, Airflow, dbt, Snowflake | Pipelines that don't break on Mondays
• Row count vs yesterday (alert on >30% change) • Null rate on key columns • Uniqueness of primary keys • Freshness (max updated_at) • Referential checks for critical joins
Priya Banerjee
Data Scientist | NLP, forecasting, Python | Ex-researcher | Women in Tech mentor
We have 4 data scientists and ~20 features reused across models. Is a feature store overkill?
Nikhil Chowdhury
Backend Engineer | Python, Django, Postgres | Fintech | Remote-first
Our ledger_entries table hit 300M rows. Monthly range partitioning on created_at + making sure every query includes a date filter = partition pruning does the rest. Gotchas: • Unique constraints must include the partition key • Create future partitions ahead of time (we use pg_partman) • Check EXPLAIN to confirm
Anika Verma
Data Analyst | SQL, Power BI, Python | Turning messy data into decisions
Calculating 7-day rolling bookings per city. Using ROWS BETWEEN 6 PRECEDING AND CURRENT ROW but I get weird results on days with no bookings. Should I use RANGE?
Reyansh Iyer
Data Engineer | Spark, Airflow, dbt, Snowflake | Pipelines that don't break on Mondays
Every few weeks a backend team would rename a column and our pipelines would break silently. What we did: 1. Producers define a contract (schema + semantics + owner) in YAML in their repo 2. CI checks on the producer side block breaking changes unless the version is bumped 3. We consume via a versioned view, never
Reyansh Iyer
Data Engineer | Spark, Airflow, dbt, Snowflake | Pipelines that don't break on Mondays
For everyone who builds and maintains data pipelines. Share your stack, ask questions, post your worst data quality story (we've all got one). What's your current stack? Ours: Airflow + Spark + dbt + Snowflake.