workOwonworkOwonA New Way to Connect
HomeFeedJobsCommunitiesEventsFeaturesAbout workOwon
HomeFeedJobsCommunitiesEventsFeaturesAbout workOwon
HomeCommunitiesData Engineering Guild

Every few weeks a backend team would rename a column and our pipelines would break silently.

RI

Reyansh Iyer

Data Engineer | Spark, Airflow, dbt, Snowflake | Pipelines that don't break on Mondays

3 Oct 2026· Data Engineering Guild

Every few weeks a backend team would rename a column and our pipelines would break silently. What we did: 1. Producers define a contract (schema + semantics + owner) in YAML in their repo 2. CI checks on the producer side block breaking changes unless the version is bumped 3. We consume via a versioned view, never the raw table 4. Breaking changes need a 2-week deprecation window Incidents from upstream changes: 11 last year, 1 this year. The cultural part (producers owning their data) was harder than the tooling.
5 reactions3 comments

Comments

  • AV

    Anika Verma3 Oct 2026

    This is exactly our problem. Did backend teams push back on the extra CI step?

  • RI

    Reyansh Iyer3 Oct 2026

    @Anika at first, yes. It helped to show them the incident list with hours lost per incident.

  • NC

    Nikhil Chowdhury3 Oct 2026

    As a backend dev, I'd honestly prefer this. I never know who uses which column.

Join the conversation

Join workOwon to react, comment and follow people and communities you care about.

Log inJoin now

Popular communities

See all