The business question data lineage was built to answer

The migration planning meeting had been running for an hour when the question of downstream impact came up. The business stakeholders had a spreadsheet — maintained by a senior data engineer who knew which data source systems fed which data pipelines by memory and had done his best to document them. Both were in the room. The spreadsheet had twelve entries. The meeting ended with a migration date and a shared confidence that the impact was understood.

Two data pipelines broke within 24 hours of the migration. Both were first-degree consumers of the migrated data source system. Both appeared in the data lineage graph. Neither appeared on the spreadsheet.

The answer to the migration impact question was already computed. Every mature data platform continuously updates a record of what depends on what — which data pipelines draw from which data source systems, which data products sit downstream, which data consumers exist at every hop. The gap is not capability. It is that the data lineage graph was never pointed at the migration planning meeting.

The reactive trap

Data lineage is categorised, almost universally, as a debugging tool. When a data pipeline breaks, data engineers open the data lineage graph to trace upstream dependencies — to find the data source system that changed, the schema that drifted, the upstream failure that propagated. The migration planning question is the same query run in the other direction and in advance: what will this change affect? In almost every data platform review I run, when I ask how migration impact is assessed, the answer is stakeholder interviews or a Confluence page maintained by hand. The data lineage graph is rarely mentioned. The categorisation holds.

The translation gap

A raw data lineage graph is not a migration impact report. The graph shows nodes and edges — what depends on what, through how many hops. What a migration planning meeting needs is different: of the downstream consumers of this data source system, which break immediately, which break within a week, and which consume it rarely enough that the risk is low?

Translating from graph to impact report requires someone who can read the data lineage and map its dependencies to business criticality and migration timing. That person is almost never defined in advance. The data engineering team owns the data lineage graph. The business and IT leadership own the migration decision. The translation between them sits with nobody — so it either happens informally when the right data engineer is asked the right question, or it does not happen and the migration creates surprises.

The substitute

When the data lineage graph is not consulted, migration planning teams replace it with stakeholder interviews, Confluence pages maintained by hand, and tribal knowledge from data engineers who remember which systems connect to which. All three are a point-in-time snapshot. A data source system's consumer list changes every time a new data pipeline is built, a data product is extended, or a downstream team adds a dependency without informing anyone upstream. The data lineage graph reflects the current state. The spreadsheet reflects the last time someone thought to update it.

The accuracy objection is worth addressing directly: the data lineage graph may not be complete — automated scanning misses custom scripts, manual loads, and undocumented transformations. Even an incomplete data lineage graph is typically more complete than a stakeholder spreadsheet. The gap is in the long tail. The dangerous consumers are almost always in the first-degree connections that automated scanning reliably captures.

The two data pipelines that broke above were both one hop from the migrated data source system. Both were in the data lineage graph. Neither was on the spreadsheet.

The leadership mandate

Migration planning is a business and IT leadership decision. The data lineage graph lives in the data engineering team's tooling. Nobody has made it the responsibility of anyone on the migration planning side to consult it, and the data engineering team has not been given a mandate to produce a migration impact report from it as a standard step before a migration begins. The gap is not technical. It is a process expectation that was never set — because neither function owns the bridge between them, the question of whether to consult the data lineage graph for migration impact does not formally get asked. It gets answered instead by whichever data pipelines break first.

The path forward

Before the next migration planning meeting, the question worth asking is not whether the data lineage graph is complete enough to trust. It is whether it was consulted at all. In most organisations that have it, it was not. The answer was already there. The meeting just did not include it.