Data platform migrations rarely fail on tooling. They fail on sequencing.

I walked into a data platform migration where the team had already evaluated five data platform architectures. Nobody had agreed on what any of them needed to do.

The tools had been benchmarked. The vendors had been through two rounds of demos. Data architecture proposals were drafted and circulated. And when I asked what the next data platform needed to deliver that the current one could not, the room went quiet.

I have seen this across projects that were six months old, two years old, and six years old. Different industries, different technology stacks, the same starting point: tooling before requirements. A data platform migration is a sequencing problem, not a tooling problem — and the sequence almost always starts one step too late.

The options available today — open-source, managed cloud services, commercial platforms, or combinations of all three — are generally capable enough. What fails migrations is the order of decisions: choosing a replacement before understanding the constraints, starting with high-priority data sources before the team has learned the new toolset, or committing to a timeline before testing against real-world edge cases.

The signals

Migration conversations usually start with cost. A license renewal arrives, the number looks wrong relative to what the data platform actually delivers, and the business starts asking questions. That is a valid signal, but it is rarely the only one.

A data platform built before the company's AI roadmap existed is likely missing capabilities that roadmap now needs: native support for semi-structured data, a Python execution environment for ML pipelines, open table formats that allow other engines to read the data directly, vector storage for embedding-based features. When those gaps start blocking product work, the migration conversation becomes urgent regardless of what the license costs.

Legal constraints add a third category. Data residency requirements, audit obligations, and access logging controls can each turn a migration from a preference into a deadline. A compliance review that surfaces these does not wait for a convenient engineering sprint.

Knowing which signal is driving the migration matters because it shapes the urgency, the scope, and who needs to be in the room when decisions get made.

The workshop

Before evaluating any new tooling, the right step is a structured conversation with key stakeholders from both the business and engineering sides. The goal is not to produce a shortlist of tools. It is to define what the next version of the data platform needs to do.

That means mapping the existing business needs — which data flows are critical, which can tolerate disruption, which are growing fastest — against the engineering requirements the current data platform is failing to meet: latency, scalability, user extensibility, the ability to support open formats. These two lenses together produce a constraint set that makes the tooling evaluation meaningful rather than speculative.

The pilot

Once the migration strategy is clear, the sequencing of execution matters as much as the strategy itself.

Start with data sources that are low in business priority. These are the candidates for the first end-to-end implementation in the new toolset — not because they are unimportant, but because they will surface the corner cases, the unexpected incompatibilities, the performance surprises that always emerge when a new data platform meets real production data for the first time. Learning those lessons on a non-critical data source is far cheaper than learning them on one the business cannot function without.

For the data sources that cannot afford disruption, plan a parallel run — provided the source system can handle the additional load: the new implementation runs alongside the existing one, producing the same outputs, until there is enough evidence that the new data platform handles the workload correctly. Only then does the old implementation get decommissioned.

The evaluation at the end of each pilot should be honest. If the new tooling cannot meet the requirements identified in the workshop — not just in theory, but in practice against real data volumes — that is not a failure. It is the signal to return to the workshop and adjust the approach before committing the migration more broadly.

The leadership mandate

A data platform migration is a business decision that the business should never feel. Data consumers — the analysts, the ML engineers, the product teams relying on the data — should experience continuity, not disruption. The technical complexity of the migration is the engineering team's problem to manage, not the business's problem to absorb.

That is what sequencing achieves. By starting with signals rather than tools, aligning on constraints before evaluating options, piloting on low-priority data sources before critical ones, and building in honest evaluation checkpoints, the migration becomes a series of controlled transitions rather than a high-stakes cutover.

The teams I have seen get this right share one habit: they treat the migration as a governance exercise first and a technology exercise second.

The path forward

The tooling conversation will happen — it always does. The one worth having first is whether the team has agreed on what the next data platform needs to do. That conversation is where I usually start.

Let's connect to build your data moat. 🛡️