Your data engineers are advancing on a ladder designed for someone else

The data engineer on a large data platform project I was part of had spent two years accumulating responsibilities that did not have a title. Schema governance decisions, data model reviews, mentoring the two junior data engineers who joined in the second year — all of it had arrived one request at a time. When the formal promotion conversation arrived, it had already been open for four months. Nobody could point to a definition of senior data engineer that described what they had actually built; the existing career ladder described software engineering mastery. The data engineer accepted an offer elsewhere, and the team lost the person anchoring most of its technical architecture decisions without ever having made those decisions formally theirs.

The absence of a data engineering career ladder is almost always invisible until it produces attrition — and by the time it does, the decision to leave has usually already been made. The ladder is not missing because nobody thought to build it. It is missing because one already existed. The software engineering career ladder was there, it was understood, and it was applied to data engineers on the reasonable assumption that the disciplines were similar enough. They are, until the senior level. That is where the paths diverge, and where the borrowed ladder stops serving the people it is supposed to describe.

The divergence point

At junior and mid levels, software engineering and data engineering career criteria overlap enough that a shared ladder works. The divergence is sharpest at senior. Software engineering senior points toward system architecture decisions, technical leadership over a product team, or the track toward principal engineer — skills measured in systems scope and people influence. Data engineering senior should point toward something different: data modelling depth, data platform ownership, the ability to translate business requirements into data platform decisions that data consumers can actually use, and the individual contributor track toward staff data engineer or data architect.

Most ladders do not make that distinction. The two paths look identical until a data engineer tries to advance and finds that the level definition describes software engineering mastery — not theirs.

The proxy problem

In the absence of defined data engineering criteria, managers evaluate data engineers on the criteria they have: code review standards, system design depth, delivery velocity. These are not wrong measures. They are insufficient ones. The skills that distinguish a strong senior data engineer — data modelling clarity, schema governance, ownership of data contracts and the standards that hold data consumers to them — are rarely captured on any level definition.

The result is that promotion decisions feel arbitrary to the data engineer and frustrating to the manager. The data engineer knows they are doing high-value work. The manager cannot point to a criterion that captures it. The promotion is deferred rather than made, and the deferral is read — correctly — as the absence of a visible path.

The technical leadership gap

The path from senior data engineer to data architect or staff data engineer is not a single step. Between them is a stage that most companies do not name: influencing data platform decisions across the team, leading data model reviews, setting data engineering standards and conventions, mentoring more junior data engineers — without managing a team and without the formal authority of a data architect. It is technical leadership, and it is the stage most commonly skipped.

Skip it by not naming it, and it fills informally. The strongest data engineer absorbs the responsibilities of that stage without the title, the recognition, or the defined scope. During high-demand periods, having most critical architectural decisions route through one person becomes a bottleneck rather than a contribution — which is what happened in that data platform project, and which is what I find in most teams where the ladder has not been designed.

Collapse it into management, and the data engineers who want to grow without managing — often the most technically capable — see no path that fits and go looking for one elsewhere.

The leadership mandate

The CTO owns the question of what mastery in data engineering looks like at this company. It is not a question the data engineering lead can answer alone — defining what each level requires is a remit, not a preference, and it needs organisational authority to make it stick. Most teams default to the software engineering ladder not because nobody cares but because nobody has been asked to own the answer.

The path forward

The starting point is not a full career ladder with ten levels. It is three distinctions: what separates junior from mid, mid from senior, and senior from staff or the data architect track. Making the technical leadership stage explicit — as a named phase, not an informal accumulation — closes the gap where most attrition happens.

Once those distinctions exist, the more important step is surfacing them to the data engineers they describe. A career path that exists in a document but has never been the subject of a direct conversation is not a career path the data engineer can use.