Senior Staff Engineer, Scalable Data Pipelines

Databricks Inc.; Mountain View (CA); On-site; USD 229,000 - 314,000; Full time

Job summary

Databricks is seeking a Senior Staff Engineer to serve as a technical architect and engineering leader for the Lakeflow Engineering platform. You will own the long-term technical direction for declarative pipelines, materialized views, and related data infrastructure, shaping systems used by data teams worldwide.

You will lead cross-cutting initiatives across declarative dataflow processing, resource management, and streaming and complex event processing, influencing product direction and

Qualifications

• 10+ years of experience building, operating, and evolving large-scale distributed systems.
• Deep expertise in database internals, storage systems, distributed computing, streaming systems, language/API design, or performance engineering.
• Track record of executing against a multi-year technical vision through well-sequenced incremental milestones.
• Strong algorithmic foundations with practical judgment to know when they matter.
• Bias toward customer impact over technical novelty for its own sake.
• Experience building alignment across teams and driving initiatives from conception through customer adoption.
• BS, MS, or PhD in Computer Science or a related field (or equivalent depth of experience).

Responsibilities

• Architect and deliver a highly scalable, fault-tolerant platform for declarative pipelines and materialized views serving thousands of production customers.
• Turn ambiguous, large-scale technical challenges into clear execution plans that ship incrementally and hold up at exabyte scale.
• Bridge the gap between research and production by bringing ideas from the academic literature or internal R&D into systems that solve real customer problems.
• Lead deep systems work: performance diagnosis on large production clusters, low-level debugging, and optimization that moves the needle on cost and latency.
• Shape product direction alongside engineering and product leadership, owning technical strategy from design through delivery.
• Drive complex, multi-team technical initiatives that require sustained coordination and clear technical judgment.

Skills

Distributed systems

Database internals

Streaming systems

Performance engineering

Education

CS degree

Job description

Databricks is seeking a Senior Staff Engineer to serve as a technical architect and engineering leader for the Lakeflow Engineering platform. You will own the long-term technical direction for declarative pipelines, materialized views, and related data infrastructure, shaping systems used by data teams worldwide.

You will lead cross-cutting initiatives across declarative dataflow processing, resource management, and streaming and complex event processing, influencing product direction and