Staff Software Engineer - Data Ingestion & Lakeflow Lead

Databricks; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Databricks is seeking senior engineers for the Lakeflow Connect team to design and build high-performance data ingestion and processing at scale. You will work on incremental data capture, log parsing, and low-level system debugging to move data from OLTP sources into the lakehouse with minimal impact on production systems.

You will influence roadmaps, lead cross-team initiatives, and mentor junior engineers while delivering robust ingestion capabilities across dashboards, notebooks, SQL, and AI

Qualifications

• 15+ years of industry experience building and supporting large-scale distributed systems.
• Experience with database replication, backup, and transaction recovery at major vendors (SQL Server, Oracle, IBM, etc).
• Comfortable planning towards a multi-year vision with incremental deliverables.
• Motivated by delivering customer value and impact.
• Strong foundation in algorithms and data structures and their real-world use cases.
• Experience driving company initiatives toward customer satisfaction.

Responsibilities

• Solve real business needs at large scale by applying software engineering.
• Deliver a highly scalable, available, and fault-tolerant engine processing hundreds of TB of data daily across thousands of customers.
• Lead architecture design and influence product roadmap; own new projects.
• Mentor teammates, lead sprint planning, and participate in project planning.
• Investigate production issues and drive de-risking through prototyping and validation.

Skills

Distributed systems

Algorithms & data structures

Production support

Technical leadership

Tools

SQL Server

Oracle

IBM DB2

Job description

Databricks is seeking senior engineers for the Lakeflow Connect team to design and build high-performance data ingestion and processing at scale. You will work on incremental data capture, log parsing, and low-level system debugging to move data from OLTP sources into the lakehouse with minimal impact on production systems.

You will influence roadmaps, lead cross-team initiatives, and mentor junior engineers while delivering robust ingestion capabilities across dashboards, notebooks, SQL, and AI