Engineering Leader, Pipelines & ETL Infrastructure

Visa Hunt; San Francisco, Mountain View (CA, CA); On-site; USD 228,600 - 317,233; Full time

Job summary

Databricks is seeking an Engineering Leader to spearhead the Pipelines Engine team for the Lakeflow platform, driving next-generation ETL features and high-performance workloads. The role requires building a world-class team and collaborating with product management and customers to deliver scalable data infrastructure.

The candidate will oversee recruitment, mentorship, and architectural decisions while ensuring the software is robust and operable, balancing long-term strategy with immediate

Qualifications

• Minimum 3 years of experience in managing top-tier engineering teams.
• 5+ years building data infrastructure systems such as Apache Spark or database internals.
• A passion for database systems, storage systems, distributed systems, or performance optimization.
• Experience working with product management, and directly with customers; ability to understand customer needs.
• Experience building and managing teams in a complex technical domain, such as distributed data systems or database internals.
• Expertise in attracting, hiring and coaching engineers, who will meet hiring standards.

Responsibilities

• Lead an engineering team building the next-generation ETL features for the Databricks Lakeflow platform.
• Oversee sustained recruitment of top-tier talent, and upskilling talent on the team.
• Build processes to implement product vision and strategy, according to organizational goals and priorities.
• Build software that is not just high quality but easy to operate.
• Manage technical debt, including long-term technical architecture decisions and balance product roadmap.

Skills

Engineering leadership

Data infrastructure

Distributed systems

Performance optimization

Customer engagement

Hiring and coaching engineers

Tools

Apache Spark

Database internals

Job description

Databricks is seeking an Engineering Leader to spearhead the Pipelines Engine team for the Lakeflow platform, driving next-generation ETL features and high-performance workloads. The role requires building a world-class team and collaborating with product management and customers to deliver scalable data infrastructure.

The candidate will oversee recruitment, mentorship, and architectural decisions while ensuring the software is robust and operable, balancing long-term strategy with immediate