Lead Data Platform Engineer — Real-Time Data & AI

Elastic; Mountain View (CA); Presencial; USD 113.000 - 179.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health coverage

Flexible schedules

Vacation days

Volunteer time

Parental leave

Stock program

401k with match

Donation matching

Descripción de la vacante

Elastic is seeking a Lead Data Engineer to join our Data Engineering & Architecture team. You’ll help grow a data platform that makes it easier for teams across Elastic to work with data confidently, transforming it into trustworthy, actionable insights.

You’ll design, maintain, and optimize pipelines and infrastructure powering the data platform, from ingestion to delivery, using Airflow, Spark, Iceberg, Kafka, Flink and dbt Labs, while partnering with stakeholders to align data with business

Formación

• 8+ years of experience in data engineering or related roles.
• Experience with Apache Airflow for orchestrating data workflows.
• Experience with Spark, Flink, Kafka, and dbt Labs for data processing and transformation.
• Understanding of Apache Iceberg for data table formats and data warehouse/lakehouse strategies.
• Skilled in using Terraform for infrastructure-as-code deployments.
• Proficient in Bash, Python, and SQL.
• Knowledge of databases including OLAP/OLTP/Document/Vector and GCP for cloud management.
• Bonus: data security and SOX compliant data; Bonus: Elastic stack experience.

Responsabilidades

• Design, maintain, and improve pipelines and infrastructure powering the data platform across the full data lifecycle.
• Collaborate with partners to understand data needs, define data contracts, and document data flows.
• Review code and promote best practices; ensure platform security and governance.
• Support compliance with platform policies and help optimize performance of data pipelines.

Conocimientos

8+ years data eng

Airflow

Spark

Flink

Kafka

dbt Labs

Iceberg

Terraform

Bash

Python

SQL

GCP

SOX compliance

Elastic stack

Descripción del empleo

Elastic is seeking a Lead Data Engineer to join our Data Engineering & Architecture team. You’ll help grow a data platform that makes it easier for teams across Elastic to work with data confidently, transforming it into trustworthy, actionable insights.

You’ll design, maintain, and optimize pipelines and infrastructure powering the data platform, from ingestion to delivery, using Airflow, Spark, Iceberg, Kafka, Flink and dbt Labs, while partnering with stakeholders to align data with business