Senior Data Engineer: Lakehouse & Spark Pipelines Lead

Apple Inc.; Cupertino (CA); Presencial; USD 185 000 - 325 000; Tempo integral

Resumo da oferta

Apple Inc. is seeking a senior data engineer to own the data lakehouse, turning device and compute events into reliable, fast-to-query tables.

You will build Spark pipelines, design Iceberg tables, and enable dashboards through Trino while maintaining data quality and schema evolution. You’ll collaborate with data analysts, full-stack engineers, and application teams, taking over ingestion from Kafka and optimizing performance.

Qualificações

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 7+ years of professional data engineering experience building production data pipelines at scale.
• Proficiency in Java or Scala, and strong SQL.
• Production experience with Apache Spark, including batch and streaming.
• Strong data modeling skills with analytics schemas on Iceberg or lakehouse formats.
• Experience building data quality checks and monitoring for production pipelines.
• Productive use of AI coding tools with validation of generated code.

Responsabilidades

• Build and run Spark streaming and batch pipelines in Scala or Java, including backfills.
• Design lakehouse tables and layouts to enable fast querying via Trino.
• Validate incoming events, handle schema changes, and manage dead-letter routing.
• Define and enforce data quality with a data analyst, monitor freshness and completeness.
• Partner with dashboards engineers to design tables and queries behind dashboards.
• Take over Kafka-to-Iceberg ingestion and improve performance and cost.
• Join on-call rotation for data lakehouse incidents and mentor engineers.

Conhecimentos

Apache Spark

Java/Scala

SQL

Data modeling

Data quality checks

AI coding tools

Kubernetes

Iceberg

Trino

Kafka

Flink

Formação académica

Bachelor's degree in Computer Science, Engineering, or related field

Ferramentas

Apache Iceberg

Trino

Kafka

Flink

Kubernetes

Descrição da oferta de emprego

Apple Inc. is seeking a senior data engineer to own the data lakehouse, turning device and compute events into reliable, fast-to-query tables.

You will build Spark pipelines, design Iceberg tables, and enable dashboards through Trino while maintaining data quality and schema evolution. You’ll collaborate with data analysts, full-stack engineers, and application teams, taking over ingestion from Kafka and optimizing performance.