Senior Data Lakehouse Engineer Spark Iceberg Kafka

Apple, Inc.

Cupertino (CA)

In loco

USD 185.000 - 325.000

Tempo pieno

2 giorni fa

Candidati tra i primi

Aumenta le possibilità di ricevere una risposta da questo datore di lavoro

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Vantaggi offerti da questo lavoro

Stock options

Comprehensive medical and dental

Relocation assistance

Descrizione del lavoro

Apple, Inc. is seeking a senior data engineer to own the data lakehouse that turns raw device and compute events into reliable, fast-to-query tables. You will design and run Spark pipelines, shape Iceberg-backed schemas, and enable trusted datasets for dashboards and analytics.

Collaboration with a data analyst and a full-stack engineer will be essential to success. You will also take ownership of Kafka-ingested events, ensure schema evolution, and drive data quality with CI checks, monitoring,

Competenze

Bachelor's degree in Computer Science, Engineering, or a related field.
7+ years of professional data engineering experience building production data pipelines at scale, including diagnosing and improving performance of data jobs and queries, or equivalent.
Production experience with Apache Spark, including batch and streaming (Structured Streaming) pipelines.
Strong data modeling skills, with experience designing analytics schemas on Iceberg or comparable lakehouse table formats.
Proficiency in Java or Scala, and strong SQL.
Experience building data quality checks and monitoring for production pipelines, including freshness and completeness alerting and reconciliation between source and derived data.
Productive use of AI coding tools in real delivery, with the discipline to validate generated code and queries.
Excellent communication and critical thinking skills in a team environment.

Mansioni

Build and run Spark streaming and batch pipelines in Scala or Java, including backfills when data issues are found or new transformations are added for past events.
Design lakehouse tables and layouts (partitioning, sort order, compaction, schema evolution) so dashboards can query Iceberg quickly through Trino.
Validate incoming events, handle schema changes, and route malformed or unknown events to a dead-letter path for review and replay.
Define and enforce data quality with a data analyst: keep schemas and data contracts under version control with compatibility checks in CI, monitor freshness, completeness, duplicates, and late arrivals, and alert the right owner.
Partner with the full-stack engineer who builds the dashboards, designing the tables and queries behind them and using AI coding tools to move faster while checking the results.
Take over the Apache Flink job that ingests events from Kafka into Iceberg.
Improve performance and cost by right-sizing jobs, fixing skew and small-file problems, and tuning queries and storage.
Join the on-call rotation for Data Lakehouse incidents, review code and designs, and mentor other engineers.