Senior Data Platform Engineer — Lakehouse & Spark Expert

Zeta-Global; San Francisco (CA); Hybrid; USD 165,000 - 185,000; Full time

Benefits offered by this job

Unlimited PTO

Medical coverage

Equity plan

Discounts & pet insurance

Job summary

Zeta Global seeks a Senior Data Platform Engineer to design and operate the data infrastructure supporting its AI-enabled marketing cloud. You will own Lakehouse architecture, distributed compute, data quality, governance, lineage, security, and the platform's operational systems, collaborating with data scientists, data engineers, and product teams.

Hybrid role in CA offices; you will optimize pipelines, deploy scalable data workloads, and mentor peers while contributing to open‑source projects

Qualifications

• 7+ years of software engineering experience.
• Strong experience in Python, Scala, Java, or another JVM language.
• Experience with Spark or another distributed data‑processing framework.
• Experience building or operating production data platforms, backend systems, or distributed systems.
• Familiarity with Lakehouse architecture, Apache Iceberg, data quality, governance, lineage, metadata, query engines, or real‑time OLAP.
• Working knowledge of AWS services such as EC2, S3, Athena, and IAM.
• Experience improving production reliability through monitoring, alerting, deployment automation, and incident response.
• Strong analytical, communication, and collaboration skills.
• Experience with Kubernetes, Terraform, Airflow, Trino, ClickHouse, Starrocks, or Pinot is a plus.

Responsibilities

• Build and operate Lakehouse, Spark, and distributed data‑processing systems.
• Improve the performance, reliability, cost, and operational visibility of platform workloads.
• Develop capabilities for data quality, governance, lineage, metadata, discoverability, security, and access control.
• Build the data context required for trusted reporting, ML models, and AI agents.
• Evaluate compute and query engines for read‑heavy workloads, assessing cost, performance, reliability, and operational fit.
• Assess real‑time OLAP technologies and identify appropriate production use cases.
• Improve deployment, observability, incident response, developer tooling, and platform operations.
• Partner with data scientists and data engineers to improve their experience using the platform.
• Contribute fixes and improvements to relevant open‑source projects, including Spark, Iceberg, and Airflow.
• Mentor engineers through code reviews, pairing, design discussions, and knowledge‑sharing sessions.

Skills

Python

Scala/Java

Spark

Distributed systems

AWS

Lakehouse

Data governance

Kubernetes

Terraform

Airflow

Tools

Apache Iceberg

Airflow

Trino

ClickHouse

StarRocks

Pinot

Job description

Zeta Global seeks a Senior Data Platform Engineer to design and operate the data infrastructure supporting its AI-enabled marketing cloud. You will own Lakehouse architecture, distributed compute, data quality, governance, lineage, security, and the platform's operational systems, collaborating with data scientists, data engineers, and product teams.

Hybrid role in CA offices; you will optimize pipelines, deploy scalable data workloads, and mentor peers while contributing to open‑source projects