Senior Lakehouse & Spark Data Platform Engineer

Zeta Global; San Francisco (CA); Hybrid; USD 165,000 - 185,000; Full time

Benefits offered by this job

Unlimited PTO

Medical coverage

Dental coverage

Vision coverage

Employee equity

Stock purchase plan

Employee discounts

Virtual wellness classes

Pet Insurance

Job summary

Zeta Global is seeking a Senior Data Platform Engineer to build and operate data infrastructure supporting analytics, ML, and AI systems. The role covers Lakehouse architecture, Spark workloads, and production-grade data platforms, with a hybrid stance out of the SF/SM offices in California.

Responsibilities include improving performance, reliability, security, and governance, plus collaborating with data scientists and engineers to deliver cost-effective, scalable data solutions.

Qualifications

• 7+ years of software engineering experience.
• Strong experience in Python, Scala, Java, or another JVM language.
• Experience with Spark or another distributed data-processing framework.
• Experience building or operating production data platforms, backend systems, or distributed systems.
• Familiarity with Lakehouse architecture, Apache Iceberg, data quality, governance, lineage, metadata, or real-time OLAP.
• Working knowledge of AWS services such as EC2, S3, Athena, and IAM.
• Experience improving production reliability through monitoring, alerting, deployment automation, and incident response.
• Strong analytical, communication, and collaboration skills.
• Experience with Kubernetes, Terraform, Airflow, Trino, ClickHouse, StarRocks, or Pinot is a plus.

Responsibilities

• Build andoperate Lakehouse, Spark, and distributed data-processing systems.
• Improve the performance, reliability, cost, and operational visibility of platform workloads.
• Develop capabilities for data quality, governance, lineage, metadata, discoverability, security, and access control.
• Build the data contextrequiredfor trusted reporting, ML models, and AI agents.
• Evaluate compute and query engines for read-heavy workloads, assessing cost, performance, reliability, and operational fit.
• Assess real-time OLAP technologies and identify appropriate production use cases.
• Improve deployment, observability, incident response, developer tooling, and platform operations.
• Partner with data scientists and data engineers to improve their experience using the platform.
• Contribute fixes and improvements to relevant open-source projects, including Spark, Iceberg, and Airflow.
• Mentor engineers through code reviews, pairing, design discussions, and knowledge-sharing sessions.

Skills

Python/Scala/Java

Spark

Distributed data-processing

Data platforms

AWS (EC2/S3)

Kubernetes

Terraform

Airflow

Trino/ClickHouse/StarRocks/Pinot

Tools

Kubernetes

Terraform

Airflow

Trino

ClickHouse

StarRocks

Pinot

Job description

Zeta Global is seeking a Senior Data Platform Engineer to build and operate data infrastructure supporting analytics, ML, and AI systems. The role covers Lakehouse architecture, Spark workloads, and production-grade data platforms, with a hybrid stance out of the SF/SM offices in California.

Responsibilities include improving performance, reliability, security, and governance, plus collaborating with data scientists and engineers to deliver cost-effective, scalable data solutions.