Senior Big Data Engineer - Data Pipelines & ML Enablement

ZipRecruiter, Inc.; Santa Monica (CA); Hybrid; USD 190,000 - 250,000; Full time

Benefits offered by this job

Competitive compensation

Benefits package

Flexible vacation & PTO

Employer-matched 401(k)

Job summary

ZipRecruiter, Inc. is seeking a Big Data Engineer to join the Tech Enablement team in a hybrid environment.

You will design, build, and operate data pipelines and data lake infrastructure that power customer-facing products, with emphasis on scalability and reliability. You will work on data ingestion, batch and streaming processing, MLOps, and data documentation tooling for Sift and related platforms, partnering with engineering and data science teams to accelerate data-driven decisions.

Qualifications

• 4+ years of professional software development experience, including building and operating data pipelines that feed into a customer facing product.
• Hands‑on Apache Spark experience: writing, tuning and debugging batch and/or streaming jobs at scale
• Experience deploying and operating containerized services and batch jobs
• Solid computer science fundamentals: data structures, algorithms, distributed systems, and testing
• Professional software development experience, including building and operating data pipelines and infrastructure that feed into a customer facing product

Responsibilities

• Data Ingestion & Datalake Management: Design, build, deploy, and maintain the data pipelines and systems that make web events available in our datalake and via event streams for various downstream applications to process.
• Big Data Processing: You’ll tackle challenges with large, evolving datasets, using a mix of batch and streaming techniques to make critical trade-offs between speed, efficiency, and reliability.
• MLOps: Design, build, deploy, and maintain frameworks and systems that accelerate ML development across teams and products at ZipRecruiter, with the goals of improving the velocity, quality, observability, and reliability of ML development and ML powered products.
• Data Documentation Framework: Design, build, deploy, and maintain frameworks and systems to improve the comprehensiveness and quality of data documentation at ZipRecruiter through processes and tooling that allows both humans and automated processes to easily add, maintain, and discover documentation.
• Data Analysis Agent: Design, build, deploy, and maintain an agentic data exploration and analysis tool, Sift, to democratize access to data, collaborate on insights, and accelerate data driven decision making.

Skills

4+ years of software development

Apache Spark

Data pipelines

CS fundamentals

Education

BS/MS/PhD in CS/Math/Physics or related field

Tools

Docker

Kubernetes

AWS (S3/IAM/EC2/MSK)

Parquet/ Iceberg/ Delta Lake

Job description

ZipRecruiter, Inc. is seeking a Big Data Engineer to join the Tech Enablement team in a hybrid environment.

You will design, build, and operate data pipelines and data lake infrastructure that power customer-facing products, with emphasis on scalability and reliability. You will work on data ingestion, batch and streaming processing, MLOps, and data documentation tooling for Sift and related platforms, partnering with engineering and data science teams to accelerate data-driven decisions.