Remote Big Data Engineer, Tech Enablement & MLOps

HR Tech Job; Austin (TX); Hybrid; USD 135,000 - 185,000; Full time

Benefits offered by this job

Competitive compensation

Exceptional benefits package

Flexible Vacation & Paid Time Off

Employer-matched 401(k) plan

Job summary

ZipRecruiter is seeking a Big Data Engineer for the Tech Enablement team to help build scalable data infrastructure and ML tooling. You will design and maintain data pipelines, implement batch and streaming processing, and contribute to data documentation and the Sift data exploration tool.

This role supports a hybrid US-based setup with remote options and a competitive compensation package. The team focuses on data ingestion, processing, and ML enablement, delivering reliable pipelines and

Qualifications

• 2+ years of professional software development experience with data pipelines feeding a customer product.
• Hands-on Apache Spark experience: batch and/or streaming jobs at scale.
• Experience deploying and operating containerized services and batch jobs.
• Solid CS fundamentals: data structures, algorithms, distributed systems, and testing.
• Professional software development experience, including building and operating data pipelines and infrastructure that feed into a customer facing product.

Responsibilities

• Design, build, deploy, and maintain the data pipelines and systems that make web events available in our datalake.
• Process large, evolving datasets using batch and streaming techniques.
• Design, build, deploy, and maintain ML development frameworks across teams and products.
• Create data documentation frameworks to improve comprehensiveness and quality of data documentation.
• Build, deploy, and maintain an agentic data exploration tool, Sift, to democratize data access.

Skills

Spark

Data pipelines

Distributed systems

Testing

Data structures

Education

BS/MS/PhD in CS/Math/Physics or related field

Tools

AWS

Kafka

Airflow

Kubernetes

Docker

Iceberg/Delta Lake

Job description

ZipRecruiter is seeking a Big Data Engineer for the Tech Enablement team to help build scalable data infrastructure and ML tooling. You will design and maintain data pipelines, implement batch and streaming processing, and contribute to data documentation and the Sift data exploration tool.

This role supports a hybrid US-based setup with remote options and a competitive compensation package. The team focuses on data ingestion, processing, and ML enablement, delivering reliable pipelines and