Remote Big Data Engineer - Tech Enablement

Remote; San Francisco (CA); Hybrid; USD 135,000 - 185,000; Full time

Benefits offered by this job

Competitive compensation

Exceptional benefits package

Flexible Vacation & Paid Time Off

Employer-matched 401(k) plan

Job summary

ZipRecruiter is hiring a Big Data Engineer in Tech Enablement to help scale data pipelines and ML tooling. You will design and operate data ingestion and processing systems, enable ML development across teams, and build documentation and discovery tools for data assets.

The role emphasizes data-intensive engineering within a hybrid, US-friendly environment, offering competitive compensation and benefits.

Qualifications

• 2+ years of professional software development experience including building and operating data pipelines feeding a customer-facing product.
• Hands-on Apache Spark experience: writing, tuning and debugging batch/streaming jobs at scale.
• Experience deploying and operating containerized services and batch jobs.
• Solid CS fundamentals: data structures, algorithms, distributed systems, testing.
• Professional software development experience building and operating data pipelines and infrastructure for customer-facing products.

Responsibilities

• Ingest and manage data in datalake and event streams for downstream apps.
• Process large evolving datasets using batch and streaming techniques for speed and reliability.
• Design, build, and maintain ML development acceleration frameworks and systems (MLOps).
• Create data documentation frameworks to improve data quality and discoverability.
• Develop an agentic data exploration tool (Sift) to democratize data access and insights.

Skills

Apache Spark

Data pipelines

Distributed systems

CS fundamentals

Software development

Customer-facing product experience

Education

BS/MS/PhD in CS/Math/Physics

Tools

Docker

Kubernetes

Apache Airflow

Kafka

Spark Structured Streaming

Iceberg/Delta Lake

AWS (S3/IAM/EC2)

Job description

ZipRecruiter is hiring a Big Data Engineer in Tech Enablement to help scale data pipelines and ML tooling. You will design and operate data ingestion and processing systems, enable ML development across teams, and build documentation and discovery tools for data assets.

The role emphasizes data-intensive engineering within a hybrid, US-friendly environment, offering competitive compensation and benefits.