Senior Data Infrastructure Engineer - Streaming & Batch

Devopsroles; Mountain View (CA); Hybrid; USD 175,000 - 221,000; Full time

Benefits offered by this job

Health, dental, and vision coverage

Top-tier 401(K) with company matching

Paid time off and holidays

FSA, HSA and commuter benefits

Team activity budget

Job summary

NewsBreak seeks a senior data engineer in Mountain View to own and scale the streaming and batch data pipelines powering our content intelligence platform. You design Kafka pipelines, build Spark jobs, and collaborate with data scientists for reliable analytics at scale.

You will drive observability with data quality checks, define SLOs, and optimize storage and partitioning. You will mentor juniors and work across teams to raise the platform bar.

Qualifications

• 5+ years of data engineering experience with production pipelines in distributed environments.
• Hands-on Spark and Kafka at scale; proven incident debugging.
• Experience with Hadoop, Presto/Trino, and Flink.
• Proficiency in Python and SQL; Scala a plus.
• Experience in AWS (S3, EMR, Glue, Redshift) and cloud environments.
• Track record improving efficiency, scalability, and stability of data systems.

Responsibilities

• Own the streaming data backbone with high-throughput Kafka pipelines to data warehouse and analytics.
• Build and optimize batch Spark jobs processing billions of rows daily, with reliable backfill and cost awareness.
• Implement data quality checks, freshness monitors, and anomaly alerts; define data SLOs.
• Model structured and unstructured data; serve analytics and product use cases and align schemas for future needs.
• Collaborate with data scientists and platform engineers; unblock downstream teams and reduce ticket queues.
• Improve efficiency through partitioning, scheduling, and tiered storage; contribute to shared infrastructure including Airflow.

Skills

Spark

Kafka

Hadoop

Presto/Trino

Flink

Python

SQL

Scala

AWS

Education

BS/MS in Computer Science or equivalent

Tools

Airflow

Job description

NewsBreak seeks a senior data engineer in Mountain View to own and scale the streaming and batch data pipelines powering our content intelligence platform. You design Kafka pipelines, build Spark jobs, and collaborate with data scientists for reliable analytics at scale.

You will drive observability with data quality checks, define SLOs, and optimize storage and partitioning. You will mentor juniors and work across teams to raise the platform bar.