Senior Data Infrastructure Engineer - Real-Time Platform

Peregrine; San Francisco (CA); On-site; USD 200,000 - 275,000; Full time

Benefits offered by this job

Benefits

Equity (if applicable)

Bonus (if applicable)

Job summary

Peregrine is seeking a Staff Data Infrastructure Engineer in San Francisco. The role focuses on designing high-throughput data integration systems and optimizing distributed processing pipelines. Candidates should have over 8 years of experience in data infrastructure, with expertise in Apache Spark and AWS. A background in Computer Science or Engineering and skills in Python or Scala are required. The salary range is $200,000–$275,000 annually, plus benefits and equity where applicable.

Qualifications

• 8+ years of experience with large-scale data infrastructure systems.
• Deep understanding of Apache Iceberg and schema evolution.
• Extensive experience with distributed data processing.

Responsibilities

• Design and operate a high-throughput data integration platform.
• Architect a scalable data storage layer for petabyte scale.
• Optimize distributed data processing pipelines.

Skills

Data infrastructure design

Apache Spark

Real-time data integration

Cloud platforms (AWS)

Python/Scala

Education

Degree in Computer Science or Engineering

Tools

Apache Kafka

Airflow

Kubernetes

Job description

Peregrine is seeking a Staff Data Infrastructure Engineer in San Francisco. The role focuses on designing high-throughput data integration systems and optimizing distributed processing pipelines. Candidates should have over 8 years of experience in data infrastructure, with expertise in Apache Spark and AWS. A background in Computer Science or Engineering and skills in Python or Scala are required. The salary range is $200,000–$275,000 annually, plus benefits and equity where applicable.