Staff Data Infrastructure Engineer — Real-Time Pipelines

Peregrinetechnologies; San Francisco (CA); On-site; USD 200,000 - 275,000; Full time

Benefits offered by this job

Benefits

Equity (if applicable)

Bonus (if applicable)

Job summary

Peregrinetechnologies is seeking a Staff Data Infrastructure Engineer in San Francisco. You will hold deep ownership over the data layer, architecting systems to ingest and serve large volumes of operational data, enabling clients to make fast, confident decisions.

The ideal candidate will have 8+ years of experience, deep expertise in Apache Iceberg and Spark, and would thrive in tackling complex, open-ended data challenges. The position offers a competitive salary range of $200,000 - $275,000 annually.

Qualifications

• 8+ years of experience architecting and operating large-scale data infrastructure systems.
• Deep expertise with open table formats, particularly Apache Iceberg.
• Extensive hands-on experience with Apache Spark for batch and streaming.

Responsibilities

• Design and operate a high-throughput, real-time data integration platform.
• Build and optimize distributed data processing pipelines.
• Collaborate with cross-functional teams on data integration patterns.

Skills

Data Infrastructure Design

Real-time Data Processing

Apache Spark

AWS GovCloud

Python/Scala Programming

Education

Degree in Computer Science or Engineering

Tools

Apache Kafka

Apache Iceberg

Airflow

Kubernetes

Job description

Peregrinetechnologies is seeking a Staff Data Infrastructure Engineer in San Francisco. You will hold deep ownership over the data layer, architecting systems to ingest and serve large volumes of operational data, enabling clients to make fast, confident decisions.

The ideal candidate will have 8+ years of experience, deep expertise in Apache Iceberg and Spark, and would thrive in tackling complex, open-ended data challenges. The position offers a competitive salary range of $200,000 - $275,000 annually.