Staff Data Infra Engineer — Real-Time, Scalable Data Platform

Peregrine; Washington; On-site; USD 200,000 - 275,000; Full time

Benefits offered by this job

Equity compensation

Bonus eligibility

Health benefits

Job summary

Peregrine seeks a Staff Data Infrastructure Engineer to own the data layer underpinning its AI-enabled platform. You will design and operate real-time data ingestion, storage, and serving systems at scale, enabling customers to make fast, confident decisions.

This senior IC role requires deep expertise in open table formats, Spark, Kafka, and cloud-native data architectures. Located in San Francisco, New York, or Washington DC with on-site work; competitive salary, equity, and bonus potential

Qualifications

• 8+ years of experience architecting large-scale data infrastructure in production environments.
• Expertise with open table formats, Apache Iceberg, including schema evolution and partitioning.
• Extensive hands-on experience with Spark for batch and streaming data processing at scale.
• Strong real-time data integration experience using Kafka, Flink, or equivalents.
• Experience with data pipeline orchestration using Airflow or similar tools.
• Proficient in Python and/or Scala, writing production-quality code.
• Experience with AWS cloud and data lake architectures (S3).
• Kubernetes and containerized deployment of data workloads.
• Degree in CS/Engineering or related field, or equivalent practical experience.

Responsibilities

• Designing and operating a high-throughput, real-time data integration platform across diverse environments.
• Architecting a scalable open table format layer for petabyte-scale storage.
• Building and optimizing distributed data processing pipelines with Spark and streaming tech.
• Driving performance, reliability, and cost efficiency across the data stack.
• Collaborating with platform and product teams to define data contracts and schemas.
• Establishing best practices and tooling to raise data infrastructure quality.

Skills

Data infrastructure

Ambiguity handling

Technical vision

End-to-end ownership

Operational excellence

Education

Degree in CS/Engineering or related field

Tools

Apache Iceberg

Apache Spark

Apache Kafka

Apache Flink

Airflow

Kubernetes

AWS/S3

Job description

Peregrine seeks a Staff Data Infrastructure Engineer to own the data layer underpinning its AI-enabled platform. You will design and operate real-time data ingestion, storage, and serving systems at scale, enabling customers to make fast, confident decisions.

This senior IC role requires deep expertise in open table formats, Spark, Kafka, and cloud-native data architectures. Located in San Francisco, New York, or Washington DC with on-site work; competitive salary, equity, and bonus potential