Staff Data Infrastructure Engineer — Real‑Time Platform

Doist; Washington (District of Columbia); On-site; USD 200,000 - 275,000; Full time

Job summary

Peregrine Technologies in Washington DC is seeking a Staff Data Infrastructure Engineer to own the data layer that underpins our real-time platform. You will architect and build systems to ingest, store, and serve massive volumes of operational data, enabling customers to make fast, informed decisions.

This senior IC role demands deep technical judgment and ownership across the stack, from data contracts to deployment.

Qualifications

• 8+ years of experience architecting and operating large-scale data infrastructure
• Deep expertise with open table formats, particularly Apache Iceberg and related storage
• Extensive hands-on experience with Apache Spark for batch and streaming processing
• Strong background in real-time data integration and stream processing (Kafka/Flink)
• Solid experience with data pipelines orchestration using Airflow or similar tools
• Strong software engineering fundamentals in Python and/or Scala with production-quality code
• Extensive experience with AWS and S3-based data lake architectures
• Experience with Kubernetes and containerized data workloads

Responsibilities

• Designing and operating a high-throughput, real-time data integration platform across diverse customer environments
• Architecting a scalable open table format layer for reliable data storage at petabyte scale
• Building and optimizing distributed data processing pipelines with Apache Spark and adjacent streaming technologies
• Driving performance, reliability, and cost efficiency across the full data infrastructure stack
• Collaborating with platform and product engineering teams to define data contracts, schemas, and integration patterns
• Establishing best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization

Skills

Data architecture

Real-time processing

Distributed systems

Cloud engineering

Education

Bachelor's degree in CS/Engineering or related

Tools

Apache Iceberg

Apache Spark

Apache Kafka

Apache Flink

Airflow

Python

Scala

AWS

Kubernetes

Job description

Peregrine Technologies in Washington DC is seeking a Staff Data Infrastructure Engineer to own the data layer that underpins our real-time platform. You will architect and build systems to ingest, store, and serve massive volumes of operational data, enabling customers to make fast, informed decisions.

This senior IC role demands deep technical judgment and ownership across the stack, from data contracts to deployment.