Staff Software Engineer (Data Infrastructure)

Peregrine; York and North Yorkshire; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Health insurance

Unlimited PTO

401k

Wellness stipend

HSA

Parental leave

Professional development

Recreational perks

Job summary

Peregrine is seeking a Staff Data Infrastructure Engineer to own the data layer powering its products. You will architect and operate systems ingesting, storing, and serving massive real-time data, enabling customers to make fast, informed decisions.

Join a team that values ownership, collaboration, and empathy. You will work onsite with peers, tackle complex challenges, and influence design decisions across the stack using AWS GovCloud, Iceberg, Spark, Kafka, Airflow, and Kubernetes.

Qualifications

• Degree in Computer Science, Engineering, or related field.
• Strong background in real-time data integration and stream processing.
• Experience with distributed data processing and big data stacks.

Responsibilities

• Designing and operating a high-throughput, real-time data integration platform.
• Architecting a scalable open table format layer for petabyte-scale storage.
• Building and optimizing distributed data pipelines with Spark and streaming tech.
• Driving performance, reliability, and cost efficiency of the data stack.
• Collaborating with platform and product teams to define data contracts and schemas.

Skills

Real-time data integration

Apache Spark

Apache Kafka

Iceberg

Kubernetes

Python/Scala

Data pipelines

AWS GovCloud

Data infra design

Education

Degree in Computer Science or related field

Tools

Apache Iceberg

Airflow

Kubernetes

Apache Spark

Apache Kafka

AWS GovCloud

Job description

• As an engineering team, we believe strongly that empathy improves our solutions. Seeing how people use the product is a priority and the way we get to the right answer
• Engineers will have the opportunity to work closely with our team onsite to understand the variety of use cases that Peregrine serves
• We value both ownership and collaboration—you will take full responsibility for major features and work closely with other engineers to drive them to completion
• We believe that humility and empathy are essential for building the right solutions—you will collaborate directly with our deployment team and users as we iterate to solve their problems. Perseverance and creativity are crucial to executing our vision
• We are looking for a Staff Data Infrastructure Engineer to join our growing team, where you will have deep ownership over the data layer that underpins everything Peregrine does
• You will architect and build the systems that ingest, store, and serve massive volumes of real-time operational data — enabling our customers to make critical decisions with speed and confidence
• You will tackle a wide range of complex challenges, including:
• Designing and operating a high-throughput, real-time data integration platform across diverse customer environments
• Architecting a scalable open table format layer for reliable data storage at petabyte scale
• Building and optimizing distributed data processing pipelines with Apache Spark and adjacent streaming technologies
• Driving performance, reliability, and cost efficiency across the full data infrastructure stack
• Collaborating with platform and product engineering teams to define data contracts, schemas, and integration patterns
• Establishing best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization
• Our stack is constantly evolving but is built on AWS GovCloud, Apache Iceberg, Apache Spark, Apache Kafka, Airflow, Kubernetes, and more

Benefits

• Health insurance: Enjoy top-notch health, dental, and vision insurance. Your health matters
• Unlimited PTO: Assume responsibility for how much time you take off. Trust is everything
• Competitive 401k: Leverage a Peregrine-sponsored 401k plan. And think of your future
• Wellness stipend: Use Peregrine’s wellness stipend to focus your body and mind
• HSA: Set aside money to pay for qualified medical expenses with a HSA
• Parental leave: Because we want to support the Peregrine parents out there
• Professional development: Access a range of professional development courses and resources
• Recreational perks: Enjoy a whole host of travel and leisure discounts, and explore new things

This is a senior individual contributor role for someone who thrives on hard technical problems and brings the experience and judgment to shape foundational infrastructure decisionsCommitted to operational excellence — you build things you’re proud to operateThrive on ambiguity and are energized by defining the right solution to hard, open-ended problemsDeep passion for data infrastructure — you care about building systems that are correct, fast, and resilient at scaleStrong technical vision with the ability to translate complex data requirements into clean, durable infrastructure designsDesire to own significant portions of the data stack end-to-end, from ingestion to servingStrong background in real-time data integration and stream processing, leveraging technologies such as Apache Kafka, Apache Flink, or equivalentsExtensive hands-on experience with Apache Spark for batch and streaming data processing at scaleExperience with Kubernetes and containerized deployment of data workloads8+ years of experience architecting and operating large-scale data infrastructure systems in production environmentsExtensive experience with AWS or comparable cloud platforms, including S3-based data lake architecturesDegree in Computer Science, Engineering, or a related field, or equivalent practical experienceSolid experience with data pipeline orchestration using Airflow or similar toolsDeep expertise with open table formats, particularly Apache Iceberg — including schema evolution, partitioning strategies, compaction, and time travelStrong software engineering fundamentals in Python and/or Scala, with a track record of writing production-quality code