Staff Data Infrastructure Engineer — Real-Time Pipelines

Peregrinetechnologies; New York (NY); On-site; USD 200,000 - 275,000; Full time

Benefits offered by this job

Benefits

Equity (if applicable)

Bonus (if applicable)

Job summary

Peregrinetechnologies in New York is seeking a Staff Data Infrastructure Engineer to lead the architecture and development of our data systems. This role involves designing high-throughput data integration platforms and optimizing data processing pipelines using technologies like Apache Spark.

The ideal candidate has over 8 years of experience in data infrastructure, is passionate about operational excellence, and has a strong background in real-time data processing. The compensation includes a salary range of $200,000 to $275,000 annually, along with benefits and potential equity.

Qualifications

• 8+ years of experience with large-scale data infrastructure systems.
• Experience with Apache Iceberg and schema evolution.
• Hands-on experience with Apache Spark for data processing.

Responsibilities

• Architect and build the data layer for operational data.
• Design high-throughput real-time data integration platforms.
• Optimize distributed data processing pipelines.

Skills

Data infrastructure design

Real-time data processing

Apache Spark

Apache Kafka

Python

AWS

Apache Airflow

Education

Degree in Computer Science, Engineering, or related field

Tools

Apache Iceberg

Kubernetes

Job description

Peregrinetechnologies in New York is seeking a Staff Data Infrastructure Engineer to lead the architecture and development of our data systems. This role involves designing high-throughput data integration platforms and optimizing data processing pipelines using technologies like Apache Spark.

The ideal candidate has over 8 years of experience in data infrastructure, is passionate about operational excellence, and has a strong background in real-time data processing. The compensation includes a salary range of $200,000 to $275,000 annually, along with benefits and potential equity.