Real-Time Data Infrastructure Engineer — Own the Stack

Peregrinetechnologies; San Francisco (CA); On-site; USD 160,000 - 220,000; Full time

Benefits offered by this job

Equity options

Health benefits

Job summary

Peregrinetechnologies in San Francisco is looking for a Data Infrastructure Engineer to own the data layer essential for their operations. You will tackle challenges in real-time data processing and contribute to foundational infrastructure decisions.

Ideal candidates will have 2-5 years of experience in large-scale data systems, proficiency in technologies like Apache Spark and AWS, and a strong background in Python or Scala. The salary range for this position is between $160,000 and $220,000 annually, along with benefits.

Qualifications

• 2-5 years of experience operating large-scale data infrastructure systems in production environments.
• Experience with open table formats, particularly Apache Iceberg.
• Extensive hands-on experience with Apache Spark for batch and streaming data processing.

Responsibilities

• Design and operate a real-time data integration platform.
• Architect a scalable open table format layer.
• Build and optimize distributed data processing pipelines.

Skills

Large-scale data infrastructure operation

Apache Iceberg

Apache Spark

Real-time data integration and stream processing

Python and/or Scala

AWS

Kubernetes

Education

Degree in Computer Science, Engineering, or related field

Tools

Apache Kafka

Airflow

Job description

Peregrinetechnologies in San Francisco is looking for a Data Infrastructure Engineer to own the data layer essential for their operations. You will tackle challenges in real-time data processing and contribute to foundational infrastructure decisions.

Ideal candidates will have 2-5 years of experience in large-scale data systems, proficiency in technologies like Apache Spark and AWS, and a strong background in Python or Scala. The salary range for this position is between $160,000 and $220,000 annually, along with benefits.