Petabyte-Scale Data Platform Engineer

HRB; Toronto; On-site; CAD 80,000 - 120,000; Full time

Job summary

A technology company in Toronto seeks a skilled engineer to build scalable data platforms. You will design and optimize the data platform that powers our API, handling data streaming and storage. Candidates should have 3+ years in data engineering, expertise in Java and Python, and experience with tools such as Kafka and Flink. Join us to tackle complex backend challenges and contribute to impactful product development.

Qualifications

• 3+ years of experience in platform engineering or data engineering.
• 2+ years of experience designing and optimizing data pipelines at TB to PB scale.
• Familiarity with lake-house architectures like Iceberg and Delta.
• Experience with real-time data processing tools like Kafka, Flink and Spark.

Responsibilities

• Own projects enhancing data replication, storage, and reporting capabilities.
• Design scalable storage solutions for handling petabytes of data.
• Develop and maintain real-time data systems.
• Write clean, maintainable code in Java and Python.

Skills

Platform engineering

Data engineering

Java

Python

System design

Problem-solving

Tools

Apache Iceberg

Apache Kafka

Apache Flink

AWS S3

AWS DynamoDB

Apache Spark

AWS CDK

Job description

A technology company in Toronto seeks a skilled engineer to build scalable data platforms. You will design and optimize the data platform that powers our API, handling data streaming and storage. Candidates should have 3+ years in data engineering, expertise in Java and Python, and experience with tools such as Kafka and Flink. Join us to tackle complex backend challenges and contribute to impactful product development.