Lead Data Platform Architect – Real-Time, TB-PB Scale

HRB; Toronto; On-site; CAD 100,000 - 130,000; Full time

Job summary

A technology company is seeking an experienced engineer to develop scalable data platforms. Responsibilities include enhancing data replication, optimizing pipelines, and managing infrastructure for high availability. The ideal candidate has over 6 years in data engineering, with strong skills in Java and real-time workflows. This role requires collaboration across teams and a strategic approach to architecture.

Qualifications

• 6+ years of experience in platform engineering or data engineering.
• 4+ years of experience designing and optimizing data pipelines at TB to PB scale.
• Proficient in Java with a focus on clean, maintainable code.
• Strong problem-solving skills and a collaborative mindset.

Responsibilities

• Own projects aimed at enhancing data replication, storage, and reporting capabilities.
• Build and optimize efficient streaming and batch data pipelines.
• Design scalable storage solutions for handling petabytes of IoT data.
• Develop and maintain real-time data systems.

Skills

Java

Python

Data engineering

System design

Real-time workflows

Big data processing

Collaboration

Tools

AWS S3

AWS Kinesis

Apache Kafka

Apache Flink

Apache Iceberg

Job description

A technology company is seeking an experienced engineer to develop scalable data platforms. Responsibilities include enhancing data replication, optimizing pipelines, and managing infrastructure for high availability. The ideal candidate has over 6 years in data engineering, with strong skills in Java and real-time workflows. This role requires collaboration across teams and a strategic approach to architecture.