Data Platform Engineer - Scalable ETL/Spark (Remote)

Lookout Inc; Remote; CAD 130,000 - 160,000; Full time

Job summary

A leading cloud security company in Canada is seeking a Staff Software Engineer to join their Data Engineering team. The role involves developing scalable data engines, managing data from various services, and ensuring high performance across distributed systems. Candidates should have strong software engineering skills, expertise in big data technologies, and familiarity with AWS services. The position offers a competitive salary and the chance to work on critical security solutions.

Benefits

Competitive salary

Bonus and equity options

Health benefits

Qualifications

• 5-8+ years of experience developing and maintaining software.
• Hands-on knowledge of Spark and streaming workloads.
• Experience in performance tuning of distributed platforms.

Responsibilities

• Create and maintain code and repositories.
• Develop and contribute to the test framework.
• Conduct functional and system testing on data pipelines.
• Build automation tests for CI/CD pipeline.

Skills

Strong software engineering fundamentals

Experience with data platforms

Broad understanding of cloud architecture

Rich experience with big data platforms

Spark and Kafka experience

Administration of JVM-based systems

Advanced SQL knowledge

Excellent communication skills

Experience in ETL testing

CI/CD tools knowledge

Education

BS degree in Computer Science or similar

Tools

Scala

Java

Python

Gradle

Maven

AWS

Docker

Job description

A leading cloud security company in Canada is seeking a Staff Software Engineer to join their Data Engineering team. The role involves developing scalable data engines, managing data from various services, and ensuring high performance across distributed systems. Candidates should have strong software engineering skills, expertise in big data technologies, and familiarity with AWS services. The position offers a competitive salary and the chance to work on critical security solutions.