Databricks Data Engineer - Spark, Python, AWS Lakehouse

The Citco Group; Toronto; On-site; CAD 80,000 - 110,000; Full time

Job summary

A global financial services firm in Toronto is seeking a Data Engineer to build data pipelines and transform data into actionable insights. You will work in a cross-functional team using agile methodologies, focusing on modern data engineering practices within an AWS cloud environment. The ideal candidate has 4 years of experience and strong skills in Python and Spark, as well as knowledge of Databricks and AWS services. The firm offers a supportive working environment and various benefits.

Benefits

Training and education support

Flexible working arrangements

Diversity and inclusion initiatives

Qualifications

• 4 years of experience in data engineering.
• 1 year of hands-on experience with Databricks platform.
• Strong programming skills in Python.

Responsibilities

• Develop and maintain data pipelines using Databricks Lakehouse and Delta Lake.
• Implement ETL / ELT workflows using Spark (Python) in Databricks environment.
• Create and optimize Spark jobs for efficient data processing.

Skills

Python

Spark

AWS

Data modelling

ETL / ELT principles

CI / CD

Agile methodologies

SQL

Real-time data processing

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

AWS S3

Delta Lake

Apache Hive

NoSQL

Hadoop

Kafka

Job description

A global financial services firm in Toronto is seeking a Data Engineer to build data pipelines and transform data into actionable insights. You will work in a cross-functional team using agile methodologies, focusing on modern data engineering practices within an AWS cloud environment. The ideal candidate has 4 years of experience and strong skills in Python and Spark, as well as knowledge of Databricks and AWS services. The firm offers a supportive working environment and various benefits.