Sr. Data Engineer

Finance Professionals Inc.; Toronto; On-site; CAD 90,000 - 130,000; Full time

Job summary

A leading financial institution in Downtown Toronto is seeking a Senior Data Engineer. This role involves designing robust ETL processes, building data pipelines, and collaborating with architects to ensure compliance with best practices. The ideal candidate will have 5-7 years of experience in data engineering and proficiency in big data technologies such as Spark and Hadoop.

Qualifications

• 5-7 years of experience in data engineering roles.
• Solid experience with big data technologies.
• Strong understanding of data modelling methodologies.

Responsibilities

• Design and maintain ETL processes.
• Build and optimize data pipelines.
• Collaborate with architects to ensure data quality.

Skills

Building batch and real-time data pipelines

Proficiency in SQL

Python or Scala

Experience with cloud-based data platforms

CI/CD tools

Docker and Kubernetes

Tools

Spark

Hadoop

Airflow

NiFi

Kafka

Snowflake

Databricks

AWS

Azure

GCP

Job description

Overview

Our client, a leading financial institution in Downtown Toronto, is looking for a Senior Data Engineer to work with business stakeholders and cross-functional teams to understand data requirements and deliver scalable data solutions. The successful candidate will have the opportunity to work with one of the Top 5 Banks in Canada.

Responsibilities

• Design, develop, and maintain robust ETL processes to extract, transform, and load data from various sources into our data platform.
• Build large-scale batch and event-driven data pipelines using cloud and on-premises hybrid data platform topology.
• Work closely with data architects to review solutions and data models and ensure adherence to data platform architecture guidelines and engineering best practices.
• Take ownership of end-to-end deliverables and ensure high-quality software development while fulfilling all operational and functional requirements promptly.
• Implement and enforce data quality standards and best practices while collaborating with data governance teams to ensure compliance with data policies and regulations.
• Optimize data integration workflows for performance and reliability.
• Troubleshoot and resolve data integration and data processing issues.
• Leverage best practices in continuous integration and delivery using Data Ops pipelines.
• Apply design-thinking and agile mindset in working with other engineers and business stakeholders to continuously experiment, iterate, and deliver on new initiatives.
• Stay informed about emerging technologies and trends in the data engineering domain.
• Lead, mentor, and inspire a team of data engineers to achieve high performance levels.

Must-Have Skills

• 5-7 years of experience building batch and real-time data pipelines leveraging big data technologies and distributed data processing using Spark, Hadoop, Airflow, NiFi, and Kafka.
• Proficiency in writing and optimizing SQL queries and at least one programming language like Python and/or Scala.
• Experience with cloud-based data platforms (Snowflake, Databricks, AWS, Azure, GCP).
• Expertise using CI/CD tools and working with Docker and Kubernetes platforms.
• Experience with the following DevOps and agile best practices.
• Experience with data modelling tools and methodologies.

Nice-To-Have Skills

• Experience with OpenShift, S3, Trino, Ranger and Hive
• Knowledge of machine learning and data science concepts and tools.
• Knowledge with BI & Analytics tools such as Tableau and Superset

Education

Finance professional is committed to creating an inclusive environment where all team members and clients feel like they belong. We seek applicants with a wide range of abilities and we provide an accessible candidate experience. We advocate for you and welcome anyone regardless of race, color, religion, national origin, sex, physical or mental disability, or age.