Sr. Data Engineer

AutoTrader.ca; Ontario; On-site; CAD 100,000 - 140,000; Full time

Job summary

A leading company in the data technology space is seeking a Sr. Data Engineer to manage core data systems. This role involves coding and deploying Data Movement and Integration, while collaborating with various teams to optimize workflows and ensure data integrity. With attractive benefits and a competitive salary including bonuses, this is an excellent opportunity for experienced professionals looking to impact data-driven strategies.

Benefits

Gym discounts

Employee and Family Assistance programs

Virtual wellness events

Training budgets

Internal training programs

3% pension matching

Income protection options

Qualifications

• 10+ years of IT experience in large-scale architecture, operations, and design.
• 3+ years with Cloud stacks experience (AWS, Azure, GCP).
• Proficiency in SQL, Scala, Python, Spark, Kafka.

Responsibilities

• Develop and maintain scalable data pipelines and build new integrations.
• Implement processes to monitor data quality and ensure accuracy.
• Collaborate with analytics to improve data models for decision-making.

Skills

Communication

Analytical

Problem-Solving

Attention to Detail

Organizational

Multi-Tasking

Education

University Degree or College Equivalent

Tools

Databricks

DataStage

SVN

Git

Jenkins

Azure DevOps

Job description

Experience

Leveraging AI, Generative AI (GenAI) to enhance engineering productivity, automate repetitive tasks, and optimize workflows. Candidates should demonstrate the ability to integrate AI-driven solutions into their daily work—such as code generation, debugging, reviews, documentation, and decision support—to improve efficiency for themselves and their teams. A proactive approach to exploring and implementing AI tools that drive innovation and streamline development processes is highly valued.

Sr. Data Engineer Role Overview

The Sr. Data Engineer is responsible for our core data systems – Data Warehouse / Data Lake / BI. This position involves coding, reviewing, and accepting Data Movement and Data Integration code for production deployment. The candidate will be involved from project inception through design, development, and into post-production support, ensuring high-quality outputs that do not impact production stability. The role requires utilizing a full suite of integration tools and technologies while adhering to configuration management and security best practices. It encompasses design, development, and operational responsibilities.

Responsibilities

1. Develop and maintain scalable data pipelines and build new integrations to support increasing data volume and complexity.
2. Collaborate with analytics and business teams to improve data models that support business intelligence, fostering data-driven decision-making.
3. Implement processes and systems to monitor data quality, ensuring data accuracy and availability for stakeholders.
4. Write unit and integration tests, contribute to engineering documentation, and maintain work records.
5. Perform data analysis to troubleshoot and resolve data-related issues.
6. Work closely with frontend and backend engineers, product managers, and analysts.
7. Define data assets, documentation, and transformation jobs to support data models.
8. Design data integrations and data quality frameworks.
9. Evaluate open-source and vendor tools for data lineage.
10. Develop long-term data platform architecture strategies in collaboration with business and engineering teams.

Qualifications

1. University Degree or College Equivalent
2. 10+ years of IT experience in large-scale architecture, operations, and design
3. 3+ years supporting applications with Cloud stacks (AWS, Azure, GCP)
4. 3+ years with data integration/ETL tools (e.g., Databricks, DataStage)
5. Knowledge of SDLC in waterfall and agile methodologies
6. Hands-on experience with source code management (SVN, Git) and CI tools (Jenkins, Azure DevOps)
7. Proficiency in SQL, Scala, Python, Spark, Kafka, and related languages
8. Experience with high-scale data processing and distributed applications
9. Cloud environment experience (Azure, GCP preferred)
10. Experience with enterprise solution analysis, development, and implementation
11. Experience with transaction processing software
12. Understanding workload management, schedulers, and distributed architectures
13. Experience with Java EE technologies and scripting languages (bash, PERL, Python)
14. Knowledge of data warehousing, analytics, BI, and visualization tools
15. SQL experience with relational databases
16. 5+ years experience on Linux, AIX, z/OS

Knowledge, Skills, and Abilities

1. Excellent communication skills (written and oral) and interpersonal skills
2. Strong analytical and problem-solving skills with flexibility
3. Attention to detail
4. Strong organizational and multi-tasking skills

What’s in it for you…

- Benefits from Day 1 include gym discounts, Employee and Family Assistance programs, virtual wellness events, training budgets, and internal training programs.

Financial Planning

- 3% pension matching and various income protection options to help invest in your future.

Competitive Salary

- Includes an annual bonus structure, 3% CPP matching, and other benefits.