Senior Data Engineer, AI Centre of Excellence

Ernst & Young Advisory Services Sdn Bhd; Toronto; On-site; CAD 80,000 - 120,000; Full time

Job summary

A leading global consultancy is seeking a Senior Data Engineer to design and optimize data pipelines for advanced AI applications. The ideal candidate will have 4-6 years of experience in data engineering, proficiency in Azure Data Factory, and a strong command of MLOps processes. This role offers comprehensive benefits and an inclusive work environment.

Benefits

Comprehensive medical, dental, and prescription drug coverage

Mental health benefits

Employee Assistance Program

Group savings plans

Qualifications

• 4-6 years of professional experience in data engineering with a focus on ML Ops and data pipelines.
• Hands-on experience in ETL processes and large-scale data integration.
• Strong analytical skills to troubleshoot data-related issues.

Responsibilities

• Design, build, and maintain scalable data pipelines using Azure Data Factory and Databricks.
• Lead optimization of ETL processes ensuring data quality.
• Implement robust MLOps workflows for model deployment and monitoring.

Skills

Azure Data Factory

Databricks

Apache Spark

SQL

Python

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Azure

AWS

Google Cloud

Job description

At EY, we’re all in to shape your future with confidence.

We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.

Join EY and help to build a better working world.

The Opportunity

As a Senior Data Engineer (ML Ops & Pipelines) at the AI Centre of Excellence (COE) within the Office of the Chief Technology Officer at EY, you will play a pivotal role in architecting, developing, and optimizing scalable data pipelines and MLOps solutions to power advanced AI and machine learning applications.

Your Key Responsibilities

• Pipeline Architecture & Development: Design, build, and maintain scalable, reliable, and high-performance data pipelines using Azure Data Factory, Databricks, and Spark to support machine learning and analytics workloads.
• ETL & Data Integration: Lead the creation and optimization of ETL processes for structured and unstructured data, ensuring data quality, lineage, and compliance with enterprise standards.
• Feature Store Management: Develop and manage feature stores to streamline the reusability and governance of engineered features for AI and ML models.
• ML Ops and Automation: Implement robust MLOps workflows that enable continuous integration, deployment, monitoring, and retraining of ML models in production environments.
• Collaboration: Partner with data scientists, ML engineers, and business stakeholders to translate analytic requirements into scalable data solutions, and provide guidance on best practices for data engineering in an AI context.
• Communication: Excellent communication skills to collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders.
• Cloud and Security: Ensure secure data movement and storage by applying security best practices and compliance protocols within Azure cloud environments.
• Continuous Improvement: Stay current with advancements in data engineering, MLOps, and cloud technologies, and proactively apply new techniques to enhance existing solutions.

Skills and Attributes for Success

• Education: Bachelor’s or Master’s degree in Computer Science, Information Management, Data Engineering, or a closely related technical field.
• Experience: 4-6 years of professional experience in data engineering, with a strong emphasis on ML Ops, data pipelines, and large-scale ETL/ELT processes.
• Technical Skills: Advanced proficiency in Azure Data Factory, Databricks, and Apache Spark for designing and implementing complex data pipelines and distributed data processing jobs; expertise in ETL/ELT processes, data wrangling, and integrating diverse data sources into unified, high-quality datasets suitable for AI and analytics applications.
• MLOps and Workflow Management: Hands-on experience with feature store management and supporting ML and AI workflows in production environments; solid understanding of MLOps principles, including CI/CD for machine learning, automation of model deployment, and monitoring practices.
• Programming and Scripting: Proficient in SQL and Python for data transformation, orchestration, and automation tasks; familiarity with additional programming languages (e.g., Scala, R) is a plus.
• Cloud Technologies: Experience with cloud-native tools and services (e.g., Azure, AWS, Google Cloud) for data storage, processing, and orchestration.
• Data Security and Compliance: Working knowledge of data security, privacy, and compliance regulations (e.g., GDPR, HIPAA) in cloud-based solutions, ensuring data governance and protection.
• Analytical and Problem-Solving Skills: Strong analytical skills to troubleshoot data-related issues and optimize data workflows for performance and efficiency.

What We Offer You

The EY benefits package is designed to support your physical, emotional, financial, and social wellbeing.

We offer comprehensive medical, dental, and prescription drug coverage, as well as mental health benefits, a robust Employee Assistance Program and group savings plans to promote your overall wellbeing.

Get involved in meaningful volunteering through EY Ripples and make a positive impact in the community.

Diversity and Inclusion at EY

Diversity and inclusiveness are at the heart of who we are and how we work.

We’re committed to fostering an environment where differences are valued, policies and practices are equitable, and our people feel a sense of belonging.

EY is an equal opportunities employer and welcomes applications from all candidates.