Senior Data Engineer/Data Architect at Oshawa

CPUS Engineering Staffing Solutions Inc.; Oshawa; Vor Ort; CAD 90.000 - 130.000; Vollzeit

Zusammenfassung

Ontario Power Generation (OPG) is seeking a Senior Data Engineer to design, build, and maintain data lake and data warehouse products that support analytics and reporting. You will implement robust ELT/ETL pipelines, ensure data security, and collaborate with data architects and analysts to enable scalable, reliable data-driven applications.

The role emphasizes production-grade pipelines, data quality, and governance, with a focus on Microsoft Azure services, Databricks, and modern data

Qualifikationen

• 4-year university degree in computer science or related field.
• Over 4 years experience building data pipelines and data processing frameworks.
• Experience as a Data Engineer building data pipelines.
• Fluent in Python, PySpark, SparkSQL and SQL.
• Experience with Azure Data Factory, ADLS, Synapse Analytics and Databricks.
• Experience building data pipelines for Data Lakehouses and Data Warehouses.
• Strong understanding of data structures and data processing frameworks.
• Knowledge of data governance and data quality principles.
• Strong communication to translate technical details to stakeholders.

Aufgaben

• Build and productionize modular, scalable data ELT/ETL pipelines and data infrastructure.
• Ingest and curate data pipelines for a single source of truth for BI and reporting.
• Collaborate with Data Architect and security teams to ensure data in transit and at rest is secure.
• Prepare and optimize datasets with lineage and quality controls.
• Support BI analysts in data modeling for visualization and reporting.
• Provide Tier 2 production support for ingestion, transformation, and performance issues.
• Contribute to data catalog maintenance and metadata management.
• Develop optimized data pipelines using Python, Spark and SQL across XML/CSV/JSON/REST APIs.
• Document pipelines and use source control for maintainable code.

Kenntnisse

Python

PySpark

SparkSQL

SQL

Azure Data Factory

Azure Data Lake

Azure Synapse Analytics

Databricks

Data modeling

Data governance

Communication

Ausbildung

4-year University education in CS/Software Eng

Tools

ADLS

Data Lakehouses

Power BI

Jobbeschreibung

Ontario Power Generation (OPG) is looking for a dynamic, strategic and results‑driven professional to join our team in the role of Senior Data Engineer.

Job Overview

Reporting to the Senior Manager, Data Lake & Platform Engineering, this position is responsible to build and support the data driven applications which enable innovative, customer centric digital experiences. You will build reliable, supportable & performant data lake & data warehouse products to meet the organization’s need for data to drive reporting analytics, applications, and innovation. You will employ best practice in development, security and accessibility to achieve the highest quality of service for our customers.

Key Accountabilities

• Build and productionize modular and scalable data ELT/ETL pipelines and data infrastructure leveraging the wide range of data sources across the organization.
• Implement data ingestion and curation data pipelines that offer an integrated, business-centric single source of truth for business intelligence, reporting, and downstream system use, in collaboration with Data Architect.
• Work closely with Data Architect, infrastructure and cyber teams to ensure data is secure in transit and at rest.
• Clean, prepare and optimize datasets for performance, ensuring lineage and quality controls are applied throughout the data integration cycle.
• Support Business Intelligence Analysts in modelling data for visualization and reporting, using dimensional data modeling and aggregation optimization methods.
• Provide production support for issues related to ingestion, data transformation and pipeline performance, data accuracy and integrity.
• Collaborate with data architect, business analysts, data scientists, data engineers, data analysts, solution architects and data modelers to develop data pipelines to feed our data marketplace.
• Assist in identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, re‑designing infrastructure for greater scalability, etc.
• Work with tools in the Microsoft Stack; Azure Data Factory, Azure Data Lake, Azure SOL Databases, Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Collibra, and Power Bl.
• Work within the agile SCRUM work management framework in delivery of products and services, including contributing to feature & user story backlog item development, and utilizing related Kanban/SCRUM toolsets.
• Assist in building data catalog and maintenance of relevant metadata for datasets published for enterprise use.
• Develop optimized, performant data pipelines and models at scale using technologies such as Python, Spark and SOL, consuming data sources in XML, CSV, JSON, REST APls, or other formats.
• Document as-built pipelines and data products within the product description, and utilize source control to ensure a maintainable code‑base.
• Implement orchestration of data pipeline execution to ensure data products meet customer latency expectations, dependencies are managed, and datasets are as up-to-date as possible, with minimal disruption to end‑customer use.
• Create tooling to help with day to day tasks, and reduce toil via automation wherever possible.
• Work with Continuous Integration/Continuous Delivery and DevOps pipelines to automate infrastructure, code delivery and product enhancement isolation and proper release management and versioning.
• Monitor the ongoing operation of in‑production solutions, assist in troubleshooting issues, and provide Tier 2 support for datasets produced by the team, on an as‑required basis.
• Implement and manage appropriate access to data products via role‑based access control.
• Write and perform automated unit and regression testing for data product builds, assist with user acceptance testing and system integration testing as required, and assist in design of relevant test cases.
• Participate in peer code review sessions, and approve non-production pull requests.

Qualifications

• 4-year University education in computer science, computer/software engineering or other relevant programs within data engineering, data analysis, artificial intelligence, or machine learning.
• Over 4 years experience building data pipelines and creating data processing frameworks using Python, PySpark, SparkSQL and SQL.
• Experience as a Data Engineer building data pipelines.
• Fluent in creating data processing frameworks using Python, PySpark, SparkSQL and SQL.
• Experience with Azure Data Factory, ADLS, Synapse Analytics and Databricks.
• Experience building data pipelines for Data Lakehouses and Data Warehouses.
• Good understanding of data structures and data processing frameworks.
• Knowledge of data governance and data quality principles.
• Effective communication skills to translate technical details to non-technical stakeholders.

The successful candidate will exhibit uncompromising integrity and commitment to upholding corporate values, and the OPG Code of Business Conduct.

OPG is committed to fostering an inclusive, equitable, and accessible environment. If you require accommodation during the selection process, please contact [e-mail] .

What makes a career at OPG different?

With operations across Ontario, OPG is one of the most diverse power producers in North America. As the largest generator in Ontario, we meet approximately 50% of the province’s electricity needs, largely from low-carbon sources like nuclear and hydro.

As we work to achieve our vision of Electrifying life in one generation, OPG and our family of companies are also helping advance the development of new low-carbon technologies such as Small Modular Reactors (SMRs), refurbishment projects, and electrification initiatives to help power the growing demands of a growing economy. Join OPG and make history.

The Darlington New Nuclear Project is leading the way in the advancement of Small Modular Reactor (SMR) technology in Canada – the future of nuclear power generation! Using new and innovative technologies such as small modular reactors, hydrogen and battery storage, while expanding our hydroelectric and nuclear portfolios, we are changing the electricity landscape in Ontario.

The base salary range considers many factors including, but not limited to experience, education, and training, including any collective agreement requirements for union represented positions. It is not typical for the salary to be offered near the top of the range, and salary is dependent on numerous factors. For management roles, the base salary range does not represent the total compensation package. The total compensation package for regular full-time management roles includes pay-for-performance programs for annual and medium time periods. Maintaining a high-performance culture and excellence is a core expectation of every member of our leadership team and is rewarded through the established compensation framework.

OPG is committed to employment equity. As such, we encourage applicants from equity‑seeking communities (Indigenous Peoples, racialized persons, persons with disabilities, and women). We strongly believe that alleviating the under‑representation of equity‑seeking individuals will create a stronger OPG team and allow us to better serve the needs of our diverse communities.

In order to fulfill the above‑mentioned purpose, priority in hiring may be given to qualified persons who self‑identify as a member of equity‑seeking groups as identified in the application process. This initiative constitutes a special program under the Human Rights Code/Canadian Human Rights Act.

All job postings at OPG are to fill existing vacancies within our organization.

OPG may use artificial intelligence (AI) tools as part of the applicant screening process. However, applications will also be reviewed by a member of our Recruitment team to ensure a fair and thorough assessment.

Please submit your application online at https://jobs.opg.com/ . OPG thanks all those who apply; however, only candidates considered for an interview will be contacted.