OPEN: Staff Data Engineer / Data Architect (Azure / Databricks)

Cpus Engineering Staffing Solutions Inc.; Oshawa; On-site; CAD 172,800 - 211,200; Full time

Job summary

Cpus Engineering Staffing Solutions Inc. is seeking a Staff Data Engineer/Data Architect in Oshawa. You will lead the architecture, design, and implementation of scalable data pipelines to deliver top-notch data products.

The position requires extensive knowledge in data modeling and cloud technologies, with 6 to 8 years of relevant experience. This role is hybrid, allowing for 3 days of remote work.

Qualifications

• Extensive knowledge in designing data models to solve business problems.
• Experience guiding data lake ingestion and data modeling projects in a cloud environment.
• 6 to 8 years in data modeling and solution architecture in Big Data.

Responsibilities

• Lead architecture and design for data pipelines and data products.
• Ensure security of data in transit and at rest.
• Collaborate with analysts and engineers to develop data pipelines.

Skills

Data modeling

Cloud data solutions (Azure & Databricks)

Data pipeline design

Performance optimization

Event-driven architecture

Team leadership

Education

Four-year University education in computer science or relevant field

Tools

Azure Data Factory

Azure Data Lake

Azure SQL Databases

Azure Data Warehouse

Power BI

Job description

Location: 1908 Colonel Sam Drive, Oshawa

Work Mode: Hybrid – 3 days remote

Hours per Week: 35 hours

Hourly Rate: $92.58

Job Overview

As a Staff Data Engineer/Data Architect, you will be responsible for leading the architecture, design and delivery of scalable data pipelines and data products which enable innovative, customer‑centric digital experiences.

• You will be working as part of a cross‑discipline agile team who helps each other solve problems across all business areas.
• You will be a thought leader and subject matter expert on the data lakehouse, data warehousing and modeling activities for the team and use your influence to ensure that the team produces best‑in‑class data solutions that leverage repeatable, maintainable, and well‑documented design patterns.
• You will employ best practice in development, security, accessibility and design to achieve the highest quality of service for our customers.
• Lead the architecture, design and oversee implementation of modular and scalable data ELT/ETL pipelines and data infrastructure leveraging the wide range of data sources across the organization.
• Design curated common data models that offer an integrated, business‑centric single source of truth for business intelligence, reporting, and downstream system use.
• Work closely with infrastructure and cyber teams to ensure data is secure in transit and at rest.
• Create, guide and enforce code templates for delivery of data pipelines and transformations for structured, semi‑structured and unstructured data sets.
• Develop modeling guidelines that ensure model extensibility and reuse by employing industry standard disciplines for building facts, dimensions, bridge, aggregates, slowly changing dimensions and other dimensional and fact optimizations.
• Establish standards database system fields, including primary and natural key combinations that optimize join performance in a multi‑domain, multiple subject area physical (structured zone) and semantic model (curated zone).
• Ensure model extensibility by employing industry standard disciplines for building facts, dimensions, bridge, aggregates, slowly changing dimensions and other dimensional and fact optimizations.
• Transform data and map to more valuable and understandable semantic layer sets for consumption, transitioning from system‑centric language to business‑centric language.
• Collaborate with business analysts, data scientists, data engineers, data analysts and solution architects to develop data pipelines to feed our data marketplace.
• Introduce new technologies to the environment through research and POCs, and prepare POC code designs that can be implemented and productionized by developers.
• Work with tools in the Microsoft Stack; Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power BI.
• Work within the agile SCRUM work management framework in delivery of products and services, including contributing to feature & user story backlog item development, and utilizing related Kanban/SCRUM toolsets.
• Document as‑built architecture and designs within the product description.
• Design data solutions that enable batch, near‑real‑time, event‑driven, and/or streaming approaches depending on business requirements.
• Design & advise on orchestration of data pipeline execution to ensure data products meet customer latency expectations, dependencies are managed, and datasets are as up‑to‑date as possible, with minimal disruption to end‑customer use.
• Ensure that designs are implemented with proper attention to data security, access management, and data cataloging requirements.
• Approve pull requests related to production deployments.
• Demonstrate solutions to business customers to ensure customer acceptance and solicit feedback to drive iterative improvements.
• Assist in troubleshooting issues for datasets produced by the team (Tier 3 support), on an as‑required basis.
• Guide data modelers, business analysts and data scientists in the build of models optimized for KPI delivery, actionable feedback/writeback to operational systems and enhancing the predictability of machine learning models and experiments.

Qualifications

• Requires an extensive knowledge in designing a data model to solve a business problem, specifying a data pipeline design pattern to bring data into a data warehouse, optimizing data structures to achieve required performance, designing low‑latency and/or event‑driven patterns of data processing, and creation of a common data model to support current and future business needs.
• This knowledge is normally acquired through the completion of a four‑year University education in computer science, computer/software engineering or other relevant programs within data engineering, data analysis, artificial intelligence, or machine learning.
• Experience guiding data lake ingestion and data modeling projects in a cloud environment (Azure & Databricks).
• Experience in modeling relational and in‑memory models with star/snowflake schemas.
• Experience with designing and implementing event‑driven (pub/sub), near‑real‑time, or streaming data solutions, involving structured, semi‑structured and unstructured data across various platforms.
• A period of over 6 years and up to and including 8 years in data modeling, data warehouse design, and data solution architecture in a Big Data environment is considered necessary to gain this experience.