Senior DataOps / Cloud Data Engineer

BITS Recruiting; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

BITS Recruiting in Toronto seeks a Senior DataOps/Cloud Data Engineer to design, build, and optimize cloud-based data pipelines and lakehouse solutions. You will work hands-on with Azure Data Factory, Databricks, Informatica, Python, and SQL to enable data integration and governance for a large public-sector modernization project.

Responsibilities include ETL/ELT development, data modeling, and CI/CD with Azure DevOps, along with data lineage and collaboration with stakeholders.

Qualifications

• Senior Data Engineer, Cloud Data Engineer, or DataOps Engineer experience.
• Hands-on with Databricks, Azure Data Factory, and Informatica.
• Strong Python and SQL development, incl. T-SQL/PL-SQL.
• Experience designing Databricks Medallion Architecture.
• Knowledge of Delta Lake and Lakehouse architecture.
• Experience managing enterprise data pipelines/workflows.
• ETL/ELT development, orchestration, automation, and deployment.
• Integrating cloud platforms with on-premises sources, especially Oracle.
• Data warehousing, lakes, lakehouses, dimensional modelling, stars.
• CI/CD and Azure DevOps.
• Working with large/high-volume data environments.
• Data quality, governance, lineage, security practices.
• Troubleshooting, analytical, problem-solving, communication.
• Experience in Agile and/or Waterfall environments.

Responsibilities

• Design and optimize Azure Data Factory and Databricks pipelines from Oracle/on-prem sources to cloud lakehouse.
• Develop solutions using Databricks Medallion Architecture and Delta Lake.
• Design relational, dimensional, and enterprise data models.
• Translate Informatica ETL into Azure Data Factory and Databricks ELT.
• Create data pipelines, workflows, jobs, scripts, and automation with Python, SQL, and related tools.
• Support CI/CD, deployment automation, and data provisioning with Azure DevOps.

Skills

Senior Data Engineer

DataOps Engineer

SQL

Python

Azure Data Factory

Databricks

Informatica

Delta Lake

Lakehouse architecture

ETL/ELT development

CI/CD

Azure DevOps

Oracle

Data governance

Troubleshooting

Agile/Waterfall

Tools

Databricks

Azure Data Factory

Informatica

SSIS

Microsoft Fabric

Azure DevOps

Job description

Contract: September 1, 2026 – September 1, 2027

We are seeking a Senior DataOps / Cloud Data Engineer to support a large public‑sector data modernization initiative in Toronto. The successful candidate will design, develop, optimize, and support cloud‑based data pipelines and lakehouse solutions using Azure Data Factory, Databricks, Informatica, Python, and SQL.

This is a highly hands-on role focused on data integration, ETL/ELT development, Medallion Architecture, data migration, pipeline orchestration, automation, and DataOps.

Key Responsibilities

• Design, develop, and optimize Azure Data Factory and Databricks pipelines from Oracle and other on-premises data sources to cloud Lakehouse environments.
• Develop and implement solutions using Databricks Medallion Architecture and Delta Lake.
• Design and optimize relational, dimensional, and enterprise data models.
• Translate existing Informatica ETL processes into Azure Data Factory and Databricks ELT solutions.
• Develop data pipelines, workflows, jobs, scripts, and automation using Python, SQL, T-SQL, PL/SQL, ADF, Informatica, SSIS, and/or Microsoft Fabric.
• Implement data orchestration, parallel processing, data movement, and pipeline automation.
• Support large-scale data migration from OLTP/OLAP environments to cloud-based platforms.
• Implement data quality, validation, profiling, cleansing, monitoring, and data governance practices.
• Develop and maintain data lineage and documentation to provide traceability of data movement and transformations.
• Support CI/CD, deployment automation, and data provisioning using tools such as Azure DevOps.
• Troubleshoot production data pipeline issues and implement corrective solutions.
• Work with technical teams, business stakeholders, and project teams to support solution design and integration.
• Create technical and solution documentation, including data mappings, models, requirements, and design documentation.
• Provide technical guidance and knowledge transfer to internal team members.

Required Skills & Experience

• Strong experience as a Senior Data Engineer, Cloud Data Engineer, or DataOps Engineer.
• Hands-on experience with Databricks, Azure Data Factory, and Informatica.
• Strong development experience with Python and SQL, including T-SQL and/or PL/SQL.
• Experience designing and implementing Databricks Medallion Architecture.
• Strong knowledge of Delta Lake and Lakehouse architecture.
• Experience developing and managing enterprise data pipelines and workflows.
• Experience with ETL/ELT development, orchestration, automation, and deployment.
• Experience integrating cloud platforms with on-premises databases and data sources, particularly Oracle.
• Experience with data warehousing, data lakes, lakehouses, dimensional modelling, fact/dimension models, and star schemas.
• Experience with CI/CD and Azure DevOps.
• Experience working with large and complex datasets and high-volume data environments.
• Experience with data quality, data governance, data lineage, and data security practices.
• Strong troubleshooting, analytical, problem-solving, and communication skills.
• Experience working in Agile and/or Waterfall environments.

AI-enabled tools may be used to sort applications based on job-related criteria. All AI generated results are vetted by our team and the decision of which candidates move forward is always made by a human.