Senior DataOps & Cloud Data Engineer - Azure & Databricks

BITS Recruiting; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

BITS Recruiting in Toronto seeks a Senior DataOps/Cloud Data Engineer to design, build, and optimize cloud-based data pipelines and lakehouse solutions. You will work hands-on with Azure Data Factory, Databricks, Informatica, Python, and SQL to enable data integration and governance for a large public-sector modernization project.

Responsibilities include ETL/ELT development, data modeling, and CI/CD with Azure DevOps, along with data lineage and collaboration with stakeholders.

Qualifications

• Senior Data Engineer, Cloud Data Engineer, or DataOps Engineer experience.
• Hands-on with Databricks, Azure Data Factory, and Informatica.
• Strong Python and SQL development, incl. T-SQL/PL-SQL.
• Experience designing Databricks Medallion Architecture.
• Knowledge of Delta Lake and Lakehouse architecture.
• Experience managing enterprise data pipelines/workflows.
• ETL/ELT development, orchestration, automation, and deployment.
• Integrating cloud platforms with on-premises sources, especially Oracle.
• Data warehousing, lakes, lakehouses, dimensional modelling, stars.
• CI/CD and Azure DevOps.
• Working with large/high-volume data environments.
• Data quality, governance, lineage, security practices.
• Troubleshooting, analytical, problem-solving, communication.
• Experience in Agile and/or Waterfall environments.

Responsibilities

• Design and optimize Azure Data Factory and Databricks pipelines from Oracle/on-prem sources to cloud lakehouse.
• Develop solutions using Databricks Medallion Architecture and Delta Lake.
• Design relational, dimensional, and enterprise data models.
• Translate Informatica ETL into Azure Data Factory and Databricks ELT.
• Create data pipelines, workflows, jobs, scripts, and automation with Python, SQL, and related tools.
• Support CI/CD, deployment automation, and data provisioning with Azure DevOps.

Skills

Senior Data Engineer

DataOps Engineer

SQL

Python

Azure Data Factory

Databricks

Informatica

Delta Lake

Lakehouse architecture

ETL/ELT development

CI/CD

Azure DevOps

Oracle

Data governance

Troubleshooting

Agile/Waterfall

Tools

Databricks

Azure Data Factory

Informatica

SSIS

Microsoft Fabric

Azure DevOps

Job description

BITS Recruiting in Toronto seeks a Senior DataOps/Cloud Data Engineer to design, build, and optimize cloud-based data pipelines and lakehouse solutions. You will work hands-on with Azure Data Factory, Databricks, Informatica, Python, and SQL to enable data integration and governance for a large public-sector modernization project.

Responsibilities include ETL/ELT development, data modeling, and CI/CD with Azure DevOps, along with data lineage and collaboration with stakeholders.