Senior Azure Data Ingestion Engineer (IDMC/Databricks)

Brainhunter Systems Ltd

Toronto

On-site

CAD 90,000 - 130,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Canada Life Data team seeks a Data Ingestion Engineer to design, implement, and maintain pipelines that ingest data from diverse sources into our data platforms using IDMC on Azure, aided by Data Factory and Databricks.

You will work with cross-functional teams to ensure data quality, monitoring, and scalable operations while supporting incident response, governance, and security best practices.

Qualifications

  • 5+ years in data operations or development (ETL, Data Warehouse).
  • 2+ years of recent development experience in IDMC, MS-Azure, and Databricks.
  • Experience designing and implementing data ingestion systems with MS-Azure/Databricks.
  • Experience with Kafka, CI/CD and CDC tools preferred.

Responsibilities

  • Design, develop, test, and support end-to-end ingestion pipelines using IDMC in MS Azure data platform that supports structured and unstructured data from multiple sources using Azure Data Factory and Databricks.
  • Collaborate with cross-functional teams to understand data requirements and design efficient ingestion patterns and workflows.
  • Implement and maintain controls and alerts to monitor data pipelines and proactively identify data issues; ensure data quality and integrity.
  • Collaborate with support teams to triage incidents and resolve major incidents, outages, or disruptions.
  • Automate processes to improve reliability, efficiency, and scalability using MS Azure tools.
  • Conduct performance and capacity assessments and develop plans to improve performance and reduce costs.
  • Focus on data governance, compliance controls, and security measures to protect sensitive data.
  • Apply best practices for scalable, reusable, and reliable data pipelines, and document configurations and procedures.

Skills

ETL/Data Warehouse
IDMC
Azure/Data Factory
Databricks
Kafka
CI/CD
CDC
Python
SQL
Monitoring pipelines

Tools

Informatica IDMC
Azure Data Factory
Databricks
Kafka
CI/CD tooling

Job description

Canada Life Data team seeks a Data Ingestion Engineer to design, implement, and maintain pipelines that ingest data from diverse sources into our data platforms using IDMC on Azure, aided by Data Factory and Databricks.

You will work with cross-functional teams to ensure data quality, monitoring, and scalable operations while supporting incident response, governance, and security best practices.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Ingestion Engineer
Data Ingestion Engineer

Brainhunter Systems Ltd • Toronto

On-site
CAD 90,000 - 130,000
ETL & Data Ingestion Engineer
ETL & Data Ingestion Engineer

Brainhunter Systems Ltd • Toronto

Hybrid
CAD 70,000 - 117,000
Hybrid work arrangement
Azure Data Ingestion Engineer - Hybrid (IDMC/Databricks)
Azure Data Ingestion Engineer - Hybrid (IDMC/Databricks)

Brainhunter Systems Ltd • Toronto

Hybrid
CAD 90,000 - 130,000
“Data Ingestion Engineer\"
“Data Ingestion Engineer\"

Brainhunter Systems Ltd • Toronto

Hybrid
CAD 90,000 - 130,000
Senior Azure Data Engineer - Lead Data Pipelines (Contract)
Senior Azure Data Engineer - Lead Data Pipelines (Contract)

Capgemini • Brampton

On-site
CAD 87,000 - 135,000
Medical benefits
Dental benefits
Vision benefits
+1
Azure Databricks Engineer
Azure Databricks Engineer

Data Elephant • Calgary

On-site
CAD 120,000 - 180,000
Senior Data Engineer, Databricks & Azure Expert
Senior Data Engineer, Databricks & Azure Expert

NTT America, Inc. • Toronto

On-site
CAD 89,000 - 111,000
Health and dental coverage
Remote work allowance
Retirement savings
+4
Senior Data Solutions Lead – Azure, Databricks & ETL (Remote)
Senior Data Solutions Lead – Azure, Databricks & ETL (Remote)

Canadian Imperial Bank of Commerce (Canada) • Toronto

Hybrid
CAD 140,000 - 180,000
Competitive salary
Pension plan
Employee share purchase plan
+2
Data Engineering Consultant: Lakehouse on Databricks
Data Engineering Consultant: Lakehouse on Databricks

Avanade Inc. • New Brunswick

On-site
CAD 97,000 - 115,000
Azure Databricks Platform Architect
Azure Databricks Platform Architect

Capgemini • Canada

On-site
CAD 228,000 - 252,000
Paid time off
Medical & dental coverage
Retirement savings plans
+2