ETL & Data Ingestion Engineer

Brainhunter Systems Ltd

Toronto

Hybrid

CAD 70,000 - 117,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Hybrid work arrangement

Job summary

Brainhunter Systems Ltd. seeks a Data Ingestion Engineer to design, build, and maintain ingestion pipelines within our Azure/Databricks ecosystem. The role focuses on reliable data flow from various sources into our data platforms, with emphasis on governance, security, and performance.

You will collaborate with cross-functional teams, automate pipelines, and implement monitoring and incident response to ensure data integrity and scalability. The position is hybrid with on-site days in Canada.

Qualifications

  • 5+ years in data ops or development (ETL, Data Warehouse).
  • 2+ years in IDMC, MS Azure and Databricks.
  • Experience designing data ingestion systems with MS Azure/Databricks.
  • Proficient in Python or SQL; strong data quality controls.
  • Experience with monitoring, incident and change management.

Responsibilities

  • Design, develop, test, and support end-to-end ingestion pipelines on MS Azure.
  • Collaborate with cross-functional teams to define data requirements and workflows.
  • Implement controls, alerts, and data quality checks across pipelines.
  • Triaging incidents and supporting rapid resolution of major outages.
  • Automate operations to improve reliability, efficiency, and scalability.
  • Perform performance and capacity assessments to reduce costs.
  • Apply data governance and security measures to protect sensitive data.
  • Document configurations, processes, and troubleshooting steps.

Skills

ETL
Data Warehouse
IDMC
MS Azure
Databricks
Kafka
CI/CD
CDC
Python
SQL
Data pipelines monitoring
Incident management
Collaboration

Education

Bachelor equivalent in CS/IS/SE

Tools

Informatica IDMC
Azure Data Factory
Databricks
Kafka

Job description

Requirement Summary
  • Job-Role/Title: Data Ingestion Engineer
  • Anticipated Start Date: 3-4 weeks from offer (pending background checks)
  • Job Type: Contract Opportunity. Initial contract for a 3-month term, with good possibilities of further extensions.
  • Job Location: Toronto, ON / London, ON
  • Work Style: Hybrid Work - 3 days per week in the office required (Tuesday and Thursday are mandatory).
  • Work Hours: 37.5 hours/week, typically Monday-Friday, 9:00 am to 5:00 pm
  • Contractor to work extra time or be on-call: Yes, on-call rotation (Monday to Sunday) might be required on a need basis.
  • Rate Cap: Up to CA$ 85/Hr. (on Incorp), depending on relevant experience. The final compensation will be negotiated between the contractor and recruiter based on experience and engagement terms.
  • Interview: Teams interviews; 2 interviews
NOTE
  • Successful personnel must have an active Reliability Status Security Clearance / be willing to undergo the Security Clearance process
  • Eligibility Criteria: The consultant must be a resident of Canada for at least 5+ years to be eligible to apply for the client's Reliability Status Security Clearance.
  • References Required (Yes/No): YES, must share at least 2 professional references from recent projects.
Required Experience, Skills, and Qualifications
  • 5+ years of prior experience in a data operations or development role (e-g ETL, Data Warehouse) with 2+ years of recent development experience in IDMC, MS Azure, and Databricks
  • Demonstrated experience in designing and implementing data ingestion systems using MS-Azure /Databricks
  • Experience in Kafka, CI/CD, and CDC tools preferred
  • Solid programming experience with Python or SQL
  • Experience in setting up monitoring and controls for MS Azure data pipelines (prior experience with Data warehousing/ other data technologies is accepted)
  • Strong knowledge in service management, including incident and change management procedures.
  • Problem-solving skills and the ability to troubleshoot complex issues.
  • Contributing to positive team culture and promoting common goals, team success, and business outcomes
  • Valuing and appreciating different views and bringing in a collaborative approach to achieving consensus.
  • Encourages inclusivity and a sense of unity within all operational areas of the Canada Data team.

___________________________________________________________

Position Overview

As a Data Ingestion Engineer, you will join a dynamic and high-performing Canada Life Data team dedicated to providing Data and AI solutions by ensuring the robustness, scalability, and reliability of data ingestion systems within our organization. Your key focus will be on designing, implementing, and maintaining the pipelines necessary for efficient and seamless data ingestion from various sources into our data platforms. You will partner with cross-functional teams, enterprise, and portfolio architects to enhance operational performance, troubleshoot issues, and implement best practices for data reliability.

  • Design, develop, test, and support end-to-end ingestion pipelines using Informatica IDMC in the MS Azure data platform that supports the consumption of structured and unstructured data from various sources with the use of Azure Data Factory and Databricks
  • Collaborate with cross-functional teams to understand data requirements, design efficient data ingestion patterns and workflows.
  • Implement and maintain controls and alerts to monitor data pipelines to proactively identify data-related issues. Ensure data quality and integrity are maintained across the pipelines.
  • Collaborate with support teams to triage incidents to quickly resolve major incidents, system outages, or disruptions.
  • Automate processes to support smooth operation of data pipelines to improve reliability, efficiency, and scalability using MS Azure tools.
  • Conduct performance and capacity assessments, identify bottlenecks, and develop plans to improve performance and reduce operational costs.
  • Focus on implementing data governance and compliance controls and security measures to protect sensitive data.
  • Apply best practices for creation and maintenance of data pipelines, including scalability, reusability, and reliability.
  • Document system configurations, processes, and troubleshooting procedures.

___________________________________________________________

Thank you for your interest in working with us. We'll contact only those candidates who are shortlisted for the next steps in the hiring process.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Ingestion Engineer
Data Ingestion Engineer

Brainhunter Systems Ltd • Toronto

On-site
CAD 90,000 - 130,000
“Data Ingestion Engineer\"
“Data Ingestion Engineer\"

Brainhunter Systems Ltd • Toronto

Hybrid
CAD 90,000 - 130,000
Senior Azure Data Ingestion Engineer (IDMC/Databricks)
Senior Azure Data Ingestion Engineer (IDMC/Databricks)

Brainhunter Systems Ltd • Toronto

On-site
CAD 90,000 - 130,000
OPEN: Data Engineer
OPEN: Data Engineer

Cpus Engineering Staffing Solutions Inc. • Oshawa

On-site
CAD 80,000 - 100,000
Data Engineer
Data Engineer

Hirevantis LLC • Toronto

Hybrid
CAD 105,000 - 135,000
Health benefits
Hybrid/Remote across Canada
Data Engineering Developer
Data Engineering Developer

Osedea • Montreal

Hybrid
CAD 90,000 - 120,000
Competitive salary RRSP
Flexible hours
8 weeks remote work
+5
Senior Data Platform Engineer
Senior Data Platform Engineer

Canada Life • London

Hybrid
CAD 85,000 - 135,000
Career development
Health & wellness benefits
Volunteer day
+3
Data Engineering Intern
Data Engineering Intern

Bright Network Consulting • Canada

Remote
CAD 25,000 - 39,000
Senior Data Engineer
Senior Data Engineer

Cpus Engineering Staffing Solutions Inc. • Pickering

Hybrid
CAD 90,000 - 120,000
Azure Data Migration Engineer Contract (1775)
Azure Data Migration Engineer Contract (1775)

Direct IT Recruiting Inc. • Toronto

Hybrid
CAD 80,000 - 100,000