Data Engineer/Lead

Kumaran Systems; Toronto; On-site; CAD 58,000 - 60,000; Full time

Job summary

An IT consulting firm in Toronto is looking for a Data Engineer at a mid-senior level. The candidate will develop a comprehensive technical migration plan using Azure technologies, design efficient data pipelines, and ensure data quality throughout. A bachelor's degree and at least four years of experience with Azure Databricks are essential. Strong problem-solving and communication skills are required for this full-time role.

Qualifications

• Min. 4 years of experience with Azure Databricks.
• Mastery of SQL required.
• Strong problem-solving skills and ability to translate business requirements into actionable solutions.

Responsibilities

• Develop a technical plan for migration to Azure's Data Factory.
• Design scalable data pipelines using Azure Databricks.
• Ensure data quality and integrity throughout the data pipeline.

Skills

Azure Databricks

Data Modelling

PySpark

SQL

Python

Data Visualization

Technical Leadership

Problem Solving

Communication

Team Collaboration

Education

Bachelor’s degree or equivalent experience

Job description

Get AI-powered advice on this job and more exclusive features.

• Develop a comprehensive technical plan for the migration, including data ingestion, transformation, storage, and access control in Azure's Data Factory and data lake
• Design and implement scalable and efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks
• Developing scalable and re-usable frameworks for ingesting data sets
• Ensure data quality and integrity throughout the entire data pipeline, implementing robust data validation and cleansing mechanisms
• Working with event-based/streaming technologies to ingest and process data
• Provide technical guidance and support to the team, resolving any technical challenges or issues that may arise during the migration and post-migration phases
• Stay up to date with the latest advancements in cloud computing, data engineering, and analytics technologies, and recommend best practices and industry standards for implementing the data lake solution.

Requirements:

• Min. 4 years of Experience working with Azure Databricks
• Experience in Data Modelling & Source System Analysis
• Familiarity with PySpark.
• Mastery of SQL.
• Experience with Python programming language used for data Engineering purposes
• Ability to conduct data profiling, cataloguing, and mapping for technical design and construction of technical data flows.
• Experience in data visualization/exploration tools.
• Providing Technical leadership and guidance to Teams
• Strong problem-solving skills and the ability to translate business requirements into actionable data science solutions
• Excellent communication skills, with the ability to effectively convey complex ideas to technical and non-technical stakeholders
• Strong team player with excellent interpersonal and collaboration skills
• Ability to manage multiple projects simultaneously and deliver high-quality results within specified timelines.

Education:

• Bachelor’s degree/University degree or equivalent experience.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Kumaran Systems by 2x

Sign in to set job alerts for “Data Engineer” roles.

Software Engineer, Backend (All Levels / All Teams)

Software Engineer I, Entry Level (Fall 2024-Spring 2025) - Toronto

Toronto, Ontario, Canada CA$58.00-CA$60.00 15 hours ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.