Databricks Data Engineer

Rubicon Path; Toronto; On-site; CAD 80,000 - 110,000; Full time

Job summary

A technology solutions firm based in Toronto is seeking a Data Engineer to design and build scalable data pipelines using Databricks and Azure. The ideal candidate will have over 5 years of experience with Databricks, strong analytical skills, and a firm grasp of big data technologies. This role requires hands-on experience in troubleshooting DevOps pipelines and integration with business intelligence tools like Power BI.

Qualifications

• 5+ years of hands-on experience with Databricks.
• Experience performing data analysis and data exploration.
• Experience in working in an agile delivery environment.

Responsibilities

• Design & develop custom frameworks/libraries for high throughput.
• Architect and implement scalable data ingestion pipelines.
• Transform raw datasets into structured data lakes.

Skills

Big Data technology

Databricks

Azure Data Factory

Data analysis

Python

Unix

Complex SQL

Apache PySpark

DevOps troubleshooting

Critical thinking

Education

Bachelors Degree in IT or related field

Tools

Databricks SQL

Job description

Who we are looking for

This is a hands-on Data Engineer position. We are looking for candidate with good knowledge on Bigdata technology and strong development experience with Databricks.

What you will be responsible for

As Data Engineer you will

• Design & Develop custom high throughput and configurable frameworks/libraries
• Architect and implement scalable data ingestion pipelines using Azure Data Factory and Databricks
• Transform raw datasets into structured data lakes (Medallion architecture: bronze, silver, gold)
• Convert Hive Metastore to Unity Catalog and configure associated permissions
• Design and secure Delta tables with appropriate schema and access controls
• Enable business intelligence through Power BI integration and dashboard-ready datasets

What we value

These skills will help you succeed in this role

• Experience performing data analysis and data exploration
• Experience working in an agile delivery environment
• Strong knowledge on Databricks SQL/Scala - Data engineering pipeline
• Strong Experience in Unix, Python and complex SQL
• Strong critical thinking, communication, and problem-solving skills
• Strong hands-on experience in troubleshooting DevOps pipelines and Azure services
• Azure cloud,Apache PySpark

Education & Preferred Qualifications

• Bachelors Degree level qualification in a computer or IT related subject
• 5+ years of Databricks hands on experience
• Data Bricks Data Engineer Professional Certification