Sr Data Engineer -

iLink Digital; Toronto; On-site; CAD 110,000 - 165,000; Full time

Job summary

iLink Digital is seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks for our clients in Toronto.

You will own end-to-end data pipelines, ML workflows, and production AI systems, collaborating with data scientists and business teams to deliver robust, cost-efficient solutions including Delta Lake, MLflow, and MLOps.

Qualifications

• 5+ years of Databricks & Apache Spark.
• Strong Python & PySpark.
• Experience with Delta Lake & Lakehouse.
• MLflow & MLOps experience.
• Cloud platform (AWS/Azure/GCP).
• Git & CI/CD.

Responsibilities

• Design and implement scalable ETL pipelines using Databricks & Spark.
• Build Lakehouse architecture using Delta Lake.
• Develop and deploy ML models using MLflow.
• Implement MLOps pipelines for training, testing, and serving models.
• Optimize cluster performance and reduce compute cost.
• Build RAG and LLM-based solutions using Mosaic AI.
• Integrate analytics with BI tools (Power BI, Tableau).
• Implement data governance using Unity Catalog.
• Collaborate with Data Scientists and Business teams.
• Ensure data quality, security, and compliance.

Skills

Databricks & Spark

Python & PySpark

Delta Lake

MLflow

MLOps

Cloud platforms

Git & CI/CD

LLMs / Generative AI

RAG pipelines

Vector databases

Deep Learning

Power BI integration

Tools

Power BI

Tableau

Unity Catalog

Job description

Job Summary

We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems.

Key Responsibilities

• Design and implement scalable ETL pipelines using Databricks & Spark


• Build Lakehouse architecture using Delta Lake


• Develop and deploy ML models using MLflow


• Implement MLOps pipelines for training, testing, and serving models


• Optimize cluster performance and reduce compute cost


• Build RAG and LLM-based solutions using Mosaic AI


• Integrate analytics with BI tools (Power BI, Tableau)


• Implement data governance using Unity Catalog


• Collaborate with Data Scientists and Business teams


• Ensure data quality, security, and compliance

Required Skills

Mandatory

• 5+ years of Databricks & Apache Spark


• Strong Python & PySpark


• Experience with Delta Lake & Lakehouse


• MLflow & MLOps experience


• Cloud platform (AWS/Azure/GCP)


• Git & CI/CD

Preferred

• Experience with LLMs & Generative AI


• RAG pipelines & Vector Databases


• Deep Learning frameworks


• Databricks certifications


• Power BI integration

Requirements

Job Summary

We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems.

Key Responsibilities

• Design and implement scalable ETL pipelines using Databricks & Spark


• Build Lakehouse architecture using Delta Lake


• Develop and deploy ML models using MLflow


• Implement MLOps pipelines for training, testing, and serving models


• Optimize cluster performance and reduce compute cost


• Build RAG and LLM-based solutions using Mosaic AI


• Integrate analytics with BI tools (Power BI, Tableau)


• Implement data governance using Unity Catalog


• Collaborate with Data Scientists and Business teams


• Ensure data quality, security, and compliance

Required Skills

Mandatory

• 5+ years of Databricks & Apache Spark


• Strong Python & PySpark


• Experience with Delta Lake & Lakehouse


• MLflow & MLOps experience


• Cloud platform (AWS/Azure/GCP)


• Git & CI/CD

Preferred

• Experience with LLMs & Generative AI


• RAG pipelines & Vector Databases


• Deep Learning frameworks


• Databricks certifications


• Power BI integration