Senior Databricks Data Engineer: Lakehouse & Streaming Lead

KData Inc.; Brampton; On-site; CAD 150,000 - 210,000; Full time

Job summary

KData Inc. in Brampton East, Canada, is seeking a Senior, hands-on Databricks Data Engineer who lives and breathes code, query optimization, and modern data architecture. You will write production PySpark/SQL, tune Databricks clusters, and build streaming and batch pipelines.

You will own end-to-end pipeline execution from ingestion to curated Gold layer models, driving modernization of our Lakehouse platform and enforcing data governance, quality, and security.

Qualifications

• 8+ years in data engineering with 4+ years on Databricks.
• Fluent in PySpark and Python with SQL proficiency.
• Deep experience with Delta Lake and Unity Catalog.
• Hands-on cloud experience (AWS/Azure/GCP) with Databricks.
• Strong CI/CD and software engineering practices.

Responsibilities

• Hands-On pipeline development and lakehouse architecture.
• Build batch and real-time streaming pipelines.
• Optimize Spark performance and manage costs.
• Implement data governance, security, and QA.
• CI/CD automation and DevOps for Databricks assets.

Skills

Data engineering

PySpark

SQL

Python

CI/CD

Git workflows

Data modeling

Tools

Delta Lake

Delta Live Tables

Unity Catalog

Auto Loader

Databricks Workflows

Airflow

Terraform

Job description

KData Inc. in Brampton East, Canada, is seeking a Senior, hands-on Databricks Data Engineer who lives and breathes code, query optimization, and modern data architecture. You will write production PySpark/SQL, tune Databricks clusters, and build streaming and batch pipelines.

You will own end-to-end pipeline execution from ingestion to curated Gold layer models, driving modernization of our Lakehouse platform and enforcing data governance, quality, and security.