Databricks Data Engineer

Viva Tech Solutions; Toronto; On-site; CAD 90,000 - 130,000; Full time

Job summary

Viva Tech Solutions in Toronto, Canada, seeks a data engineer with strong experience in Azure Databricks, PySpark, and data lakehouse architectures. The role requires hands-on ETL/ELT development, ADLS expertise, and NG/CI-CD practices.

The ideal candidate will also bring SQL performance tuning skills, data migration experience, and BigQuery migration knowledge to ensure seamless data workflows and scalable solutions in a fast-paced environment.

Qualifications

• Experience with Azure Databricks including Unity Catalog and Data Lakehouse Architecture.
• Strong PySpark and Spark SQL expertise.
• ETL/ELT development experience.
• Experience with Azure Data Lake Storage (ADLS).
• Familiar with Git and CI/CD practices.
• SQL optimization and performance tuning skills.
• Experience in data migration and reconciliation frameworks.
• GCP BigQuery migration experience is essential.

Skills

Azure Databricks

PySpark

Spark SQL

ETL/ELT Development

Azure Data Lake Storage

Git

CI/CD

SQL performance tuning

Data Migration

Data Reconciliation

BigQuery Migration

Tools

Azure Databricks

Unity Catalog

Job description

• Proven experience with Azure Databricks, including Unity Catalog and Data Lakehouse Architecture.
• Strong proficiency in PySpark and Spark SQL.
• Expertise in ETL/ELT Development.
• Experience with Azure Data Lake Storage (ADLS).
• Familiarity with Git and CI/CD practices.
• SQL Optimization and Performance Tuning skills.
• Experience in Data Migration and Reconciliation Frameworks.
• GCP BigQuery Migration Experience is essential.