Databricks Data Engineer | Azure, PySpark & CDC Pipelines

V-Soft Consulting Group, Inc.; Canada; Presencial; CAD 95 000 - 130 000; Tempo integral

Resumo da oferta

V-Soft Consulting Group, Inc. is seeking a Databricks Data Engineer to design, build, and optimize end-to-end data pipelines in an Azure environment. The role emphasizes PySpark, SQL, and Kafka streaming to deliver reliable data solutions for retail datasets.

You will implement CDC-based pipelines, ensure data quality, and collaborate across teams to move from development to production with CI/CD practices. Familiarity with Delta Lake and data modeling is a plus.

Qualificações

• Hands-on Databricks experience with ability to contribute from day one.
• End-to-end data pipelines with CDC capabilities.
• Strong PySpark and SQL; Python a plus.
• Azure cloud experience and Databricks integration.
• Familiarity with GitHub CI/CD pipelines for deployment.

Responsabilidades

• Design, build, and optimize end-to-end ETL/ELT pipelines using Databricks and Spark (PySpark).
• Develop and maintain scalable data solutions on Azure with Databricks, Delta Lake, and Kafka.
• Build robust CDC-enabled pipelines for real-time data processing.
• Enhance existing pipelines and rapidly understand current codebases to deliver improvements.
• Collaborate with architects, developers, and product owners in Agile/Scrum ceremonies.

Conhecimentos

Databricks

CDC pipelines

PySpark

SQL

Azure

Kafka

GitHub CI/CD

Data architecture

Performance optimization

Pipeline scalability

Ferramentas

Databricks Platform

Delta Lake

Kafka

Descrição da oferta de emprego

V-Soft Consulting Group, Inc. is seeking a Databricks Data Engineer to design, build, and optimize end-to-end data pipelines in an Azure environment. The role emphasizes PySpark, SQL, and Kafka streaming to deliver reliable data solutions for retail datasets.

You will implement CDC-based pipelines, ensure data quality, and collaborate across teams to move from development to production with CI/CD practices. Familiarity with Delta Lake and data modeling is a plus.