Cloud Data Engineer - PySpark, SQL & Snowflake Expert

Alltech Consulting Services; Mississauga; Hybrid; CAD 100,000 - 130,000; Full time

Job summary

A consulting firm specializing in data solutions is seeking a Cloud Data Engineer to build and optimize data pipelines, design and deploy Spark programs in Databricks, and ensure data integrity. Candidates should have over 5 years of experience in Python and SQL, along with robust knowledge in ETL processes and cloud data platforms like Snowflake and Synapse. This role is critical in managing and optimizing enterprise-level cloud data solutions.

Qualifications

• 5+ years experience in Python coding.
• 5+ years in SQL Server based development of large datasets.
• 5+ years with ETL pipeline development using Databricks Pyspark.
• Experience with cloud data warehouses and architectures.

Responsibilities

• Build and optimize data pipelines for data ingestion and transformation.
• Develop Spark programs in Databricks to process large data volumes.
• Monitor and troubleshoot Databricks jobs.
• Implement data management best practices.

Skills

Python

SQL

Pyspark

Data Warehouse Concepts

ETL Pipelines

Databricks

Cloud Data Platforms

Dimensional data modeling

Airflow

Tools

SQL Server

Synapse

Big Query

Redshift

Snowflake

Job description

A consulting firm specializing in data solutions is seeking a Cloud Data Engineer to build and optimize data pipelines, design and deploy Spark programs in Databricks, and ensure data integrity. Candidates should have over 5 years of experience in Python and SQL, along with robust knowledge in ETL processes and cloud data platforms like Snowflake and Synapse. This role is critical in managing and optimizing enterprise-level cloud data solutions.