Databricks engineer with Python, Pyspark experience

Veracity Software Inc; Ontario; Remote; CAD 80,000 - 120,000; Full time

Job summary

A leading IT services company is seeking a skilled Databricks Engineer to design and optimize data pipelines and workflows. The ideal candidate will have expertise in Python and PySpark, working within an Azure environment. This full-time position offers the opportunity to collaborate with various stakeholders and contribute to innovative data solutions.

Qualifications

• Strong experience with Databricks, Python, and PySpark.
• Hands-on experience with Apache Spark and big data processing.
• Proficiency in SQL and Azure cloud platform.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks, PySpark, and Python.
• Implement data transformation, cleansing, and integration solutions.
• Collaborate with data engineers and analysts.

Skills

Databricks

Python

PySpark

SQL

Tools

Apache Spark

Azure

Job description

Databricks Engineer with Python and PySpark Experience

1 month ago - Be among the first 25 applicants

Get AI-powered advice on this job and access more exclusive features.

Role: Databricks Engineer with Python and PySpark Experience

Location: Canada/Remote

Contract

Job Description

We are seeking a skilled Databricks Engineer with expertise in Python and PySpark to design, develop, and optimize data pipelines and workflows. The ideal candidate will have experience working with Apache Spark, ETL processes, and big data architectures within an Azure environment.

Key Responsibilities

1. Develop and maintain scalable data pipelines using Databricks, PySpark, and Python.
2. Work with structured and unstructured data in cloud-based environments.
3. Implement data transformation, cleansing, and integration solutions.
4. Collaborate with data engineers, analysts, and stakeholders to meet business requirements.

Requirements

1. Strong experience with Databricks, Python, and PySpark.
2. Hands-on experience with Apache Spark and big data processing.
3. Proficiency in SQL and working with the Azure cloud platform.
4. Experience with ETL development and performance tuning.
5. Knowledge of data lake, data warehouse, and data modeling principles.

Additional Details

• Seniority Level: Mid-Senior level
• Employment Type: Full-time
• Job Function: Other
• Industries: IT Services and IT Consulting