Databricks Engineer

Nous Infosystems; Toronto; On-site; CAD 60,000 - 100,000; Full time

Job summary

An innovative firm is seeking a skilled Databricks Engineer to design, develop, and optimize data pipelines and workflows. This entry-level position offers a unique opportunity to work with cutting-edge technologies in an Azure environment, focusing on big data architectures and ETL processes. You will be responsible for developing scalable data pipelines, collaborating with engineers and analysts, and implementing data transformation solutions. Join a dynamic team that values creativity and technical expertise, and make a significant impact in the IT services and consulting industry. If you are passionate about data engineering and eager to grow your skills, this role is perfect for you.

Qualifications

• Strong experience with Databricks, Python, and PySpark.
• Hands-on experience with Apache Spark and big data processing.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks, PySpark, and Python.
• Collaborate with data engineers and analysts to meet business requirements.

Skills

Python

PySpark

ETL processes

Apache Spark

SQL

Tools

Databricks

Azure

Job description

Skilled Databricks Engineer with expertise in Python and PySpark to design, develop, and optimize data pipelines and workflows. Experience working with Apache Spark, ETL processes, and big data architectures in an Azure environment.

Key Responsibilities:

1. Develop and maintain scalable data pipelines using Databricks, PySpark, and Python
2. Work with structured and unstructured data in cloud-based environments
3. Implement data transformation, cleansing, and integration solutions
4. Collaborate with data engineers, analysts, and stakeholders to meet business requirements

Requirements:

1. Strong experience with Databricks, Python, and PySpark
2. Hands-on experience with Apache Spark and big data processing
3. Proficiency in SQL and working with Azure cloud platform
4. Experience with ETL development and performance tuning
5. Knowledge of data lake, data warehouse, and data modeling principles

Seniority level

Entry level

Employment type

Contract

Job function

Engineering and Information Technology

Industries

IT Services and IT Consulting