Data Engineer: Pipelines, ETL & Cloud Data Mastery

Spait Infotech; Canada; On-site; CAD 90,000 - 130,000; Full time

Job summary

Spait Infotech is seeking a data engineer to design and maintain scalable data pipelines and ETL workflows. You will extract data from diverse sources, process it with Python, SQL, and PySpark, and build data warehouses and lakes. Collaboration with data science and BI teams is essential.

You will implement Airflow workflows, ensure data quality, and optimize pipelines across cloud platforms. Experience with Spark/Hadoop, databases, and CI/CD tooling is preferred.

Qualifications

• Strong knowledge of Python, SQL, and PySpark.

Responsibilities

• Design, develop, and maintain scalable data pipelines.
• Build and manage ETL/ELT workflows.
• Extract, transform, and load data from multiple sources.
• Develop data processing solutions using Python, SQL, and PySpark.
• Work with structured and unstructured data.
• Design and maintain data warehouses and data lakes.
• Develop automated workflows using Apache Airflow.
• Integrate data from APIs, databases, and external systems.
• Ensure data quality, accuracy, and consistency.
• Monitor, troubleshoot, and optimize data pipelines.
• Implement data validation, governance, and security.
• Collaborate with Data Scientists, Data Analysts, and BI teams.

Skills

Python

SQL

PySpark

ETL/ELT

Airflow

Kafka

Data warehousing

Git

Linux

Shell scripting

Tools

Apache Airflow

Apache Spark

Hadoop

Git

GitHub

Docker

Kubernetes

Linux

Shell

Snowflake

Databricks

Redshift

BigQuery

MySQL

PostgreSQL

MongoDB

AWS

Azure

GCP

AWS Glue

Azure Data Factory

Job description

Spait Infotech is seeking a data engineer to design and maintain scalable data pipelines and ETL workflows. You will extract data from diverse sources, process it with Python, SQL, and PySpark, and build data warehouses and lakes. Collaboration with data science and BI teams is essential.

You will implement Airflow workflows, ensure data quality, and optimize pipelines across cloud platforms. Experience with Spark/Hadoop, databases, and CI/CD tooling is preferred.