ETL Developer

Diverse Lynx; Mississauga; On-site; CAD 75,000 - 95,000; Full time

Job summary

A leading company in the Peel Region is seeking an experienced ETL Developer proficient in PySpark and Python. The role involves creating ELT data pipelines, optimizing Spark clusters, and utilizing various ETL tools and technologies. The ideal candidate will have a strong background in data processing and security systems, contributing to a diverse workforce.

Qualifications

• Strong hands-on experience in developing ETL solutions.
• Knowledge of S3 buckets, Kubernetes, and OpenShift.

Responsibilities

• Creating ELT data pipelines and optimizing Spark clusters.
• Utilizing various ETL tools and security systems.

Skills

ETL Solutions

PySpark

Python

Tools

IBM DataStage

Talend

Apache Airflow

SSIS

Tableau

CyberArk

Kubernetes

OpenShift

MongoDB

Hive

Job description

Job Description

We are seeking a candidate with strong hands-on experience in developing ETL solutions using PySpark and Python.

The responsibilities include:

1. Creating ELT data pipelines and optimizing Spark clusters in containerized environments.
2. Utilizing knowledge of S3 buckets, Kubernetes, OpenShift, and NAS storage systems.
3. Developing with ETL tools such as IBM DataStage, Talend, Apache Airflow, SSIS, etc.
4. Working with security systems (CyberArk), file transfer protocols (SFTP), reporting tools (Tableau), and file systems (NAS/S3, MSaaS, MongoDB, Hive) in conjunction with ETL pipelines.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without discrimination. We evaluate applicants based on their ability, competence, and proven capability to perform the role. We support a diverse workforce across all levels.