Pyspark ETL Developer

ISG Search Inc; Toronto; Presencial; CAD 90.000 - 120.000; Jornada completa

Descripción de la vacante

ISG Search Inc. seeks a data engineer for a 5 month contract with a government health data organization. You will work on PySpark pipelines, write modular Python code, and leverage AWS EMR and Airflow to process large datasets.

You will collaborate with analytics teams to deliver clean, well-documented ETL artifacts and support data products through code reviews and troubleshooting.

Formación

• Proficient in Python and PySpark for big data processing.
• Strong SQL skills with dimensional modeling and data warehouses.
• Hands-on experience with AWS EMR and Airflow.
• Experience in Linux shell scripting and documenting ETL artifacts.

Responsabilidades

• Understand systems and business processes for the assigned portfolio.
• Collaborate with client analytics teams to develop and support Python/PySpark solutions.
• Provide code quality guidance and participate in code reviews.
• Analyze software issues and assist in fixes, create project documentation.

Conocimientos

Python

PySpark

SQL

Dimensional modeling

Analytical data warehouses

Herramientas

AWS EMR

Apache Airflow

AWS Cloud

Linux shell scripting

Descripción del empleo

Duration:

5 Month contract

Must have:

• Python and PySpark programing languages to manipulate data and draw insights from large datasets
• Experience with AWS EMR, AWS Cloud
• Hands-on programming skills, able to write modular, maintainable code, preferably Python & SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses
• Experience with Apache Airflow
• Experience with Python, PySpark programming in the context of data processing / data science applications
• Profound knowledge and experience in shell scripting in Linux/Unix environment. Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL environment

Our client...

A government health data organization.

Responsibilities:

• Understand systems and Business processes related to the products in their assigned portfolio
• Works closely with client analytical teams to develop, maintain and support products using Python and/or R and PySpark programming languages.
• Provides support and guidance on creating top-quality clean code and participates in code review.
• Analyses Software deficiencies/problems and support to fix them
• Prepares documentation, reports, and presentations related to projects and activities.