Pyspark ETL Developer

isgSearch; Toronto; Presencial; CAD 96.000 - 131.000; Jornada completa

Descripción de la vacante

isgSearch is seeking a data engineering contractor to manipulate large datasets using Python and PySpark, leveraging AWS EMR and Airflow. The role involves collaborating with client analytics teams, maintaining code quality, and documenting ETL processes.

Strong SQL and dimensional modeling skills are required, with Linux shell scripting proficiency. The client is a government health data organization, offering a 5-month contract in Toronto, Ontario.

Formación

• Python and PySpark skills to manipulate data and draw insights from large datasets.
• Experience with AWS EMR and AWS Cloud.
• Hands-on programming skills, modular and maintainable code, preferably Python & SQL; strong SQL and dimensional modeling knowledge.
• Experience with Apache Airflow.
• Experience with Python and PySpark in data processing / data science applications.
• Proficiency in shell scripting in Linux/Unix and documenting ETL design and artifacts.

Responsabilidades

• Understand systems and business processes related to the products in the assigned portfolio.
• Collaborate with client analytical teams to develop, maintain and support products using Python, R and PySpark.
• Provide support and guidance on creating clean code and participate in code reviews.
• Analyze software deficiencies and assist in fixes.
• Prepare documentation, reports and presentations related to projects and activities.

Conocimientos

Python

PySpark

AWS EMR

AWS Cloud

SQL

Dimensional modeling

Apache Airflow

Linux shell scripting

ETL documentation

Herramientas

AWS EMR

Apache Airflow

Linux shell scripting

SQL tooling

Descripción del empleo

• Python and PySpark programing languages to manipulate data and draw insights from large datasets
• Experience with AWS EMR, AWS Cloud
• Hands-on programming skills, able to write modular, maintainable code, preferably Python & SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses
• Experience with Apache Airflow
• Experience with Python, PySpark programming in the context of data processing / data science applications
• Profound knowledge and experience in shell scripting in Linux/Unix environment. Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL environment

Duration:

5 Month contract

Must have:

• Python and PySpark programing languages to manipulate data and draw insights from large datasets
• Experience with AWS EMR, AWS Cloud
• Hands-on programming skills, able to write modular, maintainable code, preferably Python & SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses
• Experience with Apache Airflow
• Experience with Python, PySpark programming in the context of data processing / data science applications
• Profound knowledge and experience in shell scripting in Linux/Unix environment. Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL environment

Our client…

A government health data organization.

Responsibilities:

• Understand systems and Business processes related to the products in their assigned portfolio
• Works closely with client analytical teams to develop, maintain and support products using Python and/or R and PySpark programming languages.
• Provides support and guidance on creating top-quality clean code and participates in code review.
• Analyses Software deficiencies/problems and support to fix them
• Prepares documentation, reports, and presentations related to projects and activities.