Senior PySpark Developer: Scale Data Pipelines

Maplesoft Group; Canada; Remote; CAD 90,000 - 120,000; Full time

Job summary

A tech consulting company in Canada is seeking a Senior PySpark Developer to enhance their pharmaceutical data management system. The role involves developing efficient data pipelines and automating testing processes. Candidates should possess extensive experience in PySpark, Python, and SQL, along with strong problem-solving skills. The company values a diverse workforce and provides an inclusive work environment.

Qualifications

• Extensive experience developing PySpark data pipelines.
• Strong expertise in Python and SQL development.
• Proven experience with automated testing and DevOps practices.
• Ability to manage workload and prioritize tasks.
• Strong analytical skills to translate business requirements.

Responsibilities

• Develop and maintain information systems as part of the technical team.
• Work closely with business partners to support solutions using Python and PySpark.
• Create and execute automated test plans for coverage improvement.
• Document product design and ETL mappings as directed by the IT team.

Skills

PySpark data pipelines

Python

SQL development

Automated testing

DevOps practices

Troubleshooting

Problem-solving skills

Strong communication skills

Job description

A tech consulting company in Canada is seeking a Senior PySpark Developer to enhance their pharmaceutical data management system. The role involves developing efficient data pipelines and automating testing processes. Candidates should possess extensive experience in PySpark, Python, and SQL, along with strong problem-solving skills. The company values a diverse workforce and provides an inclusive work environment.