Remote PySpark Data Engineer — High-Impact ETL

Altis Technology; United States; On-site; USD 85,000 - 106,000; Full time

Job summary

Altis Technology is seeking an experienced Data Engineer to optimize large-scale ETL pipelines and convert Python code to PySpark for a healthcare data context across Canada. This remote contract role focuses on improving processing efficiency and reliability of nightly workflows.

You will collaborate with business and IT teams, enhancing data processing speed, scalability, and pipeline automation, while maintaining data integrity and quality for analytics.

Qualifications

• 7+ years of Apache Spark (PySpark) experience with large-scale data processing.
• 3+ years Python development, ETL and data transformation.
• 3+ years with GitHub/GitLab for version control.
• Strong ETL, testing, validation and data pipeline integration.
• Asset: AWS EMR, Airflow, S3, Glue, Redshift, Lake Formation and IaC tools such as Terraform/CloudFormation.

Responsibilities

• Convert existing Python code into optimized PySpark for ETL pipelines.
• Test, validate, and implement code changes to maintain data integrity.
• Review data enrichment and nightly pipeline performance improvements.
• Develop code optimizations for speed, scalability, and efficiency.
• Document processes and suggest improvements to data workflows.
• Collaborate with IT to enhance pipeline automation as time permits.

Skills

PySpark

Python

GitHub/GitLab

ETL pipelines

Spark performance

Airflow

AWS EMR

AWS data services

Terraform/CloudFormation

Communication

Job description

Altis Technology is seeking an experienced Data Engineer to optimize large-scale ETL pipelines and convert Python code to PySpark for a healthcare data context across Canada. This remote contract role focuses on improving processing efficiency and reliability of nightly workflows.

You will collaborate with business and IT teams, enhancing data processing speed, scalability, and pipeline automation, while maintaining data integrity and quality for analytics.