Tech Lead, Big Data Engineer: PySpark/AWS + Learning & Health

Ulvior; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

SoftServe University

Medical insurance

Food vouchers

Extra days off

Dare to Care

Anniversary Program

Job summary

Ulvior is hiring a data platform engineer to support and extend an AWS-based data platform. You will build data pipelines with Python, PySpark, and Spark, orchestrated by Airflow, and leverage AWS Glue, S3, Athena, and Aurora PostgreSQL.

The role requires strong SQL, ETL, data quality practices, and hands-on Docker/Terraform deployment, plus CI/CD using GitHub Actions. Travel to the United States for a three-month business trip in Q1 2027 is expected.

Qualifications

• 5+ years of experience in Big Data or Data Engineering.
• Strong Python, PySpark and Spark for large-scale processing.
• Hands-on with AWS Glue, S3, Athena, Aurora PostgreSQL and Lambda.
• Experience with Apache Airflow for orchestration and scheduling.
• Strong SQL, data ingestion, ETL and data quality practices.
• Experience with Docker, Terraform and CI/CD pipelines (GitHub Actions).
• Familiarity with Python testing tools (pytest, Pydantic, ruff, mypy).
• Knowledge of healthcare data regulations (HIPAA/HITRUST) is a plus.
• Willingness to travel to the United States for a 3-month trip in Q1 2027.

Responsibilities

• Support, maintain, and enhance an AWS-based data platform and ingestion pipelines.
• Develop and optimize data processing workflows using Python, PySpark, Spark and AWS Glue.
• Monitor and troubleshoot data pipelines for reliability, performance, and data quality.
• Work with AWS services such as S3, Aurora PostgreSQL, Athena, Lambda, SQS, SNS and EventBridge.
• Maintain and improve workflow orchestration with Apache Airflow.
• Support infrastructure and deployment using Docker, Terraform and Amazon EKS.
• Apply software engineering best practices: automated testing, code quality checks, CI/CD.
• Use Claude Code to aid implementation, testing and code reviews.
• Collaborate with remote teams to resolve issues and improve platform stability.
• Support data security, governance and compliance within a regulated healthcare environment.

Skills

Big Data / Data Engineering experience

Python / PySpark / Spark

AWS data services (Glue, S3, Athena, R

Airflow

SQL / ETL / data quality

CI/CD with GitHub Actions

Docker / Terraform / EKS

Python testing tools (pytest, Pydantic

HIPAA / HITRUST familiarity

Tools

Docker

Terraform

Amazon EKS

GitHub Actions

Claude Code

Job description

Ulvior is hiring a data platform engineer to support and extend an AWS-based data platform. You will build data pipelines with Python, PySpark, and Spark, orchestrated by Airflow, and leverage AWS Glue, S3, Athena, and Aurora PostgreSQL.

The role requires strong SQL, ETL, data quality practices, and hands-on Docker/Terraform deployment, plus CI/CD using GitHub Actions. Travel to the United States for a three-month business trip in Q1 2027 is expected.