Lead Data Engineer — AWS Cloud Pipelines & Airflow

EPAM Systems Inc; México; Remote; MXN 420,000 - 650,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Upskilling and certification courses

LinkedIn Learning access

Global career opportunities

Job summary

EPAM Systems Inc. is seeking a hands-on Lead Data Engineer to design and maintain scalable data pipelines on AWS.

You will leverage Python, Pandas, and PySpark to build robust data solutions and orchestrate workflows using MWAA, while driving automation with Terraform and GitHub Actions. You will work within an Agile environment, review code, promote best practices, and collaborate with cross-functional teams across engineering, product, and business units.

Qualifications

• 5+ years of professional experience in data engineering or related field.
• Experience leading and managing development teams.
• Strong Python programming skills for data engineering tasks.
• Hands-on PySpark experience for large-scale data processing.
• Experience with AWS MWAA or Apache Airflow to coordinate data workflows.
• Terraform automation of infrastructure and CI/CD pipelines with GitHub Actions.
• DevOps practices applied to data engineering environments.
• Knowledge of data quality, testing, and monitoring practices.

Responsibilities

• Build scalable data pipelines on AWS infrastructure.
• Develop data processing solutions using Python, Pandas, and PySpark.
• Coordinate and manage workflows with AWS MWAA.
• Query data efficiently using AWS Athena when applicable.
• Automate infrastructure with Terraform and GitHub Actions.
• Contribute to testing, monitoring, and data quality safeguards.
• Review code and promote engineering best practices within the team.
• Collaborate with engineering, product, and business in an Agile setting.

Skills

Python

PySpark

AWS

Airflow

Pandas

Terraform

GitHub Actions

CI/CD

DevOps

Data engineering

Agile/Scrum

Gen AI

GitHub Copilot

Tools

MWAA

Job description

EPAM Systems Inc. is seeking a hands-on Lead Data Engineer to design and maintain scalable data pipelines on AWS.

You will leverage Python, Pandas, and PySpark to build robust data solutions and orchestrate workflows using MWAA, while driving automation with Terraform and GitHub Actions. You will work within an Agile environment, review code, promote best practices, and collaborate with cross-functional teams across engineering, product, and business units.