Senior Data Software Engineer – Build Scalable Data Pipelines

EPAM Systems Inc; México; Remote; MXN 900,000 - 1,200,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Upskilling and certification courses

LinkedIn Learning access

Job summary

EPAM Systems Inc. seeks a Senior Data Software Engineer to architect scalable data pipelines and data solutions powering critical business decisions.

You will collaborate with a disciplined team focused on clean, well-tested code and robust engineering practices, tackling large-scale data problems with Lakeflow, Delta Lake, and Databricks. You will design ingestion workflows, perform data quality checks, participate in code reviews, and contribute to architecture evolution within an agile

Qualifications

• At least 3 years of hands-on data engineering experience.
• Strong experience with PySpark and Spark SQL for large-scale data workflows.
• Experience building and overseeing Lakeflow pipelines and Jobs for data orchestration.
• Proficient with Auto Loader and JDBC/CDC ingestion for various sources.
• Experience with Delta Lake and Unity Catalog for storage and governance.
• Experience writing unit tests with pytest to ensure code reliability.
• Familiarity with Git and deployment workflows.

Responsibilities

• Architect and construct scalable data pipelines across multiple platforms.
• Build and sustain data ingestion workflows from diverse sources.
• Write clean, high-performing, well-tested code.
• Collaborate with cross-functional teams to shape data solutions.
• Monitor and optimize data pipeline performance and reliability.
• Participate in code reviews to maintain quality.
• Diagnose and fix data quality issues to ensure accuracy across datasets.
• Contribute to evolving data architecture practices and standards.
• Create documentation for technical workflows and solutions.
• Operate within an agile framework and participate in sprint planning.

Skills

PySpark

Spark SQL

Data pipelines

Unit testing with pytest

Git

English (B2+)

Delta Lake

Unity Catalog

DataStage migration

Tools

Auto Loader

JDBC/CDC ingestion

Delta Lake

Unity Catalog

Databricks

Lakeflow

Job description

EPAM Systems Inc. seeks a Senior Data Software Engineer to architect scalable data pipelines and data solutions powering critical business decisions.

You will collaborate with a disciplined team focused on clean, well-tested code and robust engineering practices, tackling large-scale data problems with Lakeflow, Delta Lake, and Databricks. You will design ingestion workflows, perform data quality checks, participate in code reviews, and contribute to architecture evolution within an agile