Senior Data Software Engineer: Architect Scalable Pipelines

EPAM Systems Inc; Colombia; Remote; COP 90,000,000 - 150,000,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Learning & certification

Job summary

EPAM Systems Inc. is seeking a Senior Data Software Engineer to design scalable data pipelines and dependable data flows that power critical business decisions. You will collaborate with a team focused on clean, thoroughly tested code and disciplined engineering practices.

The role offers opportunities to work on large-scale data projects, using PySpark, Delta Lake, and Databricks, while contributing to data governance and quality across multiple platforms.

Qualifications

• 3+ years of hands-on experience with PySpark and Spark SQL.
• Experience building Lakeflow pipelines and data orchestration.
• Experience with Delta Lake and Unity Catalog for storage and governance.
• Experience with JDBC/CDC ingestion (DB2, SQL Server).
• Experience writing unit tests with pytest.
• Git and deployment workflows.

Responsibilities

• Architect and construct scalable data pipelines across platforms.
• Build and sustain data ingestion workflows from multiple sources.
• Produce clean, well-tested code grounded in solid software engineering.
• Collaborate with cross-functional teams to translate data needs into technical solutions.
• Monitor data pipeline performance and resolve issues to ensure dependability.
• Participate in code reviews to maintain quality and share knowledge.
• Diagnose and fix data quality problems to ensure accuracy.
• Document technical workflows and solutions for onboarding and knowledge sharing.
• Engage in agile sprint planning and related team activities.

Skills

PySpark

Spark SQL

Data pipelines

Unit testing

Git

English (B2+)

Tools

Auto Loader

JDBC/CDC

Delta Lake

Unity Catalog

Databricks

DataStage

Job description

EPAM Systems Inc. is seeking a Senior Data Software Engineer to design scalable data pipelines and dependable data flows that power critical business decisions. You will collaborate with a team focused on clean, thoroughly tested code and disciplined engineering practices.

The role offers opportunities to work on large-scale data projects, using PySpark, Delta Lake, and Databricks, while contributing to data governance and quality across multiple platforms.