Lead Data Engineer: Scalable Pipelines & Lakehouse

EPAM Systems Inc; Colombia; Remote; COP 292,047,000 - 486,744,000; Full time

Benefits offered by this job

Healthcare benefits

Employee financial programs

Paid time off and sick leave

Upskilling, reskilling and certific.

LinkedIn Learning access

Global career opportunities

Volunteer and community involvement

Job summary

EPAM Systems Inc. seeks a Lead Data Software Engineer to design and build scalable data pipelines and governance-ready data platforms. You will lead a team, implement PySpark/Spark SQL solutions, and ensure reliable data movement across multiple platforms.

Join an agile environment that emphasizes clean code, rigorous testing, and continuous learning. The role offers opportunities to influence data architecture and work on high-impact, international projects with top brands.

Qualifications

• Minimum 5 years of hands-on data engineering experience.
• Experience leading and overseeing development teams (≥1 year).
• Proficient in building scalable data pipelines with PySpark/Spark SQL.
• Experience with Delta Lake and Unity Catalog for governance.

Responsibilities

• Design and build scalable data pipelines across platforms.
• Develop ingestion processes from multiple source systems.
• Collaborate with cross-functional teams to translate data requirements.
• Review code and participate in sprint rituals.

Skills

PySpark

Spark SQL

Delta Lake

Unity Catalog

Unit tests with Pytest

Git

Agile methodologies

English (B2)

Lakeflow

Tools

Auto Loader

JDBC/CDC ingestion

Databricks

DB2

SQL Server

Job description

EPAM Systems Inc. seeks a Lead Data Software Engineer to design and build scalable data pipelines and governance-ready data platforms. You will lead a team, implement PySpark/Spark SQL solutions, and ensure reliable data movement across multiple platforms.

Join an agile environment that emphasizes clean code, rigorous testing, and continuous learning. The role offers opportunities to influence data architecture and work on high-impact, international projects with top brands.