Lead Data Engineer: Databricks & Spark Platform Architect

EPAM Systems, Inc.; United States; Remote; USD 150,000 - 190,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

LinkedIn Learning access

Global career opportunities

Volunteer opportunities

Employee groups

Job summary

EPAM Systems, Inc. is seeking a Lead Data Engineer to shape scalable ingestion, transformation, and governed data products on Databricks. You will couple hands-on delivery with technical leadership and architecture to enable self-service analytics and AI-assisted engineering.

Responsibilities include designing ingestion and transformation frameworks on Databricks, building batch and streaming pipelines, and mentoring engineers across teams to elevate data platforms and semantic models.

Qualifications

• 5+ years of data engineering experience with production-grade platforms.
• Advanced Apache Spark and PySpark expertise for scalable pipeline development.
• Strong SQL skills for transformation, modeling, and performance tuning.

Responsibilities

• Design scalable ingestion and transformation frameworks on Databricks.
• Lead end-to-end data products from ingestion through governed consumption layers.
• Define and enforce Databricks best practices for medallion architecture.
• Mentor engineers and drive reusable data engineering frameworks.

Skills

Databricks

Spark

PySpark

SQL

Data ingestion

Data transformation

Data modeling

Data quality

Leadership

Mentoring

Tools

Scala

Job description

EPAM Systems, Inc. is seeking a Lead Data Engineer to shape scalable ingestion, transformation, and governed data products on Databricks. You will couple hands-on delivery with technical leadership and architecture to enable self-service analytics and AI-assisted engineering.

Responsibilities include designing ingestion and transformation frameworks on Databricks, building batch and streaming pipelines, and mentoring engineers across teams to elevate data platforms and semantic models.