Senior Databricks Data Engineer | PySpark & Azure Expert

EPAM Systems, Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

EPAM Systems, Inc. seeks an experienced Senior Data Engineer / Databricks Engineer to join a project collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer.

The platform runs on Azure Databricks at scale and handles tens of terabytes daily. The role focuses on reliability, performance, and cost optimization across multi-region deployments (Europe, North America, China).

Qualifications

• 3+ years of hands-on data engineering experience.
• Expertise in Azure Databricks and PySpark.
• Advanced Python development and performance optimization.
• Experience with large-scale data pipelines and production support.
• English proficiency at B2 or higher.

Responsibilities

• Monitor health, stability, and performance of the production data platform.
• Investigate and resolve production incidents and bottlenecks.
• Analyze workloads, optimize PySpark and Databricks jobs for cost and speed.
• Support and improve ~200 existing Databricks jobs and pipelines.
• Design scaling strategies for predictable infrastructure across regions.

Skills

Distributed data processing

Data pipeline design

Problem solving

English communication

Tools

Azure Databricks

PySpark

GitHub Actions

CI/CD pipelines

Job description

EPAM Systems, Inc. seeks an experienced Senior Data Engineer / Databricks Engineer to join a project collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer.

The platform runs on Azure Databricks at scale and handles tens of terabytes daily. The role focuses on reliability, performance, and cost optimization across multi-region deployments (Europe, North America, China).