Senior Data Engineer/ Databricks, PySpark, Python

EPAM Systems; Turkey; On-site; TRY 4,442,000 - 6,417,000; Full time

Benefits offered by this job

Upskilling program

Mentoring programs

Certification support

English courses

Private health insurance

Well-being events

Job summary

EPAM Systems is seeking a Senior Data Engineer / Databricks Engineer to join a project focused on collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer. The platform operates at scale in Azure Databricks, running about 200 Databricks jobs across Europe, North America, and China, with a new China deployment planned.

The role emphasizes building scalable, cost-aware infrastructure, improving observability, and optimizing PySpark workloads for

Qualifications

• 3+ years hands-on data engineering experience.
• Experience with Azure Databricks and PySpark.
• Strong Python development skills.
• Experience optimizing large-scale Spark workloads.
• Background in production data platforms and monitoring.

Responsibilities

• Monitor health, stability, and performance of production data platform.
• Investigate and resolve performance issues and incidents.
• Analyze Databricks workloads and optimize PySpark jobs for performance and cost.
• Support 200+ existing Databricks jobs and pipelines.
• Design scaling strategies with cost-focused infrastructure.

Skills

Azure Databricks

PySpark

Python

Performance optimization

Monitoring & observability

Cloud cost optimization

English (B2)

Tools

GitHub Actions

Job description

We are looking for an experienced Senior Data Engineer / Databricks Engineer to join a project focused on collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer.

The platform operates at a significant scale, processing tens of terabytes of telemetry data in Azure Databricks and running approximately 200 Databricks jobs. The solution is currently in the production maintenance and evolution phase and is deployed across multiple regions, including Europe, North America, and China, with an additional deployment in China planned in the near future.

A key focus of the role will be ensuring that the system can scale reliably while keeping infrastructure and operational costs predictable.

Responsibilities

• Monitor the health, stability, and performance of the production data platform
• Investigate and resolve performance issues and production incidents
• Analyze Databricks workloads and identify performance bottlenecks
• Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency
• Support and improve approximately 200 existing Databricks jobs and associated data pipelines
• Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded
• Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs
• Track and optimize Databricks and Azure resource consumption
• Improve platform observability, monitoring, alerting, and operational processes
• Ensure consistency across multiple regional deployments in Europe, North America, and China
• Identify opportunities for technical improvements, automation, and reduction of operational overhead

Requirements

• 3+ years of hands‑on experience in data engineering
• Expertise in Azure Databricks
• Proficiency in PySpark and distributed data processing
• Advanced skills in Python development
• Understanding of performance optimization for large-scale Spark workloads
• Background in operating and troubleshooting production data platforms
• Knowledge of monitoring, observability, and capacity planning combined with cloud cost optimization
• Ability to analyze existing systems, identify bottlenecks, and propose pragmatic improvements
• English proficiency at an Upper-Intermediate level (B2) or higher

Nice to have

• Familiarity with Microsoft Azure services and cloud infrastructure
• Experience building and maintaining CI/CD pipelines using GitHub Actions
• Exposure to large-scale telemetry, IoT, or automotive data
• Background in managing Databricks environments with a large number of scheduled jobs and pipelines
• Showcase of multi-region or geographically distributed cloud deployments

We offer

• CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT
  • Diversity of tasks and projects
  • Assessment center for objective review of competency level
  • Personal development plan
  • Mentoring programs and leadership development
  • Certification and professional development support
  • Access to learning platforms including more than 2,500 internal courses
  • English courses taught by certified teachers
• CORPORATE BENEFITS
  • Extra leave days
  • Referral bonuses
• COMPENSATION PACKAGE
  • Competitive compensation paid in USD
  • Regular salary and performance reviews
• MEDICAL & HEALTHCARE
  • Private health insurance
  • Well-being events
• WORKING ENVIRONMENT
  • Recreation areas and kitchens
  • Tea, coffee and snacks
  • Sports equipment and game consoles
  • IT Equipment
  • Microsoft’s Software Assurance Home Use Program (HUP)

Please note that our Talent Attraction Team reviews applications and CVs submitted in English.

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI‑Native enterprises, driving measurable value from innovation and digital investments.