Lead Data Engineer: Spark, Databricks & Kubernetes (On-Prem)

EPAM Systems; Turkey; On-site; TRY 4,425,000 - 6,391,000; Full time

Benefits offered by this job

Extra leave days

Referral bonuses

Private health insurance

Well-being events

IT Equipment

Job summary

EPAM Systems, Inc. seeks a Lead Data Software Engineer to drive a migration of an existing data analytics platform to a cloud-agnostic, on-prem capable solution. You will develop streaming and batch Spark pipelines on Kubernetes and guide a growing team.

The role requires 5+ years with Spark/Databricks/Kubernetes, Python proficiency, and strong communication. Collaboration across refinement, planning, and demos is essential in a fast-evolving environment.

Qualifications

• 5+ years of experience with Apache Spark, Databricks, and Kubernetes.
• Proficiency in Python.
• English proficiency at B2 level or higher.
• Experience with Confluent/Apache Kafka.

Responsibilities

• Migrate data analytics platform to a cloud-agnostic solution with on-prem deployment.
• Develop streaming and batch Spark pipelines running on Kubernetes.
• Guide and mentor team members; participate in refinement, planning and demos.
• Investigate and resolve bugs; perform unit testing to ensure code quality.

Skills

Apache Spark

Databricks

Kubernetes

Python

Confluent Kafka

Tools

Argo CD

Ansible

Kafka Connect

KSQL

Kafka Streams

Job description

EPAM Systems, Inc. seeks a Lead Data Software Engineer to drive a migration of an existing data analytics platform to a cloud-agnostic, on-prem capable solution. You will develop streaming and batch Spark pipelines on Kubernetes and guide a growing team.

The role requires 5+ years with Spark/Databricks/Kubernetes, Python proficiency, and strong communication. Collaboration across refinement, planning, and demos is essential in a fast-evolving environment.