Lead Data Platform Engineer — Spark, Kafka & Kubernetes

EPAM Systems; Turkey; On-site; TRY 5,348,000 - 8,751,000; Full time

Benefits offered by this job

Private health insurance

Learning & development

Mentoring programs

English courses

Job summary

EPAM Systems in Turkey is seeking a Lead Data Software Engineer to drive the migration of an existing data analytics platform to a cloud-agnostic solution ready for on-premises deployment. You will implement streaming and batch Spark pipelines on Kubernetes, develop the data generator framework, and mentor the team while collaborating with the team lead.

The role requires 5+ years with Spark/Databricks/Kubernetes, Python proficiency, Kafka experience, and EPC Data Engineering coursework

Qualifications

• 5+ years of experience with Apache Spark, Databricks, and Kubernetes (K8S).
• Proficiency in Python.
• Experience with Confluent/Apache Kafka.
• Completed EPAM Data Engineering course or equivalent qualification.

Responsibilities

• Implement streaming and batch Spark pipelines running on Kubernetes.
• Develop objects of the data generator framework.
• Deploy and test solutions on local and development environments.
• Enhance and refine the current actively evolving solution.
• Investigate and resolve bugs.
• Conduct unit testing to ensure code quality.
• Participate in refinement, planning, and demo sessions.
• Guide and mentor team members while syncing with the team lead.

Skills

Apache Spark

Databricks

Kubernetes

Python

English (B2+)

Education

EPAM Data Engineering course

Tools

Confluent Kafka

Kafka Connect / KSQL / Kafka Streams

Job description

EPAM Systems in Turkey is seeking a Lead Data Software Engineer to drive the migration of an existing data analytics platform to a cloud-agnostic solution ready for on-premises deployment. You will implement streaming and batch Spark pipelines on Kubernetes, develop the data generator framework, and mentor the team while collaborating with the team lead.

The role requires 5+ years with Spark/Databricks/Kubernetes, Python proficiency, Kafka experience, and EPC Data Engineering coursework