Lead Data Software Engineer with Databricks, Apache Kafka, Apache Spark, Kubernetes

EPAM Systems; Turkey; Vor Ort; TRY 4.425.000 - 6.391.000; Vollzeit

Benefits dieser Stelle

Extra leave days

Referral bonuses

Private health insurance

Well-being events

IT Equipment

Zusammenfassung

EPAM Systems, Inc. seeks a Lead Data Software Engineer to drive a migration of an existing data analytics platform to a cloud-agnostic, on-prem capable solution. You will develop streaming and batch Spark pipelines on Kubernetes and guide a growing team.

The role requires 5+ years with Spark/Databricks/Kubernetes, Python proficiency, and strong communication. Collaboration across refinement, planning, and demos is essential in a fast-evolving environment.

Qualifikationen

• 5+ years of experience with Apache Spark, Databricks, and Kubernetes.
• Proficiency in Python.
• English proficiency at B2 level or higher.
• Experience with Confluent/Apache Kafka.

Aufgaben

• Migrate data analytics platform to a cloud-agnostic solution with on-prem deployment.
• Develop streaming and batch Spark pipelines running on Kubernetes.
• Guide and mentor team members; participate in refinement, planning and demos.
• Investigate and resolve bugs; perform unit testing to ensure code quality.

Kenntnisse

Apache Spark

Databricks

Kubernetes

Python

Confluent Kafka

Tools

Argo CD

Ansible

Kafka Connect

KSQL

Kafka Streams

Jobbeschreibung

We are seeking a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as DataFactory, Databricks, EventHub, and Cassandra, to a cloud-agnostic platform capable of on-premises deployment.ResponsibilitiesImplement streaming and batch Spark pipelines running on K8SDevelop objects of the data generator frameworkDeploy and test solutions on local and development environmentsEnhance and refine the current actively evolving solutionInvestigate and resolve bugsConduct unit testing to ensure code qualityParticipate in refinement, planning, and demo sessionsGuide and mentor team members while syncing with the team leadRequirements5+ years of experience with Apache Spark, Databricks, and Kubernetes (K8S)Proficiency in PythonExperience with Confluent/Apache KafkaCompleted EPAM Data Engineering course or equivalent qualificationCapability to quickly adapt and dive into the active implementation phase of a projectAbility to work independently while coordinating with a team leadStrong communication skills and a proactive, hands-on, team-player attitudeEnglish proficiency at B2 level or higherNice to haveKnowledge of JavaFamiliarity with Kafka Connect, KSQL, and Kafka StreamsBackground in Ansible and Argo CDUnderstanding of TDDWe offerCONTINUOUS UPSKILLING, LEARNING & DEVELOPMENTDiversity of tasks and projectsAssessment center for objective review of competency levelPersonal development planMentoring programs and leadership developmentCertification and professional development supportAccess to learning platforms including more than 2,500 internal coursesEnglish courses taught by certified teachersCORPORATE BENEFITSExtra leave daysReferral bonusesCOMPENSATION PACKAGECompetitive compensation paid in USDRegular salary and performance reviewsMEDICAL & HEALTHCAREPrivate health insuranceWell-being eventsWORKING ENVIRONMENTRecreation areas and kitchensTea, coffee and snacksSports equipment and game consolesIT EquipmentMicrosoft’s Software Assurance Home Use Program (HUP)Please note that our Talent Attraction Team reviews applications and CVs submitted in English.EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.