Lead Data Software Engineer

EPAM Systems; Chennai District; On-site; INR 2,500,000 - 5,000,000; Full time

Job summary

EPAM Systems is seeking an experienced Lead Data Software Engineer to design and deliver data solutions across diverse projects. The role emphasizes leading data engineering efforts using Spark, Python, and cloud services on AWS/Azure.

The ideal candidate will drive end-to-end data platforms, collaborate across teams, and ensure scalable, secure, and high-performance data applications. Strong leadership and a strategic mindset are valued.

Qualifications

• 8-12 years of experience in Data and related technologies.
• Expertise in Apache Spark.
• Proficiency in Hadoop and Hive.
• Skills in Python.
• Competency using AWS and Azure cloud data services.

Responsibilities

• Develop end-to-end Data solutions tailored to complex business demands.
• Collaborate with cross-functional teams to understand project needs and ensure exceptional software solutions.
• Utilize knowledge of Apache Spark, Python, Azure and AWS to design scalable and efficient systems.
• Ensure the performance, security and scalability of Data applications.
• Stay updated on emerging trends and innovations in Data technologies to improve development practices.

Tools

Apache Spark

Python

Hadoop

Hive

AWS

Azure

Docker

Kubernetes

Job description

We are looking for an experienced Lead Data Software Engineer to join our dynamic team and address rewarding challenges. As a Lead Engineer, you will play a key role in designing and delivering Data solutions across diverse projects. The ideal candidate should have significant expertise in Data and associated technologies, particularly Apache Spark, Python, Azure and AWS.

Responsibilities

• Develop end-to-end Data solutions tailored to complex business demands
• Collaborate with cross-functional teams to understand project needs and ensure exceptional software solutions
• Utilize knowledge of Apache Spark, Python, Azure and AWS to design scalable and efficient systems
• Ensure the performance, security and scalability of Data applications
• Stay updated on emerging trends and innovations in Data technologies to improve development practices

Requirements

• 8-12 years of strong experience in Data and related technologies
• Expertise in Apache Spark
• Proficiency in Hadoop and Hive
• Skills in Python
• Competency in using native Cloud data services such as AWS and Azure

Nice to have

• Background in machine learning algorithms and methods
• Proficiency in data visualization tools like Tableau or Power BI
• Familiarity with real-time data processing frameworks like Apache Flink or Apache Storm
• Capability to work with containerization technologies such as Docker or Kubernetes

Technologies

• Hadoop
• Hive