Data Engineer(Senior/Lead Engineer)

Epam Systems; Hyderabad, Pune District, Gurugram District; Hybrid; INR 1,500,000 - 2,200,000; Full time

Job summary

EPAM Systems in Hyderabad is seeking a Senior/Lead Data Engineer to design and implement large-scale data platforms. You will work across data ingestion, processing, and analytics, leveraging Spark, PySpark, and cloud services to deliver data-driven solutions.

The role requires hands-on coding in Python, expertise with Hadoop ecosystems, and the ability to lead a team in a hybrid work setup with on-site days.

Qualifications

• 5-14 years of Big Data or data-related experience.
• Expert understanding of distributed computing principles.
• Expert knowledge of Apache Spark.
• Hands-on programming with Python.
• Proficiency with Hadoop v2, Map Reduce, HDFS.
• Experience with building stream-processing systems (Storm or Spark Streaming).
• Experience with messaging systems (Kafka or RabbitMQ).
• Good understanding of SQL queries, joins, stored procedures, relational schemas.
• Experience with NoSQL databases (HBase, Cassandra, MongoDB).
• Knowledge of ETL techniques and frameworks.
• Performance tuning of Spark jobs.
• Experience with native cloud data services (Azure Databricks).
• Ability to lead a team efficiently.

Responsibilities

• Lead a data engineering team efficiently.
• Design and implement Big Data solutions for large-scale data platforms.
• Integrate data from multiple sources (RDBMS, ERP, files).
• Tune and optimize Spark jobs for performance.
• Work with cloud data services like Azure Databricks.
• Collaborate on building streaming data pipelines (Spark Streaming, Storm).

Skills

Python

Spark

PySpark

SQL

Data Structures & Algorithms

AGILE Methodology

Tools

Hadoop

Spark Streaming

Kafka

Hive/Impala

NoSQL

Azure Databricks

Cloud (AWS/GCP)

Job description

About EPAM Systems:

EPAM Systems is a leading global digital engineering and consulting company with over 60,000+ employees across 50+ countries. We partner with top enterprises to design, build, and optimize data-driven solutions. Within EPAMs Data & Analytics practice, our focus is on building modern data platforms and models that enable large-scale analytics and AI initiatives.

Exp: 5-14years, Data Engineer(Senior/Lead Engineer)

Location: Hyderabad, Chennai, Coimbatore, Pune, Gurgaon, Bangalore

Job Description:

1. 5-14 Years of in Big Data or Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop(optional)
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Experience with messaging systems, such as Kafka or RabbitMQ(optional)
8. Good understanding of Big Data querying tools, such as Hive, and Impala(optional)
9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
10. Good understanding of SQL queries, joins, stored procedures, relational schemas
11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB(optional)
12. Knowledge of ETL techniques and frameworks
13. Performance tuning of Spark Jobs
14. Experience with native Cloud data services Azure databricks
15. Ability to lead a team efficiently
16. Experience with designing and implementing Big data solutions(optional)
17. Knowledge on data structure and algorithms
18. Practitioner of AGILE methodology

Key skills:

Hands-on with Python Programming, Spark, PySpark, SQL , databricks and AWS/GCP

Azure profiles wont be considered now.

Notice Period: 0-40 days. Serving notice period beyond 30 days is accepted.

Hybrid model: 3 days from office.

In-person Interview date and location: 17th Oct 2026(Saturday) and EPAM Hyderabad

Regards,

EPAM Team