Big Data Developer

Epam Systems; Pune District; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Epam Systems in Pune is seeking a Senior Big Data Engineer with 5.6+ years of experience to lead data projects and build scalable Spark-based solutions. You will work with Python, Spark, Azure Databricks, and cloud data services while mentoring teammates in an AGILE environment.

The role emphasizes distributed computing, data integration from multiple sources, and NoSQL/Hadoop ecosystem familiarity to drive end-to-end data platforms.

Qualifications

• 5.6+ years of Big Data and data-related technology experience.
• Expert level understanding of distributed computing principles.
• Expert level knowledge and experience in Apache Spark.
• Hands-on programming with Python.

Responsibilities

• Lead a team efficiently to deliver Big Data solutions.
• Design and implement scalable Big Data architectures and pipelines.
• Practise AGILE methodologies in project delivery.

Skills

Python

Spark

Azure

Databricks

Hadoop

Kafka

Spark Streaming

SQL

NoSQL

AGILE

Tools

HDFS

MapReduce

Hive/Impala

Kafka

Job description

Job description

Greetings form EPAM Systems!!

Job Description:

1. 5.6+ Years of in Big Data & Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Experience with messaging systems, such as Kafka or RabbitMQ
8. Good understanding of Big Data querying tools, such as Hive, and Impala
9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
10. Good understanding of SQL queries, joins, stored procedures, relational schemas
11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
12. Knowledge of ETL techniques and frameworks
13. Performance tuning of Spark Jobs
14. Experience with native Cloud data services AWS Databricks & GCP
15. Ability to lead a team efficiently
16. Experience with designing and implementing Big data solutions
17. Practitioner of AGILE methodology

Mandatory skills:

Python, Spark, Azure, Databricks