Big Data Developer

Epam Systems; Bengaluru; Hybrid; INR 2,400,000 - 3,600,000; Full time

Job summary

Epam Systems in Bengaluru is seeking a seasoned Big Data Developer to join our data engineering team. The role focuses on building and optimizing distributed data processing systems, leveraging Spark, Python, and Azure Databricks to deliver scalable solutions.

The candidate should bring 6+ years of hands-on experience with Hadoop ecosystems, streaming, and data integration from diverse sources, while applying AGILE practices to drive timely, robust results.

Qualifications

• 6+ years of experience in Big Data & Data related technology.
• Expert level understanding of distributed computing principles.
• Expert level knowledge and experience in Apache Spark.
• Hands on programming with Python.
• Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop.
• Experience with building stream-processing systems using technologies such as Apache Storm or Spark-Streaming.
• Experience with messaging systems such as Kafka or RabbitMQ.
• Good understanding of Big Data querying tools such as Hive and Impala.
• Experience with integration of data from multiple data sources such as RDBMS, ERP, Files.
• Good understanding of SQL queries, joins, stored procedures, relational schemas.
• Experience with NoSQL databases such as HBase, Cassandra, MongoDB.
• Knowledge of ETL techniques and frameworks.
• Performance tuning of Spark Jobs.
• Experience with native Cloud data services Azure Databricks.
• Ability to lead a team efficiently.
• Experience with designing and implementing Big data solutions.
• Practitioner of AGILE methodology.

Skills

Python

Spark

Azure

Databricks

Tools

Hadoop v2

Map Reduce

HDFS

Sqoop

Kafka

RabbitMQ

Hive

Impala

NoSQL (HBase/Cassandra/MongoDB)

Job description

Job description

Your Profile is Shortlisted for Big Data Developer, please find the JD below.

Job Description:

1. 6+ Years of in Big Data & Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Experience with messaging systems, such as Kafka or RabbitMQ
8. Good understanding of Big Data querying tools, such as Hive, and Impala
9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
10. Good understanding of SQL queries, joins, stored procedures, relational schemas
11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
12. Knowledge of ETL techniques and frameworks
13. Performance tuning of Spark Jobs
14. Experience with native Cloud data services Azure Databricks
15. Ability to lead a team efficiently
16. Experience with designing and implementing Big data solutions
17. Practitioner of AGILE methodology

Mandatory skills:

Python, Spark, Azure, Databricks