Big Data Developer( GCP)

Epam Systems; Hyderabad; Hybrid; INR 1,400,000 - 2,800,000; Full time

Benefits offered by this job

Health benefits

Retirement benefits

Paid time off

Flexible benefits

Job summary

Epam Systems is seeking an experienced big data professional to join our Hyderabad team. You will work on complex data platforms, leveraging Spark, Python and Hadoop ecosystem to deliver scalable data pipelines and analytics solutions.

You will lead technical efforts, collaborate with cross-functional teams, and drive modern data architecture initiatives across geographies, while embracing AGILE practices and continuous learning.

Qualifications

• 6-15 years of big data and data-related technology experience.
• Expert level understanding of distributed computing principles.
• Expert level knowledge and experience in Apache Spark.
• Hands on programming with Python.
• Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop.
• Experience with building stream-processing systems such as Storm or Spark-Streaming.
• Experience with messaging systems such as Kafka or RabbitMQ.
• Good understanding of Big Data querying tools such as Hive and Impala.
• Experience with data integration from multiple sources (RDBMS, ERP, files).
• Strong SQL queries, joins, stored procedures, relational schemas.
• Experience with NoSQL databases such as HBase, Cassandra, MongoDB.
• Knowledge of ETL techniques and frameworks.
• Performance tuning of Spark Jobs.
• Experience with AWS Databricks and Azure Databricks.
• Ability to lead a team efficiently.
• Experience with designing and implementing Big Data solutions.
• Practitioner of AGILE methodology.

Responsibilities

• Lead a team to deliver scalable big data solutions.
• Design and implement data pipelines and data models.
• Collaborate with cross-functional teams across geographies.
• Drive performance tuning and optimization of Spark jobs.
• Promote Agile practices and quality engineering.

Skills

Spark

Python

Distributed computing

Agile

Leadership

Big data

Streaming

Tools

Hadoop v2

MapReduce

HDFS

Sqoop

Hive

Impala

Spark Streaming

Kafka

RabbitMQ

NoSQL

SQL

ETL

AWS Databricks

Azure Databricks

Job description

Job Description:

1. 6-15 Years of in Big Data & Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Experience with messaging systems, such as Kafka or RabbitMQ
8. Good understanding of Big Data querying tools, such as Hive, and Impala
9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
10. Good understanding of SQL queries, joins, stored procedures, relational schemas
11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
12. Knowledge of ETL techniques and frameworks
13. Performance tuning of Spark Jobs
14. Experience with native Cloud data services AWS Databricks, Azure databricks
15. Ability to lead a team efficiently
16. Experience with designing and implementing Big data solutions
17. Practitioner of AGILE methodology

WE OFFER:

1. Opportunity to work on technical challenges that may impact across geographies
2. Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications
3. Opportunity to share your ideas on international platforms
4. Sponsored Tech Talks & Hackathons
5. Possibility to relocate to any EPAM office for short and long-term projects
6. Focused individual development

Benefit package:

• Health benefits, Medical Benefits
• Retirement benefits
• Paid time off
• Flexible benefits

1. Forums to explore beyond work passion (CSR, photography, painting, sports, etc.)