Azure Data Engineer

Epam Systems; Hyderabad; Híbrido; INR 1 200 000 - 2 400 000; Tempo integral

Resumo da oferta

EPAM Systems in Hyderabad, India is seeking an experienced Big Data Developer to design and implement data platforms using Spark, Hadoop, and cloud data services. The role requires leadership of a team and hands-on Python development to build scalable data pipelines and streaming solutions.

You will work with SQL, NoSQL databases, and data integration from multiple sources, following AGILE practices in a fast-paced environment.

Qualificações

• 6+ years of Big Data and data-related technology experience.
• Expert understanding of distributed computing principles.
• Expert knowledge of Apache Spark.
• Hands-on Python programming.
• Experience with Hadoop v2, MapReduce, HDFS, Sqoop.
• Experience building stream-processing systems (Storm or Spark-Streaming).
• Experience with Kafka or RabbitMQ.
• Strong SQL, joins, and relational schemas.
• Experience with NoSQL (HBase, Cassandra, MongoDB).
• Knowledge of ETL techniques and Azure Databricks.

Responsabilidades

• Lead design and implementation of big data solutions.
• Mentor and guide a team of data engineers.
• Develop scalable data pipelines and data integration across sources.
• Apply AGILE methodologies in delivery.

Conhecimentos

Big Data

Spark

Python

Distributed computing

AGILE

Team leadership

SQL

NoSQL

Kafka

Azure Databricks

Ferramentas

Hadoop v2

Hive

Impala

Sqoop

Storm / Spark-Streaming

HDFS

MongoDB

Cassandra

HBase

Descrição da oferta de emprego

Job description

Greetings form EPAM Systems!!

Your Profile is Shortlisted for Big Data Developer, please find the JD below.

Job Description:

1. 6+ Years of in Big Data & Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Experience with messaging systems, such as Kafka or RabbitMQ
8. Good understanding of Big Data querying tools, such as Hive, and Impala
9. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
10. Good understanding of SQL queries, joins, stored procedures, relational schemas
11. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
12. Knowledge of ETL techniques and frameworks
13. Performance tuning of Spark Jobs
14. Experience with native Cloud data services Azure Databricks
15. Ability to lead a team efficiently
16. Experience with designing and implementing Big data solutions
17. Practitioner of AGILE methodology

Mandatory skills:

Python, Spark, Azure, Databricks