Aws Data Engineer

Agile Ventures; Gurugram District; Hybrid; INR 1,500,000 - 2,100,000; Full time

Job summary

Agile Ventures in Gurugram, India, is seeking an experienced Data Software Engineer to build and optimize large-scale data platforms. You will work on Spark, Hadoop, NoSQL databases, and cloud data services to deliver robust data solutions.

The role requires strong Python coding, SQL proficiency, and the ability to lead a team in an AGILE environment. Join us to design cutting-edge data architectures and drive data-driven outcomes.

Qualifications

• 5–12 years in Big Data and related technologies.
• Expert in distributed computing and Spark.
• Hands-on Python programming and SQL.

Responsibilities

• Lead data engineering initiatives and Big Data solutions.
• Design and implement data pipelines and streaming systems.
• Collaborate with stakeholders using AGILE methodologies.

Skills

Big Data

Distributed computing

Python

SQL

Agile

Team leadership

Big data design

Tools

Apache Spark

Hadoop

Hive/Impala

NoSQL

Spark Streaming / Storm

Cloud (AWS/Azure)

Job description

Data Software Engineering

Job Description:

1. 1. 5-12 Years of in Big Data & Data related technology experience
2. 2. Expert level understanding of distributed computing principles
3. 3. Expert level knowledge and experience in Apache Spark
4. 4. Hands on programming with Python
5. 5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. 6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. 7. Good understanding of Big Data querying tools, such as Hive, and Impala
8. 8. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
9. 9. Good understanding of SQL queries, joins, stored procedures, relational schemas
10. 10. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
11. 11. Knowledge of ETL techniques and frameworks
12. 12. Performance tuning of Spark Jobs
13. 13. Experience with native Cloud data services AWS or AZURE
14. 14. Ability to lead a team efficiently
15. 15. Experience with designing and implementing Big data solutions
16. 16. Practitioner of AGILE methodology