Data Engineer Architect

Manatal; Hyderabad; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

Manatal is seeking a Data Engineer in Hyderabad, India, with over 8 years of experience in building large-scale data pipelines using technologies such as Hadoop and Spark. The ideal candidate will have strong skills in NoSQL databases and Python, along with a solid understanding of software engineering principles. Excellent communication skills and experience with cloud platforms like AWS or GCP are a plus. This role offers the opportunity to work in a dynamic environment focusing on scalable distributed systems.

Qualifications

• 8+ years of hands-on experience with data pipelines in distributed environments.
• Proficient in Python and familiar with modern engineering practices.
• Experience with cloud platforms like AWS or GCP is beneficial.

Responsibilities

• Design and build large scale data pipelines.
• Optimize scalable distributed systems for high data volumes.
• Integrate multiple applications using APIs.

Skills

Data pipeline development

NoSQL databases

Python programming

API integration

Cloud platforms (AWS, GCP)

Software engineering principles

Data modeling

Tools

Hadoop

Spark

Kafka

Hive

GitHub

Job description

• Responsibilities:8+ years of hands-on Experience with crafting and building large scale data pipelines in distributed environments with technologies such as Hadoop, Spark, Kafka, Hive etc.
• Experience with NoSQL datastores like Cassandra, Elasticsearch, HBase, MongoDB.
• Proven skills in designing, tuning & optimizing scalable, highly available distributed systems which can handle high data volumes.
• Strong understanding of software engineering principles and fundamentals including data structures and algorithms.
• Strong experience of working with APIs and integrating multiple applications together.
• Proficient & hands-on in Python is a must.
• Good Experience with modern engineering practices and technologies including CICD, Cloud native development, social coding (github), chatbots and more.
• Good data modelling experience to address scale and read/write performance.
• Excellent written and oral communication skills on both technical and non-technical topics.
• Excellent general analytical & problem solving skills.
• Experience with cloud computing platform like AWS or GCP is a plus.