Data Engineer

indiumsoft; Hyderabad; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Indiumsoft in Hyderabad is seeking a Data Engineer to design, build, and maintain scalable data pipelines using PySpark and Python. You will work with business teams to translate data needs into reliable solutions and help build data warehouse capabilities.

The role requires strong SQL, ETL/ELT expertise, experience with Hadoop and Big Data tools, and the ability to handle large datasets in distributed environments. Good communication and business understanding are essential.

Qualifications

• Strong hands-on experience with PySpark and Python.
• Advanced proficiency in SQL.
• Solid experience in ETL processes and data warehousing.
• Familiarity with Hadoop ecosystem and Big Data technologies.
• Experience working with large datasets in distributed environments.
• Good communication and business understanding.

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python
• Develop and optimize complex SQL queries for large datasets
• Implement and manage ETL/ELT processes ensuring data quality and reliability
• Collaborate with business and product teams to translate data requirements into solutions
• Build and maintain data warehouse solutions
• Handle large-scale data processing using Hadoop/Big Data technologies
• Perform performance tuning and optimization of data workflows

Skills

PySpark

Python

SQL

ETL

Data warehousing

Hadoop

Distributed systems

Communication

Tools

Airflow

Kafka

Spark

Job description

Job Title: Data Engineer Job Description:

• Design, build, and maintain scalable data pipelines using PySpark and Python
• Develop and optimize complex SQL queries for large datasets
• Implement and manage ETL/ELT processes ensuring data quality and reliability
• Collaborate with business and product teams to translate data requirements into solutions
• Build and maintain data warehouse solutions
• Handle large-scale data processing using Hadoop/Big Data technologies
• Perform performance tuning and optimization of data workflows

Required Skills:

• Strong hands-on experience with PySpark and Python
• Advanced proficiency in SQL
• Solid experience in ETL processes and data warehousing
• Familiarity with Hadoop ecosystem and Big Data technologies
• Experience working with large datasets in distributed environments
• Good communication and business understanding

Good to Have:

• Experience with Apache Airflow
• Exposure to cloud platforms (AWS, GCP, Azure)
• Knowledge of data lakes and modern data architectures
• Experience with streaming tools (Kafka, Spark Streaming)