Senior ETL / Data Engineer

Durapid Technologies Pvt Ltd; India; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Durapid Technologies Pvt Ltd is seeking a Senior ETL / Data Engineer with hands-on experience in PySpark, Apache Spark, and modern cloud data platforms. The role focuses on designing scalable ETL pipelines, optimizing data processing jobs, and implementing data integration across cloud environments.

You will work with NoSQL and Graph Databases, build robust data models, and ensure data quality across large-scale data lake and warehouse ecosystems. Strong SQL and Python skills are essential.

Qualifications

• 7+ years of experience in Data Engineering / ETL development.
• Strong hands-on experience with PySpark and Apache Spark.
• Experience with cloud data platforms such as Databricks, Snowflake, or Amazon Redshift.
• Solid background in ETL/ELT pipeline development.
• Hands-on experience with NoSQL databases.
• Experience with Graph Databases and graph data modeling.
• Strong SQL and Python skills.
• Experience with data transformation, data integration, and performance optimization.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Python.
• Develop and optimize large-scale data processing jobs using Apache Spark / PySpark.
• Work with cloud data platforms such as Databricks, Snowflake, and Amazon Redshift.
• Build data ingestion and transformation pipelines from structured and unstructured data sources.
• Work with NoSQL databases for high-volume and distributed data workloads.
• Design and implement solutions using Graph Databases and graph data models.
• Perform data transformation, cleansing, validation, reconciliation, and quality checks.
• Optimize ETL pipelines, Spark jobs, queries, and data storage for performance and scalability.
• Implement data integration across databases, APIs, files, data lakes, and cloud platforms.
• Troubleshoot data pipeline failures and perform root-cause analysis.
• Collaborate with Data Architects, Data Scientists, BI teams, and application teams.

Skills

PySpark

Apache Spark

SQL

Python

ETL/ELT

NoSQL databases

Graph Databases

Cloud platforms

Job description

Job Summary

We are looking for an experienced Senior ETL / Data Engineer with strong expertise in PySpark, modern cloud data platforms, NoSQL databases, Graph Databases, and large-scale ETL pipelines. The ideal candidate should have hands-on experience designing, developing, and optimizing scalable data processing solutions across cloud data warehouse and data lake environments.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Python.
• Develop and optimize large-scale data processing jobs using Apache Spark / PySpark.
• Work with cloud data platforms such as Databricks, Snowflake, and Amazon Redshift.
• Build data ingestion and transformation pipelines from structured and unstructured data sources.
• Work with NoSQL databases for high-volume and distributed data workloads.
• Design and implement solutions using Graph Databases and graph-based data models.
• Perform data transformation, cleansing, validation, reconciliation, and quality checks.
• Optimize ETL pipelines, Spark jobs, queries, and data storage for performance and scalability.
• Implement data integration across databases, APIs, files, data lakes, and cloud platforms.
• Troubleshoot data pipeline failures and perform root-cause analysis.
• Collaborate with Data Architects, Data Scientists, BI teams, and application teams.

Required Skills

• 7+ years of experience in Data Engineering / ETL development.
• Strong hands-on experience with PySpark / Apache Spark.
• Experience with one or more cloud data platforms: Databricks, Snowflake, or Amazon Redshift.
• Strong experience in ETL/ELT pipeline development.
• Hands-on experience with NoSQL databases.
• Experience with Graph Databases and graph data modeling.
• Strong SQL and Python skills.
• Experience with data transformation, data integration, and performance optimization.
• Good understanding of data warehousing and data lake concepts.
• Experience working with large-scale datasets and distributed processing environments.

Good to Have

Databricks | Snowflake | Amazon Redshift | PySpark | Apache Spark | NoSQL | Graph Database | Python | SQL | ETL/ELT | Data Lake | Data Warehouse | Cloud Data Engineering