Senior Data Engineer

Calsoft; Pune District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Calsoft is seeking an experienced Data Engineer in Pune with 5+ years of hands-on experience designing, developing, and maintaining scalable data pipelines and ETL solutions.

The ideal candidate will have strong expertise in AWS or Azure, ETL, SQL, Python, and working knowledge of GCP, and will work on data integration across multiple sources with a focus on data quality and performance.

Qualifications

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with ETL/ELT processes.
• Strong SQL skills, including query optimization.
• Good hands-on experience with Python for data engineering.
• Strong experience with GCP, AWS or Azure.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data integration solutions using AWS or Azure cloud services.
• Write complex and optimized SQL queries for data extraction, transformation, and analysis.
• Develop and maintain data processing solutions using Python.
• Work with structured and semi-structured data from multiple sources.
• Perform data transformation, cleansing, validation, and quality checks.
• Monitor and troubleshoot data pipelines and resolve production issues.
• Collaborate with Data Architects, Developers, Analysts, and stakeholders.
• Ensure data pipelines are scalable, reliable, secure, and optimized for performance.
• Support cloud-based data platform development and migration initiatives.

Skills

ETL/ELT

SQL

Python

GCP

AWS/Azure

Data Warehousing

Tools

Airflow

Apache Spark/PySpark

Docker

Git

CI/CD

BigQuery

Redshift

S3

Lambda

Dataflow

Synapse

Azure Data Factory

Azure Data Lake

Job description

Notice Period:

Immediate Joiner

Employment Type:

Full-Time

Role Overview

We are looking for an experienced Data Engineer with 5+ years of hands‑on experience in designing, developing, and maintaining scalable data pipelines and ETL solutions. The ideal candidate should have strong expertise in AWS or Azure, ETL, SQL, and Python, along with working knowledge of GCP.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.


• Develop data integration solutions using AWS or Azure cloud services.


• Write complex and optimized SQL queries for data extraction, transformation, and analysis.


• Develop and maintain data processing solutions using Python.


• Work with structured and semi‑structured data from multiple sources.


• Perform data transformation, cleansing, validation, and quality checks.


• Monitor and troubleshoot data pipelines and resolve production issues.


• Collaborate with Data Architects, Developers, Analysts, and other stakeholders.


• Ensure data pipelines are scalable, reliable, secure, and optimized for performance.


• Support cloud‑based data platform development and migration initiatives.

Required Skills

• 5+ years of experience in Data Engineering.


• Strong hands‑on experience with ETL/ELT processes.


• Strong SQL skills, including query optimization.


• Good hands‑on experience with Python for data engineering.


• Strong experience with GCP, AWS or Azure.


• Understanding of Data Warehousing and Data Modeling concepts.


• Experience working with large‑scale data and production data pipelines.

Good to Have

• Experience with AWS Glue / S3 / Lambda / Redshift.


• Experience with Azure Data Factory / Azure Data Lake / Synapse.


• Knowledge of GCP BigQuery / Cloud Storage / Dataflow.


• Experience with Apache Spark / PySpark.


• Knowledge of Airflow or other workflow orchestration tools.


• Exposure to Git, CI/CD, Docker, or cloud‑native technologies.

Candidate Profile

• Strong problem‑solving and analytical skills.


• Good communication and stakeholder management skills.


• Ability to work independently and as part of a technical team.