Senior Data Engineer - AWS

Elementskill; Pune District; On-site; INR 1,500,000 - 2,600,000; Full time

Job summary

Elementskill is seeking a Senior Data Engineer in Pune to drive scalable data platforms on AWS and GCP. The role focuses on data ingestion, secure data transfer, and building fault-tolerant pipelines with Python-based automation.

You will collaborate with cross-functional teams, own end-to-end architecture, and optimize cloud workloads for performance and cost while ensuring data quality and governance. Strong communication and independence are essential.

Qualifications

• 5+ years of experience in Data Engineering, Cloud Engineering, or a related field.
• Strong hands-on experience with AWS.
• Proven experience designing and implementing AWS architecture and infrastructure for data workloads.
• Good understanding of AWS services.
• Experience with cloud networking, security, authentication, authorization, and access control.
• Experience with infrastructure automation and Infrastructure as Code (IaC), preferably Terraform.
• Strong understanding of data quality, monitoring, observability, and production operational best practices.
• Experience organizing GCP environments to meet data sovereignty and data residency requirements.
• Strong experience building data ingestion pipelines and data processing workflows.
• Hands-on experience designing and implementing secure data transfer and proxy solutions.
• Strong proficiency in Python and experience building production-grade applications and data pipelines.
• Excellent communication skills and ability to work independently with minimal supervision.

Responsibilities

• AWS Cloud Engineering: Build and operate data solutions using S3, Glue, Lambda, Redshift, and CloudWatch.
• Apply security, access-control, and cloud data management practices.
• Optimize cloud workloads for performance, reliability, and cost.
• Data Ingestion & Pipeline Engineering: Design robust ingestion pipelines for structured and unstructured data; build production-grade ingestion frameworks for moving data to GCP or similar storage.
• Develop scalable and fault-tolerant data pipelines with validation, monitoring, and error handling.
• Optimize ingestion workflows for performance, reliability, and cost; ensure data quality and traceability throughout ingestion process.
• Python Development & Automation: Develop production-grade data engineering applications and automation using Python; build reusable libraries and services.
• Automate infrastructure and data engineering processes to improve operational efficiency; implement logging, monitoring, error handling, and testing for Python services.

Skills

AWS

GCP

Python

Terraform

Data Pipelines

Tools

Terraform

Airflow

CI/CD

Snowflake

Job description

About The Role

We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in AWS Cloud, cloud infrastructure, data ingestion, and Python. The ideal candidate will have a proven track record of designing and implementing scalable data platforms, organizing GCP environments to meet data sovereignty requirements, and building secure and reliable data ingestion and transfer solutions.

About The Role

We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in AWS Cloud, cloud infrastructure, data ingestion, and Python. The ideal candidate will have a proven track record of designing and implementing scalable data platforms, organizing GCP environments to meet data sovereignty requirements, and building secure and reliable data ingestion and transfer solutions.

Key Responsibilities

• AWS Cloud Engineering :
• Work with AWS services such as S3, Glue, Lambda, Redshift, and CloudWatch to build and operate data solutions.
• Apply appropriate security, access-control, and cloud data management practices.
• Optimize cloud workloads for performance, reliability, and cost.
• Data Ingestion & Pipeline Engineering :
• Design and implement robust data ingestion pipelines for structured and unstructured data sources.
• Build production-grade ingestion frameworks to reliably process and move data into GCP or similar storage.
• Develop scalable and fault-tolerant data pipelines with appropriate validation, monitoring, and error handling.
• Optimize data ingestion workflows for performance, reliability, and cost.
• Ensure data pipelines maintain data quality, integrity, and traceability throughout the ingestion process.
• Python Development & Automation :
• Develop production-grade data engineering applications and automation using Python.
• Build reusable libraries, services, and utilities to support data ingestion and processing workflows.
• Automate infrastructure and data engineering processes to improve operational efficiency.
• Implement appropriate logging, monitoring, error handling, and testing for Python-based services and pipelines.
• Collaboration & Ownership :
• Work closely with Data Engineers, Software Engineers, Cloud/Infrastructure Engineers, Security teams, and business stakeholders.
• Take ownership of technical initiatives from architecture and design through production deployment and ongoing support.

Requirements

• 5+ years of experience in Data Engineering, Cloud Engineering, or a related field.
• Strong hands-on experience with AWS.
• Proven experience designing and implementing AWS architecture and infrastructure for data workloads.
• Good understanding of AWS services.
• Experience with cloud networking, security, authentication, authorization, and access control.
• Experience with infrastructure automation and Infrastructure as Code (IaC), preferably Terraform.
• Strong understanding of data quality, monitoring, observability, and production operational best practices.
• Experience organizing GCP environments to meet data sovereignty and data residency requirements.
• Strong experience building data ingestion pipelines and data processing workflows.
• Hands-on experience designing and implementing secure data transfer and proxy solutions.
• Strong proficiency in Python and experience building production-grade applications and data pipelines.
• Excellent communication skills and ability to work independently with minimal supervision.

(ref:hirist.tech)