Senior Data Engineer

AagatiServe Pvt Ltd; Delhi; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

AagatiServe Pvt Ltd is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and infrastructure on AWS. The role focuses on delivering high-quality data solutions that power analytics, AI, and business insights.

You will work with stakeholders, mentor juniors, and ensure data quality, security, and cost efficiency across environments. Proficiency in PySpark, SQL, and AWS services is essential, with a passion for robust data platforms.

Qualifications

• Minimum 5–8 years of experience in data engineering.
• Strong hands-on PySpark, SQL, and AWS capabilities.
• Experience building scalable data pipelines and data platforms.
• Familiarity with data governance, security, and cost optimization.
• Excellent communication and teamwork.

Responsibilities

• Design, develop, and optimize large-scale data pipelines using PySpark and AWS.
• Collaborate with stakeholders to translate data requirements into reliable solutions.
• Implement ETL/ELT processes to ingest, transform, and deliver data from multiple sources.
• Ensure data quality, integrity, and performance across systems.
• Mentor junior engineers and participate in code reviews and process improvements.
• Monitor pipelines for scalability, availability, and cost efficiency.
• Contribute to architecture and governance of the data platform.

Skills

PySpark

SQL

AWS

Airflow

Data modeling

Data warehousing

ETL

Git

CI/CD

Step Functions

Tools

S3

Glue

EMR

Redshift

Lambda

Athena

Step Functions

Docker

Kubernetes

Terraform

Job description

We are looking for a Senior Data Engineer to join our data engineering team and help design, build, and maintain scalable data pipelines and infrastructure on AWS. The ideal candidate will have strong hands‑on experience in PySpark, SQL, and AWS services, with a passion for building high-quality data solutions that power analytics, AI, and business insights.

Key Responsibilities

• Design, develop, and optimize large-scale data pipelines using PySpark and AWS technologies.
• Work with stakeholders to understand data requirements and deliver efficient, reliable, and reusable data solutions.
• Implement ETL/ELT processes to ingest, transform, and deliver data from multiple sources.
• Ensure data quality, integrity, and performance across all systems and environments.
• Collaborate with data scientists, analysts, and business teams to support analytics and reporting needs.
• Monitor and troubleshoot data pipelines, ensuring scalability, availability, and cost efficiency.
• Contribute to the architecture and design of the company’s data platform, ensuring best practices in security, performance, and governance.
• Mentor junior engineers and participate in code reviews, technical discussions, and process improvements.

Required Skills & Experience

• 5-8 years of experience as a Data Engineer or in a similar role.
• Strong expertise in PySpark for data transformation and processing.
• Advanced proficiency in SQL (query optimization, performance tuning, etc.).
• Hands‑on experience with AWS services such as S3, Glue, EMR, Redshift, Lambda, Athena, and Step Functions.
• Strong understanding of data modeling, data warehousing, and ETL best practices.
• Familiarity with version control (Git) and CI/CD pipelines for data engineering workflows.
• Experience with workflow orchestration tools (e.g., Airflow, AWS Step Functions).
• Knowledge of data governance, security, and compliance best practices.
• Excellent communication, problem‑solving, and analytical skills.

Preferred Qualifications

• Experience with streaming technologies (Kafka, Kinesis, Spark Streaming).
• Knowledge of containerization (Docker, Kubernetes) and infrastructure as code (Terraform, CloudFormation).
• Exposure to machine learning pipelines or data lakehouse architectures.
• AWS Certified Data Analytics or Solutions Architect certification.