Senior AWS Data Engineer

Durapid Technologies Pvt Ltd

Bengaluru

On-site

INR 2,500,000 - 4,500,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Durapid Technologies Pvt Ltd in Bengaluru, India is seeking a Senior AWS Data Engineer to design and optimize scalable data pipelines across AWS environments. You will work with Python, SQL and PySpark to build reusable components and ensure reliable data foundations for analytics and AI‑driven solutions.

The role emphasizes hands‑on execution, independent work with clients and collaboration with platform teams. Strong knowledge of Spark, AWS services and data governance is required.

Qualifications

  • Strong hands-on experience with AWS data engineering and building scalable pipelines.
  • Proficient in Python, SQL and PySpark for data processing and ETL/ELT workflows.
  • Experience optimizing Spark jobs, queries and cloud cost efficiency.

Responsibilities

  • Design, develop and maintain scalable ETL/ELT pipelines using AWS services.
  • Build reusable data engineering components with Python, SQL and PySpark.
  • Optimize Spark jobs, SQL queries and data processing workloads.
  • Analyse Spark plans, data skew, memory, and resource usage.
  • Improve partitioning, file sizing, compression, caching and joins.
  • Tune AWS Glue, Redshift, Athena and other AWS data services.
  • Implement automated testing, data validation, monitoring and recovery.
  • Troubleshoot production issues and perform root-cause analysis.
  • Establish performance benchmarks and measure runtime and cost improvements.
  • Apply data modelling, metadata management, security and governance best practices.
  • Maintain runbooks and engineering standards; mentor engineers and collaborate with stakeholders.

Skills

AWS data engineering
Advanced SQL
Python
PySpark
Apache Spark
Airflow
Data modeling
Performance tuning
CI/CD

Tools

S3
Glue
Redshift
Athena
Lambda
Git
Parquet
Iceberg
Kinesis
Kafka

Job description

We are looking for a Senior AWS Data Engineer with strong expertise in building, optimising and maintaining scalable data pipelines across AWS environments. The ideal candidate should have deep hands‑on experience with Python, SQL, PySpark, Apache Spark and AWS data services, along with the ability to troubleshoot complex workloads and improve pipeline performance, reliability and cost efficiency.

The role requires a technically strong individual contributor who can work independently with clients and engineering teams, provide technical guidance, and deliver reliable data foundations for analytics and AI‑driven solutions.

Key Responsibilities
  • Design, develop and maintain scalable ETL/ELT pipelines using AWS services.

  • Build reusable data engineering components using Python, SQL and PySpark.

  • Optimise complex SQL queries, Spark jobs and distributed data processing workloads.

  • Analyse Spark execution plans, data skew, shuffle operations, memory utilisation and resource consumption.

  • Improve partitioning, file sizing, compression, caching and join strategies.

  • Tune AWS Glue, Amazon Redshift, Amazon Athena and other AWS processing services.

  • Implement automated testing, data validation, monitoring, exception handling and recovery mechanisms.

  • Troubleshoot production issues, perform root‑cause analysis and implement preventive solutions.

  • Establish performance benchmarks and measure improvements in runtime, throughput and cloud costs.

  • Apply data modelling, metadata management, security and governance best practices.

  • Maintain technical documentation, operational runbooks and engineering standards.

  • Mentor engineers and collaborate with clients, platform teams and business stakeholders.

Mandatory Skills
  • Strong hands‑on experience in AWS data engineering: S3, Glue, Redshift, Athena and Lambda.

  • Advanced SQL with strong query optimisation and troubleshooting skills.

  • Proficiency in Python and PySpark.

  • Strong knowledge of Apache Spark and distributed data processing.

  • Experience with Apache Airflow or equivalent orchestration tools.

  • Understanding of data lakes, lakehouse architectures, data warehouses and dimensional modelling.

  • Experience with Git, CI/CD, automated testing and production support.

  • Practical experience in performance optimisation of large‑scale data pipelines.

Good to Have
  • Apache Kafka and/or Amazon Kinesis.

  • Amazon Bedrock, LLMs, embeddings and vector databases.

  • Excel/CSV ingestion, template‑driven processing and schema evolution.

  • Experience with columnar formats such as Parquet and Iceberg.

Soft Skills
  • Strong communication, comprehension and articulation skills.

  • Ability to work independently as a technical consultant and individual contributor.

  • Strong stakeholder management, persuasion and problem‑solving skills.

  • Ability to guide, mentor and inspire engineering teams.

  • Strong ownership, analytical thinking and client‑facing capabilities.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AWS Data Engineer
AWS Data Engineer

Durapid Technologies Pvt Ltd • Hyderabad

On-site
INR 900,000 - 1,350,000
Sr. Data Engineer
Sr. Data Engineer

Minfy Technologies • Gurugram District

On-site
INR 1,800,000 - 2,400,000
Senior Data Engineer
Senior Data Engineer

AagatiServe Pvt Ltd • Delhi

On-site
INR 1,800,000 - 2,400,000
Aws Data Engineer
Aws Data Engineer

Qentelli LLC • Bengaluru

On-site
INR 1,400,000 - 2,200,000
Lead AWS Data Engineer
Lead AWS Data Engineer

Weekday • Bengaluru

On-site
INR 1,000,000 - 1,500,000
AWS Data Engineer
AWS Data Engineer

ExlService Holdings, Inc. • Bengaluru

On-site
INR 1,200,000 - 2,200,000
AWS Pyspark Developer
AWS Pyspark Developer

Tata Consultancy Services Limited • Indore District

On-site
INR 1,200,000 - 2,400,000
AWS Data Engineer
AWS Data Engineer

Cognizant Technology Solutions Corporation • Chennai

Hybrid
INR 1,200,000 - 1,800,000
AWS Data Engineer
AWS Data Engineer

Cognizant Technology Solutions Corporation • Chennai

Hybrid
INR 1,000,000 - 1,800,000
Senior Data Engineer (AWS)
Senior Data Engineer (AWS)

Objectways • Bengaluru

On-site
INR 3,500,000 - 5,500,000