A complete application in a minute
Tailored resume and cover letter, ready to send.
Durapid Technologies Pvt Ltd in Bengaluru, India is seeking a Senior AWS Data Engineer to design and optimize scalable data pipelines across AWS environments. You will work with Python, SQL and PySpark to build reusable components and ensure reliable data foundations for analytics and AI‑driven solutions.
The role emphasizes hands‑on execution, independent work with clients and collaboration with platform teams. Strong knowledge of Spark, AWS services and data governance is required.
We are looking for a Senior AWS Data Engineer with strong expertise in building, optimising and maintaining scalable data pipelines across AWS environments. The ideal candidate should have deep hands‑on experience with Python, SQL, PySpark, Apache Spark and AWS data services, along with the ability to troubleshoot complex workloads and improve pipeline performance, reliability and cost efficiency.
The role requires a technically strong individual contributor who can work independently with clients and engineering teams, provide technical guidance, and deliver reliable data foundations for analytics and AI‑driven solutions.
Design, develop and maintain scalable ETL/ELT pipelines using AWS services.
Build reusable data engineering components using Python, SQL and PySpark.
Optimise complex SQL queries, Spark jobs and distributed data processing workloads.
Analyse Spark execution plans, data skew, shuffle operations, memory utilisation and resource consumption.
Improve partitioning, file sizing, compression, caching and join strategies.
Tune AWS Glue, Amazon Redshift, Amazon Athena and other AWS processing services.
Implement automated testing, data validation, monitoring, exception handling and recovery mechanisms.
Troubleshoot production issues, perform root‑cause analysis and implement preventive solutions.
Establish performance benchmarks and measure improvements in runtime, throughput and cloud costs.
Apply data modelling, metadata management, security and governance best practices.
Maintain technical documentation, operational runbooks and engineering standards.
Mentor engineers and collaborate with clients, platform teams and business stakeholders.
Strong hands‑on experience in AWS data engineering: S3, Glue, Redshift, Athena and Lambda.
Advanced SQL with strong query optimisation and troubleshooting skills.
Proficiency in Python and PySpark.
Strong knowledge of Apache Spark and distributed data processing.
Experience with Apache Airflow or equivalent orchestration tools.
Understanding of data lakes, lakehouse architectures, data warehouses and dimensional modelling.
Experience with Git, CI/CD, automated testing and production support.
Practical experience in performance optimisation of large‑scale data pipelines.
Apache Kafka and/or Amazon Kinesis.
Amazon Bedrock, LLMs, embeddings and vector databases.
Excel/CSV ingestion, template‑driven processing and schema evolution.
Experience with columnar formats such as Parquet and Iceberg.
Strong communication, comprehension and articulation skills.
Ability to work independently as a technical consultant and individual contributor.
Strong stakeholder management, persuasion and problem‑solving skills.
Ability to guide, mentor and inspire engineering teams.
Strong ownership, analytical thinking and client‑facing capabilities.