Senior AWS EMR Engineer

Web Spiders Group

Kolkata

Presencial

INR 1.800.000 - 3.600.000

Jornada completa

Hace 6 días
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

Web Spiders Group in Kolkata is seeking a Senior AWS EMR Engineer with deep hands-on experience in AWS EMR, Apache Spark, Hadoop, and large-scale distributed data processing. The role requires building, managing, optimizing and troubleshooting production-grade data workloads on AWS with EMR clusters and Spark workloads.

The ideal candidate will design scalable data pipelines, optimize performance and cost, and collaborate with cross-functional teams.

Formación

  • 5+ years hands-on data engineering with AWS EMR and Spark.
  • Strong knowledge of Hadoop ecosystem and distributed processing.
  • Proficiency in PySpark or Scala for large-scale workloads.
  • Experience with S3 data lakes and containerized/cloud workloads.

Responsabilidades

  • Design, develop, deploy and optimize large-scale data workloads on AWS EMR and Spark.
  • Build and maintain distributed data processing solutions using Spark/Hadoop.
  • Develop and optimize Spark jobs for performance, scalability and cost.
  • Work with PySpark/Scala for processing and transformation.
  • Configure and manage EMR clusters according to workload needs.
  • Optimize applications for resource usage and execution speed.
  • Troubleshoot EMR, Spark, Hadoop and production data issues.
  • Integrate EMR workloads with S3 and AWS services (Glue, Step Functions, Airflow/MWAA).
  • Monitor workloads and implement robust logging and fault tolerance.

Conocimientos

AWS EMR
Apache Spark
Hadoop
PySpark
Scala
ETL/ELT
S3
Glue
Airflow
MWAA
Step Functions
Lambda
Redshift
CloudWatch

Herramientas

CloudWatch
Glue
Step Functions
MWAA

Descripción del empleo

Web Spiders is looking for a Senior AWS EMR Engineer with strong hands‑on experience in AWS EMR, Apache Spark, Hadoop, and large‑scale distributed data processing. The ideal candidate will have experience building, managing, optimizing, and troubleshooting production‑grade data processing workloads on AWS, with a strong understanding of EMR clusters, Spark workloads, data pipelines, performance optimization, scalability, reliability, and cost efficiency.

Web Spiders is looking for a Senior AWS EMR Engineer with strong hands‑on experience in AWS EMR, Apache Spark, Hadoop, and large‑scale distributed data processing.

The ideal candidate will have experience building, managing, optimizing, and troubleshooting production‑grade data processing workloads on AWS, with a strong understanding of EMR clusters, Spark workloads, data pipelines, performance optimization, scalability, reliability, and cost efficiency.

If AWS EMR + Spark/Hadoop is your core expertise, we'd love to hear from you.

5+ Years Experience | Kolkata – Work from Office

* Immediate joiners preferred.*

Working Hours: Ability to work in the US Eastern Time Zone. Depending on project requirements, this may be adjusted to a half‑day IST + half‑day US EST schedule.


What You'll Do:
  • Design, develop, deploy, and optimize large‑scale data processing workloads using AWS EMR and Apache Spark.
  • Build and maintain distributed data processing solutions using Spark/Hadoop.
  • Develop and optimize Spark jobs for performance, scalability, reliability, and cost efficiency.
  • Work with PySpark/Scala for distributed data processing and transformation.
  • Configure and manage EMR clusters based on workload and processing requirements.
  • Optimize Spark applications, including resource utilization, partitioning, joins, caching, and execution performance.
  • Troubleshoot EMR, Spark, Hadoop, and production data‑processing issues.
  • Work with Amazon S3 as a scalable data lake/storage layer.
  • Integrate EMR workloads with AWS services such as Glue, Lambda, Step Functions, and Airflow/MWAA.
  • Monitor data‑processing workloads and implement appropriate logging, error handling, and operational controls.
  • Optimize cloud workloads for performance, scalability, reliability, and AWS cost.
  • Collaborate with Data Engineering, Cloud, AI/ML, and Product teams to deliver reliable data‑processing solutions.
Must‑Have Skills:
  • 5+ years of hands‑on experience in Data Engineering / Big Data Engineering.
  • Strong hands‑on experience with AWS EMR.
  • Strong experience with Apache Spark and distributed data processing.
  • Strong understanding of Hadoop ecosystem and distributed computing concepts.
  • Strong programming experience with PySpark and/or Scala.
  • Experience working with Amazon S3 and AWS‑based data lakes.
  • Experience troubleshooting and optimizing Spark/EMR workloads.
  • Strong understanding of ETL/ELT concepts and large‑scale data processing.
  • Experience with production data pipelines and performance optimization.
Good to Have:
  • AWS Glue
  • Apache Airflow / MWAA
  • AWS Step Functions
  • AWS Lambda
  • Amazon Redshift
  • Experience with Spark performance tuning and cluster optimization.
  • Experience with CI/CD and deployment of data‑processing applications.
  • AWS Certified Data Engineer or another relevant AWS certification.
Interview Process
  • Application review
  • 5–10 minute initial screening call with the TA team
  • Technical interviews {Domain specific}
  • Practical test conducted in the presence of a panel member
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

AWS Data Engineer (Kafka & PySpark)
AWS Data Engineer (Kafka & PySpark)

Tata Consultancy Services Limited • Kolkata District, Chennai, Bengaluru

Presencial
INR 1.200.000 - 2.400.000
Aws Data Engineer
Aws Data Engineer

RITWIK IT SERVICES PRIVATE LIMITED • Bengaluru

Híbrido
INR 1.500.000 - 2.500.000
AWS Data Engineer
AWS Data Engineer

Softview Infotech • Chennai District

Presencial
INR 1.200.000 - 2.400.000
AWS Pyspark Developer
AWS Pyspark Developer

Tata Consultancy Services Limited • Indore District

Presencial
INR 1.200.000 - 2.400.000
Big Data Applications Engineer_AWS
Big Data Applications Engineer_AWS

Zorba AI • Hyderabad

Presencial
INR 1.800.000 - 2.600.000
Senior Data Engineer
Senior Data Engineer

RITWIK IT SERVICES PRIVATE LIMITED • Bengaluru

Híbrido
INR 4.500.000 - 6.500.000
AWS Data Engineer
AWS Data Engineer

VMC Soft Technologies • Chennai District

Presencial
INR 1.080.000 - 1.320.000
Sr. Data Engineer
Sr. Data Engineer

Minfy Technologies • Gurugram District

Presencial
INR 1.800.000 - 2.400.000
Aws Data Engineer
Aws Data Engineer

Qentelli LLC • Bengaluru

Presencial
INR 1.400.000 - 2.200.000
AWS Big Data Engineer
AWS Big Data Engineer

Zorba AI • Hyderabad

Presencial
INR 1.500.000 - 2.500.000