MODULE LEAD - AWS

Happiest Minds Technologies; Pune District; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Happiest Minds Technologies in Pune is seeking a Data Engineer to design and implement scalable data pipelines, optimize PySpark workloads, and enable real-time cloud data processing on AWS.

You will collaborate with data science, product, and governance teams to deliver secure, reliable data platforms and maintain end-to-end data lifecycle.

Qualifications

• Strong experience in Python and SQL with PySpark optimization.
• Experience with cloud-based data processing in real-time environments.
• Hands-on knowledge of distributed data processing and data pipelines.

Responsibilities

• Design and implement scalable data engineering solutions on the Cloud.
• Translate business needs into robust data pipelines for large data sets.
• Build and维护 end-to-end data pipelines using Spark, PySpark, and Airflow.
• Collaborate with Product Managers, Architects, and Data teams to deliver engineering solutions.
• Contribute to data governance and platform reliability.

Skills

Python

SQL

Spark

PySpark

Real-time cloud experience

Tools

AWS EMR

Redshift

Databricks

Airflow

Job description

Data Engineer

VERY Strong IN Python, and SQL, Spark / Pyspark (Must know PySpark OptimiZation)

Real time cloud experience - AWS

Must have experience in AWS EMR , Redshift, Databricks , airflow.

Roles & Responsibilities

We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to -

• Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment
• Understand and translate business needs into data engineering solutions
• Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams
• Partner with cross-functional stakeholder including Product managers, Architects, Data Quality engineers, Application and Quantitative Science end users to deliver engineering solutions
• Contribute to defining data governance across the data platform