MODULE LEAD - AWS

Happiest Minds Technologies; Maharashtra; On-site; INR 1,100,000 - 1,700,000; Full time

Job summary

Happiest Minds Technologies is seeking a Data Engineer to design and build scalable data pipelines on AWS. You will contribute to the full lifecycle of the data engineering platform and collaborate with product managers and data science teams.

Key requirements include strong Python and SQL skills, PySpark optimization, and hands-on experience with AWS EMR, Redshift, Databricks and Airflow. Real-time data processing and cloud-based data architectures are valued.

Qualifications

• Strong Python and SQL skills with Spark/PySpark experience.
• Experience with real-time data processing on AWS cloud.
• Hands-on with AWS EMR, Redshift, Databricks, and Airflow.

Responsibilities

• Design and implement scalable data engineering solutions on the Cloud.
• Translate business needs into data engineering solutions.
• Build large-scale data pipelines for big data sets using distributed processing techniques.
• Collaborate with Product managers, Architects, Data Quality engineers, and data science teams.
• Contribute to defining data governance across the data platform.

Skills

Python

SQL

PySpark

Spark optimization

Data pipelines

Cloud experience

Tools

AWS

AWS EMR

Redshift

Databricks

Airflow

Job description

Data Engineer

VERY Strong IN Python, and SQL, Spark / Pyspark (Must know PySpark OptimiZation)

Real time cloud experience - AWS

Must have experience in AWS EMR , Redshift, Databricks , airflow.

Roles & Responsibilities

We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to -

• Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment
• Understand and translate business needs into data engineering solutions
• Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams
• Partner with cross-functional stakeholder including Product managers, Architects, Data Quality engineers, Application and Quantitative Science end users to deliver engineering solutions
• Contribute to defining data governance across the data platform