Data Engineer

Happiest Minds Technologies; Pune District; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Happiest Minds Technologies in Pune is seeking a data engineer with strong Python, SQL, and Spark/PySpark expertise. You will work on real-time cloud data processing using AWS services and modern data platforms.

Responsibilities include designing scalable data pipelines, translating business needs into engineering solutions, and partnering with cross-functional teams to deliver secure data infrastructure. Prior experience with AWS EMR, Redshift, Databricks and Airflow is highly valued.

Qualifications

• Strong Python and SQL skills with Spark/PySpark optimization.
• Experience with real-time cloud data processing on AWS.
• Hands-on experience with EMR, Redshift, Databricks and Airflow.

Responsibilities

• Join Data Engineering to build and maintain data platform end-to-end.
• Design scalable, reliable, and secure data solutions in the cloud.
• Translate business needs into data engineering solutions.
• Build large-scale data pipelines for big data across teams.
• Collaborate with product managers, architects, and data quality teams.

Skills

Python

SQL

PySpark

Spark

AWS

Airflow

Databricks

Tools

AWS EMR

Redshift

Databricks

Airflow

Job description

Job Description

• VERY Strong IN Python, and SQL, Spark / Pyspark (Must know PySpark OptimiZation)
• Real time cloud experience - AWS
• Must have experience in AWS EMR , Redshift, Databricks , airflow.

Roles & Responsibilities

1. We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team, you will get an opportunity to -
2. Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment
3. Understand and translate business needs into data engineering solutions
4. Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams
5. Partner with cross-functional stakeholder including Product managers, Architects, Data Quality engineers, Application and Quantitative Science end users to deliver engineering solutions
6. Contribute to defining data governance across the data platform