Lead AWS Data Engineer

Careernet; Hyderabad, Pune District, Chennai District; On-site; INR 1,500,000 - 2,100,000; Full time

Benefits offered by this job

Git

CI/CD

Job summary

Careernet is hiring a data engineer to design and build scalable data pipelines on AWS and Databricks. The role focuses on Python, PySpark, SQL, and data modelling to prepare warehouse-ready datasets for reporting and analytics.

Ideal candidates have 6-12 years of data engineering experience, strong AWS and Databricks skills, and proficiency in Airflow for orchestrating data workflows. Bachelor's degree required, Master's preferred.

Qualifications

• 6-12 years of relevant data engineering experience.
• Strong hands-on experience with AWS, Python, PySpark, SQL, Databricks, data modelling, and Airflow.
• Experience designing cloud-based data platforms and building scalable data processing solutions.

Responsibilities

• Design and build scalable data pipelines and transformation workflows on AWS and Databricks.
• Develop efficient Python, SQL, and PySpark solutions for batch data processing and data preparation.
• Implement robust data models and warehouse-ready datasets to support reporting and analytics use cases.
• Orchestrate and maintain scheduled data workflows using Airflow and related cloud services.
• Partner with engineering and analytics teams to deliver trusted, reusable data assets across the platform.

Skills

Pyspark

AWS

Databricks

Data Modelling

SQL

Python

Airflow

GIT

CICD

Education

Bachelor's degree in Computer Science or related field

Master's degree preferred

Tools

Git

CI/CD

Job description

Key Skills: Pyspark, AWS, Databricks, Data Modelling, SQL, Python, Airflow, GIT, CICD

Roles and Responsibilities:

• Design and build scalable data pipelines and transformation workflows on AWS and Databricks
• Develop efficient Python, SQL, and PySpark solutions for batch data processing and data preparation
• Implement robust data models and warehouse-ready datasets to support reporting and analytics use cases
• Orchestrate and maintain scheduled data workflows using Airflow and related cloud services
• Partner with engineering and analytics teams to deliver trusted, reusable data assets across the platform

Skills Required:

• 6-12 years of relevant data engineering experience
• Strong hands-on experience with AWS, Python, PySpark, SQL, Databricks, data modelling, and Airflow
• Experience designing cloud-based data platforms and building scalable data processing solutions
• Ability to write complex SQL and optimize data transformations for large datasets
• Experience delivering production-grade data pipelines and integrating data from multiple sources

Good to Have:

• Git
• CI/CD

Education: Bachelor's degree in Computer Science or related field; Master's degree preferred