Data Engineer

Careernet; Chennai District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Careernet is seeking a senior Data Engineer in Chennai to develop and optimize data pipelines, leveraging PySpark, Databricks, and AWS cloud services. You will build scalable batch and streaming jobs, write efficient SQL, and collaborate with analytics teams to deliver clean, well-modeled data for downstream consumption.

The role requires 7–10 years of data engineering experience, strong Python and SQL skills, and hands-on Databricks/AWS expertise. Bachelor's degree in CS/Engineering is required.

Qualifications

• 7 to 10 years of data engineering experience.
• Strong hands-on experience with Python, PySpark, Databricks, and MySQL.
• Experience designing and supporting data pipelines and cloud data platforms.
• Ability to work with large datasets and optimize SQL and Spark-based processing.
• Experience with AWS-based data environments and production-grade data delivery.

Responsibilities

• Develop and maintain data pipelines for ingestion, transformation, and delivery across enterprise data platforms.
• Build scalable Spark-based processing jobs using Python and PySpark in Databricks.
• Write and optimize SQL queries for data extraction, transformation, and analytical use cases.
• Support cloud-based data workflows on AWS services used for storage, processing, and orchestration.
• Partner with engineering and analytics teams to deliver well-modeled datasets for downstream consumption.

Skills

Pyspark

Python

Data modelling

Data warehousing

SQL

Education

Bachelor's degree in Computer Science, Engineering, or a related field

Tools

Databricks

MySQL

AWS Cloud

Git

Job description

Key Skills:

Pyspark, MySQL, Databricks, Python, Data Warehouse, AWS Cloud, Git, Data Modelling

Roles and Responsibilities:

• Develop and maintain data pipelines for ingestion, transformation, and delivery across enterprise data platforms.
• Build scalable Spark-based processing jobs using Python and PySpark in Databricks.
• Write and optimize SQL queries for data extraction, transformation, and analytical use cases.
• Support cloud-based data workflows on AWS services used for storage, processing, and orchestration.
• Partner with engineering and analytics teams to deliver well-modeled datasets for downstream consumption.

Skills Required:

• 7 to 10 years of relevant data engineering experience.
• Strong hands-on experience with Python, PySpark, Databricks, and MySQL.
• Experience designing and supporting data pipelines and cloud data platforms.
• Ability to work with large datasets and optimize SQL and Spark-based processing.
• Experience with AWS-based data environments and production-grade data delivery.

Good to Have:

• Data Modelling
• Data Warehouse
• AWS Cloud
• Git

Education:

Bachelor's degree in Computer Science, Engineering, or a related field.