Data Engineer

Expleo; Chennai; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Expleo is seeking a Data Engineer to design, build, and maintain scalable data pipelines and platforms in Chennai. You will work closely with data scientists, analysts, and business stakeholders to ensure reliable, high‑quality data across our ecosystem.

The role requires 4–6 years of hands‑on data engineering experience with PySpark, SQL, and Databricks, plus experience with AWS/Azure cloud platforms. Collaboration with cross‑functional teams is key to success.

Qualifications

• 4–6 years of hands‑on data engineering experience.
• Strong proficiency in PySpark for large‑scale data processing.
• Advanced SQL skills including window functions, CTEs, and complex joins.
• Solid Databricks experience including Delta Lake, workflows, and Unity Catalog.
• Experience with cloud platforms (AWS/Azure).

Responsibilities

• Design and develop robust ETL/ELT pipelines using PySpark and SQL for large‑scale data processing.
• Build, manage, and optimise data workflows on Databricks, including notebooks, jobs, and cluster configurations.
• Collaborate with analytics and ML teams to deliver clean, well‑documented datasets.
• Monitor pipeline performance, troubleshoot failures, and ensure data quality and integrity.
• Define and implement data governance and best practices across the data platform.
• Participate in design reviews and contribute to architectural decisions.

Skills

PySpark

SQL

Databricks

AWS/Azure

Python

Git

Tools

Databricks

AWS

Azure

Python

Git

Job description

Overview

Data Engineer

4–6 years experience / Chennai

We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines and platforms. You will work closely with data scientists, analysts, and business stakeholders to ensure reliable, high-quality data across our ecosystem.

Responsibilities

Responsibilities

• Design and develop robust ETL/ELT pipelines using PySpark and SQL for large-scale data processing.
• Build, manage, and optimise data workflows on Databricks, including notebooks, jobs, and cluster configurations.
• Collaborate with analytics and ML teams to deliver clean, well-documented datasets.
• Monitor pipeline performance, troubleshoot failures, and ensure data quality and integrity.
• Define and implement data governance and best practices across the data platform.
• Participate in design reviews and contribute to architectural decisions.

Requirements

• 4–6 years of hands‑on experience in data engineering or a related role.
• Strong proficiency in PySpark for large-scale distributed data processing.
• Advanced SQL skills — query optimisation, window functions, CTEs, and complex joins.
• Solid experience with Databricks — Delta Lake, workflows, Unity Catalog preferred.
• Experience working with cloud platforms (AWS/Azure).

Core skills

PySpark/SQL/Databricks/AWS / Azure /Python/Git

Nice to have

• Experience with streaming frameworks such as Apache Kafka or Spark Streaming.
• Exposure to ML pipelines and feature engineering workflows.

Experience

Requirements

• 4–6 years of hands‑on experience in data engineering or a related role.
• Strong proficiency in PySpark for large-scale distributed data processing.
• Advanced SQL skills — query optimisation, window functions, CTEs, and complex joins.
• Solid experience with Databricks — Delta Lake, workflows, Unity Catalog preferred.
• Experience working with cloud platforms (AWS/Azure).