Databricks Data Engineer

EXL; Pune District, Gurugram District, Bengaluru; Hybrid; INR 1,100,000 - 1,700,000; Full time

Job summary

EXL in Pune is seeking a Data Engineer with 4–9 years of experience to design, build, and optimize scalable data pipelines across multiple environments. The role emphasizes PySpark, Python, SQL, and hands-on work with Azure cloud data services to ensure data quality and efficient processing.

You will contribute to data modeling, ETL/ELT implementations, and automation, while collaborating in Agile teams to deliver reliable data solutions within a dynamic, fast-paced setting.

Qualifications

• 4–9 years of Data Engineering experience with Databricks and Azure Cloud.
• Strong proficiency in PySpark, Python, and SQL.
• Experience with ETL/ELT pipelines and data warehouse concepts.

Responsibilities

• Develop and maintain scalable data pipelines for diverse systems.
• Implement data quality checks and validate large datasets.
• Monitor data jobs, troubleshoot issues, and minimize downtime.
• Audit data processes for compliance with internal standards.
• Work within Agile teams on sprints and backlog refinement.

Skills

PySpark

Python

SQL

Data modeling

ETL/ELT

Automation

Agile/Scrum

Education

Bachelor's degree in Computer Science / IT

Tools

Databricks

Azure Data Factory

Azure DataBricks

Azure Data Lake

Delta Lake

Job description

Role & responsibilities

• Develop and maintain scalable and efficient data pipelines to ensure seamless data flow across various systems adddress any issues or bottlenecks in existing pipelines.
• Implement robust data checks to ensure the accuracy and integrity of data. Summarize and validate large datasets to ensure they meet quality standards.
• Monitor data jobs for successful completion. Troubleshoot and resolve any issues that arise to minimize downtime and ensure continuity of data processes.
• Regularly review and audit data processes and pipelines to ensure compliance with internal standards and regulatory requirements.
• Familiar with working on Agile methodologies - scrum, sprint planning, backlog refinement etc.

Candidate Profile

• 4-9 years experience on Data Engineering role working with Databricks & Azure Cloud technologies.
• Bachelors degree in computer science, Information Technology, or related field.
• Strong proficiency in PySpark, Python, SQL.
• Strong experience in data modeling, ETL/ELT pipeline development, and automation.
• Hands-on experience with performance tuning of data pipelines and workflows.
• Proficient in working on Azure cloud components Azure Data Factory, Azure DataBricks, Azure Data Lake etc.
• Experience with data modeling, ETL processes, Delta Lake and data warehousing.