Data Engineer

EXL; Gurugram District; On-site; INR 2,800,000 - 4,000,000; Full time

Job summary

EXL is seeking a Data Engineer in India to design, develop, and optimize scalable data pipelines using Databricks and Azure. You will ensure data quality, implement validation rules, and maintain accuracy across large datasets.

You will own monitoring and reliability of data platforms, drive governance, collaborate with architects, analysts, and product owners in Agile environments, and mentor junior engineers to raise the team's technical bar.

Qualifications

• 6–10 years of data engineering experience with Databricks and Azure.
• Bachelor's degree in CS, IT, or related field.
• Strong PySpark, Python, and SQL skills.
• Experience with ETL/ELT, data modeling, and automation.
• Experience with Delta Lake and data warehousing.
• Strong analytical and problem-solving abilities.

Responsibilities

• Design, develop and optimize scalable data pipelines.
• Enforce data quality frameworks and validation.
• Monitor data platforms with observability and SLAs.
• Govern data pipelines with audits and governance.
• Collaborate with architects, analysts and product owners in Agile.
• Mentor junior engineers and promote best practices.

Skills

Databricks

Azure Cloud

PySpark

Python

SQL

Data Modeling

ETL/ELT

Delta Lake

Delta Live Tables

Automation

Problem-solving

Education

Bachelor's degree in Computer Science, IT, or related field

Tools

Delta Lake

Delta Live Tables

Autoloader

Unity Catalog

Job description

We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for development and optimization of data pipelines, implementing robust data checks, and ensuring the accuracy and integrity of data flows. This role is critical in supporting data-driven decision-making processes, especially in the context of our insurance-focused business operations.

Key Responsibilities:

• Lead the design, development, and optimization of scalable, high-performance data pipelines, ensuring reliable data ingestion and transformation across distributed systems, while proactively identifying and resolving performance bottlenecks and system inefficiencies.
• Define and enforce enterprise-grade data quality frameworks, implementing advanced validation rules, anomaly detection, and reconciliation processes to ensure data accuracy, completeness, and integrity across large-scale datasets.
• Own end-to-end monitoring and reliability of data platforms, establishing observability practices (alerts, SLAs, dashboards) and driving root-cause analysis to minimize downtime and ensure seamless continuity of critical data workflows.
• Govern and continuously improve data pipelines through periodic audits and reviews, ensuring adherence to internal standards, data governance policies, and regulatory compliance, while driving standardization and best practices across teams.
• Collaborate with cross-functional stakeholders (architects, analysts, product owners) to align data solutions with business objectives, and actively contribute to backlog prioritization, sprint planning, and technical problem-solving in Agile environments.
• Mentor junior engineers and drive technical excellence, promoting reusable frameworks, coding standards, and best practices to improve overall team productivity and solution quality.

Candidate Profile:

• 6-10 years' experience on Data Engineering role working with Databricks & Azure Cloud technologies.
• Bachelor's degree in computer science, Information Technology, or related field.
• Strong proficiency in PySpark, Python, SQL.
• Strong experience in data modeling, ETL/ELT pipeline development, and automation
• Hands-on experience with performance tuning of data pipelines and workflows
• Experience with data modeling, ETL processes, Delta Lake and data warehousing.
• Experience on Delta Live Tables, Autoloader & Unity Catalog.
• Preferred - Knowledge of the insurance industry and its data requirements.
• Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
• Excellent communication and problem-solving skills to work effectively with diverse teams
• Excellent problem-solving skills and ability to work under tight deadlines.