Senior Databricks Data Engineer: Azure & Delta Lake

EXL; United States; On-site; USD 63,000 - 86,500; Full time

Job summary

EXL Service is seeking a Senior Databricks Data Engineer with 10–12 years of experience to design, develop, and optimize enterprise data platforms and large-scale ETL solutions. The role emphasizes Databricks, Spark, Delta Lake, and Azure Data Platform to deliver analytics-ready datasets for reporting, AI/ML, and analytics initiatives.

The candidate will build scalable data pipelines, real-time architectures, and reusable data integration frameworks, while ensuring data quality, security, and

Qualifications

• 10-12 years of data engineering experience with enterprise-scale ETL and data platforms.
• Deep expertise in Databricks, Spark, Delta Lake, and Azure Data Platform.
• Proficiency with PySpark, Spark SQL, Python, and SQL.
• Experience building batch, real-time, and CDC data pipelines in cloud environments.
• Certifications in Azure Data Engineer or Databricks preferred.

Responsibilities

• Design, develop, and optimize large-scale data pipelines using Databricks, Spark, and Delta Lake.
• Implement cloud-native data solutions and improve platform performance.
• Build scalable ingestion frameworks for diverse data sources and formats.
• Develop data quality, lineage, monitoring, and error handling.
• Develop Databricks Workflows, SQL Jobs, and automation for production workloads.
• Integrate REST APIs and process JSON/XML data for analytics-ready datasets.
• Collaborate across teams to ensure reliable data products for analytics, reporting, and AI/ML.

Skills

Databricks

Spark

Delta Lake

Azure Data Platform

PySpark

Spark SQL

Python

Azure DevOps

Git

Ab Initio

Tools

Azure DevOps

Ab Initio

Job description

EXL Service is seeking a Senior Databricks Data Engineer with 10–12 years of experience to design, develop, and optimize enterprise data platforms and large-scale ETL solutions. The role emphasizes Databricks, Spark, Delta Lake, and Azure Data Platform to deliver analytics-ready datasets for reporting, AI/ML, and analytics initiatives.

The candidate will build scalable data pipelines, real-time architectures, and reusable data integration frameworks, while ensuring data quality, security, and