Senior Data Engineer

EXL; Uttar Pradesh; On-site; INR 900,000 - 1,500,000; Full time

Job summary

EXL is seeking a Data Engineer with strong expertise in Databricks, data engineering, and healthcare data to design scalable, secure data platforms supporting healthcare analytics. You will design ingestion, transformations, Delta Lake, Unity Catalog, and pipelines using PySpark and SQL, ensuring HIPAA compliance and data governance.

You will collaborate with data scientists, analysts, and business stakeholders across Azure/AWS/GCP cloud environments to deliver high-performance solutions for

Qualifications

• 3–6 years of experience with Databricks and data engineering.
• Expertise in Lakehouse architecture and Delta Lake.
• Strong PySpark and SQL skills.
• Experience with Unity Catalog and data governance.
• Experience designing enterprise-scale data platforms.
• Knowledge of ETL/ELT and data modeling.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Experience with data integration and orchestration tools.
• Knowledge of CI/CD, Git, and DevOps practices.
• Strong understanding of data security and access control.
• Experience with Agile/Scrum methodologies.

Responsibilities

• Design and lead Databricks Lakehouse for large-scale healthcare data platforms.
• Design data ingestion, transformation, storage, and consumption patterns using Databricks.
• Develop Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables/Lakeflow.
• Integrate data from multiple healthcare sources, including clinical, claims, eligibility, provider, and patient systems.
• Design robust ETL/ELT pipelines using PySpark, SQL, and Databricks.
• Establish data quality, validation, reconciliation, lineage, and governance frameworks.
• Design secure healthcare data solutions while considering HIPAA and PHI/PII protection requirements.
• Work with cloud platforms such as Azure, AWS, or GCP to build scalable data solutions.
• Collaborate with data engineers, data scientists, analysts, product owners, and business stakeholders.
• Define technical standards, architectural patterns, and best practices for Databricks development.
• Lead performance optimization and cost-management initiatives across the Databricks platform.
• Provide technical leadership and mentoring to data engineering teams.
• Participate in Agile ceremonies and contribute to technical planning and roadmap discussions.

Skills

PySpark

SQL

Data governance

HIPAA compliance

Agile/Scrum

Data security

ETL/ELT design

Tools

Databricks

Delta Lake

Unity Catalog

Databricks Workflows

Delta Live Tables

Lakeflow

Job description

We are looking for a Data Engineer with strong expertise in Databricks, data engineering, and healthcare data. The ideal candidate will be responsible for designing scalable, secure, and high-performance data platforms supporting healthcare analytics and data integration.

The candidate should have strong knowledge of modern data architecture, Lakehouse architecture, SQL, Python, data governance, and healthcare domain workflows.

Responsibilities

• Design and lead Databricks Lakehouse for large-scale healthcare data platforms.
• Design data ingestion, transformation, storage, and consumption patterns using Databricks.
• Develop Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables/Lakeflow.
• Integrate data from multiple healthcare sources, including clinical, claims, eligibility, provider, and patient systems.
• Design robust ETL/ELT pipelines using PySpark, SQL, and Databricks.
• Establish data quality, validation, reconciliation, lineage, and governance frameworks.
• Design secure healthcare data solutions while considering HIPAA and PHI/PII protection requirements.
• Work with cloud platforms such as Azure, AWS, or GCP to build scalable data solutions.
• Collaborate with data engineers, data scientists, analysts, product owners, and business stakeholders.
• Define technical standards, architectural patterns, and best practices for Databricks development.
• Lead performance optimization and cost-management initiatives across the Databricks platform.
• Provide technical leadership and mentoring to data engineering teams.
• Participate in Agile ceremonies and contribute to technical planning and roadmap discussions.

Qualifications

• Strong hands-on experience with Databricks.
• 3-6 years of Experience in Databricks
• Expertise in Lakehouse architecture and Delta Lake.
• Strong PySpark and SQL skills.
• Experience with Unity Catalog and data governance.
• Experience designing enterprise-scale data platforms.
• Strong knowledge of ETL/ELT and data modeling.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Experience with data integration and orchestration tools.
• Knowledge of CI/CD, Git, and DevOps practices.
• Strong understanding of data security and access control.
• Experience with Agile/Scrum methodologies.