Area Manager-Data Engineering-Cloud Data Engineering

EXL Talent Acquisition Team; Dadri; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

EXL Talent Acquisition Team in India seeks an experienced Azure Data Engineer to design, develop, test, and deploy robust data pipelines using Azure Databricks, PySpark, and Spark SQL for large volumes of data.

You will implement workflow orchestration with Azure Data Factory and Databricks Workflows, build and maintain CI/CD pipelines, model healthcare data, and design enterprise BI architectures with data warehouses and marts, delivering insightful dashboards.

Qualifications

• 6+ years of experience as a Data Engineer with a proven track record of building data pipelines in the cloud.
• 3+ years hands-on experience with Azure Databricks.
• Strong experience with core Azure data services - Azure Data Lake Storage (Gen 2), Azure data factory.
• Proficiency in Python and expert-level skills in SQL.
• Solid understanding of data warehousing concepts, dimensional modeling (star/snowflake schemas) and data architecture principles.
• Experience with CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) and version control systems (Git).
• Demonstrated experience and knowledge of strategic problem solving and frameworks, and project management skills
• Excellent written and verbal communication with the ability to establish credibility and strong relationships with stakeholders

Responsibilities

• Design, develop, test, and deploy scalable data pipelines using Azure Databricks and Spark SQL.
• Implement workflow orchestration using Azure Data Factory and Databricks Workflows to automate ETL/ELT processes.
• Maintain CI/CD pipelines for Databricks notebooks, scripts, and configurations using Azure DevOps or GitHub Actions.
• Develop and maintain healthcare data models, including data entities, relationships, and data flows, to support healthcare analytics and decision-making.
• Design and implement enterprise-wide BI architectures. Develop and maintain data warehouses and data marts.
• Develop and maintain reports, dashboards, and data visualizations. Collaborate with stakeholders to understand business needs and provide data-driven insights

Skills

Azure Databricks

Python

SQL

Azure Data Factory

Azure Data Lake Storage

CI/CD

GitHub Actions

Jenkins

Git

Spark SQL

Databricks

Dimensional Modeling

Tools

Azure DevOps

GitHub

Jenkins

Databricks

Job description

Role Title: Azure Data Engineer

BU/Segment:Healthcare Analytics

Responsibilities

• Design, develop, test, and deploy robust and scalable data pipelines using Azure Databricks, PySpark, and Spark SQL for processing large volumes of structured and unstructured data.
• Implement workflow orchestration using Azure Data Factory and Databricks Workflows to automate ETL/ELT processes.
• Implement and maintain CI/CD pipelines for Databricks notebooks, scripts, and configurations using Azure DevOps or GitHub Actions, promoting code across dev, test, and prod environments.
• Develop and maintain healthcare data models, including data entities, relationships, and data flows, to support healthcare analytics and decision-making.
• Design and implement enterprise-wide BI architectures. Develop and maintain data warehouses and data marts.
• Develop and maintain reports, dashboards, and data visualizations. Collaborate with stakeholders to understand business needs and provide data-driven insights

Person attributes

• 6+ years of experience as a Data Engineer with a proven track record of building data pipelines in the cloud.
• 3+ years of hands-on experience withAzure Databricks
• Strong experience with core Azure data services - Azure Data Lake Storage (Gen 2), Azure data factory.
• Proficiency inPythonand expert-level skills inSQL
• Solid understanding of data warehousing concepts, dimensional modeling (star/snowflake schemas), and data architecture principles
• Experience with CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) and version control systems (Git).
• Demonstrated experience and knowledge of strategic problem solving and frameworks, and project management skills
• Excellent written and verbal communication with the ability to establish credibility and strong relationships with stakeholders