Azure Data Engineer

Metlife; Hyderabad; Hybrid; INR 1,500,000 - 2,100,000; Full time

Job summary

MetLife is seeking an Azure Data Engineer to design, build, and support enterprise-scale cloud data platforms. The role requires hands-on expertise in Azure, Databricks, Spark, Lakehouse architecture, and modern data pipelines.

You will collaborate with engineers, business stakeholders, and DevOps teams in an Agile environment to deliver secure, analytics-ready data solutions, implement CI/CD, data governance, RBAC, and cost optimization.

Qualifications

• Expert-level experience with Azure Databricks, Synapse, Data Factory, PySpark, Spark SQL, and ADLS Gen2.
• Strong knowledge of Delta Lake, Lakehouse architecture, Unity Catalog, data governance, and lineage.
• Advanced SQL across Oracle, SQL Server, PostgreSQL, and cloud platforms.
• Hands-on experience with CI/CD, Git, GitHub, and Azure DevOps.

Responsibilities

• Architect and deliver scalable data lakehouse solutions using Azure services.
• Build curated data products, data marts, and analytics-ready datasets for analysts and reporting teams.
• Develop batch and near-real-time pipelines with Databricks, Spark, PySpark, SQL, and Synapse.
• Implement ingestion frameworks with data quality, lineage, validation, observability, and reusable patterns.
• Apply data modeling, partitioning, schema evolution, and cost optimization.
• Implement CI/CD for data workloads with automation and governance.
• Use Entra ID, RBAC, Key Vault, Managed Identity, and governance controls to protect data platforms.
• Leverage Copilot and AI workflows to accelerate development.

Skills

Azure Databricks

Azure Synapse Analytics

Azure Data Factory

PySpark

Spark SQL

ADLS Gen2

Delta Lake

Lakehouse architecture

Unity Catalog

Medallion architecture

CDC

SCD

Data quality frameworks

Data lineage

Advanced SQL

CI/CD

Git/GitHub/Azure DevOps

Tools

Git

GitHub

Azure DevOps

CI/CD

Secure coding

Monitoring & dashboards

Job description

Azure Data Engineer

An Azure Data Engineer will design, build, and support enterprise-scale cloud data platforms. This role requires hands-on expertise in Azure, Databricks, Spark, Lakehouse architecture, and modern data pipelines, along with the ability to collaborate with engineers, business stakeholders, and DevOps teams to deliver secure, reliable, analytics-ready data solutions in an Agile environment.

Key Responsibilities

• Architect and deliver scalable data lakehouse solutions using Azure Data Lake Storage, Azure Databricks, Delta Lake, Unity Catalog, and Medallion architecture.
• Build curated data products, data marts, and analytics-ready datasets for analysts, data scientists, and enterprise reporting teams.
• Develop reliable batch and near-real-time pipelines using Databricks, Apache Spark, PySpark, SQL, Azure Data Factory, Synapse, and related Azure services.
• Implement ingestion frameworks for structured, semi-structured, and unstructured data while ensuring data quality, lineage, validation, observability, and reusable patterns.
• Apply data modeling, partitioning, schema evolution, performance tuning, cost optimization, error handling, and operational readiness practices.
• Implement CI/CD for data workloads using Azure DevOps or GitHub Actions, including automated testing, deployment, release governance, and secure engineering practices.
• Use Entra ID, RBAC, secrets management, Key Vault, Managed Identity, and governance controls to protect data platforms.
• Leverage GitHub Copilot, Microsoft Copilot, Databricks Assistant, and AI-powered workflows to accelerate SQL/PySpark development, ETL delivery, test data generation, documentation, troubleshooting, and productivity.

Candidate Qualifications

• Expert-level experience with Azure Databricks, Azure Synapse Analytics, Azure Data Factory, PySpark, Spark SQL, and Azure Data Lake Storage Gen2.
• Strong knowledge of Delta Lake, Lakehouse architecture, Unity Catalog, data governance, Medallion architecture, CDC, SCD, data quality frameworks, reconciliation, and lineage.
• Advanced SQL expertise across Oracle, SQL Server, PostgreSQL, and modern cloud data platforms.
• Hands-on experience with Informatica PowerCenter is preferred, especially for migration programs.
• Experience with Git, GitHub, Azure DevOps, CI/CD, automated deployment, secure coding, monitoring, alerting, and operational dashboards.
• Strong understanding of Azure security services including Key Vault, Managed Identity, Entra ID, and role-based access controls.

Tech Stack

• Data Engineering: Azure Databricks, Azure Data Factory, Azure Synapse, ADLS Gen2, Python, PySpark, SQL, Spark SQL, ETL/ELT, orchestration, metadata management, and data modeling.
• Lakehouse: Delta Lake, Bronze/Silver/Gold layers, schema evolution, SCD, reusable ingestion frameworks, and performance tuning.
• DevOps & Quality: Git, GitHub, Azure DevOps, CI/CD, automated testing, release pipelines, SonarQube, secure coding, Azure Monitor, Application Insights, logs, alerts, and pipeline health checks.