Technical Lead Azure Databricks_3417

Allianz; Thiruvananthapuram; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Allianz Group is seeking a senior platform engineer to build and operate data platforms powering thousands of colleagues. You will work hands-on with Databricks, Azure Synapse, and Microsoft Fabric, delivering production-grade pipelines and governance across the data stack.

This role blends engineering, data, and operations, requiring strong communication in English, experience with CI/CD and IaC, and the ability to mentor engineers while partnering with Operating Entities.

Qualifications

• Bachelor/Master in CS/IS/Engineering or equivalent.
• 5+ years building data/platforms in enterprise or regulated environments.
• Strong Databricks experience: Delta Lake, Unity Catalogue, pipelines.
• End-to-end data integration: ETL/ELT, Azure Data Factory, Airflow; batch/streaming.
• Proficient in Python and SQL; production-grade code.
• Experience with CI/CD, source control, and IaC (Terraform, Bicep).
• Knowledge of security, governance, and data lineage.
• Excellent English communication.

Responsibilities

• Consult with Operating Entities on core Databricks deployments.
• Build data integration pipelines connecting source systems to central platforms.
• Configure high availability, access controls, and observability.
• Extend the platform to AI/GenAI scenarios using Azure AI Foundry.
• Translate business needs into platform implementations and reusable solutions.
• Implement governance, security, and data controls for regulated data.
• Develop CI/CD and infrastructure-as-code pipelines for fast, secure deployments.
• Collaborate with security, risk, compliance, architecture, and governance teams.
• Deliver reusable components and reference architectures for rollout.
• Support OE rollout and build business-specific use cases atop the platform.
• Mentor engineers and drive tooling standards and improvements.

Skills

Databricks

Delta Lake

Unity Catalogue

ETL/ELT design

Azure Data Factory

Airflow

Python

SQL

CI/CD pipelines

Infrastructure-as-code

Terraform

Bicep

Security governance

Data governance

Data quality

English communication

Kubernetes

Docker

Azure Synapse

Microsoft Fabric

Azure Monitor

Databricks Certified Data Engineer

Microsoft Azure Data Engineer

Education

Bachelor's or Master’s in CS/IS/Engineering

Tools

Databricks

Azure Synapse

Microsoft Fabric

Airflow

Terraform

Bicep

MLflow

Kubernetes

Docker

Job description

Role Description & Responsibilities

Are you energised by building and implementing the data platforms that thousands of colleagues depend on at one of the world's largest financial services groups?

If so, this role puts you at the centre of that work. You will implement and engineer the platforms behind the Allianz Data Platform, working hands‑on with Databricks, Azure Synapse, and Microsoft Fabric, and turning platform capabilities into services that Operating Entities can adopt with confidence.

The role sits at the crossroads of engineering, data, and operations, which means the pipelines, integrations, and controls you build shape how thousands of colleagues across Allianz Group turn data into decisions.

To get there, you will work forward with the Operating Entities themselves, assisting in the deployment and configuration of platform components, integration of source systems, and extending our offering into so that analytics and AI use cases run on solid, governed foundations.

What we value most is your ability to configure production‑grade solutions, reason clearly about architecture and trade‑offs, work through security, compliance, and data governance requirements, and take ownership of first deployment and configuration through to stable operation.

As part of this role you will:

• Consult with OEs in the implementation and configuration of core platform components across Databricks from initial deployment through to stable production.
• Assist in building data integration pipelines that connect OE source systems to central platforms, using both batch and streaming patterns.
• Configure platform capabilities with high availability, strong access controls, and end-to-end observability.
• Extend the platform offering into AI and GenAI scenarios, including Azure AI Foundry, so that model‑based use cases run on governed foundations.
• Partner directly with Operating Entities to translate business needs into working platform implementations and reusable solutions.
• Implement governance, security, and data control layers in line with financial services expectations for data handling, auditability, and operational resilience.
• Build and refine CI/CD and infrastructure‑as‑code pipelines to keep deployment cycles fast, secure, and repeatable.
• Collaborate closely with security, risk, compliance, architecture, and data governance teams to clear the path for platform adoption.
• Deliver reusable components, reference architectures, and engineering best practices that accelerate platform rollout across the group.
• Support the rollout of central platform solutions to the OEs, and build business‑specific use cases on top of them.
• Coach and mentor engineers across the organisation, and drive continuous improvement in tooling, standards, and ways of working.

Basic Qualifications

• Bachelor/Master degree in Computer Science, Information Systems, Engineering, or an equivalent technical discipline.
• 5+ years of hands‑on experience implementing or engineering data and analytics platforms in enterprise environments, ideally within financial services, insurance, or another regulated industry.
• Significant experience with Databricks, including Delta Lake, Unity Catalogue, workflows, and production pipelines.
• Strong data integration skills, covering ETL and ELT design, orchestration (e.g., Azure Data Factory, Airflow), and both batch and streaming sources.
• Solid engineering skills in Python and SQL, with experience building and maintaining production‑grade code.
• Experience with CI/CD pipelines, source control management, and infrastructure‑as‑code (e.g., Terraform, Bicep).
• Working knowledge of platform security and governance concepts, including access controls, encryption, and data lineage.
• Familiarity with end‑to‑end (E2E) Data Product Management, including data governance, quality, and lifecycle management.
• Excellent communication and stakeholder management skills in English (additional languages are a plus).

Preferred Qualifications

• 7+ years of experience in platform engineering, data engineering, or cloud infrastructure.
• Exposure to Azure AI Foundry or comparable AI platform tooling, including the deployment of model‑based services.
• Familiarity with MLOps practices and tools such as MLflow, feature stores, and model lifecycle governance.
• Experience with Kubernetes, Docker, and cloud‑native architectures.
• Hands‑on familiarity with the wider modern data stack (e.g., Spark, dbt, Kafka, Snowflake, or similar).
• Practical experience with Azure data services such as Azure Synapse Analytics and Microsoft Fabric.
• Experience with observability and monitoring tools such as Azure Monitor, Datadog, or Prometheus.
• Relevant certifications such as Databricks Certified Data Engineer, Microsoft Azure Data Engineer, or equivalent.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.

At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

Join us. Let's care for tomorrow.