Technical Lead Azure Databricks_3417

Allianz Services; Thiruvananthapuram; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Allianz Services seeks a senior data platform engineer to build and operate production-grade data platforms for Allianz Group. You will design, implement and run platforms behind the Allianz Data Platform, working hands-on with Databricks, Delta Lake, Unity Catalogue, workflows, and production pipelines.

You will partner with operating entities to deploy core components, integrate source systems, and extend offerings to AI/GenAI scenarios with secure governance.

Qualifications

• Bachelor/Master degree in Computer Science, Information Systems, Engineering, or equivalent.
• 5+ years hands-on experience building data and analytics platforms in enterprise environments.
• Significant Databricks experience: Delta Lake, Unity Catalogue, workflows, pipelines.
• Strong data integration skills: ETL/ELT, orchestration (Airflow, ADF).
• Python and SQL production-grade coding skills.
• Experience with CI/CD, source control, and infrastructure-as-code (Terraform, Bicep).
• Knowledge of security, governance, data lineage, and auditability.
• Excellent English communication and stakeholder management.

Responsibilities

• Consult with Operating Entities to implement core platform components across Databricks from initial deployment to production.
• Assist in building data integration pipelines connecting source systems to central platforms (batch and streaming).
• Configure platform capabilities for high availability, access controls, and observability.
• Extend platform into AI/GenAI scenarios, including Azure AI Foundry with governed foundations.
• Translate business needs into platform implementations and reusable solutions.
• Implement governance, security, and data control layers per financial services requirements.
• Develop CI/CD and infrastructure-as-code pipelines for fast, secure deployments.
• Collaborate with security, risk, compliance, architecture, and data governance teams.
• Deliver reusable components, reference architectures, and best practices for rollout across Allianz.
• Mentor engineers and drive tooling improvements and standards.

Skills

Python

SQL

CI/CD

Data governance

Stakeholder management

Education

Bachelor/Master in Computer Science or equivalent

Tools

Databricks

Delta Lake

Unity Catalogue

Azure Data Factory

Airflow

Terraform

Bicep

Azure Synapse

Microsoft Fabric

Kubernetes

Docker

MLflow

Job description

Job Description:

Role Description & Responsibilities

Are you energised by building and implementing the data platforms that thousands of colleagues depend on at one of the worlds largest financial services groups?

If so, this role puts you at the centre of that work. You will implement and engineer the platforms behind the Allianz Data Platform, working hands-on with Databricks, Azure Synapse, and Microsoft Fabric, and turning platform capabilities into services that Operating Entities can adopt with confidence.

The role sits at the crossroads of engineering, data, and operations, which means the pipelines, integrations, and controls you build shape how thousands of colleagues across Allianz Group turn data into decisions.

To get there, you will work forward with the Operating Entities themselves, assisting in the deployment and configuration of platform components, integration of source systems, and extending our offering into so that analytics and AI use cases run on solid, governed foundations.

What we value most is your ability to configure production-grade solutions, reason clearly about architecture and trade-offs, work through security, compliance, and data governance requirements, and take ownership of first deployment and configuration through to stable operation.

As part of this role you will:

• Consult with OEs in the implementation and configuration of core platform components across Databricks from initial deployment through to stable production.
• Assist in building data integration pipelines that connect OE source systems to central platforms, using both batch and streaming patterns.
• Configure platform capabilities with high availability, strong access controls, and end-to-end observability.
• Extend the platform offering into AI and GenAI scenarios, including Azure AI Foundry, so that model-based use cases run on governed foundations.
• Partner directly with Operating Entities to translate business needs into working platform implementations and reusable solutions.
• Implement governance, security, and data control layers in line with financial services expectations for data handling, auditability, and operational resilience.
• Build and refine CI/CD and infrastructure-as-code pipelines to keep deployment cycles fast, secure, and repeatable.
• Collaborate closely with security, risk, compliance, architecture, and data governance teams to clear the path for platform adoption.
• Deliver reusable components, reference architectures, and engineering best practices that accelerate platform rollout across the group.
• Support the rollout of central platform solutions to the OEs, and build business-specific use cases on top of them.
• Coach and mentor engineers across the organisation, and drive continuous improvement in tooling, standards, and ways of working.

Basic Qualifications.

• Bachelor/Master degree in Computer Science, Information Systems, Engineering, or an equivalent technical discipline.
• 5+ years of hands-on experience implementing or engineering data and analytics platforms in enterprise environments, ideally within financial services, insurance, or another regulated industry.
• Significant experience with Databricks, including Delta Lake, Unity Catalogue, workflows, and production pipelines.
• Strong data integration skills, covering ETL and ELT design, orchestration (e.g., Azure Data Factory, Airflow), and both batch and streaming sources.
• Solid engineering skills in Python and SQL, with experience building and maintaining production-grade code.
• Experience with CI/CD pipelines, source control management, and infrastructure-as-code (e.g., Terraform, Bicep).
• Working knowledge of platform security and governance concepts, including access controls, encryption, and data lineage.
• Familiarity with end-to-end (E2E) Data Product Management, including data governance, quality, and lifecycle management.
• Excellent communication and stakeholder management skills in English (additional languages are a plus).

Preferred Qualifications

• 7+ years of experience in platform engineering, data engineering, or cloud infrastructure.
• Exposure to Azure AI Foundry or comparable AI platform tooling, including the deployment of model-based services.
• Familiarity with MLOps practices and tools such as MLflow, feature stores, and model lifecycle governance.
• Experience with Kubernetes, Docker, and cloud-native architectures.
• Hands-on familiarity with the wider modern data stack (e.g., Spark, dbt, Kafka, Snowflake, or similar).
• Practical experience with Azure data services such as Azure Synapse Analytics and Microsoft Fabric.
• Experience with observability and monitoring tools such as Azure Monitor, Datadog, or Prometheus.
• Relevant certifications such as Databricks Certified Data Engineer, Microsoft Azure Data Engineer, or equivalent.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.

At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

Join us. Lets care for tomorrow.

Requirements: