Technical Lead Azure Databricks_3417

Allianz Technology; Maharashtra; On-site; INR 3,500,000 - 6,500,000; Full time

Job summary

Allianz Technology seeks a senior data platform engineer to design, build, and operate scalable data platforms for a large financial services group.

You will integrate Databricks, Delta Lake, Unity Catalogue, and orchestration tools to deliver production pipelines with strong security and governance, enabling analytics and AI use cases across operating entities.

Qualifications

• Bachelor's or Master's in Computer Science, Information Systems, Engineering, or equivalent.
• 5+ years building data/analytics platforms in enterprise settings, ideally in financial services.
• Strong experience with Databricks, Delta Lake, Unity Catalogue, and production pipelines.
• Data integration design with ETL/ELT, orchestration (Airflow/Azure Data Factory).
• Proficient in Python and SQL with production-grade code.
• Experience with CI/CD and infrastructure-as-code (Terraform/Bicep).
• Knowledge of security, governance, data lineage and auditability.
• Excellent English communication and stakeholder management.

Responsibilities

• Consult with operating entities on core platform deployment from initial setup to production.
• Build data integration pipelines connecting source systems to central platforms.
• Configure high availability, access controls, and end-to-end observability.
• Extend platform capabilities into AI/GenAI using Azure Foundry for governed use cases.
• Translate business needs into reusable platform implementations and reference architectures.
• Implement governance, security, data control layers per financial services norms.

Skills

Databricks

Python

SQL

CI/CD

Security & governance

Data governance

Stakeholder comms

Education

Bachelor's/Master's in CS/IS/Engineering

Tools

Delta Lake

Unity Catalogue

Azure Data Factory

Airflow

Terraform

Bicep

Kubernetes

Docker

Azure Synapse

Microsoft Fabric

Spark

dbt

Kafka

Snowflake

MLflow

Job description

Role Description & Responsibilities

Are you energised by building and implementing the data platforms that thousands of colleagues depend on at one of the world's largest financial services groups?

Are you energised by building and implementing the data platforms that thousands of colleagues depend on at one of the world's largest financial services groups? If so, this role puts you at the centre of that work. You will implement and engineer the platforms behind the Allianz Data Platform, working hands‑on with Databricks, Azure Synapse, and Microsoft Fabric, and turning platform capabilities into services that Operating Entities can adopt with confidence.

The role sits at the crossroads of engineering, data, and operations, which means the pipelines, integrations, and controls you build shape how thousands of colleagues across Allianz Group turn data into decisions.

To get there, you will work forward with the Operating Entities themselves, assisting in the deployment and configuration of platform components, integration of source systems, and extending our offering into so that analytics and AI use cases run on solid, governed foundations.

What we value most is your ability to configure production‑grade solutions, reason clearly about architecture and trade‑offs, work through security, compliance, and data governance requirements, and take ownership of first deployment and configuration through to stable operation.

As Part Of This Role You Will

• Consult with OEs in the implementation and configuration of core platform components across Databricks from initial deployment through to stable production.
• Assist in building data integration pipelines that connect OE source systems to central platforms, using both batch and streaming patterns.
• Configure platform capabilities with high availability, strong access controls, and end‑to‑end observability.
• Extend the platform offering into AI and GenAI scenarios, including Azure AI Foundry, so that model‑based use cases run on governed foundations.
• Partner directly with Operating Entities to translate business needs into working platform implementations and reusable solutions.
• Implement governance, security, and data control layers in line with financial services expectations for data handling, auditability, and operational resilience.
• Build and refine CI/CD and infrastructure‑as‑code pipelines to keep deployment cycles fast, secure, and repeatable.
• Collaborate closely with security, risk, compliance, architecture, and data governance teams to clear the path for platform adoption.
• Deliver reusable components, reference architectures, and engineering best practices that accelerate platform rollout across the group.
• Support the rollout of central platform solutions to the OEs, and build business‑specific use cases on top of them.
• Coach and mentor engineers across the organisation, and drive continuous improvement in tooling, standards, and ways of working.

Basic Qualifications.

• Bachelor/Master degree in Computer Science, Information Systems, Engineering, or an equivalent technical discipline.
• 5+ years of hands‑on experience implementing or engineering data and analytics platforms in enterprise environments, ideally within financial services, insurance, or another regulated industry.
• Significant experience with Databricks, including Delta Lake, Unity Catalogue, workflows, and production pipelines.
• Strong data integration skills, covering ETL and ELT design, orchestration (e.g., Azure Data Factory, Airflow), and both batch and streaming sources.
• Solid engineering skills in Python and SQL, with experience building and maintaining production‑grade code.
• Experience with CI/CD pipelines, source control management, and infrastructure‑as‑code (e.g., Terraform, Bicep).
• Working knowledge of platform security and governance concepts, including access controls, encryption, and data lineage.
• Familiarity with end‑to‑end (E2E) Data Product Management, including data governance, quality, and lifecycle management.
• Excellent communication and stakeholder management skills in English (additional languages are a plus).

Preferred Qualifications

• 7+ years of experience in platform engineering, data engineering, or cloud infrastructure.
• Exposure to Azure AI Foundry or comparable AI platform tooling, including the deployment of model‑based services.
• Familiarity with MLOps practices and tools such as MLflow, feature stores, and model lifecycle governance.
• Experience with Kubernetes, Docker, and cloud‑native architectures.
• Hands‑on familiarity with the wider modern data stack (e.g., Spark, dbt, Kafka, Snowflake, or similar).
• Practical experience with Azure data services such as Azure Synapse Analytics and Microsoft Fabric.
• Experience with observability and monitoring tools such as Azure Monitor, Datadog, or Prometheus.
• Relevant certifications such as Databricks Certified Data Engineer, Microsoft Azure Data Engineer, or equivalent.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.

At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

Join us. Let's care for tomorrow.