Senior Databricks & AI Architect_D3006

Allianz Technology

Thiruvananthapuram

On-site

INR 5,000,000 - 9,000,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Allianz Technology in India, Kerala invites a Senior Databricks & AI Architect to lead our data platform and AI roadmap. You will architect Spark-based systems, drive FinOps, optimize performance, and shape agentic AI workflows for insurance processes.

You will guide end-to-end data architectures on Databricks, champion LLM orchestration, RAG patterns, and robust data modelling, working with insurance domain concepts and ensuring governance and security excellence.

Qualifications

  • 8+ years in Data Engineering/Architecture with 3+ years in Databricks leadership.
  • Expert in Spark (PySpark/Scala), Delta Lake, and Databricks SQL.
  • Proven AI solutions experience; Agentic AI landscape knowledge.
  • Databricks FinOps, cost management, and performance tuning.
  • Strong data modelling; ETL/ELT template frameworks.
  • Insurance/FinServ domain experience preferred.

Responsibilities

  • Architect and govern end-to-end data architectures on the Databricks Platform.
  • Lead design of AI workflows focusing on Agentic AI and LLM orchestration.
  • Own FinOps and performance: cluster policies, Photon optimisations, ROI.
  • Standardise data engineering templates and robust data modelling.
  • Translate insurance concepts into scalable technical schemas and AI logic.
  • Ensure governance and security with Databricks Unity Catalog.

Skills

Spark framework
PySpark
Scala
Delta Lake
Databricks SQL
LangChain
LlamaIndex
Agentic AI
RAG patterns
FinOps
Cost management

Education

Bachelor’s or Master’s in CS/DS
Databricks Certifications

Tools

Databricks
Photon engine
Z-Ordering
Liquid Clustering

Job description

Position Overview

We are seeking a visionary Senior Databricks & AI Architect to lead the evolution of our modern data platform and AI roadmap. In this role, you will bridge the gap between complex insurance data domains and cutting-edge AI execution. You will be responsible for designing scalable Spark-based architectures, implementing rigorous FinOps and optimisation strategies, and pioneering Agentic AI workflows to automate high-value insurance processes (e.g., claims, underwriting, and risk assessment).

Key Responsibilities
  • Architectural Leadership: Design and govern end-to-end data architectures on the Databricks Data Intelligence Platform, ensuring seamless integration with existing insurance ecosystems.
  • AI Innovation: Lead the design of emerging AI workflows, specifically focusing on Agentic AI (multi-agent systems), LLM orchestration (LangChain/LlamaIndex), and RAG patterns tailored for insurance policy and regulatory documents.
  • Databricks FinOps & Performance: Own the platform’s efficiency. Implement cluster policies, Photon engine optimisations, and cost-attribution models to ensure high performance with a focus on ROI.
  • Data Engineering Excellence: Standardise data engineering templates (Medallion architecture) and implement robust data modelling (Star/Snowflake, Vault 2.0) to handle structured and unstructured insurance data.
  • Domain Alignment: Translate complex insurance concepts (Premiums, Claims, Actuarial models) into scalable technical schemas and AI logic.
  • Governance & Security: Ensure all designs comply with industry regulations using Databricks Unity Catalog.
Required Skills & Experience
  • Experience: 8+ years in Data Engineering/Architecture, with at least 3+ years of hands‑on leadership within the Databricks ecosystem.
  • Core Tech: Expert‑level knowledge of the Spark framework (PySpark/Scala), Delta Lake, and Databricks SQL.
  • AI/ML Expertise: Proven experience building and deploying AI solutions. Deep understanding of the Agentic AI landscape—moving beyond simple chatbots to autonomous, goal‑oriented agents.
  • FinOps Mastery: Specific experience in Databricks cost management, instance selection, and performance tuning (Z‑Ordering, Liquid Clustering).
  • Data Management: Strong background in data modelling and the creation of reusable ETL/ELT templates and frameworks.
  • Insurance Domain: Prior experience in the Insurance or FinServ sector is highly preferred; you must understand the nuances of policy‑centric data.
Education & Certifications
  • Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
  • Preferred: Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.

At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

Join us. Let's care for tomorrow.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks & AI Architect_D3006
Senior Databricks & AI Architect_D3006

Allianz • Pune

On-site
INR 18,000,000 - 26,000,000
Allianz - Senior Databricks & AI Architect
Allianz - Senior Databricks & AI Architect

Allianz PNB Life • Thiruvananthapuram

On-site
INR 2,500,000 - 4,500,000
Assistant Data Architect (Data Scientist)_ 3399
Assistant Data Architect (Data Scientist)_ 3399

Allianz • Pune

On-site
INR 1,800,000 - 2,800,000
Assistant Data Architect (Data Scientist)_ 3399
Assistant Data Architect (Data Scientist)_ 3399

Allianz Technology • Pune District

On-site
INR 1,200,000 - 2,400,000
Allianz - Technical Lead - AI Development
Allianz - Technical Lead - AI Development

Allianz PNB Life • Pune District

On-site
INR 3,500,000 - 7,000,000
Solution Architect [SF & AI]_3451
Solution Architect [SF & AI]_3451

Allianz • Pune

On-site
INR 4,200,000 - 7,000,000
Manager- Business Intelligence
Manager- Business Intelligence

Mployee.me • Pune

On-site
INR 1,800,000 - 2,800,000
Manager- Business Intelligence
Manager- Business Intelligence

Allianz Services • Pune

On-site
INR 1,200,000 - 2,300,000
AI Tech Lead Java &Web_3008
AI Tech Lead Java &Web_3008

Allianz • India

Hybrid
INR 4,000,000 - 6,000,000
Hybrid work model
Pension
Employee shares program
+2
AI Tech Lead (Data Engineering)_D3006
AI Tech Lead (Data Engineering)_D3006

Allianz • Pune

On-site
INR 4,200,000 - 6,400,000