Lead Databricks Engineer [T500-29323]

Talent500; Hyderabad; On-site; INR 1,500,000 - 2,700,000; Full time

Job summary

Zurich's actuarial IT Solutions unit is seeking a Databricks Engineer to design, build, and maintain scalable data pipelines and analytical workloads on Databricks, enabling reliable ingestion, transformation, and reporting for actuarial and financial platforms.

You will work with PySpark, Spark SQL, Delta Lake, and cloud ecosystems (Azure/AWS), ensuring data quality, governance, and performance, while collaborating with actuaries, testers, and architects to translate requirements into robust

Qualifications

• 5 years of experience in data engineering or analytics engineering.
• 1
• Hands-on experience building Databricks solutions in enterprise environments.
• 1
• Bachelor’s degree in Computer Science, Engineering, Mathematics, Data Science, or related field.

Responsibilities

• Design, develop, and maintain data pipelines and analytical workloads on Databricks.
• Implement transformations using PySpark / Spark SQL and Delta Lake.
• Support end-to-end data flows across ingestion, curation, and analytics layers.
• Contribute to common Databricks development frameworks and standards.

Skills

PySpark

Python

SQL

Data Pipelines

Data Governance

Azure/AWS

Education

Bachelors in CS/Engineering/Math/Data Science

Tools

Databricks

Unity Catalog

Delta Lake

Azure/AWS

Job description

Talent500 is hiring for one of its clients.

Own Zurich's global SAP strategy & relationship to enable cost-effective model to offer the Group functions and BUs. Operate and implement standard processes in SAP in alignment with our Business Partners.

Drive transformation and deliver services with full accountability from service and application design, delivery and operation, as well as financial responsibility, security and compliance aspects with a single point of entry to our business partners in Group functions or to the business units. Focus on digitalization and automation, as well developing capabilities and managing technology assets to drive value for Zurich.

Who are we:

Actuary IT Solutions is Zurich’s global capability within Finance, Investment & Treasury (FIT), responsible for running, stabilizing, and evolving shared actuarial platforms that support IFRS and Group Reporting across business units.

We provide a unified and scalable operating model covering service management, platform operations, governance, compliance, and modernization—so actuarial teams can focus on business outcomes with confidence in platform reliability and control.

Our portfolio includes Zurich’s core actuarial platforms such as ACE, GNLA, GRAND, and GLAD, operated as shared services with clear ownership, measurable KPIs, and standardized processes. We continuously modernize this landscape to improve resilience, speed, cost efficiency, and long term sustainability, while reducing operational risk and complexity.

India delivery is an integral part of our global model, serving as a fully embedded delivery and capability hub. Teams work alongside global partners to support run operations, change, and transformation initiatives, contributing to scalability, resilience, and cost efficiency across the actuarial ecosystem.

Why you’ll love working here:

• Enterprise critical impact – Work on platforms that underpin IFRS and Group Reporting across Zurich
• Platform thinking, not one off fixes – Focus on standardization, reuse, and long term operational health
• AI enabled ways of working – AI first engineering, testing, operations, and knowledge management
• Clear ownership and accountability – Well defined service models, KPIs, and decision rights
• Global collaboration – Work with diverse, highly skilled teams across locations and disciplines
• Ownership culture – Trusted to make decisions, solve real problems, and see tangible outcomes

If you want to build deep expertise in actuarial platforms, work on services that truly matter to Zurich’s financial reporting and regulatory outcomes, and grow in a forward looking, AI enabled global team—Actuary IT Solutions is the place to do it.

Role Purpose:

The Databricks Engineer designs, builds, and maintains scalable, secure, and audit ready data pipelines and analytical workloads on Databricks. The role supports actuarial and financial platforms by enabling reliable data ingestion, transformation, calculation, and reporting, with strong focus on data quality, performance, and governance.

Key Accountabilities:

• Design, develop, and maintain data pipelines and analytical workloads using Databricks.
• Implement transformations using PySpark / Spark SQL and Delta Lake.
• Support end-to-end data flows across ingestion, curation, and analytics layers.
• Contribute to the evolution of common Databricks development frameworks and standards.

Data Quality, Performance & Reliability:

• Implement robust data validation, logging, and error handling mechanisms.
• Optimize workloads for performance, scalability, and cost efficiency.
• Ensure pipelines are reliable, recoverable, and observable.

Security, Governance & Audit:

• Implement and operate data governance controls using Unity Catalog (access control, lineage, auditability).
• Ensure solutions comply with internal control, segregation of duties, and audit requirements.
• Support traceability of data and results for regulatory and business reviews.
• Work closely with actuarial SMEs, testers, and architects to translate requirements into technical solutions.
• Support SIT, UAT, and release activities by resolving data, performance, or pipeline related issues.
• Contribute to documentation, technical reviews, and knowledge sharing.

Skills & Experience:

Required Experience:

• Approximately 5 years of experience in data engineering or analytics engineering roles.
• Handson experience building solutions on Databricks in enterprise environments.

Technical Skills:

• Solid coding experience in Python (PySpark) and SQL.
• Experience with Delta Lake, data modelling, and data transformation patterns.
• Experience building and operating data pipelines in cloud environments (Azure or AWS).
• Familiarity with version control and CI/CD practices.

Databricks & Platform Knowledge:

• Experience working with Unity Catalog or similar data governance frameworks is an advantage.
• Understanding of performance tuning, cluster usage, and workload optimization.
• Exposure to actuarial, financial, or risk related data domains is a plus, but not mandatory.

Professional Skills:

• Strong analytical and problem-solving skills.
• Structured and quality focused approach to engineering.
• Effective communicator able to work with technical and nontechnical stakeholders.
• Open, learning oriented mindset and willingness to continuously improve.

Education:

• Bachelor’s degree in Computer Science, Engineering, Mathematics, Data Science, or a related field (or equivalent experience).

This role suits an engineer who enjoys Handson development, values data quality and governance, and wants to build reliable Databricks platforms that support critical actuarial and financial processes at scale.

Candidate Data Privacy Notice:

Applicability of This Notice: This

job posting relates to opportunities with Zurich Digital International Private Ltd. (“Zurich”) and is published with the support of ANSR Inc., (“ANSR”) an authorised recruitment service provider engaged by Zurich for its hiring activities.

Participation in the Recruitment Process:

When you apply for this role, personal data provided as part of your application (such as your name, email address, contact details, address, financial information, background information, medical history, and details of previous employers/employment) is collected for the purposes of recruitment and selection and may be reviewed by Zurich, and transferred/disclosed to Zurich’s affiliates, subsidiaries, and related entities and ANSR (“Transferees”), solely for recruitment and selection purposes (in accordance with their respective privacy policies).

The Transferees maintain at least the same level of protection for your data as maintained by Zurich, and do not further transfer/disclose/share or publish your data.

For information on how your personal data is processed by ANSR, including your rights and how to contact the relevant data protection office, please review:

– ANSR Privacy Policy: ANSR Privacy Policy - Ensuring Data Security”

• You have the option not to provide your data (in which case we may not be able to process your application);
• You have the option to review, correct/amend your data;
• You have the option to withdraw your consent from processing your data;
• Your data is retained until the purpose for its collection is served; and
• Zurich uses reasonable security measures to help protect against the unauthorized access, loss, misuse and alteration of the personal information under our control. However, no method of transmission over the internet, or method of electronic storage, is completely secure.

By submitting your interest in any of our vacancies, you consent to the collection, storage, transfer, disclosure, and processing of your personal information and/or sensitive personal data or information.