PHI - Lead- Data Engineer

prudential; Mumbai; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Prudential Health India is seeking a Lead/Data Engineer to join the Experience team in Mumbai. The role focuses on building a cloud-native, microservices-oriented data platform for health insurance, with emphasis on reusable components and rapid delivery.

The candidate will work on data pipelines, ETL processes, and integration of various data sources, ensuring quality and security while collaborating with multiple stakeholders.

Qualifications

• Deep hands-on experience in data architecture.
• Experience building cloud-native data platforms.
• Proficiency with ETL processes and data pipelines.

Responsibilities

• Design, develop, and maintain robust data pipelines for batch and streaming data processing.
• Architect reusable data solutions and development frameworks.
• Collaborate with cross-functional teams to understand data requirements and ensure scalable solutions.
• Implement data quality, governance, and security practices across the data stack.

Job description

Prudential's purpose is to be partners for every life and protectors for every future. Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured, for our people, customers, and partners. We provide a platform for our people to do their best work and make an impact to the business, and we support our people's career ambitions. We pledge to make Prudential a place where you can Connect, Grow, and Succeed.

Prudential (UK) in partnership with HCL group plans to set-up a standalone Indian health insurance company to address the growing healthcare needs of the Indian consumer.

This joint venture will combine Prudential's global expertise in insurance and financial services with HCL Group's experience in technology and healthcare solutions.

Prudential, with its longstanding presence in India, already operates two leading businesses in life insurance and asset management with the ICICI Group. Prudential was also the proud sponsor of the 1983 Cricket World Cup, India's first World Cup Victory!

Prudential Health India is a Zero to One team undertaking a no-legacy, greenfield health insurance deployment in India, building journeys that truly empathize with the customer and offer a differentiated experience.

To partner with us in this mission, we are looking for a talented to join our Experience team Name in Mumbai.

Lead/Data Engineer
Note: The title will depend on (1) Experience (2) Expertise and (3) Performance. So, the title could be:
(Senior Manager) Lead Data Engineer
(Manager) Data Engineer

Deep technology role
Experience: 12 - 15 years.
Location: Mumbai and/or Bangalore
Work from office only

Job Profile Summary:

PHI intends to build a cloud-native, microservices-oriented, loosely coupled & open technology platform, which is tightly aligned to health-insurance domain, and built expecting to be reused while anticipating change .

The PHI platform will be made up of multiple applications supporting different business functions, which are well-integrated and well-orchestrated.

The applications could be COTS (Common-Off-The-Shelf) vendor software, or Prudential Group software capabilities, or software built by in-house PHI engineering team.

All applications need to adopt common services, platforms, architectural principles, and design patterns.

The right candidate will be accountable to deliver technology artefacts to business stakeholders, in the fastest time possible , with least gaps , best quality , and clarity on how the technology and business requirements can be delivered and tested at pace, with minimal gaps, and best quality.

Requirement gaps, change requests, non-integrated journeys, bugs in UAT, NFR failures - all these would signal poor quality deliverables by this candidate.

Job Description:
Deeply understand the long-term architectural direction, emphasizing reusable components and interactions between various applications and accordingly architect data platform needs.
Collaborate with TDMs, BAs, data engineers, data owners, business stakeholders, and Solution designers to understand business requirements to architect data solutions considering functionality, interoperability, performance, scalability, reliability, availability, and other applicable criteria.
Conceptualize and engineer reusable data solution and development frameworks.
Use data-driven insights to guide the development of programs and apps that meet user needs.
Follow and contribute to defining and ensuring adherence to engineering and coding standards.
Stay innovative by building world-class propositions using bleeding-edge technologies, frameworks, and best practices to bring differentiation through technology.
Design, develop, and maintain robust data pipelines for batch and streaming data processing, ensuring scalability and reliability .
Implement ETL processes to gather, transform, and load data from various sources into data warehouses and data lakes .
Utilize data engineering technologies such as Parquet format for efficient data storage, Iceberg open table format for managing large-scale datasets .
Leverage Google Cloud Platform services including Google Cloud Storage, Google Cloud Pub/Sub, Google Cloud Dataflow, Google Cloud Dataproc, Google Cloud Composer, and Google Cloud Bigtable
Ensure data quality and governance by implementing monitoring, measuring, and improving data quality throughout its lifecycle
Develop and maintain data integration workflows using APIs and standard toolsets
Collaborate with cross-functional teams to solve enterprise problems related to data storage, exposure, and sharing in line with security and compliance requirements
Build and optimize data infrastructure to support advanced analytics and insights
Implement and maintain components of big data technologies for both exploratory and production data science platforms.

Who we are looking for:

Technical Skills & work experience:

MH: Proven hands-on experience of data architecture