Senior Data Engineer-1

Prudential (Cambodia) Life Assurance; Bengaluru; On-site; INR 8,876,000 - 14,793,000; Full time

Job summary

Prudential Singapore is seeking a senior data engineer to design, build, and maintain batch and near-real-time data pipelines on the Unified Data Platform (UDP). You will develop data ingestion, transformations, and publishing processes to support business operations, analytics, AI, and commissions calculations.

You'll work with business, finance, technology, and data teams to transform source data into trusted datasets, ensure data quality and security, and optimize pipelines for performance

Qualifications

• 5+ years in data engineering or software engineering.
• Hands-on experience delivering and supporting production data pipelines and enterprise data platforms.
• Experience designing ETL/ELT solutions, data models, transformations, and workflow orchestration.
• Good understanding of data quality, observability, lineage, metadata, data contracts, and access controls.
• Experience with Delta Lake, Unity Catalog, Git, automated testing, CI/CD, and cloud data platforms.
• Understanding of APIs, file-based integration, event processing, and secure data exchange.
• Good knowledge of Agile delivery, software engineering practices, and production support.
• Strong problem analysis, troubleshooting, communication, and collaboration skills.
• Insurance or financial services experience is an advantage.

Responsibilities

• Design, build, test, deploy, and maintain batch and near-real-time data pipelines on UDP platform.
• Develop data ingestion, transformation, enrichment, reconciliation, and publishing processes.
• Build and maintain data processing for commissions calculations, including rules, rates, adjustments, incentives, deductions, and calculation outputs.
• Implement reusable data models and curated datasets aligned with enterprise and UDP standards.
• Ensure data accuracy, completeness, lineage, traceability, security, and auditability across pipelines and calculation processes.
• Implement automated data quality checks, controls, alerts, and exception handling.
• Optimize pipeline performance, reliability, scalability, and cost.
• Use source control, automated testing, CI/CD, environment controls, and deployment standards.
• Monitor production pipelines, investigate failures and data discrepancies, and provide operational support.
• Contribute to the enhancement and maintenance of core UDP platform capabilities, engineering standards, and reusable components.

Skills

Python

SQL

Databricks

Spark

Data pipeline development

Education

Bachelor in Computer Science, Data Engineering, Information Systems, Engineering or equivalent

Tools

Git

CI/CD

Delta Lake

Unity Catalog

Automation testing

Cloud data platforms

Job description

Prudential's purpose is to be partners for every life and protectors for every future. Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured, for our people, customers, and partners. We provide a platform for our people to do their best work and make an impact to the business, and we support our people's career ambitions. We pledge to make Prudential a place where you can Connect, Grow, and Succeed.

In this role, you will design, build, maintain, and support data pipelines and core data capabilities on Prudential Singapore's Unified Data Platform (UDP). You will deliver reliable and auditable data products supporting business operations, analytics, AI, and commissions calculations.

As part of this dynamic role, you will work closely with business, finance, technology, data, and platform teams to transform source data into trusted and reusable datasets across the organization.

Job Description:

• Design, build, test, deploy, and maintain batch and near-real-time data pipelines on the UDP platform.
• Develop data ingestion, transformation, enrichment, reconciliation, and publishing processes.
• Build and maintain data processing for commissions calculations, including rules, rates, adjustments, incentives, deductions, and calculation outputs.
• Implement reusable data models and curated datasets aligned with enterprise and UDP standards.
• Ensure data accuracy, completeness, lineage, traceability, security, and auditability across pipelines and calculation processes.
• Implement automated data quality checks, controls, alerts, and exception handling.
• Optimize pipeline performance, reliability, scalability, and cost.
• Use source control, automated testing, CI/CD, environment controls, and deployment standards.
• Monitor production pipelines, investigate failures and data discrepancies, and provide operational support.
• Contribute to the enhancement and maintenance of core UDP platform capabilities, engineering standards, and reusable components.

Who we are looking for:

• Strong hands-on experience with Python, SQL, Databricks, Spark, and data pipeline development.
• Experience designing ETL/ELT solutions, data models, transformations, and workflow orchestration.
• Experience implementing complex calculation logic, reconciliation, and financial or commissions-related data processing.
• Good understanding of data quality, observability, lineage, metadata, data contracts, and access controls.
• Experience with Delta Lake, Unity Catalog, Git, automated testing, CI/CD, and cloud data platforms.
• Understanding of APIs, file-based integration, event processing, and secure data exchange.
• Good knowledge of Agile delivery, software engineering practices, and production support.
• Strong problem analysis, troubleshooting, communication, and collaboration skills.
• Insurance or financial services experience is an advantage.

Working Experience:

• At least 5 years of experience in data engineering or software engineering.
• Hands-on experience delivering and supporting production data pipelines and enterprise data platforms.

Education:

• Bachelor in Computer Science, Data Engineering, Information Systems, Engineering or equivalent.

Language:

Fluent written and spoken English.

Prudential is an equal opportunity employer. We provide equality of opportunity of benefits for all who apply and who perform work for our organisation irrespective of sex, race, age, ethnic origin, educational, social and cultural background, marital status, pregnancy and maternity, religion or belief, disability or part-time / fixed-term work, or any other status protected by applicable law. We encourage the same standards from our recruitment and third-party suppliers taking into account the context of grade, job and location. We also allow for reasonable adjustments to support people with individual physical or mental health requirements.