Data Engineer

Milliman Ireland; Gurgaon; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Milliman Ireland is seeking a Data Engineer to support data ingestion, transformation, and governance within a Microsoft Fabric-centric environment. You will work with actuarial, product, IT, and security teams to build reliable data pipelines and maintain Lakehouse tables.

Role involves profiling data, standardizing fields, and documenting data quality findings while ensuring secure connections and controlled access for datasets used by actuarial stakeholders.

Qualifications

• Foundational experience in data pipelines, ETL/ELT, or cloud data platforms.
• SQL and Python proficiency; Spark/notebooks experience preferred.
• Understand data modeling, data quality checks, and table maintenance.

Responsibilities

• Ingest data from cloud storage into Microsoft Fabric via ADF pipelines.
• Maintain bronze, silver, and gold Lakehouse tables per the data model.
• Develop data profiling scripts to assess structures, quality, and metrics.
• Standardize source data fields and support daily transformation work.
• Document data lineage, assumptions, and quality findings for actuarial users.
• Support Fabric workspace setup, permissions, and secure connections.
• Assist cross-functionally with product build, BI access, and IT/security.

Skills

Foundational data engineering

SQL

Python

Data profiling

Azure Fabric / ADF / OneLake

Data quality checks

Documentation

Tools

Spark

Notebooks

Git

Azure DevOps

Power BI

Job description

Milliman is among the world's largest providers of actuarial and related products and services. The firm has consulting practices in healthcare, property & casualty insurance, life insurance and financial services, and employee benefits. Founded in 1947, Milliman is an independent firm with offices in major cities around the globe.

Role Summary

The Data Engineer will support data ingestion, transformation, profiling, documentation, and Microsoft Fabric environment operations for a data and actuarial discovery sandbox. This role will work closely with senior data engineering, actuarial, product, IT, and security teams to help build reliable data pipelines, maintain Lakehouse tables, validate data quality, and support controlled access to curated datasets.

Key Responsibilities

• Support ingestion of sample source data from cloud storage into Microsoft Fabric using ADF pipelines, OneLake landing folders, and Lakehouse tables.
• Assist with the creation and maintenance of bronze, silver, and gold tables based on the agreed discovery data model.
• Build and run data profiling scripts and notebooks to identify file structures, schemas, row counts, null rates, duplicate records, timestamp ranges, and field-level quality issues.
• Help standardize common source data fields such as entity identifiers, transaction or event IDs, timestamps, event types, measurement units, and geography fields.
• Support data transformation work required to prepare daily summary, event-summary, exposure, behavioral, and contextual feature datasets.
• Document source tables, transformations, assumptions, data quality findings, known limitations, and operational procedures for actuarial users.
• Support ad hoc data requests from the actuarial team during discovery and help refine datasets based on feedback.
• Assist with Microsoft Fabric workspace setup, Lakehouse configuration, pipeline permissions, access controls, and environment support.
• Help configure and validate secure connections between Microsoft Fabric and external cloud platforms, including authentication handling, service account/key usage, and privileged access management considerations.
• Monitor pipeline runs, ingestion failures, storage usage, compute activity, and basic cost/performance indicators during the discovery build.
• Provide crossover support to the product build effort, including Fabric configuration, Lakehouse setup, Power BI access support, and troubleshooting with IT/security teams.

Required Skills and Experience

• Foundational experience in data engineering, data pipelines, ETL/ELT, data transformation, or cloud data platforms.
• Working knowledge of SQL and Python; exposure to Spark, notebooks, or distributed data processing is preferred.
• Basic understanding of data modeling, data quality checks, schema validation, and table maintenance.
• Familiarity with Microsoft Azure, Microsoft Fabric, ADF pipelines, OneLake, Lakehouse tables, or similar cloud data services.
• Ability to analyze source data, identify quality issues, and document findings clearly for technical and non-technical audiences.
• Strong attention to detail, willingness to learn, and ability to work collaboratively with senior engineers and actuarial stakeholders.

Preferred Qualifications

• Exposure to insurance, actuarial, mobility, operational, or risk analytics data.
• Experience working with GCP, Azure, Power BI, or cloud-to-cloud data ingestion patterns.
• Understanding of access controls, workspace permissions, data retention, and secure data handling practices.
• Experience using Git, Azure DevOps, or similar tools for version control, task tracking, and documentation.