Senior Data Engineer

Aon; Bengaluru; Hybrid; INR 2,200,000 - 4,000,000; Full time

Job summary

Aon is seeking a Senior Data Engineer in Bangalore to design and implement scalable data pipelines on Databricks, aligned with Azure-based architecture for reinsurance analytics.

You will work with Solution Architects, Product Owners, and Business Analysts to translate requirements into reliable data solutions while ensuring data quality and governance.

Qualifications

• Bachelor's or higher degree in engineering, CS, or related field.
• 7+ years in data engineering or big data.
• Proven ETL/ELT pipeline development with large datasets.

Responsibilities

• Build and maintain scalable data ingestion and ETL/ELT pipelines on Databricks.
• Collaborate with Solution Architects, Product Owners, and Analysts.
• Support Azure-based modern architecture and data governance.
• Develop data models and ensure data quality and security.

Skills

Databricks

PySpark

Python

SQL

Azure Data Factory

Azure Data Lake

Unity Catalog

Azure OpenAI

LLM integration

AI integration

Education

Engineering/CS degree

Tools

Databricks

Azure Data Factory

Azure Data Lake Storage

Unity Catalog

Azure OpenAI

Workflow Studio

Job description

Job Title

Senior Data Engineer

Position type

Full Time

Work Location

Bangalore

Working style

Hybrid

People Manager role

No

Required education and certifications critical for the role

Any Graduate or Post-Graduate (full time)

Required years of experience

Minimum 7+ years of relevant experience

About The Role

The Senior Data Engineer will be a key contributor to the Altair modernization program, working on the Azure-based data architecture layer built on Databricks. You will design and develop scalable data pipelines that support reinsurance analytics, Property & Casualty loss ratio calculations, and end-to-end workflow automation. As an individual contributor, you will work independently on complex data engineering challenges, collaborate with cross-functional teams including Solution Architects, Product Owners, and Business Analysts, and help establish best practices for the team. This role requires strong technical skills, the ability to understand business requirements in the reinsurance domain, and a passion for building reliable, scalable data solutions.

Aon is in the business of better decisions

At Aon, we shape decisions for the better to protect and enrich the lives of people around the world. As an organization, we are united through trust as one inclusive team and we are passionate about helping our colleagues and clients succeed.

What The Day Will Look Like

• Data Pipeline Development & Integration
  • Design, develop, and maintain scalable data ingestion, transformation, and integration pipelines on Databricks using PySpark
  • Build robust ETL/ELT workflows to process structured and unstructured data from multiple upstream systems (Meridian, email, Workflow Studio, AB Connect placements)
  • Implement data imports and transformations using agreed ingestion patterns and Unity Catalog
  • Develop data flows aligned with RAPID technology stack standards and design patterns
  • Support the transition of Altair capabilities to modern Azure-based architecture

Data Modeling & Quality

• Develop and maintain data models and schemas that support reinsurance business processes and analytics
• Implement data quality frameworks, validation processes, and data governance standards
• Ensure data lineage, security, and compliance across all data assets
• Design transformations that support Property & Casualty loss ratio calculations and statistical analysis

Business Requirements & Domain Understanding

• Collaborate with business stakeholders, SMEs, and Business Analysts to understand reinsurance business processes
• Translate business requirements into technical data solutions with clear understanding of domain context
• Analyze and resolve moderately complex data challenges related to reinsurance workflows
• Partner with Product Owners and Project Managers to deliver trusted data assets

Technical Collaboration & Best Practices

• Work closely with Lead Data Engineers, Solution Architects, and QA Engineers to deliver high-quality solutions
• Participate in code reviews and promote data engineering best practices
• Contribute to the development of reusable components and design patterns
• Support knowledge transfer activities to build in-house capabilities and reduce vendor dependency
• Create technical documentation and operational procedures

AI & Automation

• Partner with AI specialists to incorporate Azure OpenAI and LLM capabilities into data solutions
• Support development of intelligent data intake capabilities that trigger workflows automatically
• Contribute to AI-assisted development approaches and automation initiatives

Platform Optimization & Support

• Monitor and optimize pipeline performance, reliability, scalability, and cost efficiency on Azure
• Troubleshoot production issues, perform root-cause analysis, and implement sustainable solutions
• Implement CI/CD processes and automation for data engineering workflows
• Identify opportunities for continuous improvement across the data ecosystem

About The Project

You will be joining the RAPID team working on the Altair modernization program. The initiative involves building a modern, Azure-based Workflow Studio that integrates multiple applications and enables end-to-end process custody from data intake through client deliverables in the reinsurance domain. You will work with a collaborative team including Solution Architects, Product Owners, Project Managers, QA Engineers, and reinsurance SMEs. The program is decomposing existing Altair capabilities and re-implementing them across Databricks (data layer), Workflow Studio (process layer), and Blueprint/Remetrica (analytics layer). Other existing projects in the ecosystem include Workflow Studio, Blueprint, Remetrica, and Altair manual processes.

Skills and experience that will lead to success

• Bachelor's degree in Engineering, Computer Science, Data Engineering, Information Systems, or related field
• 7+ years of experience in data engineering, software engineering, or big data development
• Proven experience developing ETL/ELT pipelines and working with large-scale datasets

Technical Skills

• Strong hands‑on experience with Databricks for building data pipelines and transformations
• Solid proficiency in PySpark and Python for data processing and engineering
• Strong expertise in SQL for data manipulation and analysis
• Experience with Azure data services (Azure Data Factory, Azure Data Lake Storage, etc.)
• Knowledge of Unity Catalog for data governance and metadata management
• Experience with Azure OpenAI, LLM integration, and AI‑enabled data solutions
• Experience with Genie Spaces/Agents or similar AI agent frameworks
• Understanding of data modeling, data quality, and data governance principles
• Experience with ETL/ELT workflows and data integration patterns
• Familiarity with microservices architecture and API integration
• Knowledge of cloud‑based data warehouses, data lakes, and modern data architectures

Business & Analytical Skills

• Strong ability to understand and translate business requirements into technical solutions
• Solid analytical and problem‑solving skills
• Understanding of statistical concepts and their application in data analysis
• Proven ability to work independently with minimal supervision
• Excellent communication and stakeholder collaboration skills
• Ability to handle multiple priorities in a fast‑paced environment

Preferred Qualifications

• Experience in insurance, reinsurance, or financial services domains
• Understanding of Property & Casualty insurance, loss ratio calculations, or reinsurance analytics
• Knowledge of business process management (BPM) concepts and workflow automation
• Prior involvement in modernization or cloud migration programs
• Experience working with external vendors and distributed teams
• Azure certifications (Azure Data Engineer Associate or similar)
• Familiarity with workflow orchestration tools (Airflow, Azure Data Factory, etc.)

How We Support Our Colleagues

In addition to our comprehensive benefits package, we encourage a diverse workforce. Plus, our agile, inclusive environment allows you to manage your wellbeing and work/life balance, ensuring you can be your best self at Aon. Furthermore, all colleagues enjoy two “Global Wellbeing Days” each year, encouraging you to take time to focus on yourself. We offer a variety of working style solutions, but we also recognise that flexibility goes beyond just the place of work... and we are all for it. We call this Smart Working!

Our continuous learning culture inspires and equips you to learn, share and grow, helping you achieve your fullest potential. As a result, at Aon, you are more connected, more relevant, and more valued.

We provide individuals with disabilities reasonable accommodations to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment in accordance with applicable law. Please contact us to request an accommodation on [e-mail]

Aon values an innovative, diverse workplace where all colleagues feel empowered to be their authentic selves. Aon is proud to be an equal opportunity workplace.

2586663