Senior Data Engineer

Aon Corporation; Bengaluru; Hybrid; INR 2,200,000 - 4,200,000; Full time

Benefits offered by this job

Cab facility

Job summary

Aon is seeking a Senior Data Engineer to design and operate scalable cloud-based data platforms, enabling analytics and AI-ready data products. You will work with data scientists, architects, and product owners to deliver reliable data services across the enterprise.

The role emphasizes governance, data quality, and scalable architectures, with a hybrid Bengaluru-based work arrangement and a strong focus on modern data tooling.

Qualifications

• Bachelor's or Master's in CS, Engineering, or related field.
• 5+ years of relevant experience in data engineering.
• Strong programming and data-platform design skills.
• Experience with cloud data platforms and governance.

Responsibilities

• Design, develop, deploy, and operate scalable cloud data solutions.
• Build trusted data pipelines and feature-ready datasets.
• Ensure reliability, monitoring, and governance of data products.
• Collaborate with stakeholders to deliver end-to-end data platforms.
• Mentor other engineers and contribute to standards.

Skills

Python/PySpark/Scala

SQL

Spark pipelines

DataOps

CI/CD

Mentoring

Communication

Education

Bachelor's or Master's in CS/Engineering

Tools

Databricks

Airflow

dbt

Delta Lake

Unity Catalog

Job description

Job Title – Senior Data Engineer

Position type- Full Time

Work Location- Bangalore

Working style- Hybrid

Cab Facility- Yes

Shift Time – 12:30 PM to 9:30 PM

People Manager role: No

Required education and certifications are critical for the role: bachelor’s or master’s degree in computer science, Engineering, or a related field.

Required years of experience: 5+ years of relevant experience

AON IS IN THE BUSINESS OF BETTER DECISIONS

At Aon, we shape decisions for the better to protect and enrich the lives of people around the world.

As an organization, we are committed to our purpose as one firm, united through trust as one inclusive, diverse team and we are passionate about helping our colleagues and clients succeed.

Role Overview:

Enterprise Data & Analytics Engineering team is leading the development of cloud-based products and analytics capability for Aon. With a diverse portfolio of projects, and a sizeable production estate supporting an already successful business, we are in a critical growth phase and need the best minds in industry to help advance our journey.

The D&A Data Engineering team is responsible for building the data foundation that powers analytics, operational decision-making, and AI-driven products across the organization. As a Senior Data Engineer, you will play a critical role in designing, delivering, and operating scalable data platforms and data products that enable trusted, governed, and actionable data at enterprise scale.

This role goes beyond traditional ETL development. You will help build and shape modern data architecture, implement engineering best practices, and build reusable data capabilities that support reporting, advanced analytics, machine learning, and emerging AI use cases. Working closely with business stakeholders, architects, product owners, and data scientists, you will transform complex data ecosystems into AI-ready reliable and consumable data products.

Key Responsibilities:

• Design, develop, deploy, and operate scalable cloud-based data, analytics, automation and tooling solutions using modern data platforms and Cloud services.
• Support integration of AI-enabled capabilities by providing trusted data pipelines, feature-ready datasets, APIs, and operational controls required by downstream AI, analytics, and application services
• Apply engineering best practices as you design and deliver high-quality, scalable solutions. Own your solution end-to-end through design, implementation, and operations.
• Own production reliability of data systems, including monitoring, observability, incident response, and SLA adherence
• Collaborate cross-functionally across business stakeholders, product owners, and other engineers to deliver integrated, customer-focused and resilient data solutions.
• Contribute to the evolution of Aon’s data platforms through technical feedback and innovation.
• Contribute to engineering standards, documentation, and technical mentorship within the team.

Skills/competencies required:

Pipeline Performance & Spark Optimization

• Proven experience designing and optimizing Spark pipelines against multi-terabyte datasets in production
• Deep understanding of Spark memory management, caching strategies and cluster resource allocation
• Ability to read and act on Spark UI diagnostics, query plans, and job metrics to resolve performance bottlenecks

Lakehouse & Delta Architecture

• Strong working knowledge of Delta Lake internals — MERGE optimization, Z-ordering/liquid clustering, schema evolution, Change Data Feed
• Ability to design and evolve layered, reliable data architectures (e.g., medallion or equivalent multi-stage patterns) that support reprocessing and scale without compromising data quality

Pipeline & Orchestration Design

• Experience building idempotent, restartable data pipelines with robust incremental processing
• Hands‑on experience with at least one major orchestration tool (Databricks Workflows, Airflow, or dbt), with sound judgment on tool selection

Data Governance & Platform Enablement

• Experience with Unity Catalog (or equivalent) for access control, lineage, and data discovery across teams
• Understanding of data security practices — PII tagging, masking, row/column-level access control
• Ability to design scalable ownership and self‑service access models rather than centralized, manual approval processes

Foundational / General Competencies

• Strong proficiency in Python/PySpark and/or Scala, and SQL
• Solid grasp of distributed systems concepts (partitioning, parallelism, fault tolerance)
• Experience with DataOps and Engineering practices – Git, CI/CD Pipelines, Environment management
• Ability to communicate technical tradeoffs clearly to both engineering and non‑engineering stakeholders
• Comfortable mentoring other engineers on performance, architecture, and best practices

AI-Enabled Engineering Productivity

• Hands‑on fluency with AI-assisted development tools (e.g., Claude Code, GitHub Copilot, Databricks Assistant) for code generation, refactoring, debugging, and documentation
• Track record of infusing AI tooling across the development lifecycle — design/spec drafting, coding, code review, testing, and documentation — not just at the code-generation step.