Sr Data Engineer

Aon plc

Bengaluru

Hybrid

INR 3,000,000 - 6,000,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Aon plc is seeking a Sr Data Engineer in Bengaluru to lead the design and delivery of large-scale data pipelines and data warehouses. The role requires deep experience with Apache Spark (Databricks), Databricks hands-on work, and strong SQL/database tuning.

You will collaborate with actuaries, developers, and business experts to turn data into actionable insights for web-based applications. Responsibilities include end-to-end pipeline development, data modeling, performance optimization, and

Qualifications

  • 8+ years of total experience in data integration teams.
  • 3+ years of in-depth experience developing data pipelines with Apache Spark (Databricks preferred).
  • 2+ years hands-on experience with Databricks.
  • Experience building data pipelines ingesting various data sources into a database or data lake and populating a data warehouse or data mart.
  • Strong SQL performance tuning and database performance tuning knowledge.
  • Proficiency in ETL/ELT via SSIS or similar tool.
  • Knowledge of data modelling, databases, and data warehousing.
  • Experience designing and building data warehouses and data marts.
  • Able to learn new technologies quickly and adapt to AI/LLM integrations.
  • Experience integrating with AI/LLM platforms (Azure OpenAI, OpenAI, Hugging Face, vector DBs).

Responsibilities

  • Collaborate with actuaries, developers, and business experts to deliver projects.
  • Design end-to-end data pipelines from ingestion to storage to consumption.
  • Design and implement data warehouses and data marts for data consumers.
  • Implement designs from concept to deployment and operationalization.
  • Model and organize data at macro and micro levels for consumers.
  • Tune database performance and manage data lifecycles.
  • Support and enhance existing data pipelines and databases.

Skills

Apache Spark
Databricks
SQL performance tuning
ETL/ELT design
Data modeling
Data warehousing
Data pipelines
AI/LLM integration
Object-oriented programming

Tools

Databricks
Azure OpenAI
OpenAI
Hugging Face
Vector databases

Job description

Sr Data Engineer

Responsibilities:
  • Collaborate with teams of experienced actuaries, developers and business experts to deliver projects
  • Design and propose end-to-end data pipelines ((ingestion to storage to consumption) for data projects as well as databases to support web-based applications
  • Design and implement data warehouses and data marts to serve data consumers
  • Execute implementation of the designs and bring it from design stages to implementation followed by operationalization to maintenance
  • Database design and modelling. To be able to organize data at both macro and micro level and provide logical data models for the consumers
  • Database performance tuning and data lifecycle management
  • Assist in support and enhancements of existing data pipelines and databases
Requirements:
  • 8+ years of total experience working with data integration teams.
  • 3+ years of in-depth experience developing data pipelines within an Apache Spark environment (preferably Databricks).
  • 2+ years of active hands-on work with Databricks, demonstrating in-depth experience.
  • Good experience in building data pipelines that ingests various types of data sources into a database or data lake and populating a structured warehouse or data mart that fuels the data consumer.
  • Fluent in both complex SQL query performance tuning and database performance tuning
  • Highly knowledgeable in ETL/ELT processes, both design and implementation via SSIS or some other ELT/ETL tool.
  • Good knowledge of data modelling, database technologies, data warehousing.
  • Have experience in building and designing data warehouses/data marts
  • Understand the importance of performance and able to implement best practices in ensuring performance and maintainability for data-centric projects
  • This person is required to be dynamic and be a quick eager learner as knowledge about new technologies will need to be acquired on the spot
  • Practical experience integrating with AI/LLM platforms and frameworks (e.g., Azure OpenAI, OpenAI, Hugging Face, vector databases).
Nice to haves:
  • Certifications in Databricks, cloud computing or database management.
  • Experience developing data solutions using native IaaS and PaaS solutions on AWS (RDS, S3).
  • Experience in any object-oriented programming language.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Amgen Technology • Hyderabad

Hybrid
INR 2,500,000 - 4,000,000
Associate Data Engineer
Associate Data Engineer

Amgen Inc. • Hyderabad

On-site
INR 1,800,000 - 2,600,000
Associate Data Engineer
Associate Data Engineer

Amgen Technology • Hyderabad

On-site
INR 1,200,000 - 2,000,000
Sr Data Engineer
Sr Data Engineer

The Versatile Club • India

Remote
INR 1,500,000 - 2,500,000
Sr Data Engineer (AWS | Multi-Cloud | Databricks)
Sr Data Engineer (AWS | Multi-Cloud | Databricks)

Cloudthat • Bengaluru

On-site
INR 1,400,000 - 2,800,000
Senior Data Engineer
Senior Data Engineer

CoffeeBeans • Bengaluru

On-site
INR 1,800,000 - 2,400,000
Associate Data Engineer
Associate Data Engineer

Mployee.me • Hyderabad

On-site
INR 900,000 - 1,300,000
Data Architect with Databricks
Data Architect with Databricks

Unique Erp Technologies • Hyderabad

On-site
INR 1,800,000 - 4,000,000
Senior Data Architect
Senior Data Architect

Tredence Inc. • Bengaluru

On-site
INR 2,400,000 - 4,800,000
Senior Data Engineer Databricks / PySpark
Senior Data Engineer Databricks / PySpark

Wissen Technology Pte Ltd • Bengaluru, Mumbai

On-site
INR 1,500,000 - 2,300,000