Senior Big Data Engineer - I

MetLife, Inc.; Pune District; On-site; INR 1,200,000 - 2,200,000; Full time

Job summary

MetLife is seeking a senior Data Engineer to design and deliver scalable data platforms and pipelines across Azure cloud and on-prem environments. You will own end-to-end data engineering initiatives, mentor engineers, and collaborate with governance, architecture, and analytics teams to enable advanced analytics and reporting.

The role emphasizes production-grade data platforms, real-time processing, and delivering reliable data solutions for business decision-making.

Qualifications

• Bachelor's degree in Computer Science, IT, Engineering, or equivalent.
• 8-10+ years in Data Engineering, Big Data, Cloud Data Platforms or related fields.
• Strong SQL and Python and/or Scala for large-scale data processing.
• Experience with ETL/ELT design, architecture, and estimation.
• Hands-on with Spark, Hadoop, Hive; NoSQL and distributed databases.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines across Azure and on-prem environments.
• Develop scalable data ingestion and transformation for large data volumes.
• Lead solution architecture, design, estimation, and implementation planning.
• Monitor and optimize data platforms for reliability, scalability, and performance.
• Enforce data quality, governance, security, and compliance.
• Mentor Data Engineers and communicate complex solutions to stakeholders.

Skills

SQL

Python

Spark

Hadoop

Cosmos DB

HBase

Unix Shell

Education

Bachelor's degree in CS/IT/Engineering

Tools

Apache NiFi

Kafka

Azure DevOps

Job description

The Data Engineer is responsible for designing, building, and delivering scalable data engineering solutions that enable advanced analytics, reporting, and business decision-making across the organization. This role plays a critical part in the data and analytics lifecycle by developing production-grade data platforms and pipelines leveraging Big Data technologies and cloud-based solutions.

As a senior individual contributor, this position is expected to independently solve complex business challenges, lead solution design and estimation activities, and drive engineering excellence across projects. The role provides an exciting opportunity to work with modern Azure data technologies, large-scale distributed systems, and real-time data processing frameworks while mentoring data engineers and collaborating with stakeholders across Business, Technology, Operations, and Data & Analytics functions.

Key Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines across Azure cloud and on-premises environments for batch and real-time data processing.
• Develop scalable solutions to collect, ingest, transform, and store large volumes of structured and unstructured data.
• Independently lead solution architecture, technical design, effort estimation, and implementation planning for data engineering initiatives.
• Monitor, optimize, and troubleshoot data platforms and pipelines to ensure reliability, scalability, availability, and performance.
• Implement and enforce data quality, governance, security, compliance, and operational standards.
• Build and support data lakes, data warehouses, data marts, and Medallion Architecture-based solutions.
• Develop real-time and streaming data pipelines using Apache Spark and related technologies.
• Collaborate with cross-functional teams including Data Governance, Data Quality, Data Architecture, Data Modeling, Data Science, DevOps, Business Intelligence, Operations, and Technology groups.
• Mentor and guide Data Engineers through technical leadership, knowledge sharing, and best practice adoption.
• Drive stakeholder engagement, provide technical recommendations, and effectively communicate complex technical solutions to business and technology audiences.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or an equivalent educational qualification.
• 8-10+ years of experience in Data Engineering, Big Data, Cloud Data Platforms, or related domains.
• Strong expertise in SQL and Python and/or Scala for large-scale data processing and engineering solutions.
• Extensive experience in ETL/ELT pipeline design, development, solution architecture, and estimation.
• Hands‑on experience with Big Data technologies including Apache Spark, Hadoop, and Hive.
• Experience working with NoSQL and distributed databases such as Cosmos DB and HBase.
• Deep understanding of Data Lakes, Data Warehouses, Data Marts, and Medallion Architecture.
• Strong expertise in performance optimization, scalability improvements, and data quality validation.
• Experience designing and supporting both batch and real-time streaming data pipelines using Spark.
• Excellent communication, analytical thinking, structured problem-solving, storytelling, and stakeholder management skills.
• Proven experience mentoring and developing junior and mid-level Data Engineers.
• Ability to work independently and drive end-to-end solution delivery in a senior individual contributor capacity.

Preferred Qualifications

• Experience with DevOps practices including Git, Azure DevOps, and CI/CD pipeline implementation.
• Knowledge of Kafka, MongoDB, Apache NiFi, and Unix Shell Scripting.
• Exposure to Generative AI technologies, AI-enabled data platforms, and emerging AI tools.
• Experience working within the Banking, Financial Services, and Insurance (BFSI) domain.
• Azure certifications in Data Engineering, Cloud Architecture, or related technologies.
• Experience leading cloud modernization, data transformation, and enterprise-scale analytics initiatives.
• Strong consulting mindset with the ability to influence technical and business stakeholders.

Experience

8-10+ Years of Relevant Experience

Education

Bachelor's Degree in Computer Science, Information Technology, Engineering, or Equivalent Educational Qualification.

About MetLife Recognized on Fortune magazine’s list of the "World’s Most Admired Companies" and Fortune World’s 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world’s leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East. As part of our New Frontier strategy, MetLife is building an AI-enabled, people-centered future. We’re looking for people who bring curiosity, adaptability, and a growth mindset as we use AI to enhance how we serve customers, support communities, and evolve the way work gets done. At MetLife, AI is a responsible partner that supports human judgment, creativity, and continuous improvement while helping us build trust, inclusion, and long-term value. Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we’re inspired to transform the next century in financial services. At MetLife, it’s #AllTogetherPossible .Join us!