Lead Data Engineer

lseg; India; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Hybrid working

Leave benefits

Upskilling programs

Wellbeing support

Job summary

London Stock Exchange Group (LSEG) seeks an experienced Lead Data Engineer to drive our billing transformation program in India. You will source telemetry, design scalable ETL pipelines, implement rating logic, and deliver invoice-ready data for SAP invoicing with strong data quality and governance.

You will work with AWS Glue, Snowflake, Python and SQL, building CI/CD pipelines and monitoring dashboards to ensure accuracy, auditability and timely delivery across Finance and Analytics teams.

Qualifications

• Proven hands-on data engineering experience with large-scale pipelines.
• Experience with cloud-native data processing and Snowflake.
• Strong emphasis on data quality, reconciliation, and audit controls.

Responsibilities

• Ingest telemetry data and build scalable data pipelines.
• Create curated usage datasets in Snowflake for billing.
• Implement rating logic and generate billing-ready datasets.
• Generate SAP invoice feeds and ensure data quality controls.
• Develop CI/CD pipelines for data deployments and automate tests.
• Establish monitoring, alerts, and runbooks for operations.

Skills

Data engineering

ETL/ELT development

SQL

Python

CI/CD practices

Tools

AWS Glue

Snowflake

PySpark

GitHub Actions

Jenkins

Job description

Role Overview

We are seeking an experienced Lead Data Engineer to join our billing transformation program. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing. The role requires strong experience with AWS Glue, Snowflake, Python, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, and operational monitoring to ensure billing accuracy. This is a critical role supporting a transformation from raw telemetry usage data to rated, validated, and invoice-ready billing data.

Key Responsibilities

• Data ingestion pipelines from telemetry systems.
• Curated usage datasets in Snowflake.
• Rating logic implementation and billing-ready datasets.
• SAP invoice feed generation.
• Data quality and reconciliation framework.
• Exception and error-handling process.
• CI/CD pipelines for data engineering deployments.
• Monitoring, alerting, and operational dashboards.
• Technical documentation and runbooks.

Required Skills

• Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
• Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing.
• Strong experience working with Snowflake as a data warehouse or enterprise data platform.
• Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.
• Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes.
• Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.
• Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization.
• Experience working with batch and event-driven data ingestion patterns.
• Understanding of data modelling, partitioning, performance tuning, and cost optimization.

Soft Skills

• Strong analytical skills and attention to detail-especially important for financial accuracy.
• Ability to collaborate with cross-functional teams including Finance, Engineering, and Analytics.
• Good communication skills to explain defects, logic, and test outcomes clearly.
• High ownership and adaptability, with willingness to take on both QA and minor development tasks.

Benefits

• We are an equal opportunity employer with hybrid working.
• Annual Leave and Parental Leave benefits.
• Upskilling opportunities with job-relevant training and certifications.
• Dedicated Employee Assistance and Wellbeing support.

What you'll get in return

High-impact projects : We work on a variety of pioneering AI products and leverage extensive datasets to solve complex, high-value challenges.

Competitive benefits : You will enjoy a strong compensation package, comprehensive benefits, and ongoing investment in career growth and skill development.

Industry leadership : This is an opportunity to be a founding member of the organization that's delivering brand-new products that democratize modeling and analytics solutions.

Collaborative environment : We provide opportunities for continuous learning and professional development in a work environment of dedicated, highly experienced teams.

We recognize that to attract the best talent, we need to be flexible, and we are open to discussing work arrangements with you. We take a hybrid approach to the workplace; this role is considered 'Blended', which requires attending the office at least three day per week while some teams and colleagues choose to collaborate in the office more frequently.

We're proud to have been recognised as a Great Place to Work in India '25

Learn more about life and purpose of our company directly from India colleagues' video: Bengaluru, India | Where We Work | LSEG

Career Stage

Senior Associate

London Stock Exchange Group (LSEG) Information

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership , Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basi