Lead AWS Data Engineer(Python)

LSEG; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

LSEG (London Stock Exchange Group) is looking for a Software Data Engineer for the FTSE Russell division in Bengaluru, India. This role focuses on designing scalable data pipelines, developing data workflows using AWS, and applying modern software engineering practices. Candidates should have strong Python skills and an understanding of data engineering principles. LSEG encourages applicants from all backgrounds and offers equal opportunities.

Qualifications

• Strong, clean, maintainable Python coding skills including type hints and tests.
• Solid understanding of data engineering fundamentals like schema evolution.
• Curiosity about financial markets and data analytics.

Responsibilities

• Design and develop scalable, testable data pipelines using Python and Apache Spark.
• Orchestrate data workflows with AWS tools like Glue and Lambda.
• Implement observability and basic quality checks within data flows.

Skills

Python coding skills

Understanding of ETL pipelines

Experience with Apache Spark

Familiarity with AWS tools

Agile team experience

Tools

Apache Spark

AWS (S3, Glue, Lambda, EMR)

CI/CD tools (GitLab CI, Jenkins)

Job description

Company Overview

LSEG (London Stock Exchange Group) is a global financial markets infrastructure and data provider that supports financial stability and growth worldwide.

Role Summary

Software Data Engineer – FTSE Russell (AWS environment)

Responsibilities

• Design and develop scalable, testable data pipelines using Python and Apache Spark.
• Orchestrate data workflows with AWS tools such as Glue, EMR Serverless, Lambda, and S3.
• Apply modern software engineering practices: version control, CI/CD, modular design, automated testing.
• Contribute to a lakehouse architecture with Apache Iceberg.
• Collaborate with business teams to translate requirements into data‑driven solutions.
• Implement observability and basic quality checks within data flows.
• Participate in code reviews, pair programming, and architecture discussions.
• Continuously learn about the financial indices domain and share insights with the team.

Qualifications

• Strong, clean, maintainable Python coding skills (type hints, linters, tests such as pytest).
• Solid understanding of data engineering fundamentals: batch processing, schema evolution, ETL pipelines.
• Experience or willingness to learn Apache Spark for large‑scale data processing.
• Familiarity with the AWS data stack: S3, Glue, Lambda, EMR.
• Ability to learn business context and collaborate closely with stakeholders.
• Experience working in Agile teams, valuing collaboration over solo heroics.

Nice‑to‑Haves

• Experience with Apache Iceberg or similar table formats.
• Familiarity with CI/CD tools (GitLab CI, Jenkins, GitHub Actions).
• Exposure to data quality frameworks (Great Expectations, Deequ).
• Curiosity about financial markets, index data, or investment analytics.

Equal Opportunity Statement

LSEG is an equal opportunities employer and does not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy, or disability, or any other basis protected by applicable law. We accommodate applicants' and employees' religious practices, beliefs, mental health, and physical disability needs where reasonably achievable.