Senior Lead Software Engineer

LSEG (London Stock Exchange Group); London; On-site; GBP 60,000 - 100,000; Full time

Job summary

An established industry player is seeking a Senior Lead Software Engineer to drive innovation in data engineering. In this pivotal role, you will design and implement functionalities focusing on modernizing data products and distribution channels. Your expertise will be essential in building secure data processing pipelines and optimizing data platforms using cutting-edge Azure technologies. This dynamic position offers the chance to work with semi-structured data and lead the software development lifecycle for continuous data delivery. If you thrive in fast-paced environments and are passionate about leveraging technology to transform data solutions, this role is perfect for you.

Qualifications

• Experience in Data Platforms within Financial Services, familiar with Azure’s PaaS/SaaS offerings.
• Proven experience as a data engineer with cloud-distributed data processing platforms.

Responsibilities

• Build and maintain secure production data processing pipelines on Microsoft Fabric and Azure.
• Design and operationalize large-scale enterprise data solutions using Azure data services.

Skills

Data Platforms

Cloud-Distributed Data Processing

Azure Services

Spark

SQL

Python

Scala

Java

Data Lifecycle Processes

Agile Methodologies

Tools

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

Azure Cosmos DB

Jupyter Notebooks

Job description

Join to apply for the Senior Lead Software Engineer role at LSEG (London Stock Exchange Group)

Role Description

As a Senior Lead within Software Engineering, you’ll design and implement functionalities, focusing on Data Engineering tasks. You’ll be working with semi-structured data to ingest and distribute it on a Microsoft Fabric-based platform, modernizing data products and distribution channels. You’ll drive the software development lifecycle for continuous data delivery and lead the evaluation and adoption of emerging technologies.

Key Responsibilities

1. Build and maintain secure and compliant production data processing pipelines on Microsoft Fabric and Azure to ingest, land, and transform data into data products.
2. Ensure data pipelines and data stores are high-performing, efficient, organized, and reliable, given business requirements and constraints.
3. Design, implement, monitor, and optimize data platforms to meet functional and non-functional data pipeline requirements.
4. Handle data-related implementation tasks including provisioning storage, ingesting streaming and batch data, transforming data, security, data retention, performance monitoring, and external data access.
5. Design and operationalize large-scale enterprise data solutions using Azure data and analytics services.
6. Implement data solutions leveraging Azure services like Delta.io, Lakehouse, Fabric, Azure Cosmos DB, Data Factory, Spark, and Blob storage, among others.

Skills and Experience

• Relevant experience in Data Platforms within the Financial Services industry, familiar with Azure’s PaaS/SaaS offerings.
• Proven experience as a data engineer, especially with cloud-distributed data processing platforms like Spark and open table formats such as Delta/Iceberg.
• Strong experience with Azure services: Synapse Analytics, Data Factory, Data Lake, Databricks, Purview, SQL, Cosmos DB, and more.
• Proficiency in Spark, SQL, and programming languages like Python, Scala, or Java.
• Experience in building Lakehouse architectures using open-source table formats and tools like Jupyter notebooks.
• Knowledge of security best practices in cloud environments, including Azure Key Vault, IAM, and RBAC.
• Ability to integrate and transform data from various sources for analytics solutions.
• Understanding of data lifecycle processes including retention, validation, visualization, and segmentation.
• Strategic thinking with the ability to operate in a fast-paced environment and understand business implications.
• Familiarity with Azure DevOps and Agile methodologies like SCRUM and Kanban.
• Excellent communication, presentation, and interpersonal skills.
• Self-management skills to work independently in dynamic settings.

Location

Preferred locations are London or Bangalore.