Senior Data Engineer - London Stock Exchange Group

Jobs via eFinancialCareers; London; On-site; GBP 65,000 - 100,000; Full time

Job summary

A leading financial markets infrastructure and data provider is seeking a Senior Lead within Engineering to enhance their data engineering capabilities. The role involves designing secure data processing pipelines on Azure, optimizing data solutions, and leading innovative technology adoption. This position is pivotal in ensuring continuous data delivery for financial services, offering competitive remuneration and a collaborative work culture.

Benefits

Healthcare

Retirement Planning

Paid Volunteering Days

Wellbeing Initiatives

Qualifications

• Relevant experience in data engineering roles, especially within financial services.
• Proven ability with Azure services and tools.
• Strong communication and self-management skills.

Responsibilities

• Design and implement data engineering solutions on Azure.
• Build secure, compliant data processing pipelines.
• Optimize data platforms considering business requirements.

Skills

Data Platforms

Azure PaaS/SaaS

Cloud Distributed Data Processing

Spark

Python

SQL

Security Best Practices

Agile Development

Strong Communication Skills

Job description

About Us:

Role Description:
As a Senior Lead within Engineering, you'll design and implement functionalities focusing on Data Engineering tasks. You'll work with semi-structured data to ingest and distribute it on a Microsoft Fabric-based platform, modernizing data products and distribution channels. You'll drive the software development lifecycle for continuous data delivery and lead the evaluation and adoption of emerging technologies.

Key Responsibilities:
Provide partnership and support to SMEs and Tech Leads to ensure delivery on commitments.

• Build and maintain secure, compliant production data processing pipelines on Microsoft Fabric and Azure to ingest, land, and transform data into data products.
• Ensure data pipelines and data stores are high-performing, efficient, organized, and reliable based on business requirements and constraints.
• Design, implement, monitor, and optimize data platforms to meet functional and non-functional requirements.
• Handle data-related implementation tasks including provisioning storage, ingesting streaming and batch data, transforming data, implementing security and retention policies, identifying performance bottlenecks, and setting up monitoring and telemetry.
• Design and operationalize large-scale enterprise data solutions using Azure data and analytics services.
• Implement data solutions using Azure services like Delta.io, Lakehouse, Fabric, Azure Cosmos DB, Azure Data Factory, Spark, and Azure Blob Storage, among others.

Skills and Experience:

• Relevant experience in Data Platforms within the Financial Services industry, and familiarity with Azure PaaS/SaaS offerings like Fabric, Synapse, Purview, ADF, and Data Lake Storage.
• Proven experience as a data engineer or similar role, focusing on cloud distributed data processing platforms like Spark and open table formats like Delta or Iceberg.
• Strong experience with Azure services such as Synapse Analytics, Data Factory, Data Lake, Databricks, and others.
• Proficiency in Spark, SQL, and programming languages like Python, Scala, or Java.
• Experience in building Lakehouse architectures using open-source table formats and tools like Jupyter Notebook.
• Understanding of security best practices, including Azure Key Vault, IAM, RBAC, and monitoring tools.
• Ability to integrate, transform, and consolidate data from various sources for analytics solutions.
• Knowledge of data exploration, retention, validation, visualization, and related processes.
• Strategic thinking and operational capability in a fast-paced environment.
• Ability to understand business requirements and their impact on roadmaps.
• High-level understanding of Azure DevOps and Agile methodologies like SCRUM and Kanban.
• Strong communication, presentation, documentation, and interpersonal skills.
• Self-management skills in a dynamic environment.

Location:
Preferred locations are London or Bangalore.

LSEG is a leading global financial markets infrastructure and data provider, committed to driving financial stability, empowering economies, and enabling sustainable growth.

Our values of Integrity, Partnership, Excellence, and Change underpin our culture. We foster a diverse, collaborative, and innovative environment, committed to sustainability and supporting the green economy.

LSEG offers benefits including healthcare, retirement plans, volunteering days, and wellbeing initiatives.

We are an equal opportunities employer, committed to non-discrimination and reasonable accommodations for applicants and employees.

Please review our privacy notice regarding personal data handling.

For recruitment agencies, ensure candidates are informed of this privacy notice.

More jobs from London Stock Exchange Group

Boost your career

Find thousands of opportunities by signing up to eFinancialCareers today.