Azure Platform DevOps Engineer: Data Pipelines & IaC

London Stock Exchange; Greater London; On-site; GBP 85,000 - 120,000; Full time

Job summary

London Stock Exchange is seeking a DevOps Engineer to design, build, and maintain scalable cloud infrastructure on Microsoft Azure for data platforms such as Databricks, Event Hubs and ADLS Gen2. You will implement infrastructure-as-code, automation, and modern DevOps practices to streamline deployments, ensure reliability of real-time and batch data workloads, and optimize costs.

Collaborate with engineering and data teams to enable DevOps best practices and maintain secure, cost-effective

Qualifications

• Experience designing and implementing IaC with Terraform.
• Hands-on experience with Azure cloud services and security practices.
• Strong Linux system administration and scripting skills.
• Proficiency in Python and Bash for automation.
• Solid SQL skills for data workflows and performance tuning.
• Experience with data-centric platforms: Databricks, Event Hubs, ADLS/Blob storage.
• Familiarity with containerization (Docker/Kubernetes) and CI/CD tooling.
• Experience with monitoring and log management tools (Datadog, Splunk).
• Version control with Git (GitHub/GitLab) and Azure DevOps.

Responsibilities

• Design, build, and maintain scalable Azure cloud infrastructure.
• Automate data platform workloads and deployment pipelines.
• Support real-time and batch data workloads with reliable CI/CD.
• Manage and optimize Azure-native data services and SQL platforms.
• Implement security, governance, and cost-optimization across resources.
• Collaborate with engineering and data teams to enable DevOps best practices.
• Troubleshoot production issues and ensure high availability.

Skills

Python

Bash scripting

SQL

Linux

CI/CD

Event-driven

Tools

Terraform

Azure

Databricks

Azure Event Hubs

Azure Storage

Azure SQL

PostgreSQL

Docker

Kubernetes

Git

GitHub

GitLab

Azure DevOps

Datadog

Splunk

Bash

Python

Linux

Job description

London Stock Exchange is seeking a DevOps Engineer to design, build, and maintain scalable cloud infrastructure on Microsoft Azure for data platforms such as Databricks, Event Hubs and ADLS Gen2. You will implement infrastructure-as-code, automation, and modern DevOps practices to streamline deployments, ensure reliability of real-time and batch data workloads, and optimize costs.

Collaborate with engineering and data teams to enable DevOps best practices and maintain secure, cost-effective