Senior Software Engineer - DevOps

BBC; Greater London; Hybrid; GBP 67,000 - 82,000; Full time

Job summary

The BBC is seeking a Senior Engineer to architect and develop cloud and network systems underpinning our observability and alerting platforms. You will migrate legacy influx systems to an open-source stack and integrate Loki and Tempo, ensuring high availability for millions of users.

You will collaborate across the BBC to shape the observability roadmap, work with AWS services, and contribute to a fast-moving engineering culture that values learning and innovation.

Qualifications

• Extensive experience building and operating cloud-based, highly available infrastructures.
• Strong knowledge of monitoring and observability tooling and metrics collection.
• Proficiency with IaC tools (Terraform, Ansible, CloudFormation, AWS CDK).
• Hands-on scripting in Python or Bash for automation and tooling.

Responsibilities

• Architect and scale robust, highly available infrastructure using cloud services and automation.
• Lead design and development of software meeting complex product and business needs with integrated monitoring.
• Diagnose operational issues and contribute to an on-call rotation in a collaborative, Agile environment.
• Drive engineering quality through tests, reviews, and improved SDLC processes focused on automation, IaC, and operations.
• Guide long-term technical strategy and observability roadmaps.
• Mentor junior engineers and support skill growth across technologies.

Skills

Git/GitHub

Atlassian tools

Telegraf

InfluxDB

Consul

LibreNMS

Docker

Kubernetes

Jenkins

Full stack: databases/web/http/backend

Alerta

Grafana

Prometheus

Mimir

Alloy

Terraform

Ansible

CloudFormation

AWS CDK

Python

Bash/Shell

AWS (EC2, Load Balancing, Lambda, SQS,

Tools

Terraform

Ansible

CloudFormation

AWS CDK

Docker

Kubernetes

Job description

JOB DETAILS

JOB BAND: D

CONTRACT TYPE: Permanent - Full Time

DEPARTMENT: Media Tech

LOCATION: London and UK Wide - Hybrid

PROPOSED SALARY RANGE: Up to £74,338 depending on relevant skills, knowledge, and experience. The expected salary range for this role reflects internal benchmarking and external market insights.

We're happy to discuss flexible working. If you'd like to, please indicate your preference in the application – though there's no obligation to do so now. Flexible working will be part of the discussion at offer stage.

PURPOSE OF THE ROLE

The BBC’s future depends on delivering seamless, 24/7 online services without disruptions. We build business-critical, highly available observability and alerting platforms for massive-scale performance tracking and corrective automation. We are migrating legacy influx systems to a modern open-source stack (Mimir, Prometheus, Alerta, Alloy, Grafana, LibreNMS) and integrating Loki and Tempo. As a Senior Engineer, you will architect, design, and develop these core cloud and network systems, underpinning our audience products and so directly preventing outages for millions of users.

WHY JOIN THE TEAM

Join an innovative, tight-knit team where your growth matters. You will collaborate across the BBC to shape our future observability roadmap, engineering critical high-scale services using AWS. This role offers total freedom to pitch ideas, innovate at a massive scale, and define our future enterprise monitoring ecosystem. Beyond delivering high-impact platforms, you will enjoy tailored learning opportunities to develop your skills alongside a supportive community of experienced engineers and our internal Academy ensuring currency in a rapidly evolving technical landscape.

Your Key Responsibilities And Impact

• Architect and scale robust, highly available, and user-focused infrastructure using cloud services and automation as well as lead the design and development of software that meets complex product and business needs which includes integrating monitoring and support tools.
• Be able to think out of the box and proactively diagnose and resolve operational issues whether triggered by system alerts or raised by stakeholders, contribute to team rituals and participate in a supportive agile on-call rotation.
• Collaborate with stakeholders to gather requirements to ensure platforms scale effectively and meet business demands as well as product and operational teams to ensure services are suitable and supportable.
• Own engineering quality by driving continuous improvement in engineering practices and processes such as appropriate tests using automated test-driven techniques, daily best practices, code reviews, enhancing workflows and processes and solid SDLC processes bringing a focus to automation, infrastructure as code, and operations.
• Guide the team’s long-term technical strategy and plan future observability roadmaps.
• Mentor and support junior team members to accelerate their professional learning and growth in current and new technologies

Essential Criteria

YOUR SKILLS AND EXPERIENCE – KEY CRITERIA

• While you do not need expertise in everything listed, expertise in some, familiarity with several and a willingness to learn is key:
• Tools and Services: Git / GitHub, Atlassian Tools, Telegraf, InfluxDB, Consul, LibreNMS, Docker, Kubernetes, Jenkins, Full stack: Database (Relational & Time series), Web / HTTP, backend
• Strong experience in Alerta, Grafana, Prometheus, Mimir, Alloy, Jenkins
• Infrastructure as Code: Terraform, Ansible, CloudFormation, AWS CDK
• Strong experience in Programming / Scripting Languages: Python, Bash/Shell.
• Strong production experience in AWS and Cloud: EC2, Load Balancing, Lambda, SQS, SNS, S3, RDS, CloudWatch, Route 53.
• Knowledge of system metrics and their impact on availability and performance.
• Experience with Linux Operating Systems (e.g. RedHat, CentOS, Rocky, Amazon Linux).
• Understanding of CI/CD practices and large-scale infrastructure design and support.

DESIRABLE CRITERIA

• Experience of high throughput metric ingestion at scale and distributed metrics storage / clustering
• Optional Network aspects such as HTTP, TLS, DNS, SNMP
• Working in an Agile development environment
• Enjoy collaborating in cross-functional teams and being comfortable asking questions, seeking feedback, and learning from others.

Disclaimer

This job description is a written statement of the essential characteristics of the job, with its principal accountabilities, incorporating a note of the skills, knowledge and experience required for a satisfactory level of performance. This is not intended to be a complete, detailed account of all aspects of the duties involved.

Please note: If you were to be offered this role, the BBC will conduct Employment screening checks which include Reference checks; Eligibility to work checks; and if applicable to the role, Safeguarding and Adverse media/Social media checks. Any offer made is conditional on these checks being satisfactory.

Before your start date, you may need to disclose any unspent convictions or police charges, in line with our Recruitment policy. This allows us to discuss any support you may need and assess any risks. Failure to disclose may result in the withdrawal of your offer.

For any general queries, please contact: [e-mail]

Redeployment

The BBC is committed to redeploying employees seeking suitable alternative employment within the BBC and they will be given priority consideration ahead of other applicants. Priority consideration means for those employees seeking redeployment their application will be considered alongside anyone else at risk of redundancy, prior to any individuals being considered who are not at risk.