Senior Data Engineer: Scalable Cloud Data Pipelines

BBC Group and Public Services; Salford, Greater London; Hybrid; GBP 55,000 - 65,000; Full time

Job summary

The BBC Group and Public Services is seeking a Senior Data Engineer to design, build and operate scalable cloud data pipelines and data products. The role is based in Salford or London with a hybrid working pattern, and offers a salary range of £55,000 to £65,000 depending on skills and experience.

You will collaborate with analytics, data science, marketing, brand and product teams to enable audience insight and decision making, while upholding privacy, governance and security standards across

Qualifications

• Strong experience designing, building and maintaining scalable data pipelines and integrations using Apache Airflow, Python, SQL and bash/linux.
• Experience developing cloud-native data platforms on AWS, including data warehouses such as Amazon Redshift and data modelling best practices.
• Good understanding of software engineering principles, including object-oriented development, testing practices, version control, CI/CD deployment, containerisation, and LLM-assisted coding.
• Experience delivering data integrations for CDPs and external data sources, including technologies such as mParticle, Twilio Segment, or similar.
• Desirable: knowledge of layered data architecture patterns (Medallion Architecture, Data Warehouse or Lakehouse), Streamlit/Shiny, Hadoop/Spark, and Infrastructure as Code tools like CloudFormation/Terraform

Responsibilities

• Design, build and maintain scalable cloud-based data pipelines and data products supporting audience growth, CRM activation, marketing measurement and analytics.
• Build and maintain integrations between BBC Account, mParticle, CRM platforms, marketing technologies, advertising platforms and enterprise reporting systems.
• Develop robust API-based integrations to enable secure data exchange across marketing, product, audience and analytics platforms.
• Collaborate with analysts, data scientists, engineers, product managers and stakeholders to identify opportunities for automation and improved decision making.
• Ensure all solutions comply with BBC standards for privacy, data governance, security, consent management and international regulatory compliance.

Skills

Apache Airflow

Python

SQL

Bash/Linux

AWS

Redshift

Data modelling

CI/CD

Containerisation

LLM-assisted coding

CDP integrations

mParticle

Twilio Segment

Tools

CloudFormation

Terraform

Kubernetes

Streamlit

Shiny

Job description

The BBC Group and Public Services is seeking a Senior Data Engineer to design, build and operate scalable cloud data pipelines and data products. The role is based in Salford or London with a hybrid working pattern, and offers a salary range of £55,000 to £65,000 depending on skills and experience.

You will collaborate with analytics, data science, marketing, brand and product teams to enable audience insight and decision making, while upholding privacy, governance and security standards across