Senior Data Engineer

The BDPN; Greater London; Hybrid; GBP 51,000 - 69,000; Full time

Benefits offered by this job

Flexible benefits

Hybrid work options

Annual bonus

Pension scheme

Employee discount

Job summary

Allianz in London is seeking a Senior Data Engineer to design, build and operate production-grade data and ML services, including real-time endpoints serving 10 million+ requests. You will deploy, scale, and monitor containerised apps using Docker and Kubernetes, and construct cloud solutions with Terraform.

The role emphasizes CI/CD automation via Azure DevOps or GitHub Actions, and strong Python for data pipelines.

Qualifications

• Experience deploying, monitoring and maintaining production services.
• Containerisation (Docker) & Service orchestration (Kubernetes or similar).
• Managing and maintaining real-time endpoints or APIs.
• Experience building cloud infrastructure for data or predictive services (Terraform; Azure preferred).
• CI/CD tooling experience (Azure DevOps or GitHub Actions).
• Familiarity with data governance and GDPR requirements.

Responsibilities

• Design, build, and operate production-grade data/ML services with real-time endpoints.
• Build, deploy, and orchestrate Docker containers for scalable workloads.
• Design, build, and maintain cloud solutions using Infrastructure as Code (Terraform).
• Develop monitoring to ensure reliability, performance and security of ML systems.
• Create CI/CD pipelines to automate workflows with Azure DevOps or GitHub Actions.
• Write clean, well-tested Python data pipelines and features for analytics.

Skills

Python

Agile

Communication

GDPR & Data governance

LLMs familiarity

Insurance/Financial services domain

Tools

Docker

Kubernetes

Terraform

Azure DevOps

GitHub Actions

Job description

As a Senior Data Engineer, you will play a key role in building and maintaining production‑ready data and machine‑learning systems that support critical business decisions. You’ll work acrossthe full lifecycle, from engaging with stakeholders and shaping data/model outputs, to deploying and maintaining real‑time services in a cloud environment.

A significant part of this role involves developing and evolving our cloud‑based infrastructure, including containerised applications, automated deployment pipelines, and infrastructure‑as‑code. You’ll also help shape engineering best practices and support the growth of modern, scalable data platforms.

You’ll join one of our small stream‑aligned agile teams, working closely with domain experts to understand business needs and deliver high‑impact solutions. Teams rotate gradually to broaden your exposure to new data challenges and technical domains.

Location

This role is hybrid, with the option to work from our London, Bristol, or Bournemouth offices. You’ll be required to attend the office two days per month.

Salary Information

Pay: Circa £60,000 per annum, dependent on experience, skills, and location.

Salary is one part of our wider total reward package.

What You’ll Do

Design, build, and operate production‑level data and machine‑learning services, including real‑time API endpoints that serve 10 million + requests daily

Build and deploy and orchestrate docker containers, optimizing for performance and resource utilization

Design, build and maintain cloud R&D solutions using IaC tools like Terraform

Building, improving and maintaining monitoring systems to track model performance, infrastructure health and ensure reliability, scalability, and security of ML systems

Develop CICD processes to help automateworkflows using tools like Azure DevOps

Write high‑quality, well‑tested Python for production‑grade data pipelines and services.

Deliver data extracts, transformations, and features to support modelling and analytics.

Workwithin an agile workflow, managing your ownticketsand collaborating with team members and across disciplines to deliver productsCommunicate technical decisions and findings to both technical and non‑technical audiences.

Taking time tostay up to date with the newestcloudanddatatechasyou contribute to continuous improvement across cloud tooling, engineering standards, and platform development.

About You (What We’re Looking For)

1. Experience deploying, monitoring and maintaining production services

We’re looking for engineers who have worked hands‑on with:

Containerisation (Docker) & Service orchestration (Kubernetes or similar)

Managing and maintaining real‑time endpoints or APIs

This is the most important area of experience for this role.

Experience building or supporting cloud infrastructure for data or predictive services, ideally including:

Terraform (highly desirable)

Azure (preferred), or other public cloud platforms

Infrastructure patterns for scalable, secure services

3. CI/CD Automation

Experience with:

Azure DevOps or GitHub Actions

Automated testing, packaging, and deployment processes

CI/CD experience is valuable, but not sufficient on its own without cloud or production‑deployment exposure.

We are not looking for profiles focused only on ETL.

However, we will consider candidates who bring Cloud + CI/CD experience alongside technologies such as:

Proficient SQL; familiarity with PySpark is beneficial

dbt

Microsoft Fabric

SQLMesh

5. Core Engineering Skills

Strong Python, with good documentation and unit testing practices

Comfortable in an agile delivery environment

Able to communicate clearly with non‑technical teams

Demonstrates curiosity, continuous learning, and a collaborative mindset

Understanding of GDPR and data governance

Experience in insurance or financial services

Familiarity with LLMs (e.g., GPT, OpenAI)

Knowledge of infrastructure monitoring, backup, or disaster recovery

What We Will Offer You

Recognised and rewarded for a job well done, we have a range of flexible benefits for you to choose from- so you can pick a package that’s perfect for you. We also offer flexible working options, global career opportunities across the wider Allianz Group, and fantastic career development and training. That’s on top of enjoying all the benefits you’d expect from the world’s number one insurance brand, including:

Flexible buy/sell holiday options

Annual performance related bonus

Contributory pension scheme

A discount up to 50% on a range of insurance products including car, home and pet

Our Ways of Working

Do you need flexibility with the hours you work? Let us know as part of your application and if it’s right for our customers, our business and for you, then we’ll do everything we can to make it happen.Here at Allianz, we are signatories of the ABIs flexible working charter. We believe in supporting hybrid work patterns, which balance the needs of our customers, with your personal circumstances and our business requirements. Our aim with this is to help innovation, creativity, and you to thrive - Your work life balance is important to us.

Integrity, Fairness, Inclusion & Trust

At Allianz, we believe in fostering an inclusive workforce and are proud to be an equal opportunity employer. Our commitment to equal opportunities, gender equity, and balanced gender representation, is demonstrated by our numerous accreditations: EDGE certified for gender inclusion, Women in Finance Charter members, Disability Confident employer, Stonewall Diversity Champion, Business in the Community’s Race at Work Charter signatories, and Armed Forces Covenant gold standard employer.

We embrace neurodiversity and welcome applications from neurodivergent and disabled candidates, offering tailored adjustments to ensure your success.

We encourage our employees to advocate for their needs, whether it’s assistive technology, ergonomic equipment, mentoring, coaching, or flexible work arrangements

Accessible Application for All

As part of the Disability Confident Scheme, we support candidates with disabilities or long‑term health conditions through the Offer an Interview Scheme, for those meeting the essential skills for the role.

Contact our Resourcing team to opt into this scheme or for assistance with your application, including larger text, hard copies, or spoken applications. [e-mail]

For any inquiries or to submit your application, please contact: Georgie Hill

If you are an at‑risk candidate facing potential redeployment, please include this information in your CV.