Lead Data Platform Engineer - AWS, Spark

Experian Information Solutions, Inc.

City Of London

Hybrid

GBP 90,000 - 130,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Great compensation package
Discretionary bonus plan
25 days annual leave
Pension
Bupa healthcare
Sharesave scheme

Job summary

Experian is seeking a Lead Data Platform Engineer to provide hands-on technical leadership for a fraud data capability within the Data Analytics Platform. You will guide a team to deliver secure, scalable data pipelines and cloud components, collaborating with central platform teams and data owners.

The role combines data engineering depth with cloud platform experience, DevSecOps discipline and mentoring across engineers, with a hybrid Nottingham-based work pattern (40% in the office).

Qualifications

  • Experience building data pipelines and data-lake solutions or data platform capabilities.
  • Experience with AWS data engineering services (Glue, S3, Step Functions, IAM, Lake Formation).
  • Proficiency in Scala or Python for data processing.
  • DevSecOps practices and Infrastructure as Code (Terraform, CI/CD).
  • Understanding cloud security, governance, and data privacy in platforms.

Responsibilities

  • Lead and support an agile technical team delivering data platform solutions for fraud data assets.
  • Guide the delivery of AWS-based data pipelines and data-lake components.
  • Oversee handover, ownership, maintenance and enhancements of pipelines from central DAP team.
  • Lead building, testing and maintaining data pipelines from Snowflake and internal platforms.
  • Ensure metadata is published and surfaced in the DAP catalogue.
  • Champion DevSecOps, CI/CD, IaC and cloud-native tooling to improve reliability and security.
  • Mentor and coach engineers, promoting learning and collaboration.

Skills

Data pipelines
AWS data services
DevOps / IaC
Spark / Hadoop
Scala or Python
API integration
Cloud security & governance
Mentoring

Tools

Terraform
CI/CD
Git
Jenkins
OpenShift
Kubernetes
Snowflake
S3
Lambda
EC2

Job description

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries.

Internal Grade

Internal Grade D/EB8

Job Description

Experian Credit Services & Verifications are looking for a Lead Data Platform Engineer to provide hands-on technical leadership for a new fraud data capability that will connect fraud-related data assets into Experian's Data Analytics Platform. Reporting into our Engineering Director, you will lead a technical team that supports innovation, learning and collaboration. Ensuring that the team follows the highest technical standards. You will work closely with the central Data Analytics Platform team, data owners, security, compliance, platform and delivery stakeholders to agree practical implementation approaches, align with established platform patterns, and ensure the team delivers secure, scalable and supportable solutions. This is a hands-on leadership role suited to someone who can combine strong data engineering depth with cloud platform experience, engineering discipline and team leadership. You will apply effective DevSecOps practices and contribute to technical roadmap discussions where there are shared or overlapping platform requirements, while working collaboratively with central platform teams where broader architectural ownership sits outside the local team.

This is a Hybrid, Nottingham based role (40% in the office).

Key Responsibilities
  • Lead and support an agile technical team delivering data platform solutions for fraud-related data assets.
  • Help guide the delivery of AWS-based data pipelines, data-lake components and producer-side data platform capability.
  • Support the handover, ownership, maintenance and enhancement of pipelines and cloud components initially developed by the central DAP team.
  • Lead the team in building, testing and maintaining high-quality data pipelines from source systems such as Snowflake and internal product/data platforms.
  • Ensure metadata is prepared and published accurately so assets can be surfaced through the DAP catalogue.
  • Champion DevSecOps practices, CI/CD, Infrastructure as Code and cloud-native tooling to improve reliability, security and delivery efficiency.
  • Mentor and coach senior and junior engineers, supporting innovation, learning and collaboration within the team.
  • Promote modern engineering practices, including automation and AI-assisted development where appropriate, to improve delivery speed, quality and repeatability.
Qualifications
  • Strong experience building data pipelines, data-lake solutions or data platform capabilities.
  • Strong experience with AWS cloud services relevant to data engineering and platform delivery, including services such as AWS Glue, S3, Step Functions, IAM, Lake Formation, EC2, Lambda, EventBridge and SQS.
  • Strong experience with DevOps and Infrastructure as Code tooling, including Terraform, CI/CD pipelines, Git and Jenkins.
  • Strong experience developing on Apache Spark or Hadoop.
  • Significant programming experience using Scala or Python.
  • Experience with Configuration as Code principles and API integration.
  • Strong understanding of cloud security, data privacy, access control and governance principles in data platform environments.
Desired Skills
  • AWS Developer Certification or other relevant AWS certification.
  • Strong expertise in Terraform or CloudFormation.
  • Experience with OpenShift or Kubernetes.
  • Experience with automated testing tools such as pytest or Scalatest.
  • Familiarity with code quality and security scanning tools such as SonarQube and Veracode.
  • Knowledge of RESTful APIs and microservice architectures.
  • Experience working with sensitive, governed or regulated data environments.
  • Familiarity with data catalogue, metadata management, data-marketplace or federated data-mesh concepts.
Additional Information

Benefits package includes:

  • Great compensation package and discretionary bonus plan
  • 25 days annual leave with 8 bank holidays and 3 volunteering days. You can purchase additional annual leave.
  • Core benefits include pension, bupa healthcare, sharesave scheme and more.

Our uniqueness is that we celebrate yours. Experian's people first, inclusive and purpose driven culture is multi award-winning; World's Best Workplaces 2025 (Fortune Global Top 25), Great Place To Work in 26 countries to name a few. Check out Experian Life on social or explore our Careers Site to understand why. Experian is also proud to be an Equal Opportunity and affirmative Action employer. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.

Recruitment Fraud Awareness

Recruitment process is conducted only through authorised channels. Recruitment communications will only be sent from an @experian.com email address. Experian will never ask candidates to make any payment as part of an application, interview, assessment, onboarding, or recruitment process.

Summary

Type: Full-time Function: Engineering Experience level: Not Applicable Industry: Financial Services

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Platform Engineer - AWS, Spark
Lead Data Platform Engineer - AWS, Spark

Experian • Nottingham

Hybrid
GBP 90,000 - 130,000
Discretionary bonus
25 days annual leave plus buy/sell
Pension and private healthcare
Senior Data Platform Engineer AWS, Spark
Senior Data Platform Engineer AWS, Spark

Experian Information Solutions, Inc. • City Of London

Hybrid
GBP 75,000 - 110,000
Pension
Healthcare (Bupa)
Sharesave scheme
+2
Lead Data Platform Engineer - AWS, Spark, DevSecOps
Lead Data Platform Engineer - AWS, Spark, DevSecOps

Experian Information Solutions, Inc. • City Of London

Hybrid
GBP 90,000 - 130,000
Great compensation package
Discretionary bonus plan
25 days annual leave
+3
Lead Data Platform Engineer — AWS/Spark, Hybrid (Nottingham)
Lead Data Platform Engineer — AWS/Spark, Hybrid (Nottingham)

Experian • Nottingham

Hybrid
GBP 90,000 - 130,000
Discretionary bonus
25 days annual leave plus buy/sell
Pension and private healthcare
Senior Data Platform Engineer (AWS Spark) – Fraud Data
Senior Data Platform Engineer (AWS Spark) – Fraud Data

Experian Information Solutions, Inc. • City Of London

Hybrid
GBP 75,000 - 110,000
Pension
Healthcare (Bupa)
Sharesave scheme
+2
Financial Crime Compliance Delivery Lead
Financial Crime Compliance Delivery Lead

Experian • Nottingham

Hybrid
GBP 65,000 - 90,000
Great compensation and discretionary bonus
Core benefits include pension and healthcare
25 days annual leave with bank holidays
+3
Senior Fullstack Engineer - Typescript - Python
Senior Fullstack Engineer - Typescript - Python

Experian • United Kingdom

Hybrid
GBP 70,000 - 110,000
Hybrid-based role (40% office)
25 days annual leave
Pension
+2
Senior Insider Risk Specialist
Senior Insider Risk Specialist

Experian • Nottingham

Hybrid
GBP 70,000 - 100,000
Pension
Healthcare (Bupa)
Sharesave
+2
Lead Data Engineer
Lead Data Engineer

Forsyth Barnes • Greater London

Hybrid
GBP 70,000 - 90,000
Senior FullStack Engineer - TypeScript - Python
Senior FullStack Engineer - TypeScript - Python

Experian • Greater London

Hybrid
GBP 65,000 - 90,000
Hybrid work
Bonus plan
Pension & healthcare
+1