Hybrid Data Platform Lead - AWS Spark & Data Pipelines

JobCubby; Nottingham; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Pension

Bupa healthcare

Sharesave scheme

25 days annual leave

Job summary

Experian Credit Services & Verifications in Nottingham seeks a Lead Data Platform Engineer for a 6-month fixed-term contract. Hybrid role with 40% in-office work, reporting to Engineering Director, leading an Agile team to build data pipelines and a data lake while upholding high technical standards.

You will work with AWS, Spark/Hadoop, and DevOps tooling to deliver scalable solutions, ensuring GDPR/PCI DSS compliance and collaboration across risk and security teams.

Qualifications

• DevOps & IaC tooling: Terraform, CI/CD pipelines, Git, Jenkins, Harness.
• Apache Spark / Hadoop development experience.
• AWS cloud services: Glue, S3, Step Functions.
• Data pipelines and data lake construction expertise.
• Lifecycle tools: Bitbucket & Jira.
• Scala or Python programming with SOLID principles.

Responsibilities

• Lead an Agile technical team delivering high-value data platform work.
• Build data pipelines and data lake components.
• Deliver quality software with secure, compliant practices.
• Collaborate with risk, security and compliance teams on standards.
• Identify overlapping tech needs and help roadmap technology.
• Own codebase and technical application ownership.

Skills

Terraform

CI/CD pipelines

Git

Jenkins

Harness

Apache Spark

Hadoop

AWS Glue

Amazon S3

AWS Step Functions

Data pipelines

Data lake

Bitbucket

Jira

Scala

Python

SOLID principles

Clean code

Data analysis

AWS Lambda

EventBridge

SQS

Lake Formation

Automated testing

Tools

Jira

Bitbucket

Jenkins

Job description

Experian Credit Services & Verifications in Nottingham seeks a Lead Data Platform Engineer for a 6-month fixed-term contract. Hybrid role with 40% in-office work, reporting to Engineering Director, leading an Agile team to build data pipelines and a data lake while upholding high technical standards.

You will work with AWS, Spark/Hadoop, and DevOps tooling to deliver scalable solutions, ensuring GDPR/PCI DSS compliance and collaboration across risk and security teams.