Data Engineer, Lakehouse & AI/ML Pipelines

Whitehall Resources; Cambridge; Hybrid; GBP 60,000 - 90,000; Full time

Job summary

Whitehall Resources is seeking a Data Engineer for a 15‑month contract in Cambridge with hybrid working (2–3 days onsite). You will implement scalable data processing systems, work with a data lakehouse, and collaborate to deliver data products. The role demands Python/Scala, lakehouse tech (Databricks, Spark, Kafka) and strong data governance.

IR35 inside. Responsibilities include leading data initiatives, ensuring data quality and compliance, and mentoring other engineers in a fast-paced

Qualifications

• Degree in engineering, computer science or related data-intensive field.
• Proficient in Python or Scala.
• Hands-on experience with lakehouse tech (Databricks, Spark, Kafka).
• Experience building ETL/ELT workflows in AWS.
• DataOps mindset: automation, version control, testing, monitoring, CD/CI for data systems.
• Strong communication and collaboration skills.
• Ability to present data clearly with emphasis on accuracy and quality.
• Solid understanding of data governance, security and compliance.

Responsibilities

• Build scalable, fault-tolerant data processing systems for batch and streaming workloads.
• Collaborate with engineering to own and deliver data products and data flows.
• Lead initiatives from concept to delivery and influence roadmaps.
• Ensure data quality, governance, observability and compliance across data lifecycle.
• Mentor engineers and promote engineering standards across the team.

Skills

Python

Scala

Databricks

Spark

Kafka

AWS

ETL/ELT

Data governance

MLOps

Data visualization

Education

Degree in engineering/computer science or related data-intensive field

Tools

Databricks

Spark

Kafka

Tableau/Looker/Power BI

Job description

Whitehall Resources is seeking a Data Engineer for a 15‑month contract in Cambridge with hybrid working (2–3 days onsite). You will implement scalable data processing systems, work with a data lakehouse, and collaborate to deliver data products. The role demands Python/Scala, lakehouse tech (Databricks, Spark, Kafka) and strong data governance.

IR35 inside. Responsibilities include leading data initiatives, ensuring data quality and compliance, and mentoring other engineers in a fast-paced