Lead Data Platform Engineer — Cloud, Databricks & AI

Csod; Greater London; Hybrid; GBP 95,000 - 105,000; Full time

Job summary

Csod is seeking a Lead Data Engineer to design and evolve a modern cloud-based data platform in London, with regular in-office presence in London and Eastleigh twice a month. The role focuses on trusted data products to enable analytics, ML and AI across the business.

You will mentor data engineers, build scalable pipelines and collaborate with analytics and business teams to solve data challenges while driving CI/CD and IaC best practices.

Qualifications

• Strong experience in Data Engineering and cloud-based data platforms.
• Hands-on knowledge of Databricks, Spark, SQL and Python.
• Understanding of Airflow, Terraform and modern engineering practices.
• Experience building large-scale batch and real-time data pipelines.
• Strong problem-solving and decision-making skills.
• Ability to explain technical concepts in a clear and simple way.
• Collaborative approach and enjoyment of working across teams.
• Passion for coaching, mentoring and helping others develop.
• Focus on quality, customer outcomes and continuous improvement.
• Interest in AI and emerging technologies.

Responsibilities

• Lead the development and improvement of the data platform and engineering standards.
• Mentor and support a team of data engineers to help them grow and succeed.
• Design, build and maintain scalable data pipelines and data products.
• Work with analytics, AI and business teams to solve data challenges.
• Improve platform reliability, performance, security and cost efficiency.
• Drive best practice in testing, automation, CI/CD and Infrastructure as Code.
• Support the delivery of trusted data for reporting, modelling and AI use cases.
• Help shape data strategy and assess new technologies and tools.
• Promote continuous improvement and knowledge sharing across teams.
• Ensure data governance and compliance requirements are built into solutions.

Skills

Databricks

Spark

SQL

Python

Airflow

Terraform

Data pipelines

Tools

Airflow

Terraform

Job description

Csod is seeking a Lead Data Engineer to design and evolve a modern cloud-based data platform in London, with regular in-office presence in London and Eastleigh twice a month. The role focuses on trusted data products to enable analytics, ML and AI across the business.

You will mentor data engineers, build scalable pipelines and collaborate with analytics and business teams to solve data challenges while driving CI/CD and IaC best practices.