Senior Platform Engineer: Build a Scalable Databricks Platform

Retail Insight; Greater London; Hybrid; GBP 85,000 - 110,000; Full time

Benefits offered by this job

Hybrid working (2 days in office)

25 days annual leave

Learning & development access

Private medical insurance & wellbeing

Volunteer days & charity matching

Car purchase scheme & discounts

Job summary

Retail Insight is seeking a Senior Platform Engineer to join the Data & Engineering team and advance the Databricks platform that underpins our analytics, ML, and AI initiatives. You will design, build and optimise the data platform powering our products and services, collaborating with Platform, Data and Product teams to deliver secure, high-performance data solutions.

In this senior, hands-on role you’ll tackle complex challenges, work with modern cloud technologies and contribute to

Qualifications

• Proven experience building and supporting scalable, production-grade data platforms.
• Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL.
• Good understanding of cloud-based data engineering, ideally within Azure environments.
• Experience working with Infrastructure as Code tools such as Terraform, Bicep or similar.
• Solid understanding of data modelling, ETL/ELT processes, data quality, security and governance.
• Comfortable working with technical and non-technical stakeholders to deliver high-quality solutions.

Responsibilities

• Design, build and enhance our Databricks platform, ensuring it remains secure, scalable and reliable.
• Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake.
• Contribute to platform architecture, engineering standards and best practices across the data estate.
• Collaborate with Product, Analytics and Data Science teams to deliver solutions that solve real business problems.
• Support platform operations, monitoring and optimisation to ensure high levels of performance and reliability.
• Share knowledge with colleagues and help drive continuous improvement across the engineering team.

Skills

Databricks

Spark

Delta Lake

Python

SQL

Azure

Terraform

Bicep

Data modelling

ETL/ELT

Data quality

Security and governance

Stakeholder management

Education

Degree in Computer Science or Data Engineering or related field

Tools

Terraform

Bicep

dbt

Kafka

Event Hubs

Job description

Retail Insight is seeking a Senior Platform Engineer to join the Data & Engineering team and advance the Databricks platform that underpins our analytics, ML, and AI initiatives. You will design, build and optimise the data platform powering our products and services, collaborating with Platform, Data and Product teams to deliver secure, high-performance data solutions.

In this senior, hands-on role you’ll tackle complex challenges, work with modern cloud technologies and contribute to