Senior Platform Engineer (Databricks)

Retail Insight; Greater London; Hybrid; GBP 85,000 - 110,000; Full time

Benefits offered by this job

Hybrid working (2 days in office)

25 days annual leave

Learning & development access

Private medical insurance & wellbeing

Volunteer days & charity matching

Car purchase scheme & discounts

Job summary

Retail Insight is seeking a Senior Platform Engineer to join the Data & Engineering team and advance the Databricks platform that underpins our analytics, ML, and AI initiatives. You will design, build and optimise the data platform powering our products and services, collaborating with Platform, Data and Product teams to deliver secure, high-performance data solutions.

In this senior, hands-on role you’ll tackle complex challenges, work with modern cloud technologies and contribute to

Qualifications

• Proven experience building and supporting scalable, production-grade data platforms.
• Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL.
• Good understanding of cloud-based data engineering, ideally within Azure environments.
• Experience working with Infrastructure as Code tools such as Terraform, Bicep or similar.
• Solid understanding of data modelling, ETL/ELT processes, data quality, security and governance.
• Comfortable working with technical and non-technical stakeholders to deliver high-quality solutions.

Responsibilities

• Design, build and enhance our Databricks platform, ensuring it remains secure, scalable and reliable.
• Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake.
• Contribute to platform architecture, engineering standards and best practices across the data estate.
• Collaborate with Product, Analytics and Data Science teams to deliver solutions that solve real business problems.
• Support platform operations, monitoring and optimisation to ensure high levels of performance and reliability.
• Share knowledge with colleagues and help drive continuous improvement across the engineering team.

Skills

Databricks

Spark

Delta Lake

Python

SQL

Azure

Terraform

Bicep

Data modelling

ETL/ELT

Data quality

Security and governance

Stakeholder management

Education

Degree in Computer Science or Data Engineering or related field

Tools

Terraform

Bicep

dbt

Kafka

Event Hubs

Job description

Help build the platform at the heart of Retail Insight's data strategy

We're looking for a Senior Platform Engineer to join our Data & Engineering team and play a key role in the development and evolution of our Databricks platform.

This is a senior, hands-on engineering role where you'll design, build and optimise the data platform that powers our products and services. Working alongside Platform, Data and Product teams, you'll help deliver scalable, secure and high-performance data solutions that enable analytics, machine learning and AI across the business.

If you enjoy solving complex technical challenges, working with modern cloud technologies and building robust data platforms at scale, we'd love to hear from you.

What you'll be doing

• Design, build and enhance our Databricks platform, ensuring it remains secure, scalable and reliable.
• Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake.
• Contribute to platform architecture, engineering standards and best practices across the data estate.
• Collaborate with Product, Analytics and Data Science teams to deliver solutions that solve real business problems.
• Support platform operations, monitoring and optimisation to ensure high levels of performance and reliability.
• Share knowledge with colleagues and help drive continuous improvement across the engineering team.

Our ideal candidate

• Proven experience building and supporting scalable, production-grade data platforms.
• Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL.
• Good understanding of cloud-based data engineering, ideally within Azure environments.
• Experience working with Infrastructure as Code tools such as Terraform, Bicep or similar.
• Solid understanding of data modelling, ETL/ELT processes, data quality, security and governance.
• Comfortable working with technical and non-technical stakeholders to deliver high-quality solutions.

Even better if you have

• Databricks certifications.
• Experience with Unity Catalog, MLflow or Databricks Asset Bundles.
• Familiarity with dbt or similar data transformation frameworks.
• Exposure to Kafka, Event Hubs or other streaming technologies.
• Experience working with retail or consumer packaged goods data, including point-of-sale, inventory or supply-chain datasets.
• A degree in Computer Science, Data Engineering or a related discipline, or equivalent practical experience.

Some of our extras

• Flexible Working - Enjoy a hybrid work model (typically 2 days in the office) with flexibility based on business needs, plus a work from anywhere policy to give you freedom to explore.
• Time Off - 25 days annual leave (+ bank holidays), increasing with length of service, plus an extra day off for your birthday! We also operate summer hours so you can make the most of the sunshine.
• Learning & Development - Access a vast range of courses through our learning platform and benefit from structured career progression plans to support your growth.
• Health & Wellbeing - Private Medical Insurance, a healthcare cash plan, and mental health support via Help@Hand. Plus, we'll ensure you have a safe and productive home setup with a workspace assessment.
• Giving Back - Take paid volunteer days to support your local community, donate to your chosen charity through salary sacrifice (we'll match it!), and make a difference with Give as You Earn.
• Extra Perks - A car purchase scheme to make buying a new car easier, plus access to additional benefits through our online platform, including gym and tech discounts.

Plus much more!

Our interview process

Our interview process will vary slightly depending on the role. We'll always let you know what to expect at each stage and give you plenty of opportunity to ask questions along the way.

People Team Intro Call

An initial conversation with a member of our People Team to learn more about you, your experience and what you're looking for in your next role. We'll also tell you more about Retail Insight, the opportunity and what you can expect from the interview process.

Skills-Based Interview

This is an opportunity to explore the skills most relevant to the position and understand how you approach your work. This may include a practical task, presentation or technical assessment, and we'll always explain what's involved in advance.

Values-Based Interview

Our final stage focuses on how you work, collaborate and approach different situations, aligned with our values and ways of working at Retail Insight. It's also another opportunity for you to get to know us and make sure Retail Insight feels like the right fit for you.

Be your authentic self

Retail Insight is committed to promoting equal opportunities in employment. All employees and job applicants will receive equal treatment. We actively seek to create an environment where everyone feels respected, supported and encouraged to contribute their best work.