Databricks Platform Engineer

Stadtwerke OELSNITZ/V. GmbH; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

Sagacity in London is seeking a Databricks Platform Engineer to design, build and operate the Databricks Lakehouse powering client analytics, predictive modelling, and GenAI work. You will work with Data Scientists, Data Engineers and the Analytics Manager to ensure reliability, governance and cost controls.

You will have 3+ years of hands-on Databricks production experience and be proficient with Spark, Delta Lake, Unity Catalog, Terraform and Git CI/CD workflows.

Qualifications

• 3+ years experience as a Data Platform/Data Engineer or Cloud Engineer with Databricks production experience.
• Strong knowledge of Databricks Lakehouse, Delta Lake, Apache Spark and Unity Catalog.
• Production experience managing Databricks on AWS, Azure or GCP including IAM and networking.
• Proficiency with Terraform and Git-based CI/CD workflows.
• Strong programming skills in Python and SQL; PySpark experience desirable.

Responsibilities

• Design, build, and operate the Databricks Lakehouse platform powering client analytics.
• Provision, configure and operate Databricks workspaces, clusters, SQL warehouses and pools.
• Implement Unity Catalog and security controls; manage CI/CD pipelines.
• Tune Spark/Delta/Photon workloads to reduce DBU and cloud spend.
• Monitor jobs and handle incident response; collaborate with data scientists and engineers.
• Support MLflow, model serving and feature store usage for predictive modelling and GenAI.
• Govern platform reliability, cost and performance across multiple cloud environments.

Skills

Databricks

Terraform

Git CI/CD

Python

SQL

Spark

Tools

Databricks

Terraform

Git

Job description

Databricks PlatformLocation: UKRole Type: Permanent - Full TimeJob Location: Hybrid Working - minimum of 2 days a week in our London office

Sagacity are the data intelligencepeople. Our proposition isbusinesses increase customerengagement and make moreinto the data businesses alreadyhave, and combine it with our data,insight, and action. Think of us asdata co-pilots for sales, marketing,ops, billing, credit and debt - clientstypically see 5x returns within thefirst few months.

Founded in 2005, we are acustomer-centric business witha World Class +95 Net PromoterScore. We believe data should beat the heart of every companyand while we are sector agnostic,primarily our clients are in thetelecoms & media, water, andnot for profit sectors.

We are a purpose led companyand we believe that purposecomes before profit. We workas one team both with clientsand internally, and are trusted todeliver quality in our standardsand in achieving successful clientoutcomes. We are open, honestand communicate in a jargonfree, collaborative way, with all ourteams being agile and curious, andcontinuously learning to achieveour purpose. We believe that ifwe achieve our purpose theneverything else will follow.

ABOUT SAGACITY

Clever with data... combining data,insight, and action to drive value

Responsibilities

Principal responsibilities will involve: Provision, configure and operate Databricks workspaces, clusters,

SQL warehouses and pools across our cloud environment Implement and maintain Unity Catalog: catalogs, schemas,

external locations, row/column-level security, lineage and audit

Build reusable Terraform modules for Databricks and automate

platform changes through CI/CD pipelines

Tune Spark, Delta and Photon workloads to improve performance

and reduce DBU and cloud spend

Manage identity, SSO/SCIM, service principals, secret scopes and

network isolation to meet security and compliance needs

Operate Databricks Workflows, Delta Live Tables and

orchestration integrations used by Data Scientists and Data

Support MLflow, model serving and feature store usage for predictive modelling and GenAI work

Monitor jobs, clusters and warehouses; lead incident response and post-incident reviews

QA own work and that of others

What Success Looks Like in the Role

A secure, reliable, well-governed Databricks platform that ouranalytics teams can rely on day to day

Reduced DBU and cloud spend through clear cluster policies,tuning and FinOps reporting, without slowing teams down

Faster, smoother delivery of client analytics and modelling work,with fewer platform-related blockers

Continued improvement of Sagacity’s Product Suite through arobust, scalable data and ML platform

Ability to take accountability and ownership for client and internal

Work as part of the Data & Analytics team, providing knowledgetransfer, peer reviews and mentoring to drive continuous learning

About the Role

You will design, build, and operate the Databricks Lakehouse platform that powers Sagacity’s client analytics, predictive modelling, and AI work. Working closely with our Data Scientists, Data Engineers, and Data & Analytics Manager, you will own platform reliability, governance, performance, and cost, enabling teams to deliver clear, insightful analytics to clients across financial services, retail, energy, telecoms & media, water, and the not for profit sector.

3+ years experience as a Data Platform Engineer, Cloud Engineeror Data Engineer, with hands-on Databricks production experience

Strong knowledge of the Databricks Lakehouse, Delta Lake, ApacheSpark and Unity Catalog

Production experience administering Databricks on at least onemajor cloud (AWS, Azure or GCP), including networking, IAM andstorage

Proficiency with Terraform (ideally the Databricks provider) and Gitbased CI/CD workflows

Solid programming skills in Python and SQL; comfortable withPySpark and shell scripting

Experience tuning Spark jobs and Delta tables (partitioning,

Working knowledge of SSO/SCIM, identity providers and secretsmanagement

Understanding of data security, encryption, network isolation andframeworks such as SOC 2, ISO 27001 or GDPR

Highly numerate with strong attention to detail

Solutions orientated with natural problem-solving skills

Self-motivated and pro-active, with an ability to multi-task, prioritise, be organised and plan your work

Excellent communication skills, both written and verbal, with a willingness to engage and influence others

A team player with a positive attitude; personable, adaptable and willingness to learn

Are flexible and self-motivated

Are good under pressure

Able to travel throughout the UK

Have the right to work in the UK

Have a commitment to personal development

Competencies and Behaviours

If you would like to join a unique working environment, with a sociableculture, where work is done a little bit differently – and we believebetter - then we look forward to hearing from you!

We achieve this through our corevalues:

One Team

Quality Delivery

Agile & Curious

Open, Honest, Simple Communication

Success in any business isultimately about its people; theirdedication and enthusiasm.

We recognise the value of ourpeople and their commitmentto working together to achievesuccessful outcomes.

At Sagacity, we: believe working with our clients results

coach and mentor our clients’teams so our data and solutions

believe in delivering benefits as

ABOUT SAGACITY

People at Sagacity