Databricks Platform Engineer

Stichting Codam; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Sagacity is seeking a Data Platform Engineer to design, build and operate the Databricks Lakehouse platform powering client analytics, predictive modelling, and AI work. You will collaborate with Data Scientists and Data Engineers to ensure reliability, governance, performance and cost control across financial services, retail, energy, telecoms and not-for-profit sectors.

You will contribute to secure, scalable data workflows, participate in FinOps, and help drive value through resilient

Qualifications

• 3+ years experience as a Data Platform Engineer, Cloud Engineer or Data Engineer with hands-on Databricks production experience.
• Strong knowledge of the Databricks Lakehouse, Delta Lake, Apache Spark and Unity Catalog.
• Production experience administering Databricks on AWS, Azure or GCP, including networking, IAM and storage.
• Proficiency with Terraform (Databricks provider) and Git-based CI/CD workflows.
• Solid programming skills in Python and SQL; comfortable with PySpark and shell scripting.
• Experience tuning Spark jobs and Delta tables (partitioning,).
• Working knowledge of SSO/SCIM, identity providers and secrets management.
• Understanding of data security, encryption, network isolation and frameworks such as SOC 2, ISO 27001 or GDPR.
• Highly numerate with strong attention to detail.
• Solutions orientated with natural problem-solving skills.
• Self-motivated and pro-active, with an ability to multi-task, prioritise, be organised and plan your work.
• Excellent communication skills, both written and verbal, with a willingness to engage and influence others.
• A team player with a positive attitude; personable, adaptable and willingness to learn.
• Are flexible and self-motivated; Are good under pressure; Able to travel throughout the UK; Have the right to work in the UK.

Responsibilities

• Provision, configure and operate Databricks workspaces, clusters, SQL warehouses and pools across our cloud environment.
• Implement and maintain Unity Catalog: catalogs, schemas, external locations, row/column-level security, lineage and audit.
• Build reusable Terraform modules for Databricks and automate platform changes through CI/CD pipelines.
• Tune Spark, Delta and Photon workloads to improve performance and reduce DBU and cloud spend.
• Manage identity, SSO/SCIM, service principals, secret scopes and network isolation to meet security and compliance needs.
• Operate Databricks Workflows, Delta Live Tables and orchestration integrations used by Data Scientists and Data & Analytics.
• Support MLflow, model serving and feature store usage for predictive modelling and GenAI work.
• Monitor jobs, clusters and warehouses; lead incident response and post-incident reviews.
• QA own work and that of others.

Job description

Job Information

Databricks Platform Location: UK Role Type: Permanent - Full Time Job Location: Hybrid Working - minimum of 2 days a week in our London office

ABOUT SAGACITY

Sagacity are the data intelligencepeople. Our proposition isbusinesses increase customerengagement and make moreinto the data businesses alreadyhave, and combine it with our data,insight, and action. Think of us asdata co-pilots for sales, marketing,ops, billing, credit and debt - clientstypically see 5x returns within thefirst few months.

Founded in 2005, we are acustomer-centric business witha World Class +95 Net PromoterScore. We believe data should beat the heart of every companyand while we are sector agnostic,primarily our clients are in thetelecoms & media, water, andnot for profit sectors.

We are a purpose led companyand we believe that purposecomes before profit. We workas one team both with clientsand internally, and are trusted todeliver quality in our standardsand in achieving successful clientoutcomes. We are open, honestand communicate in a jargonfree, collaborative way, with all ourteams being agile and curious, andcontinuously learning to achieveour purpose. We believe that ifwe achieve our purpose theneverything else will follow.

Clever with data... combining data,insight, and action to drive value

Responsibilities

Principal responsibilities will involve:

• Provision, configure and operate Databricks workspaces, clusters,
• SQL warehouses and pools across our cloud environment
• Implement and maintain Unity Catalog: catalogs, schemas, external locations, row/column-level security, lineage and audit
• Build reusable Terraform modules for Databricks and automate platform changes through CI/CD pipelines
• Tune Spark, Delta and Photon workloads to improve performance and reduce DBU and cloud spend
• Manage identity, SSO/SCIM, service principals, secret scopes and network isolation to meet security and compliance needs
• Operate Databricks Workflows, Delta Live Tables and orchestration integrations used by Data Scientists and Data
• Support MLflow, model serving and feature store usage for predictive modelling and GenAI work
• Monitor jobs, clusters and warehouses; lead incident response and post-incident reviews
• QA own work and that of others

What Success Looks Like in the Role

• A secure, reliable, well-governed Databricks platform that ouranalytics teams can rely on day to day
• Reduced DBU and cloud spend through clear cluster policies,tuning and FinOps reporting, without slowing teams down
• Faster, smoother delivery of client analytics and modelling work,with fewer platform-related blockers
• Continued improvement of Sagacity’s Product Suite through arobust, scalable data and ML platform
• Ability to take accountability and ownership for client and internal
• Work as part of the Data & Analytics team, providing knowledgetransfer, peer reviews and mentoring to drive continuous learning

About the Role

You will design, build, and operate the Databricks Lakehouse platform that powers Sagacity’s client analytics, predictive modelling, and AI work. Working closely with our Data Scientists, Data Engineers, and Data & Analytics Manager, you will own platform reliability, governance, performance, and cost, enabling teams to deliver clear, insightful analytics to clients across financial services, retail, energy, telecoms & media, water, and the not for profit sector.

Requirements

• 3+ years experience as a Data Platform Engineer, Cloud Engineeror Data Engineer, with hands-on Databricks production experience
• Strong knowledge of the Databricks Lakehouse, Delta Lake, ApacheSpark and Unity Catalog
• Production experience administering Databricks on at least onemajor cloud (AWS, Azure or GCP), including networking, IAM and storage
• Proficiency with Terraform (ideally the Databricks provider) and Gitbased CI/CD workflows
• Solid programming skills in Python and SQL; comfortable with PySpark and shell scripting
• Experience tuning Spark jobs and Delta tables (partitioning,
• Working knowledge of SSO/SCIM, identity providers and secretsmanagement
• Understanding of data security, encryption, network isolation andframeworks such as SOC 2, ISO 27001 or GDPR
• Highly numerate with strong attention to detail
• Solutions orientated with natural problem-solving skills
• Self-motivated and pro-active, with an ability to multi-task, prioritise, be organised and plan your work
• Excellent communication skills, both written and verbal, with a willingness to engage and influence others
• A team player with a positive attitude; personable, adaptable and willingness to learn
• Are flexible and self-motivated
• Are good under pressure
• Able to travel throughout the UK
• Have the right to work in the UK
• Have a commitment to personal development

Competencies and Behaviours

If you would like to join a unique working environment, with a sociableculture, where work is done a little bit differently – and we believe‘better’ - then we look forward to hearing from you!

Core Values

We achieve this through ourcore values:

• One Team
• Quality Delivery
• Agile & Curious
• Open, Honest, Simple Communication

Success in any business isultimately about its people; theirdedication and enthusiasm.

We recognise the value of ourpeople and their commitmentto working together to achievesuccessful outcomes.

At Sagacity, we

• believe working with our clients
• coach and mentor our clients’ teamsso our data and solutions live on
• believe in delivering benefits as we

People at Sagacity

People at Sagacity