Databricks Engineer

LA International; Wembley; On-site; GBP 70,000 - 110,000; Full time

Job summary

LA International is seeking a Databricks Engineer to start within weeks. You will architect Databricks Clean Rooms, configure Unity Catalog sharing, and implement robust privacy controls across clouds. Expect to build notebooks, enforce governance, and collaborate with legal and security teams on data-sharing requirements.

The role requires 2+ years on Databricks, 6+ years in data/platform engineering, and will involve travel to Wembley twice weekly. IR35, on-site in the UK.

Qualifications

• Strong Unity Catalog, Delta Lake, and Delta Sharing experience.
• Solid SQL and Python, plus Terraform or other infrastructure-as-code.
• Understanding of privacy-enhancing techniques and regulations (GDPR, UK GDPR, CCPA, sector rules)
• Experience with IAM, network security, and audit on AWS or Azure

Responsibilities

• Architect and configure Databricks Clean Rooms across clouds and workspaces.
• Set up Unity Catalog metastores, catalogs and sharing for controlled access.
• Build and approve clean room notebooks and queries (SQL, Python) for analytics and ML scoring.
• Apply privacy controls: row filters, column masks, thresholds, pseudonymisation.
• Define onboarding, output approval workflows and revocation processes.
• Configure audit logging, lineage and monitoring for each run.
• Coordinate with legal/privacy/security to translate data-sharing into controls.
• Produce architecture diagrams, runbooks and reference patterns for reuse.

Skills

Unity Catalog

Delta Lake

Delta Sharing

SQL

Python

Terraform

Infrastructure as Code

Privacy regs

IAM & security

AWS / Azure

Tools

Terraform

Job description

Databricks Engineer

We are looking for a Databricks engineer who can start a new assignment in the next couple of weeks. You will have at least 2 years of experience on Databricks and more than 6 years in data or platform engineering.

• Architect and configure Databricks Clean Rooms (Delta Sharing-based) across clouds and workspaces, including cross-cloud and cross-region collaboration
• Set up Unity Catalog metastores, catalogs, and sharing so only approved tables, views, and volumes enter each clean room
• Build and approve clean room notebooks and queries (SQL, Python) for joint analytics, audience overlap, and ML scoring
• Apply privacy controls: row filters, column masks, aggregation thresholds, and pseudonymisation or hashing of join keys
• Define collaborator onboarding, output approval workflows, and revocation processes
• Configure audit logging, lineage, and monitoring for every clean room run
• Work with legal, privacy, and security teams to translate data-sharing agreements into technical controls
• Produce architecture diagrams, runbooks, and reference patterns for reuse

Skills required:

• Strong Unity Catalog, Delta Lake, and Delta Sharing experience
• Solid SQL and Python, plus Terraform or other infrastructure-as-code
• Understanding of privacy-enhancing techniques and regulations (GDPR, UK GDPR, CCPA, sector rules)
• Experience with IAM, network security, and audit on AWS or Azure

The role is inside IR35 and will require travel to Wembley twice a week.

LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.

A multiple award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over consecutive years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.

Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.