Lead Databricks Data Engineer

Computer Futures; United Kingdom; Hybrid; GBP 80,000 - 100,000; Full time

Job summary

A dynamic technology recruitment agency is seeking a hands-on Lead Data Engineer to guide a small team and oversee a data platform in Warwickshire. The role involves leading engineering efforts, managing DevOps workflows, and ensuring robust configuration and documentation of Databricks environments. Ideal candidates should have strong experience in Databricks, Azure Synapse, and Azure DevOps, along with proficiency in SQL and PySpark. This is a hybrid contract position with an initial duration of 6 months.

Qualifications

• Strong experience with Databricks, Azure Synapse, and Azure DevOps.
• Proficient in SQL and PySpark.
• Proven leadership in small engineering teams.

Responsibilities

• Lead data engineering across the platform.
• Supervise DevOps processes (branching, merging, CI/CD).
• Act as Databricks DBA and manage environment promotion.
• Maintain configuration, documentation, and monitoring.
• Mentor and support Data Engineers.

Skills

Databricks

Azure Synapse

Azure DevOps

SQL

PySpark

Job description

Overview

We're seeking a hands-on Lead Data Engineer to guide a small team and oversee our data platform. You'll lead engineering efforts, manage DevOps workflows, and ensure robust configuration and documentation of Databricks environments.

Responsibilities

• Lead data engineering across the platform
• Supervise DevOps processes (branching, merging, CI/CD)
• Act as Databricks DBA and manage environment promotion
• Maintain configuration, documentation, and monitoring
• Mentor and support Data Engineers

Required Skills

• Strong experience with Databricks, Azure Synapse, and Azure DevOps
• Proficient in SQL and PySpark
• Proven leadership in small engineering teams
• Skilled in configuration management and documentation

Role Details

• Location: Warwickshire
• Type: Contract - 6 Months initially (high chance of extension)
• IR35 Status: Inside IR35
• Rate: Open to Candidate Proposals
• Work Setup: Hybrid (1 day per week on-site)