Databricks Engineer - SC Cleared

Shareforce; United Kingdom; Hybrid; GBP 80,000 - 100,000; Full time

Job summary

A technology consulting firm is seeking a SC Cleared Databricks Engineer to join their team in the UK. The role involves designing and delivering advanced Databricks solutions in the Azure ecosystem, focusing on data analytics and AI. The ideal candidate will have proven experience with Databricks and Azure, strong coding skills, and the ability to mentor junior engineers. This position offers a competitive rate and a hybrid working model.

Qualifications

• Proven experience in designing data solutions in Azure using Databricks.
• Hands-on expertise in Delta Lake and Delta Live Tables.
• Strong coding skills in Python and SQL.

Responsibilities

• Design, build, and optimise end-to-end data pipelines using Azure Databricks.
• Collaborate with stakeholders to define technical requirements.
• Drive best practices for data engineering.

Skills

Designing data solutions in Azure using Databricks

Delta Lake and Delta Live Tables

Python coding skills

SQL expertise

Lakehouse architecture understanding

Azure Data Factory knowledge

CI/CD practices with Azure DevOps

Excellent communication skills

Education

Certification in Databricks Data Engineer or Azure Data Engineer

Job description

This is an exciting contract opportunity for an SC Cleared Databricks Engineer tojoin an experienced team in a new customer engagement working at the forefront of data analytics and AI. This role offers the chance to take a key role in the design and delivery of advanced Databricks solutions within the Azure ecosystem.

Responsibilities

• Design, build, and optimise end-to-end data pipelines using Azure Databricks, including Delta Live Tables.
• Collaborate with stakeholders to define technical requirements and propose Databricks-based solutions.
• Drive best practices for data engineering.
• Help clients realise the potential of data science, machine learning, and scaled data processing within Azure / Databricks ecosystem.
• Mentor junior engineers and support their personal development.
• Take ownership for the delivery of core solution components.
• Support with planning, requirements refinements, and work estimation.

Skills & Experiences

• Proven experience designing and implementing data solutions in Azure using Databricks as a core platform.
• Hands-on expertise in Delta Lake, Delta Live Tables and Databricks Workflows.
• Strong coding skills in Python and SQL, with experience in developing modular, reusable code in Databricks.
• Deep understanding of lakehouse architecture, with a solid grasp of data warehousing, data lakes, and real-time data processing.
• Familiar with Azure Data Factory, Azure Synapse Analytics, and Microsoft Fabric.
• Good experience with CI/CD practices and tools for data platforms using Azure DevOps.
• Good knowledge on how to leverage AI to increase deployment productivity and quality.
• Excellent communication skills.
• Desirable: certification in Databricks Data Engineer and/or Azure Data Engineer are a plus.

Additional Information

• Rate offered: £500-550 per day
• IR35 Status: Outside
• Location: Hybrid with travel to client site in the North East
• Start date: January
• Duration: 3 months initial sign up with significant opportunity for extension
• Required: Active SC Clearance