Data Engineer - SC Cleared

Shareforce; North East; Hybrid; GBP 92,000 - 101,000; Part time

Job summary

Shareforce is seeking an SC Cleared Data Engineer to join an experienced team delivering a critical public sector engagement at the forefront of data analytics and AI. The role focuses on designing and delivering advanced Databricks solutions within the Azure ecosystem.

You will design, build, and optimize end-to-end data pipelines in Azure Databricks, mentor juniors, and take ownership of core components while engaging with stakeholders on requirements and estimates.

Qualifications

• Proven experience designing and implementing data solutions in Azure using Databricks as a core platform.
• Hands-on expertise in Delta Lake, Delta Live Tables and Databricks Workflows.
• Strong coding skills in Python and SQL, with experience in developing modular, reusable code in Databricks.

Responsibilities

• Design, build, and optimise end-to-end data pipelines using Azure Databricks, including Delta Live Tables.
• Collaborate with stakeholders to define technical requirements and propose Databricks-based solutions.
• Drive best practices for data engineering.
• Help clients realise the potential of data science, machine learning, and scaled data processing within Azure / Databricks ecosystem.
• Mentor junior engineers and support their personal development.
• Take ownership for the delivery of core solution components.
• Support with planning, requirements refinements, and work estimation.

Skills

Azure Databricks

Delta Live Tables

Python

SQL

Lakehouse architecture

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

CI/CD

Azure DevOps

Databricks Workflows

Databricks Data Engineer

Azure Data Engineer

Tools

Azure Databricks

Delta Lake

Delta Live Tables

Databricks Workflows

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Azure DevOps

Job description

This is an exciting contract opportunity for an SC Cleared Data Engineer tojoin an experienced team delivering a crtitical public sector engagement working at the forefront of data analytics and AI. This role offers the chance to take a key role in the design and delivery of advanced Databricks solutions within the Azure ecosystem.

Responsibilities

• Design, build, and optimise end-to-end data pipelines using Azure Databricks, including Delta Live Tables.
• Collaborate with stakeholders to define technical requirements and propose Databricks-based solutions.
• Drive best practices for data engineering.
• Help clients realise the potential of data science, machine learning, and scaled data processing within Azure / Databricks ecosystem.
• Mentor junior engineers and support their personal development.
• Take ownership for the delivery of core solution components.
• Support with planning, requirements refinements, and work estimation.

Skills & Experiences

• Proven experience designing and implementing data solutions in Azure using Databricks as a core platform.
• Hands-on expertise in Delta Lake, Delta Live Tables and Databricks Workflows.
• Strong coding skills in Python and SQL, with experience in developing modular, reusable code in Databricks.
• Deep understanding of lakehouse architecture, with a solid grasp of data warehousing, data lakes, and real-time data processing.
• Familiar with Azure Data Factory, Azure Synapse Analytics, and Microsoft Fabric.
• Good experience with CI/CD practices and tools for data platforms using Azure DevOps.
• Good knowledge on how to leverage AI to increase deployment productivity and quality.
• Excellent communication skills.
• Desirable: certification in Databricks Data Engineerand/or Azure Data Engineer are a plus.

Additional Information

• Rate offered: £500-550 per day
• IR35 Status: Outside
• Location: Hybrid working, North England
• Start date: October
• Duration: 6 months initial sign up, significant opportunity for extension
• Required: Active SC Clearance