Data Engineer - SC Cleared

ShareForce; England; On-site; GBP 92,000 - 101,000; Full time

Job summary

ShareForce seeks a SC Cleared Data Engineer to join a critical public sector engagement, delivering Databricks-based data analytics within the Azure ecosystem.

You will design and deliver end-to-end pipelines, mentor engineers, and take ownership of core solution components, with a 6-month contract and strong extension prospects.

Qualifications

• Proven track record designing data solutions in Azure with Databricks as core platform.
• Hands-on Delta Lake, Delta Live Tables, and Databricks Workflows experience.
• Strong Python and SQL coding with modular, reusable Databricks code.
• Deep understanding of lakehouse architecture, data warehousing, data lakes, and real-time processing.

Responsibilities

• Design, build, and optimise end-to-end Data pipelines using Azure Databricks (Delta Live Tables).
• Collaborate with stakeholders to define technical requirements and propose Databricks solutions.
• Help clients realise potential of data science, ML, and scaled data processing in Azure/Databricks.
• Mentor junior engineers and support their personal development.
• Own delivery of core solution components.
• Support planning, requirements refinements, and work estimation.

Skills

Databricks

Python

SQL

Azure

Delta Live Tables

Delta Lake

Data pipelines

CI/CD

Azure DevOps

AI deployment

Tools

Databricks Workflows

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Job description

This is an exciting contract opportunity for an SC Cleared Data Engineer tojoin an experienced team delivering a crtitical public sector engagement working at the forefront of data analytics and AI. This role offers the chance to take a key role in the design and delivery of advanced Databricks solutions within the Azure ecosystem.

Responsibilities:

• Design, build, and optimise end-to-end data pipelines using Azure Databricks, including Delta Live Tables.
• Collaborate with stakeholders to define technical requirements and propose Databricks-based solutions.
• Help clients realise the potential of data science, machine learning, and scaled data processing within Azure / Databricks ecosystem.
• Mentor junior engineers and support their personal development.
• Take ownership for the delivery of core solution components.
• Support with planning, requirements refinements, and work estimation.

Skills & Experiences:

• Proven experience designing and implementing data solutions in Azure using Databricks as a core platform.
• Hands-on expertise in Delta Lake, Delta Live Tables and Databricks Workflows.
• Strong coding skills in Python and SQL, with experience in developing modular, reusable code in Databricks.
• Deep understanding of lakehouse architecture, with a solid grasp of data warehousing, data lakes, and real-time data processing.
• Familiar with Azure Data Factory, Azure Synapse Analytics, and Microsoft Fabric.
• Good experience with CI/CD practices and tools for data platforms using Azure DevOps.
• Good knowledge on how to leverage AI to increase deployment productivity and quality.
• Desirable: certification in Databricks Data Engineer and/or Azure Data Engineer are a plus.

Additional Information:

• Rate offered: £500-550 per day
• Duration: 6 months initial sign up, significant opportunity for extension