SC Cleared Data Engineer - Databricks

iO Associates; Lincoln; On-site; GBP 70,000 - 100,000; Full time

Job summary

iO Associates is seeking a Senior SC Cleared Data Engineer with strong Azure Databricks expertise to join a UK Central Government team responsible for designing and developing high-volume datasets.

You will build, automate, and support data pipelines, work with Spark-based technologies, and collaborate across CI/CD, Docker, and Kubernetes environments to deliver secure data solutions.

Qualifications

• SC clearance is required or in place.
• Strong Azure Databricks expertise and Spark-based tech experience.
• Experience building, automating and supporting data pipelines.
• Familiarity with CI/CD, Docker and Kubernetes environments.

Responsibilities

• Design and develop high-volume datasets for a UK Central Government team.
• Build, automate, and support data pipelines across the data platform.

Skills

SC Cleared

Data Engineering

UK Central Government

Tools

Azure Databricks

Docker

Kubernetes

Spark

CI/CD

Job description

Senior SC Cleared Data Engineer - Azure Databricks

iO Associates are looking for an experienced SC Cleared Data Engineer with strong Azure Databricks expertise to join a Central Government team responsible for designing & developing high-volume datasets.

Extensive hands-on experience with Azure Databricks

Recent experience working within UK Central Government

Experience with Spark-based technologies

• Experience building, automating and supporting data pipelines

Familiarity with CI/CD, Docker & Kubernetes