SC Cleared Data Engineer — Azure Databricks & Pipelines

iO Associates; England; On-site; GBP 70,000 - 105,000; Full time

Job summary

iO Associates is seeking a Senior SC Cleared Data Engineer with extensive Azure Databricks experience to join a UK Central Government data team responsible for designing and developing high-volume datasets and robust data pipelines.

You will work with Spark-based technologies, build, automate and operate data pipelines, and apply CI/CD practices with Docker and Kubernetes in a secure government context.

Qualifications

• Hands-on experience with Azure Databricks.
• Experience with Spark-based technologies.
• Experience designing, building and operating data pipelines for large datasets.

Responsibilities

• Design and develop high-volume datasets and data pipelines.
• Build, automate and support data pipelines.
• Apply CI/CD practices and container orchestration in a secure government context.

Skills

Data pipelines

Cloud computing

CI/CD practices

Tools

Azure Databricks

Spark

Docker

Kubernetes

Job description

iO Associates is seeking a Senior SC Cleared Data Engineer with extensive Azure Databricks experience to join a UK Central Government data team responsible for designing and developing high-volume datasets and robust data pipelines.

You will work with Spark-based technologies, build, automate and operate data pipelines, and apply CI/CD practices with Docker and Kubernetes in a secure government context.