Senior Data Engineer – On‑Site (3d/wk) Python/Kafka/K8s

Searchability NS&D; Portsmouth; On-site; GBP 54,000 - 90,000; Full time

Job summary

Searchability NS&D is seeking an experienced Databricks Data Engineer to design and build scalable, secure data applications within Defence environments. Portsmouth customer site with 3 days on-site per week, salary up to £90k DoE plus bonus.

You will work at the intersection of Software and Data Engineering, delivering production‑grade data services, observability, CI/CD and IaC within highly governed networks.

Qualifications

• Strong Python development and software engineering experience.
• Experience building scalable data pipelines and distributed data platforms.
• Knowledge of Kafka, Spark and modern data processing technologies.
• Experience with orchestration tools like Airflow, Dagster or Prefect.
• Hands-on experience with containerised deployments (Docker/Kubernetes).
• CI/CD with production ownership and reliability focus.
• Understanding of data quality, testing and lineage.
• MLOps or production ML experience.
• Infrastructure as Code using Terraform or Ansible.
• PostgreSQL and modern data platforms.
• Experience in Defence, National Security or regulated environments.
• Familiarity with air-gapped, restricted or secure networks.
• Knowledge of Google Distributed Cloud or edge cloud technologies.

Responsibilities

• Building scalable data applications and pipelines using Python
• Developing streaming, event-driven and orchestrated data solutions
• Deploying and supporting containerised services in Kubernetes environments
• Implementing observability, monitoring and data governance best practices
• Leveraging Infrastructure as Code and CI/CD for secure deployments
• Owning solutions from design and development through to production support
• Delivering resilient data platforms within secure Defence environments

Skills

Python

Data pipelines

Kafka

Spark

Airflow

Dagster

Prefect

Docker

Kubernetes

CI/CD

Data quality

Data lineage

MLOps

Terraform

Ansible

PostgreSQL

Defence environment knowledge

Air-gapped networks

GDC / edge cloud

Tools

Docker

Kubernetes

Terraform

Ansible

Job description

Searchability NS&D is seeking an experienced Databricks Data Engineer to design and build scalable, secure data applications within Defence environments. Portsmouth customer site with 3 days on-site per week, salary up to £90k DoE plus bonus.

You will work at the intersection of Software and Data Engineering, delivering production‑grade data services, observability, CI/CD and IaC within highly governed networks.