Secure Data Engineer

Ontologize LLC; United Kingdom; On-site; GBP 90,000 - 130,000; Full time

Job summary

Capgemini in the UK offers a senior Solution Architect role focused on Azure and Databricks to design, build and secure data platforms for defence clients. You will deploy data warehouses and ML-ready pipelines, while ensuring reliability, governance and secure patterns across containerised environments.

The role blends software and data engineering, requires SC clearance eligibility, and a hybrid working pattern across offices, client sites and home.

Qualifications

• Expert level Python and strong foundations in software engineering such as object oriented design, automated testing and version control.
• Experience building pipelines using streaming frameworks, distributed processing engines and relational or analytical storage technologies.
• Experience defining and running data workflows using modern orchestration frameworks.
• Familiarity with tools and techniques for data testing, documentation and lineage.
• Understanding of how to operationalise machine learning models in production, including model packaging, monitoring and controlled deployment.
• Confidence deploying applications in containerised environments, including defining services, pods and deployment configurations.

Responsibilities

• Building Data Applications: develop modular software components for data processing and analytics with strong testing and observability.
• Streaming and Real Time Architecture: implement real-time data ingestion and event-driven patterns with reliability.
• Workflow Orchestration: programmatic data workflows, manage dependencies, scheduling and error recovery in secure environments.
• Deployment and Ownership: containerise services and deploy into secure Kubernetes or cloud environments with CI/CD principles.
• Resilience, Observability and Compliance: implement health monitoring, logging, metrics and data lineage for auditability.
• Infrastructure as Code: provision resources using IaC and work with platform teams on accredited architectures.

Skills

Python

OO design

Data pipelines

Streaming

CI/CD

Security

Kubernetes

Docker

Education

Bachelor's degree in CS/Engineering

Tools

Kafka

Spark

PostgreSQL

Airflow

Dagster

Prefect

dbt

Great Expectations

Terraform

Ansible

Docker

Kubernetes

Job description

Manchester, London, Bristol, Newcastle, Birmingham

Who You Will Be Working With...

The Data Platforms team is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety of projects and sectors. Data Platforms is the home of the Data Engineers, Platform Engineers, Solutions Architects and Business Analysts who are focused on driving our customers digital and data transformation journey using the modern cloud platforms.

We specialise on using the latest frameworks, reference architectures and technologies using AWS, Azure and GCP along with various data platforms like Databricks, Snowflake, Quantexa, Palantir, SAS.

The Focus Of Your Role

As a Solution Architect with an Azure and Databrick focus, you will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.

We are looking for a code first Data Engineer to design and build scalable and resilient data applications for Defence customers. This role sits at the intersection of Software Engineering and Data Engineering, working on mission critical systems where reliability, security and auditability are essential.

You will build data applications that process high volume and high velocity data, orchestrate complex workflows, and deploy your own solutions into secure containerised environments. You will work across the full lifecycle from development through to production, shaping architectures that support operational users, analytics, and AI enabled capabilities.

This role is hands on and engineering. You will write production grade code, contribute to secure platform patterns, and ensure that your data services run predictably in tightly governed Defence environments.

What You Will Bring

• Core Engineering
• Expert level Python and strong foundations in software engineering such as object oriented design, automated testing and version control.
• Data Engineering Stack
• Experience building pipelines using streaming frameworks, distributed processing engines and relational or analytical storage technologies.
• Examples include Kafka, Spark, PostgreSQL.
• Orchestration
• Experience defining and running data workflows using modern orchestration frameworks.
• Examples include Airflow, Dagster or Prefect.
• Data Quality and Lineage
• Familiarity with tools and techniques for data testing, documentation and lineage.
• Examples include Great Expectations or dbt.
• AI and MLOps
• Understanding of how to operationalise machine learning models in production, including model packaging, monitoring and controlled deployment.
• Containerisation and Kubernetes
• Confidence deploying applications in containerised environments, including defining services, pods and deployment configurations.
• Examples include Docker and Kubernetes.

• DevOps Mindset
• Hands on experience with CI/CD approaches and a belief in owning the services you build.
• Examples include GitLab CI, GitHub Actions or Argo.
• Nice to have
• Experience with Infrastructure as Code or configuration management tools.
• Examples include Terraform or Ansible.
• Experience working in secure, restricted or air gapped environments, including Defence networks or MODCloud aligned platforms.
• Familiarity with Google Distributed Cloud (GDC) or other edge and on premises cloud platforms used in constrained or disconnected settings.

What You Will Be Doing

Your work will involve

Building Data Applications

Developing modular and maintainable software components that process, transform and expose data for analytical, operational or AI driven use cases. You will follow strong engineering practices, with testing and observability built in from the start.

Streaming and Real Time Architecture

Designing and implementing data ingestion and event driven patterns that support real time or near real time flows, ensuring they remain reliable even under demanding operational conditions.

Workflow Orchestration

Defining data workflows programmatically and managing complex dependencies, scheduling and error recovery behaviours within secure and assured environments.

Deployment and Ownership

Containerising your own services and deploying them into secure Kubernetes or cloud environments, using CI/CD principles adapted for Defence delivery. You will own your applications in production and contribute to secure patterns for deployment.

Resilience, Observability and Compliance

Implementing health monitoring, structured logging, metrics and lineage to meet Defence requirements for auditability, security and operational assurance. You will design systems that can self heal or fail gracefully when needed.

Infrastructure as Code

Provisioning the resources your services depend on using Infrastructure as Code and working closely with platform teams to ensure alignment with accredited Defence architectures.

About Capgemini

Security Clearance: To be successfully appointed to this role, must be eligible to obtain Security Check (SC)clearance.

To obtain SC clearance, the successful applicant must have resided continuously within the United Kingdom for the last 5 years, along with other criteria and requirements.

Throughout the recruitment process, you will be asked questions about your security clearance eligibility such as, but not limited to, country of residence and nationality.

Some posts are restricted to sole UK Nationals for security reasons; therefore, you may be asked about your citizenship in the application process.

Hybrid working: The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

If you are successfully offered this position, you will go through a series of pre-employment checks, including: identity, nationality (single or dual) or immigration status, employment history going back 3 continuous years, and unspent criminal record check (known as Disclosure and Barring Service)

What we’ll offer you

You will be encouraged to have a positive work-life balance. Our hybrid-first way of working means we embed hybrid working in all that we do and make flexible working arrangements the day-to-day reality for our people. All UK employees are eligible to request flexible working arrangements.

You will be empowered to explore, innovate, and progress. You will benefit from Capgemini’s ‘learning for life’ mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard Manage Mentor, Cybersecurity qualifications and much more.

Why we’re different

At Capgemini, we help organisations across the world become more agile, more competitive, and more successful. Smart, tailored, often ground-breaking technical solutions to complex problems are the norm. But so, too, is a culture that’s as collaborative as it is forward thinking. Working closely with each other, and with our clients, we get under the skin of businesses and to the heart of their goals. You will too.

Capgemini is proud to represent nearly 130 nationalities and its cultural diversity. Our holistic definition of diversity extends beyond gender, gender identity, sexual orientation, disability, ethnicity, race, age, and religion. Capgemini views diversity as everything that makes us who we are as an organization, including our social background, our experiences in life and work, our communication styles and even our personality. These dimensions contribute to the type of diversity we value the most: diversity of thought.