Data Engineer

CACI Digital Experience (formerly Cyber-Duck); Greater London; On-site; GBP 50,000 - 70,000; Full time

Job summary

CACI Digital Experience (formerly Cyber-Duck) seeks an experienced Data Engineer to join their London team. The role involves maintaining and migrating cloud data architectures, designing data processing environments using AWS services like Glue and EMR, and building robust data pipelines with a focus on quality. Candidates should have a minimum of 4 years' experience in data engineering, be proficient in Python, SQL, and PySpark, and possess strong problem-solving skills. CACI promotes diversity and equal opportunity in hiring.

Qualifications

• 4 years of experience in a Data Engineering role.
• Strong experience with AWS data architectures and services.
• Proficiency in coding using Python, SQL, and PySpark.
• Experience with Git for version control.
• Hands-on experience with CI/CD tools.

Responsibilities

• Collaborate across departments to develop and maintain data products.
• Design and implement data processing environments using AWS PaaS.
• Build data processing and analytics pipelines.
• Design security architectures for data sensitivity.
• Enable business units by delivering manageable solutions.

Skills

AWS services (Glue, EMR, S3, etc.)

Python

SQL

PySpark

Data architecture

Agile principles

Git

CI/CD tools

Problem-solving

Data mesh understanding

Tools

CloudFormation

Terraform

Jira

Job description

Company Overview

Headquartered in London, CACI Ltd is a wholly owned subsidiary of CACI International Inc., a publicly listed company on the NYSE with annual revenue in excess of US $6.2bn and employing approximately 22,000 people worldwide. CACI Ltd is an international data and technology consultancy with £154m turnover and 1200 employees, passionate, progressive and unafraid of challenge, with a mission to use technology and data‑driven insight to make a commercial difference.

About CACI Data Engineering

CACI has implemented a Data Platform that supports and enables a Data Mesh organisation, using AWS technology to deliver an open federated Lakehouse and a unified user experience. The platform focuses on enabling decentralised management, processing, analysis and delivery of data, while enforcing federated governance across business domains. The goal is to empower multiple teams to create and manage high‑integrity data and data products that are analytics and AI ready, and to be consumed internally and externally.

What does a Data Engineer do?

A Data Engineer partners closely with business units to maintain existing cloud data architectures, and to assess, design and execute the migration of their existing cloud and on‑premise/desktop data products and workflows onto a modern cloud data platform. This involves understanding current data architectures, dependencies and transformation logic, translating that into cloud native solutions in harmony with the company data platform, governance and strategy. The role requires developing data pipelines that operate across a medallion architecture, with an obsession on data quality and integrity, while considering the cost benefit of different approaches. Skills in AWS services such as Glue, EMR, S3, MWAA (Apache Airflow), Step Functions, Redshift, SageMaker, and in traditional RDBMS (Postgres, Oracle, SQL Server) are essential; SQL, Python and PySpark are required, as is experience with IaC such as CloudFormation or Terraform.

Responsibilities Will Include

• Collaborating across CACI departments to develop and maintain data products and the data platform
• Designing and implementing data processing environments and integrations using AWS PaaS such as Glue, S3, Lambda, Fargate, EMR, SageMaker, Redshift, Aurora and Snowflake
• Data architecture and data modelling across full data lifecycles, including detailed modelling of databases and data products
• Building data processing and analytics pipelines as code, using Python, SQL, PySpark, Spark, CloudFormation, Lambda, Step Functions, and Apache Airflow
• Designing and applying security and access control architectures to secure sensitive data
• Enabling business units by working with them to deliver complete and manageable solutions, while providing support and expert advice

You Will Have

• 4 years of experience in a Data Engineering role
• Strong experience and knowledge of data architectures implemented in AWS using native services such as S3, DataZone, Glue, EMR, SageMaker, Aurora and Redshift, using Python, PySpark and SQL
• Experience developing and administrating databases, data platforms and solutions
• Good coding discipline in terms of style, structure, versioning, documentation and unit tests
• A well‑developed understanding of a data mesh organisation, as well as Master and Reference Data Management
• Experience migrating legacy to modern, as well as modern to modern
• Knowledge and experience of relational databases such as Postgres, Redshift
• Experience using Git for code versioning, and lifecycle management
• Experience operating to Agile principles and ceremonies
• Hands‑on experience with CI/CD tools such as GitLab
• Strong problem‑solving skills and ability to work independently or in a team environment
• Excellent communication and collaboration skills
• A keen eye for detail, and a passion for accuracy and correctness in numbers

While not essential, the following skills would also be useful

• Experience using Jira, or other agile project management and issue tracking software
• Experience with spatial data processing, technology and approaches
• Experience with machine learning and AI workflows
• Experience with IaC such as CloudFormation or Terraform

We are committed to creating a diverse environment and are proud to be an equal‑opportunity employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Successful candidates must have the right to work in the UK.