Lead Data Engineer

Circle Recruitment; City Of London; Remote; GBP 129,000 - 166,000; Full time

Job summary

Circle Recruitment is seeking a Lead Data Engineer for a remote contract position with UK government stakeholders. The role focuses on secure, scalable data pipelines, data governance, and AI-ready data architectures under an Umbrella solution. SC clearance is required.

The candidate will lead cross‑functional teams, establish standards, and ensure QA, data quality, and compliance across enterprise data platforms. The contract runs until April and pays £700–£900 per day inside IR35.

Qualifications

• Experience designing scalable ETL/ELT pipelines and APIs across enterprise platforms.
• Strong governance, access controls, and UK GDPR alignment expertise.
• Ability to build synthetic datasets and AI-ready data products.
• Proven capability to implement CI/CD, testing, and observability.

Responsibilities

• Lead design and delivery of secure, scalable data pipelines and integrations.
• Establish engineering standards, ensure data quality and compliance.
• Engage with senior stakeholders to communicate data risks and recommendations.
• Mentor cross-functional teams and drive secure, maintainable solutions.

Skills

Data Integration & Pipelines

Security & Governance

Data Readiness & Innovation

Engineering Standards & Observability

Technical Leadership & Stakeholder Eng

Data Engineering

Security & Compliance

Data Quality & Diagnostics

Cloud & Software Engineering

Leadership & Communication

Tools

AWS

Python

SQL

Terraform

Job description

Lead Data Engineer

Contract until April

£Market rates/Day Inside IR35

Remote

Clearance: SC

We are seeking a Lead Data Engineer our Government client on a remote basis. The Lead Data Engineer role will be an initial contract until 30/04 paying between £700 - £900/Day Inside IR35. Ideally they want to be under the top end budget.

This role is inside IR35 - Due to the service of the role it will now be based on an Umbrella solution.

Lead the design and delivery of secure, scalable data pipelines and integrations supporting enterprise AI and workflow automation. Establish engineering standards, ensure data quality and compliance, and align solutions with wider enterprise data architectures and roadmaps.

Clearance: SC

Role Profile:

• Data Integration & Pipelines: Design and implement ETL/ELT workflows, APIs, and integrations across enterprise platforms and databases, aligning with existing standards.
• Security & Governance: Implement access controls, SSO, data classification, and permission-aware AI integrations in line with UK GDPR and data protection requirements.
• Data Readiness & Innovation: Assess data quality and suitability for AI, develop synthetic test datasets in secure sandboxes, and build domain-specific knowledge and retrieval systems.
• Engineering Standards & Observability: Develop reusable, secure, maintainable solutions, including metadata management, automated testing, CI/CD, and monitoring.
• Technical Leadership & Stakeholder Engagement: Lead cross-functional engineering teams, translate business requirements into technical solutions, and communicate data risks and recommendations to senior stakeholders.

• Data Engineering: Proven expertise in scalable ETL/ELT pipelines, API integrations, vector indexing, and enterprise AI retrieval architectures.
• Security & Compliance: Strong knowledge of identity and access management, identity federation, data classification, UK GDPR, and data protection requirements.
• Data Quality & Diagnostics: Experience assessing data quality, validating AI readiness, and developing synthetic testing environments for automation and machine learning.
• Cloud & Software Engineering: Advanced AWS, Python, and SQL skills, including query optimisation, Terraform, CI/CD, automated testing, and metadata management.
• Leadership & Communication: Demonstrated ability to lead multidisciplinary teams in agile environments and translate complex technical risks into clear strategic advice.