Lead Data Engineer, AI Pathfinder, Public Sector, Remote

Bangura Solutions; United States; Remote; GBP 90,000 - 130,000; Full time

Job summary

Bangura Solutions seeks a Lead Data Engineer, AI Pathfinder to drive the design and implementation of data pipelines and integrations enabling enterprise AI capabilities. You will establish data engineering best practices, ensure data readiness and security, and coordinate work across departments to align with broader data architectures and roadmaps.

In this remote role, you will lead technical decisions, mentor engineers, and translate complex data risks into clear guidance for senior

Qualifications

• Hands-on expertise in scalable ETL/ELT pipelines and API integrations.
• Knowledge of IAM, data classification, and GDPR/DPA compliance in AI environments.
• Experience with data quality assessments and synthetic testing environments.
• Proficiency in AWS, Python, SQL, and query optimization; CI/CD with Terraform.
• Experience leading cross-functional engineering teams in agile environments.

Responsibilities

• AI Data Integration & Pipeline Design: lead automated data pipelines, ETL/ELT workflows, and API connections for enterprise AI.
• AI Data Security & Compliance: implement access controls, data quality checks, and ensure permissions are respected by AI apps.
• AI Data Innovation & Diagnostics: run early data quality diagnostics and synthetic datasets for AI automation.
• Data Development Process: maintain metadata repositories and observability pipelines across AI workflows.
• Technical & Stakeholder Communication: bridge engineering and business leads, translating data risks into actions.

Skills

ETL/ELT pipelines

API integrations

Vector indices

Data governance

UK GDPR compliance

Data quality

Synthetic testing

Leadership

Agile

Tools

AWS

Python

SQL

Terraform

Job description

Role:

Lead Data Engineer, AI Pathfinder

Contract:

Until April 2027

Location:

Remote

IR35:

In Scope

Security Clearance:

SC

Role Purpose

As a Lead Data Engineer supporting the organisation's enterprise AI buildout, this role is responsible for leading the design and implementation of complex data flows, pipelines, and integrations to enable advanced AI and automated workflow solutions. It will set data engineering best practices, ensure data readiness and security, and coordinate work to turn the organisation's operational data into secure, production-ready AI capabilities. From an enterprise perspective, the role will ensure that deliverables align and interoperate with broader departmental data architectures and roadmaps.

Key Responsibilities

• AI Data Integration & Pipeline Design: Establish organisational data integration standards and lead the engineering of automated data pipelines, ETL/ELT workflows, and API connections connecting enterprise SaaS and internal databases to AI applications whilst ensuring alignment with existing pipelines and standards.
• AI Data Security & Compliance: Implement identity and access controls, including SSO, conduct data quality, sensitivity and access checks before indexing, and ensure AI applications respect source-system permissions to prevent data leakage.
• AI Data Innovation & Diagnostics: Conduct early-stage data quality diagnostics to test dataset viability for AI automation, configure synthetic test datasets within isolated sandboxes, and structure domain-specific knowledge engines.
• Data Development Process: Establish repeatable, secure, and maintainable data solutions, managing metadata repositories and configuring observability pipelines across the organisation's AI workflows, reusing existing repositories and observability pipelines where possible.
• Technical & Stakeholder Communication: Act as the key bridge between technical engineering partners and the organisation's business leads, interpreting data requirements and translating complex data risks into clear choices for stakeholders.

Essential Criteria (Skills, Knowledge & Experience)

• Technical Mastery & Engineering Practice: Hands-on expertise in building scalable ETL/ELT pipelines, designing robust API integrations, and configuring vector indices or retrieval systems for large-scale enterprise AI architectures
• Data Governance, Security & Compliance: Expert knowledge of Identity and Access Management, identity federation, data classification standards, and UK GDPR/DPA compliance frameworks within AI environments.
• Analytical Proficiency & Data Diagnostics: Substantial experience conducting quantitative data quality assessments, structuring synthetic testing environments, and evaluating dataset readiness for complex automation or ML models.
• Languages & Software Engineering Standards: Advanced proficiency in AWS, Python, SQL, and query optimisation, alongside demonstrated experience implementing CI/CD pipelines using Terraform, automated testing, and metadata management best practices.
• Strategic Communication & Leadership: Proven ability to translate complex technical data risks into strategic guidance for senior leadership, alongside a track record of leading cross-functional engineering teams in agile environments.

Minorities, women, LGBTQ+ candidates, and individuals with disabilities are encouraged to apply.