AI Data Engineer for Scalable AI Pipelines

ABS Group; England; On-site; GBP 70,000 - 110,000; Full time

Job summary

ABS Group is seeking a Data Engineer to design and operate data foundations powering AI chat assistants, models, and AI-driven improvements for client projects. You will build robust pipelines, standardize content, and enable scalable retrieval for AI workloads, in collaboration with AI engineers, consultants, and domain experts.

The role emphasizes production-grade data assets, data quality, governance, and scalable architectures across batch and streaming workloads, with a strong focus on data

Qualifications

• Bachelor's degree in a technical field; Master’s preferred.
• 6+ years designing, building, operating production data solutions.
• Experience in data-intensive environments and cloud data services.
• Hands-on with at least one major cloud provider and managed analytics services.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines for AI workloads.
• Integrate diverse data sources into centralized platforms for RAG and search.
• Standardize, normalize, and tag content to enterprise taxonomies.
• Collaborate with AI/ML teams to productionize datasets and embeddings.
• Optimize data workflows for reliability, cost, and performance.
• Establish data quality, lineage, and governance across systems.
• Automate common data patterns to accelerate delivery for client engagements.
• Translate business requirements into robust data solutions with stakeholders.

Skills

SQL

Python

Spark

Databricks

Snowflake

BigQuery

Airflow

ETL/ELT

RAG embeddings

Data modeling

Cloud platforms

CI/CD

UK work rights

Education

Bachelor's degree in Computer Science or related field

Master's degree preferred

Tools

AWS

Azure

GCP

Azure Data Factory

dbt

S3/ADLS/GCS

Kafka/Kinesis

Job description

ABS Group is seeking a Data Engineer to design and operate data foundations powering AI chat assistants, models, and AI-driven improvements for client projects. You will build robust pipelines, standardize content, and enable scalable retrieval for AI workloads, in collaboration with AI engineers, consultants, and domain experts.

The role emphasizes production-grade data assets, data quality, governance, and scalable architectures across batch and streaming workloads, with a strong focus on data