AI Data Engineer – GCP, Python, Kafka Pipelines (Hybrid)

Searchability; Manchester, Leeds; Hybrid; GBP 138,000 - 148,000; Full time

Job summary

Searchability is seeking a Data Engineer to join an enterprise data and AI transformation programme. You will design and build scalable data pipelines in GCP, develop cloud-native solutions, and support AI/LLM and RAG initiatives across a large-scale environment.

You will work on event-driven architectures with Kafka, write robust Python and SQL, and help ensure data quality, governance and scalable data platforms. This hybrid role is based in Manchester or Leeds, with on-site 2 days per week.

Qualifications

• Strong commercial GCP experience.
• Excellent Python and SQL skills.
• Strong Kafka / event-driven architecture experience.
• Experience working within large-scale enterprise environments.
• Exposure to AI/LLM and RAG solutions.
• Experience with BigQuery, vector databases, and enterprise search.

Responsibilities

• Designing and building scalable ETL/ELT data pipelines.
• Developing cloud-native data solutions within GCP.
• Engineering data platforms to support AI/LLM and RAG solutions.
• Building large-scale data ingestion, processing and enrichment capabilities.
• Developing event-driven architectures using Kafka.
• Building automation and data engineering solutions using Python and SQL.
• Supporting enterprise search, indexing and data retrieval capabilities.
• Ensuring data quality, governance, lineage and scalability.

Skills

GCP

Python

SQL

Kafka

AI

RAG

GenAI

LLM

BigQuery

Vector Databases

Enterprise Search

Tools

BigQuery

Vector Databases

Enterprise Search

Job description

Searchability is seeking a Data Engineer to join an enterprise data and AI transformation programme. You will design and build scalable data pipelines in GCP, develop cloud-native solutions, and support AI/LLM and RAG initiatives across a large-scale environment.

You will work on event-driven architectures with Kafka, write robust Python and SQL, and help ensure data quality, governance and scalable data platforms. This hybrid role is based in Manchester or Leeds, with on-site 2 days per week.