Senior Data Engineer for Generative AI Solutions (Remote)

Pyramid Consulting, Inc; Greater London; Remote; GBP 90,000 - 130,000; Full time

Job summary

Pyramid Consulting, Inc. is seeking a Senior Data Engineer in a remote-capable role covering London, Spain, Germany, and Portugal.

The candidate will own end-to-end data engineering and AI workstreams, delivering scalable solutions using Python, SQL, Spark, and modern MLOps practices. Responsibilities include leading technical initiatives, mentoring engineers, and building Generative AI/Agentic AI capabilities with robust governance, security, and production-readiness across AWS, Azure, and GCP

Qualifications

• Degree in Computer Science, Engineering or equivalent.
• 5+ years of relevant Data Engineering experience.
• Expert-level Python and SQL experience.
• Experience with Generative AI, Agentic AI, ML, and BI solutions.
• Knowledge of RAG, embeddings, vector databases, and prompt engineering.

Responsibilities

• Design, develop, and maintain scalable data engineering and AI solutions.
• Lead technical workstreams and mentor junior engineers.
• Build Generative AI, Agentic AI, ML, and BI solutions.
• Work with Spark, LangChain, Databricks, Airflow, Dagster, Dask, Kedro.
• Develop AI capabilities including prompt engineering, embeddings, and context construction.
• Own AI feature delivery from development to production with guardrails and performance optimisation.
• Implement data security, governance, authentication, and RBAC.
• Establish MLOps/LLMOps pipelines with CI/CD and AI evaluation.

Skills

Python

SQL

Generative AI

Agentic AI

MLOps

LLMOps

CI/CD

RBAC

Data governance

Education

Degree in Computer Science, Engineering

Tools

Spark

Databricks

LangChain

Airflow

Dagster

Dask

Kedro

Terraform

Job description

Pyramid Consulting, Inc. is seeking a Senior Data Engineer in a remote-capable role covering London, Spain, Germany, and Portugal.

The candidate will own end-to-end data engineering and AI workstreams, delivering scalable solutions using Python, SQL, Spark, and modern MLOps practices. Responsibilities include leading technical initiatives, mentoring engineers, and building Generative AI/Agentic AI capabilities with robust governance, security, and production-readiness across AWS, Azure, and GCP