Data Engineer

Pyramid Consulting, Inc

Greater London

Hybrid

GBP 129,000 - 185,000

Full time

42 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Pyramid Consulting, Inc. in London seeks a contractor to write production-grade Python and develop agents to automate R&D data processing. The role requires testing, instrumentation and thorough documentation for long-term team use.

You will design data storage, transfer and transformation utilities, ensure secure data exchange with automated quality control, and collaborate with ML and software engineers to shape R&D datasets and validate metadata. 2-3 days on client site.

Qualifications

  • PhD in computational science or AI/ML with R&D data, or MSc with 2+ years of industry experience.
  • Strong Python for data engineering, including data transformation and schema management.
  • Experience with secure data exchange, quality control and data pipelines.
  • Experience with CI/CD, agile development and DevOps.
  • Ability to work independently with ambiguous requirements.

Responsibilities

  • Write production-grade Python and develop agents to automate R&D data processing with tests and documentation.
  • Design data storage, transfer and transformation utilities and secure data exchange with QC.
  • Collaborate with ML and software engineers to shape R&D datasets.
  • Parse and validate metadata and build de-identification checks to replace manual review.

Skills

GenAI automation
Python for data engineering
MLOps
CI/CD
Cloud storage
ETL/ELT pipelines
Data transformation
Documentation

Education

PhD in computational science/AI/ML
MSc with 2+ years industry experience

Tools

Google Cloud Storage
Azure Blob Storage/ADLS
Amazon S3

Job description

Location: London-based, with 2-3 days per week on client site
Duration: 6-12 month
What the contractor will do:
  • Write production-grade Python and develop agents to automate R&D data processing. The work needs to be tested, instrumented and documented well enough for the team to run long term.
  • Design data storage, transfer and transformation utilities, including secure inbound and outbound data exchange with automated quality control.
  • Work with ML engineers and software engineers to shape R&D datasets around what the models and the science need.
  • Parse and normalize metadata, validate incoming manifests against the agreed standard, and build quality and de-identification checks to replace manual review.
Required skills and experience:
  • Strong technical or platform engineering and MLOps background, combined with GenAI automation and development
  • Strong Python for data engineering, including data transformation, job monitoring and schema management
  • Experience with large binary or blob datasets in cloud object storage (Google Cloud Storage, Azure Blob Storage/ADLS, Amazon S3 or equivalent)
  • Experience building and running ETL/ELT pipelines with an orchestration framework
  • Experience with CI/CD, agile development and DevOps
  • Ability to work independently with ambiguous requirements
Education/experience:

PhD in a computational science or AI/ML with R&D data (any domain), or an MSc with 2+ years of industry experience

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

R&D Data Engineer: GenAI & MLOps (Hybrid London)
R&D Data Engineer: GenAI & MLOps (Hybrid London)

Pyramid Consulting, Inc • Greater London

Hybrid
GBP 129,000 - 185,000
Data Engineer
Data Engineer

Principle • Greater London

Hybrid
GBP 85,000 - 110,000
Data & AI Engineer
Data & AI Engineer

Digi2al • Portsmouth

On-site
GBP 112,000 - 147,000
Data Scientist
Data Scientist

LT Harper - Cyber Security Recruitment • England

On-site
GBP 60,000 - 80,000
Data & AI Engineer
Data & AI Engineer

Digi2al Limited • Portsmouth

On-site
GBP 112,000 - 147,000
AI Engineer (GenAI / LLM / RAG / Agentic AI)
AI Engineer (GenAI / LLM / RAG / Agentic AI)

Ampstek • Greater London

Hybrid
GBP 101,000 - 157,000
Data Engineer
Data Engineer

Talent • Greater London

Hybrid
GBP 90,000 - 120,000
Senior AI / Python Engineer
Senior AI / Python Engineer

GCS • City Of London

Hybrid
GBP 70,000 - 110,000
Data & Integration Engineer - AI
Data & Integration Engineer - AI

HMRC • United Kingdom

On-site
GBP 92,000 - 148,000
Data Engineer
Data Engineer

Accelero • Greater London

Hybrid
GBP 50,000 - 75,000
Real ownership and technical impact
Exposure to modern cloud and AI tooling
Hybrid working in the City of London