Data Ingestion Engineer for Scalable AI Pipelines

Reflection; Greater London; Hybrid; GBP 70,000 - 90,000; Full time

Benefits offered by this job

Top-tier compensation

Comprehensive medical, dental, vision insurance

Fully paid parental leave

Paid time off

Daily lunch and dinner provided

Job summary

Reflection seeks a Data Engineer in Greater London to build and operate large-scale data ingestion systems. You will collaborate closely with world-class researchers to create data pipelines that enhance AI model performance.

The ideal candidate has experience with web crawling technologies like Ray and Spark and enjoys hybrid roles in research and engineering. You will contribute to impactful work within a supportive environment offering top-tier compensation and comprehensive health benefits.

Qualifications

• Experience building web crawling, data ingestion, or large-scale data acquisition systems.
• Comfortable working with very large datasets (multi-TB to PB scale).
• Able to design experiments and use data to guide improvements.

Responsibilities

• Build and operate large-scale data ingestion systems for pre-training.
• Run experiments to evaluate data acquisition strategies.
• Analyze ingested data to identify gaps and areas for improvement.

Skills

Web crawling experience

Data ingestion systems experience

Familiarity with LLM training

Experience with large datasets

Excellent communication skills

Tools

Ray

Beam

Spark

Job description

Reflection seeks a Data Engineer in Greater London to build and operate large-scale data ingestion systems. You will collaborate closely with world-class researchers to create data pipelines that enhance AI model performance.

The ideal candidate has experience with web crawling technologies like Ray and Spark and enjoys hybrid roles in research and engineering. You will contribute to impactful work within a supportive environment offering top-tier compensation and comprehensive health benefits.