Data Engineer – Real-Time, Scalable Data Pipelines

Consortia Group; City Of London; Hybrid; GBP 60,000 - 75,000; Full time

Job summary

A data solutions company in London is seeking a skilled Data Engineer to work with high-volume datasets and cutting-edge technologies. In this role, you will build and maintain data pipelines, integrate real-time data feeds, and support machine learning applications. The ideal candidate has over 3 years of experience in data engineering, strong proficiency in Python and Rust, and hands-on experience with AWS, Linux, and Docker. If you are ready to advance your data engineering career, we want to hear from you!

Qualifications

• 3+ years of experience in data engineering.
• Strong proficiency in Python and Rust.
• Hands-on experience with AWS cloud, Linux, and Docker.
• Expertise in distributed computing frameworks.
• Solid understanding of ETL pipelines and database optimization.

Responsibilities

• Build and maintain high-performance data pipelines.
• Integrate real-time feeds and ensure data flow.
• Implement quality assurance measures for data integrity.
• Support machine learning applications by preparing data.

Skills

Python

Rust

AWS

Linux

Docker

Distributed computing frameworks (Ray, Dask, Apache Arrow)

Data modeling

ETL pipelines

Real-time streaming technologies

Analytical skills

Job description

A data solutions company in London is seeking a skilled Data Engineer to work with high-volume datasets and cutting-edge technologies. In this role, you will build and maintain data pipelines, integrate real-time data feeds, and support machine learning applications. The ideal candidate has over 3 years of experience in data engineering, strong proficiency in Python and Rust, and hands-on experience with AWS, Linux, and Docker. If you are ready to advance your data engineering career, we want to hear from you!