Data Engineer

VirtueTech Recruitment Group; Greater London; On-site; GBP 70,000 - 100,000; Full time

Job summary

VirtueTech Recruitment Group is seeking a highly technical data integration engineer for a cross-asset trading house in London. You will design and implement ingestion, orchestration and distribution solutions using service-based and event-driven architectures, bridging data from bespoke platforms and external providers.

You will work with enormous data volumes and high-throughput streams, focusing on reliability, observability and real-time processing across data platforms and applications.

Qualifications

• Hands-on Apache Kafka expertise for enterprise-scale data integration.
• Experience designing API-led, event-driven and service-based integration solutions.
• Proficiency in Python and SQL for data engineering workloads.

Responsibilities

• Design, build and optimize large-scale data pipelines ingesting data from distributed sources.
• Implement resilient ingestion, orchestration and distribution services with observability.
• Work on throughput, latency, data consistency, and schema evolution in real-time systems.
• Collaborate across data, application and infrastructure layers to enable robust integrations.

Skills

Apache Kafka

Event-driven integration

Python

SQL

CI/CD

Observability

Distributed systems

Job description

This engineer is needed for a cross asset trading house who are looking to consume and distribute data at scale, faster, more securely and in a replicable manner, as they continue to add trading products, acquire companies and push their SAAS platform at scale.

The emphasis is on the INTEGRATION as you will be responsible for designing and implementing resilient ingestion, orchestration and distribution solutions, using service-based and event-driven architectures. Bridging data into systems, from an extraction and distribution perspective, ensuring integration Is smooth and accurate.

Extracting Data from both bespoke internally built platforms, external market data providers, CRM’s, off-the-shelf platforms, Kafka Services & other sources. Integrating this data into a variety of different sources, ensuring observability, speed and accuracy, whilst also thinking about scale, as the firm continues to grow.

3 days from London, closest station is Bank / Liverpool Street

Experience with Apache Kafka and event-driven integration architectures & designing and implementing enterprise-scale API-led, event-driven, and service-based integration solutions across data platforms.

This role will suit a highly technical engineer to design, build and optimise large-scale data pipelines that ingest and transform data from highly distributed, event-driven systems. You’ll work with enormous volumes of data and high-throughput event streams, building resilient, scalable integration services that can process data reliably in real time and near real time. The role will involve working deep in the technology stack, tackling challenges around throughput, latency, data consistency, schema evolution, fault tolerance and observability.

You need to be be comfortable working with event-driven architectures, streaming platforms, APIs and distributed systems, and have a strong understanding of how to engineer data pipelines that remain performant and reliable at scale. This will suit a data integration engineer who enjoys solving complex engineering problems, understands the trade-offs involved in processing huge datasets, and can work across application, data and infrastructure layers to deliver robust integrations that underpin critical business processes.

• Strong hands-on experience with Apache Kafka and event-driven integration architectures.
• Proven expertise designing and implementing enterprise-scale API-led, event-driven, and service-based integration solutions across data platforms, applications, and operational systems.
• Experience implementing CI/CD pipelines, automated testing, and modern software engineering practices.
• Strong proficiency in Python and SQL for enterprise data engineering and integration workloads.
• How Integration architectures and engineering practices influence data quality, observability, lineage, traceability, and auditability.