Data Engineer - AI

Ellison Institute of Technology; Oxford; On-site; GBP 50,000 - 70,000; Full time

Job summary

A leading research institution in Oxford is seeking a Data Engineer to join their team. In this role, you will design and build distributed data systems that support groundbreaking research across various scientific domains. You will work closely with scientists and engineers, creating scalable systems to handle multimodal datasets. Strong programming skills in Python and SQL, along with experience in distributed data systems, are essential. This position offers a collaborative work environment and competitive benefits.

Benefits

Enhanced holiday pay

Pension

Private Medical Insurance

Electric Car Scheme

Therapy Services

Qualifications

• Strong programming experience in Python and SQL with focus on code quality.
• Experience in designing and optimising distributed data systems.
• Collaborative mindset and clear communication skills.

Responsibilities

• Design and build distributed data systems for scientific research.
• Architect APIs for access to multimodal datasets.
• Develop data pipelines for various types of data.

Skills

Python

SQL

Distributed data systems

Collaboration

Observability

Tools

Spark

Kafka

FastAPI

Job description

The Ellison Institute of Technology (EIT) Oxford’s purpose is to have a global impact by fundamentally reimagining the way science and technology translate into end-to-end solutions and delivering these solutions in programmes and platforms that respond to humanity’s most challenging problems.

EIT Oxford will ensure scientific discoveries and pioneering science are turned into products for the benefit of society that can have high-impact worldwide and, over time, be commercialised to ensure long-term sustainability.

Led by a world-class faculty of scientists, technologists, policy makers, economists and entrepreneurs, the Ellison Institute of Technology aims to develop and deploy commercially sustainable solutions to solve some of humanity’s most enduring challenges. Our work is guided by four Humane Endeavours: Health, Medical Science & Generative Biology, Food Security & Sustainable Agriculture, Climate Change & Managing Atmospheric CO2 and Artificial Intelligence & Robotics.

Set for completion in 2027, the EIT Campus in Littlemore will include more than 300,000 sq ft of research laboratories, educational and gathering spaces. Fuelled by growing ambition and the strength of Oxford’s science ecosystem, EIT is now expanding its footprint to a 2 million sq ft Campus across the western part of The Oxford Science Park. Designed by Foster + Partners led by Lord Norman Foster, this will become a transformative workplace for up to 7,000 people, with autonomous laboratories, purpose-built laboratories including a plant sciences building and dynamic spaces to spark interdisciplinary collaboration.

The Role:

Our Data Engineering Team builds the core data systems that power frontier research across EIT. As an early member of our Data Engineering team, you’ll design and build the platforms used by scientists and engineers in fields such as healthcare, robotics, agriculture, and AI. You’ll work alongside our MLOps and Infrastructure teams to create reliable, scalable systems capable of handling large-scale (from TB to PB+), multimodal datasets.

EIT is unique in combining foundational data from diverse disciplines into a single research ecosystem. You’ll help develop the technical foundation that makes this possible: platforms, services, APIs and distributed systems that are robust, observable and easy to work with. This is a role for engineers who think long-term and want to build a platform that will underpin the next generation of scientific and technological discovery.

Day-to-Day, You Might:

• Design and build distributed data systems that support research across EIT’s scientific domains.
• Architect APIs and services for high-throughput, low-latency access to multimodal datasets.
• Work with MLOps, Infrastructure and data engineers embedded within research teams to integrate systems into active research workflows.
• Develop pipelines for large-scale text, audio, video, imaging, sensor, and structured data on OCI.
• Add observability, monitoring, and automated quality checks to ensure the trustworthiness of every dataset.
• Contribute to an engineering culture that values maintainability, testing, clear system design, and deep collaboration with our researchers and scientists.

What Makes You a Great Fit:

• You have strong programming experience in Python and SQL, and value code quality, reliability (including testing, CI/CD) and observability as much as performance.
• You have experience designing, deploying, and optimising distributed data systems or data-intensive backend services.
• You think in terms of systems and longevity, not just one-off ETL scripts, and embrace end-to-end ownership from low-level performance to user interfaces.
• You’re a collaborative partner to Infrastructure/Ops teams and researchers; clear, respectful communicator.
• You have a low-ego, team-first mindset and help grow our engineering culture by mentoring, sharing, and elevating the work of those around you.

Great to Also Have

Nobody checks every box - if you’re not sure if you’re qualified, we still encourage you to apply.

• You’re used to working with modern tech stacks and developing for distributed systems, for example Spark/Flink/Kafka, Polars/Arrow, Airflow/Prefect.
• You’ve contributed to shared Python libraries used across multiple teams and maintained dependency and packaging standards (e.g. Poetry, pip-tools).
• You have experience integrating multimodal datasets (text, video, imaging, sensor data) into unified platforms.
• You’ve designed and optimised robust, high-performance APIs for data ingestion/consumption using tools such as FastAPI, gRPC, and GraphQL, and use tools such as Prometheus and OpenTelemetry to maintain SLAs.
• You’re curious about database internals, storage engines, and low-latency query processing.
• You’ve built web apps and dashboards using tools such as Dash or frameworks like React.
• You’ve managed schema evolution, data versioning, and governance in production with tools such as Open Policy Agent and Apache Hive Metastore.

We offer the following salary and benefits:

• Enhanced holiday pay
• Pension
• Life Assurance
• Income Protection
• Private Medical Insurance
• Hospital Cash Plan
• Therapy Services
• Perk Box
• Electric Car Scheme
• --

Why work for EIT:

At the Ellison Institute, we believe a collaborative, inclusive team is key to our success. We are building a supportive environment where creative risks are encouraged, and everyone feels heard. Valuing emotional intelligence, empathy, respect, and resilience, we encourage people to be curious and to have a shared commitment to excellence. Join us and make an impact!