Software Engineer (HPC Sustainability)

European Molecular Biology Laboratory; Cambridge; Hybrid; GBP 52,000 - 72,000; Full time

Benefits offered by this job

Hybrid working patterns

30 days annual leave

Campus shuttle and facilities

On-site nursery support

Job summary

EMBL-EBI is seeking a backend data engineer to join the HPC sub team and help measure, report, and reduce environmental impact of scientific computing. You will develop Python services for HPC telemetry, collaborate with the Green Algorithms team at Cambridge, and contribute to Grafana dashboards and real-time monitoring.

This is a hybrid role based in Cambridge/Biomed campus, with a strong focus on sustainability.

Qualifications

• Production-grade Python with data pipelines and maintainable code.
• Experience with time-series data and PostgreSQL.
• Familiarity with Kafka-based real-time pipelines.
• Proficient in data transformation using Pandas.
• Open-source contribution and CI/CD familiarity.

Responsibilities

• Develop and maintain Python backend services for HPC telemetry.
• Process energy and carbon footprint metrics per job.
• Collaborate with Cambridge GA team to integrate monitoring into GA codebase.
• Work with time series DBs for large-scale telemetry data.
• Enhance carbon and energy reporting tooling and dashboards.
• Produce technical docs and share knowledge across teams.

Skills

Python

Data pipelines

Concurrency

Markdown/Documentation

Pandas

Git

pytest

ruff

Poetry

AI-assisted development

Tools

PostgreSQL

TimescaleDB

Kafka

Grafana

Pandas (lib)

Git tooling

Job description

About the team

EMBL-EBI's HPC infrastructure underpins world-leading bioinformatics research, operating large-scale compute clusters on behalf of scientific users across the institute. The HPC sub team sits within IT and Technical Services (ITS) and is responsible for the reliability, operation, and ongoing development of these clusters and their supporting tooling.

Your role

Your role is embedded within that HPC sub team, reporting to the HPC Lead. You will combine hands-on backend development with a meaningful sustainability mission: improving how EBI measures, reports, and reduces the environmental impact of its scientific computing. A significant part of your work involves a close collaboration with the Green Algorithms team at the University of Cambridge - an internationally recognised open source project for quantifying the carbon footprint of research computing and to align EBI's tooling with the global GA ecosystem and contribute to the E-SCOUT international trial. Beyond this collaboration, you will contribute to the broader HPC sustainability agenda at EBI, including improving energy and carbon reporting, supporting user facing tooling, and helping establish best practices for sustainable HPC operations.

Key Responsibilities

• Develop and maintain Python based backend services that process HPC job telemetry and compute per job energy consumption and carbon footprint metrics.
• Contribute to the Green Algorithms open source project, working in close collaboration with the Cambridge GA team to integrate EBI's real time job monitoring capabilities into the shared GA codebase.
• Work with time series databases to ensure efficient storage, aggregation, and retrieval of large scale HPC telemetry data.
• Extend and improve carbon and energy reporting tooling, including alignment with established methodologies (e.g., regional carbon intensity, memory energy, GPU utilisation weighting).
• Support the development of Grafana based dashboards that give HPC users and stakeholders visibility into the environmental impact of their workloads.
• Contribute to wider HPC sustainability initiatives within EBI, including documentation, user guidance, and participation in the E-SCOUT international trial.
• Produce clear technical documentation and actively share knowledge within the team.

You have

• Strong experience writing production-grade Python, with a focus on data pipelines, concurrent processing, and clean, maintainable code. This is a backend data engineering role, not web API development.
• Solid experience with PostgreSQL and time-series data management. Familiarity with TimescaleDB including continuous aggregates is an advantage, as is comfort with both psycopg2 and psycopg3.
• Working knowledge of Apache Kafka as a message broker: consumer groups, offset management, and event-driven pipeline design. Kafka is a core component of EBI's real time job monitoring system.
• Confidence working with pandas DataFrames for data transformation, aggregation, and analysis used extensively in the Green Algorithms codebase.
• Proficient with Git, pytest, ruff, and Poetry. Able to contribute effectively to a collaborative open-source environment with CI/CD pipelines.
• Proficiency in agentic coding and AI-assisted development tools (e.g., Claude Code, GitHub Copilot, Cursor, or similar) to accelerate development within a well-structured codebase.
• Strong written and verbal communication skills. Able to work across two independent teams (EBI and Cambridge), translate technical trade-offs into clear

You may also have

• Prior experience contributing to open-source projects, including code review, documentation, and working within established.
• Experience building or modifying Grafana dashboards programmatically.
• Genuine interest in green computing and the environmental impact of scientific infrastructure.

Contract length: 6 month temporary contract

Salary: Grade 5 - salary from £3,452.02 after tax.

Why join us

Do something meaningful

At EMBL-EBI you can apply your talent and passion to accelerate science and tackle some of humankind's greatest challenges. EMBL-EBI, part of the European Molecular Biology Laboratory, is a worldwide leader in the storage, analysis and dissemination of large biological datasets. We provide the global research community with access to publicly available databases and tools which are crucial for the advancement of healthcare, food security, and biodiversity.

Join a culture of innovation

We are located on the Wellcome Genome Campus, alongside other prominent research and biotech organisations, and surrounded by beautiful Cambridgeshire countryside. This is a highly collaborative and inclusive community where our employees enjoy a relaxed atmosphere. We are committed to ensuring our employees feel valued, supported and empowered to reach their professional potential.

Campus life & benefits

• Flexible working arrangements - including hybrid working patterns
• Generous time off: 30 days annual leave per year, in addition to public holidays
• Campus life: Free shuttle bus to and from work, on-site library, subsidised on-site gym and cafeteria, casual dress code, extensive sports and social club activities (on campus and remotely)
• Family benefits: On-site nursery, maternity leave and holiday clubs on campus

What else you need to know

• International applicants: We recruit internationally and successful candidates are offered visa exemptions.
• Learn how EMBL-EBI makes an impact
• Diversity and inclusion: At EMBL, we believe that diverse teams drive innovation and scientific excellence. We encourage applications from candidates of all genders, identities, nationalities and/or any other diverse backgrounds.

Closing Date

30/09/2026