Software Engineer (HPC Sustainability)

1000 European Molecular Biology Laboratory; Hinxton; Hybrid; GBP 35,000 - 42,000; Full time

Benefits offered by this job

Flexible working

30 days annual leave

Shuttle bus

On-site library

Gym and cafeteria

Casual dress code

Sports & social clubs

Job summary

EMBL-EBI is seeking a backend data engineer to join the HPC sustainability initiative. You will develop Python-based services to process HPC telemetry and energy metrics, collaborate with the Green Algorithms team, and extend reporting tooling.

This role requires strong proficiency in data pipelines, time-series databases, and open-source collaboration. You will work across two teams (EBI and Cambridge) to translate trade-offs into proposals, build consensus, and contribute to open-source code

Qualifications

• Strong experience writing production-grade Python for data pipelines and backend services.
• Solid experience with PostgreSQL and time-series data management.
• Familiarity with TimescaleDB including continuous aggregates is an advantage; comfortable with psycopg2/psycopg3.
• Working knowledge of Apache Kafka as a message broker and real-time pipeline design.
• Proficiency with pandas DataFrames for data transformation and analysis.

Responsibilities

• Develop and maintain Python-based backend services that process HPC job telemetry and energy metrics.
• Contribute to the Green Algorithms open source project with Cambridge GA team.
• Work with time series databases for scalable telemetry storage, aggregation, and retrieval.
• Extend carbon and energy reporting tooling and align with regional methodologies.
• Support Grafana dashboards for environmental impact visibility.
• Contribute to documentation and cross-team collaboration including the E-SCOUT trial.
• Produce clear technical documentation and share knowledge within the team.

Skills

Python

PostgreSQL

Data pipelines

Pandas

CI/CD

Psycopg2 / Psycopg3

Apache Kafka

Timeseries data

Git

pytest

ruff

Poetry

AI-assisted tooling

Open-source collaboration

Tools

PostgreSQL

TimescaleDB

Apache Kafka

Git

pytest

ruff

Poetry

Job description

About the team

EMBL-EBI's HPC infrastructure underpins world-leading bioinformatics research, operating large-scale compute clusters on behalf of scientific users across the institute. The HPC sub team sits within IT and Technical Services (ITS) and is responsible for the reliability, operation, and ongoing development of these clusters and their supporting tooling.

Your role Your role is embedded within that HPC sub team, reporting to the HPC Lead. You will combine hands-on backend development with a meaningful sustainability mission: improving how EBI measures, reports, and reduces the environmental impact of its scientific computing. A significant part of your work involves a close collaboration with the Green Algorithms team at the University of Cambridge - an internationally recognised open source project for quantifying the carbon footprint of research computing and to align EBI's tooling with the global GA ecosystem and contribute to the E-SCOUT international trial. Beyond this collaboration, you will contribute to the broader HPC sustainability agenda at EBI, including improving energy and carbon reporting, supporting user facing tooling, and helping establish best practices for sustainable HPC operations. This is a technically demanding role for someone who is equally comfortable working independently in a production codebase and collaborating across organisational boundaries.

Key Responsibilities

• Develop and maintain Python based backend services that process HPC job telemetry and compute per job energy consumption and carbon footprint metrics.
• Contribute to the Green Algorithms open source project, working in close collaboration with the Cambridge GA team to integrate EBI's real time job monitoring capabilities into the shared GA codebase.
• Work with time series databases to ensure efficient storage, aggregation, and retrieval of large scale HPC telemetry data.
• Extend and improve carbon and energy reporting tooling, including alignment with established methodologies (e.g., regional carbon intensity, memory energy, GPU utilisation weighting).
• Support the development of Grafana based dashboards that give HPC users and stakeholders visibility into the environmental impact of their workloads.
• Contribute to wider HPC sustainability initiatives within EBI, including documentation, user guidance, and participation in the E-SCOUT international trial.
• Produce clear technical documentation and actively share knowledge within the team.

You have Strong experience writing production-grade Python, with a focus on data pipelines, concurrent processing, and clean, maintainable code. This is a backend data engineering role, not web API development. Solid experience with PostgreSQL and time-series data management. Familiarity with TimescaleDB including continuous aggregates is an advantage, as is comfort with both psycopg2 and psycopg3. Working knowledge of Apache Kafka as a message broker: consumer groups, offset management, and event-driven pipeline design. Kafka is a core component of EBI's real time job monitoring system. Confidence working with pandas DataFrames for data transformation, aggregation, and analysis used extensively in the Green Algorithms codebase. Proficient with Git, pytest, ruff, and Poetry. Able to contribute effectively to a collaborative open-source environment with CI/CD pipelines. Proficiency in agentic coding and AI-assisted development tools (e.g., Claude Code, GitHub Copilot, Cursor, or similar) to accelerate development within a well-structured codebase. Strong written and verbal communication skills. Able to work across two independent teams (EBI and Cambridge), translate technical trade-offs into clear proposals, and build consensus with stakeholders from different organisational backgrounds. You may also have Prior experience contributing to open-source projects, including code review, documentation, and working within established.

Contract length: 6 month temporary contract

Salary: Grade 5 - salary from £3,452.02 after tax.

Closing Date 30/09/2026

Why join us

Do something meaningful At EMBL-EBI you can apply your talent and passion to accelerate science and tackle some of humankind's greatest challenges. EMBL-EBI, part of the European Molecular Biology Laboratory, is a worldwide leader in the storage, analysis and dissemination of large biological datasets. We provide the global research community with access to publicly available databases and tools which are crucial for the advancement of healthcare, food security, and biodiversity.

Join a culture of innovation We are located on the Wellcome Genome Campus, alongside other prominent research and biotech organisations, and surrounded by beautiful Cambridgeshire countryside. This is a highly collaborative and inclusive community where our employees enjoy a relaxed atmosphere. We are committed to ensuring our employees feel valued, supported and empowered to reach their professional potential.

Campus life & benefits

• Flexible working arrangements - including hybrid working patterns
• Generous time off: 30 days annual leave per year, in addition to public holidays
• Campus life: Free shuttle bus to and from work, on-site library, subsidised on-site gym and cafeteria, casual dress code, extensive sports and social club activities (on campus and remotely)
• Family benefits: On-site nursery, maternity leave and holiday clubs on campus

International applicants

We recruit internationally and successful candidates are offered visa exemptions.

Diversity and inclusion

At EMBL, we believe that diverse teams drive innovation and scientific excellence. We encourage applications from candidates of all genders, identities, nationalities and/or any other diverse backgrounds.