Software Engineer (HPC Sustainability)

European Molecular Biology Laboratory; Cambridgeshire and Peterborough; Hybrid; GBP 42,000 - 54,000; Full time

Benefits offered by this job

Hybrid working patterns

30 days annual leave

Campus facilities: library, gym & cafe

On-site nursery

Job summary

EMBL-EBI is seeking a production‑grade Python backend developer for the HPC sustainability program. You will build data pipelines to process HPC telemetry, measure energy use, and quantify carbon footprint in collaboration with the Green Algorithms team at Cambridge.

The role combines independent production code work with cross‑team coordination and documentation. You will contribute to timeseries data storage and visualization, including Grafana dashboards, and help align tooling with

Qualifications

• This is production-grade Python data pipelines, not web API work.
• Experience with PostgreSQL and time-series data management, including TimescaleDB.
• Familiarity with Kafka and real-time data processing is required.
• Proficient with pandas for data transformation and analysis.

Responsibilities

• Develop and maintain Python-based backend services processing HPC telemetry and energy metrics.
• Collaborate with the Cambridge GA team to integrate real-time monitoring into the GA codebase.
• Work with time series databases for storage, aggregation, and retrieval of large telemetry data.
• Extend carbon and energy reporting tooling and support Grafana dashboards.
• Produce technical docs and share knowledge across EMBL-EBI and Cambridge teams.

Skills

Python backend

PostgreSQL

TimescaleDB

Apache Kafka

pandas

Git

pytest

ruff

Poetry

CI/CD

AI-assisted development

Tools

Grafana

Job description

About the team

EMBL-EBI's HPC infrastructure underpins world-leading bioinformatics research, operating large-scale compute clusters on behalf of scientific users across the institute. The HPC sub team sits within IT and Technical Services (ITS) and is responsible for the reliability, operation, and ongoing development of these clusters and their supporting tooling.

Your role

Your role is embedded within that HPC sub team, reporting to the HPC Lead. You will combine hands‑on backend development with a meaningful sustainability mission: improving how EBI measures, reports, and reduces the environmental impact of its scientific computing. A significant part of your work involves a close collaboration with the Green Algorithms team at the University of Cambridge - an internationally recognised open source project for quantifying the carbon footprint of research computing and to align EBI's tooling with the global GA ecosystem and contribute to the E‑SCOUT international trial. Beyond this collaboration, you will contribute to the broader HPC sustainability agenda at EBI, including improving energy and carbon reporting, supporting user facing tooling, and helping establish best practices for sustainable HPC operations.

This is a technically demanding role for someone who is equally comfortable working independently in a production codebase and collaborating across organisational boundaries.

Key Responsibilities

• Develop and maintain Python based backend services that process HPC job telemetry and compute per job energy consumption and carbon footprint metrics.
• Contribute to the Green Algorithms open source project, working in close collaboration with the Cambridge GA team to integrate EBI's real time job monitoring capabilities into the shared GA codebase.
• Work with time series databases to ensure efficient storage, aggregation, and retrieval of large scale HPC telemetry data.
• Extend and improve carbon and energy reporting tooling, including alignment with established methodologies (e.g., regional carbon intensity, memory energy, GPU utilisation weighting).
• Support the development of Grafana based dashboards that give HPC users and stakeholders visibility into the environmental impact of their workloads.
• Contribute to wider HPC sustainability initiatives within EBI, including documentation, user guidance, and participation in the E‑SCOUT international trial.
• Produce clear technical documentation and actively share knowledge within the team.

You have

• Strong experience writing production-grade Python, with a focus on data pipelines, concurrent processing, and clean, maintainable code. This is a backend data engineering role, not web API development.
• Solid experience with PostgreSQL and time‑series data management. Familiarity with TimescaleDB including continuous aggregates is an advantage, as is comfort with both psycopg2 and psycopg3.
• Working knowledge of Apache Kafka as a message broker: consumer groups, offset management, and event‑driven pipeline design. Kafka is a core component of EBI's real time job monitoring system.
• Confidence working with pandas DataFrames for data transformation, aggregation, and analysis used extensively in the Green Algorithms codebase.
• Proficient with Git, pytest, ruff, and Poetry. Able to contribute effectively to a collaborative open‑source environment with CI/CD pipelines.
• Proficiency in agentic coding and AI‑assisted development tools (e.g., Claude Code, GitHub Copilot, Cursor, or similar) to accelerate development within a well‑structured codebase.
• Strong written and verbal communication skills. Able to work across two independent teams (EBI and Cambridge), translate technical trade‑offs into clear proposals, and build consensus with stakeholders from different organisational backgrounds.

You may also have

• Prior experience contributing to open‑source projects, including code review, documentation and working within established.
• Experience building or modifying Grafana dashboards programmatically.
• Genuine interest in green computing and the environmental impact of scientific infrastructure.

Contract length

6 month temporary contract

Salary

Grade 5 – salary from £3,452.02 after tax.

Why join us

Do something meaningful

At EMBL-EBI you can apply your talent and passion to accelerate science and tackle some of humankind’s greatest challenges. EMBL-EBI, part of the European Molecular Biology Laboratory, is a worldwide leader in the storage, analysis and dissemination of large biological datasets. We provide the global research community with access to publicly available databases and tools which are crucial for the advancement of healthcare, food security, and biodiversity.

Join a culture of innovation

We are located on the Wellcome Genome Campus, alongside other prominent research and biotech organisations, and surrounded by beautiful Cambridgeshire countryside. This is a highly collaborative and inclusive community where our employees enjoy a relaxed atmosphere. We are committed to ensuring our employees feel valued, supported and empowered to reach their professional potential.

Campus life & benefits:

• Flexible working arrangements - including hybrid working patterns
• Generous time off: 30 days annual leave per year, in addition to public holidays
• Campus life: Free shuttle bus to and from work, on‑site library, subsidised on‑site gym and cafeteria, casual dress code, extensive sports and social club activities (on campus and remotely)
• Family benefits: On‑site nursery, maternity leave and holiday clubs on campus

What else you need to know

• International applicants: We recruit internationally and successful candidates are offered visa exemptions.
• Learn how EMBL-EBI makes an impact
• Diversity and inclusion: At EMBL, we believe that diverse teams drive innovation and scientific excellence. We encourage applications from candidates of all genders, identities, nationalities and/or any other diverse backgrounds.

Closing Date

30/09/2026