Postdoctoral Researcher (OCCL)

Eidgenössische Technische Hochschule Zürich; Singapore; On-site; SGD 60,000 - 84,000; Full time

Job summary

The Singapore-ETH Centre in Singapore is seeking a Postdoctoral Researcher to co-lead the development of a novel open-source accelerator communication library (OCCL) for distributed AI workloads.

You will design communication primitives and optimise patterns for ML training and inference, working across host and device kernels.

Qualifications

• PhD in Computer Science or related field.
• Strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems; excellent knowledge of C and C++.
• Good working knowledge of MPI and NCCL.
• Experience with HPC networking technologies such as UCX, NVLINK, InfiniBand, UltraEthernet or related.
• Prior knowledge of distributed ML systems and related publications desirable.
• Experience managing larger software engineering projects.

Responsibilities

• Co-lead the design and development of an open-source accelerator communication library (OCCL) for distributed AI workloads.
• Design and implement communication primitives and algorithms for training and inference.
• Optimise communication patterns for large-scale DL workloads and overlap with computation.
• Ensure portability across hardware and network substrates (UCX, NVLINK, InfiniBand, etc.).
• Evaluate performance on GPU clusters and large-scale systems; ensure robustness and usability.
• Collaborate with researchers across Singapore and Zurich; integrate OCCL with FastTrackAI ecosystem.
• Prepare and publish scientific outputs and supervise MSc/BSc students.

Skills

PhD in CS

Parallel computing

C/C++

MPI

NCCL

UCX

NVLINK

InfiniBand

HPC systems

Distributed ML

Education

PhD in Computer Science

Tools

UCX

NVLINK

InfiniBand

Job description

The Singapore-ETH Centre was established in 2010 by ETH Zurich - The Swiss Federal Institute of Technology and Singapore’s National Research Foundation (NRF), as part of the NRF’sCREATE campus. As ETH Zurich's only research centre outside of Switzerland, the centre has strengthened the research capacity of ETH Zurich to develop sustainable solutions to global challenges in Switzerland, Singapore and the surrounding regions.

The centre serves as an intellectual hub for research, bringing together principal investigators and researchers from diverse disciplines and backgrounds. To promote the exchange of ideas and expertise, our researchers actively collaborate with universities and research institutes and engage with industry and government agencies to translate knowledge to practical solutions to real-world problems.

Project background

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the \"graph of thoughts\". We are firmly in the \"Age of Computation\", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid growth in computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the \"graph of thoughts\". We are firmly in the \"Age of Computation\", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid grow thin computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

Our collaborative efforts with the A*STAR Institute of Advanced Intelligence and Computing (ASTAR IAIC) and other partners in Singapore, such as NTU, could be pivotal in this field, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while accelerating scientific productivity. To enhance the efficacy of AI for Science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration is aimed at creating new models imbued with robust prior knowledge, such as simulated behaviors defined by fundamental equations and conservation laws. Our approach involve the close collaboration of experts from A*STAR IAIC, who bring extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

The Postdoctoral Researcher will be co-leading the development of a novel open-source accelerator communication library (OCCL). This communication library should support point-to-point as well as collective communication calls. It should allow communication initiated by the host, as well as communication initiated within accelerator device kernels. It should support a wide range of devices and network substrates, such as NVLINK and next-generation ethernet networks.

Some key components in the work are:

• Co-lead the design and development of an open-source accelerator communication library (OCCL) supporting point-to-point and collective communication for distributed AI workloads.
• Design and implement communication primitives and algorithms optimised for machine learning training and inference workloads, supporting communication initiated by both host applications and accelerator device kernels.
• Investigate and optimise communication patterns for large-scale deep learning workloads, including techniques for improving efficiency through communication scheduling and overlap with computation.
• Ensure the communication library is configurable and portable across diverse hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet and related networking frameworks.
• Evaluate and optimise the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness and usability across heterogeneous computing architectures.
• Collaborate with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.
• Maintain effective communication and collaboration with supervisors, research staff and external partners distributed across Singapore and Zurich, ensuring alignment of research objectives and deliverables.
• Prepare and publish key scientific outputs, including papers documenting research outcomes and system developments.
• Contribute to the co-supervision of MSc and BSc students working on related research topics.

Profile

You will have:

• PhD in Computer Science or any related field.
• The successful candidates should have a strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems. Excellent knowledge of C and C++ is required.
• A good working knowledge of the design principles of existing high-level communication libraries, such as MPI and NCCL.
• Experience and/or knowledge of modern communication and network technologies used in HPC and ML environments, such as UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.
• Prior knowledge of communication optimisation for large-scale ML training and inference workloads is essential. A candidate with existing publications in distributed ML systems, HPC systems or related are highly desirable.
• Since the aim of this project is to build a software product, not only a prototype, prior experience in managing and carrying out larger software engineering projects is required.
• The team working on this subject is distributed across Singapore and Zurich, thus a successful candidate must both be able to work independently and communicate effectively with remote colleagues.
• Applicants need to be proficient in written and spoken English.

Workplace

We offer

• Accredited with 5 Tripartite Standards by Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.
• A diverse workplace with 32 nationalities, offeringample opportunitiesfor mutual learning.
• Positive and inclusive working environment.
• 25 days of annual leave for fixed-term contracts.
• 1 day of Birthday Leave.
• Committed to being a supportive employer as you prioritize your physical and mental wellness.
• Comprehensive healthcare insurance coverage.
• Flexible hybrid work arrangement (up to 2 days per week from home).
• Accredited with NS mark certification.

The Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated on their merits and qualifications, without regards to gender, race, age or religion.

In line with our values , ETH Zurich encourages an inclusive culture. We promote equality of opportunity, value diversity and nurture a working and learning environment in which the rights and dignity of all our staff and students are respected. Visit our Equal Opportunities and Diversity website to find out how we ensure a fair and open environment that allows everyone to grow and flourish. Sustainability is a core value for us - we are consistently working towards a climate-neutral future .

Curious? So are we.

We look forward to receiving your online application with the following documents:

• A cover letter with a specific statement of your motivation for the project and how your background and experience align with the position. Please reference the requirements outlined above and relate your own background and past experience to those.
• Your CV including the name and contact information of 2 references.
• A copy of your university transcripts as PDFs.
• Reference or recommendation letters may be included.

Please note that we exclusively accept applications submitted through our online application portal. Applications via email or postal services will not be considered.

We would like to point out that the pre-selection is carried out by the responsible recruiters and not by artificial intelligence.

Work location: 1 Create Way, CREATE Tower, Singapore 138602 (NUS University Town)
Contract Duration: 2 Years

Remuneration will be commensurate with the candidate's qualifications and relevant experience.

Further information about The Singapore-ETH Centre can be found on our website .
For further information, please contact,Mrs Iva Kabosch (ETH Zurich), [e-mail]

The Singapore-ETH Centre provides a multicultural and interdisciplinary environment to researchers working on diverse themes focussed on sustainable and liveable cities, resilient urban systems, and patient-centric healthcare. The centre is home to a community of over 100 doctoral, postdoctoral and professorial researchers working in three main programmes: Future Cities Laboratory, Future Resilient Systems, and Future Health Technologies.