Postdoctoral Researcher (OCCL)

Singapore-ETH Centre; Singapore; Hybrid; SGD 67,000 - 112,000; Full time

Benefits offered by this job

Flexible hybrid work arrangement

25 days annual leave

Comprehensive healthcare coverage

Birthday leave

Job summary

The Singapore-ETH Centre is inviting a Postdoctoral Researcher to co-lead the design and development of an open-source accelerator communication library (OCCL) for distributed AI workloads, in close collaboration with partners in Singapore and Zurich.

The successful candidate will hold a PhD in Computer Science, possess deep HPC expertise, strong C/C++ skills, and experience with MPI/NCCL and modern interconnects. A competitive salary is offered along with a flexible hybrid work arrangement.

Qualifications

• PhD in Computer Science or related field.
• Strong background in parallel computing and HPC.
• Experience with MPI/NCCL and modern interconnects.

Responsibilities

• Co-lead the design and development of OCCL for distributed AI workloads.
• Implement communication primitives and algorithms for host and device kernels.
• Optimize communication patterns for large-scale ML training and inference.
• Ensure portability across UCX, NVLINK, InfiniBand, and UltraEthernet networks.
• Collaborate with Singapore and Zurich partners; publish scientific outputs.
• Co-supervise MSc and BSc students on related topics.

Skills

C/C++ programming

Parallel computing

High-performance computing

English communication

Education

PhD in Computer Science

Tools

MPI

NCCL

UCX

NVLINK

InfiniBand

UltraEthernet

Job description

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the \"graph of thoughts\". We are firmly in the \"Age of Computation\", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid growth in computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the \"graph of thoughts\". We are firmly in the \"Age of Computation\", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid grow thin computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

Our collaborative efforts with the A*STAR Institute of Advanced Intelligence and Computing (ASTAR IAIC) and other partners in Singapore, such as NTU, could be pivotal in this field, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while accelerating scientific productivity. To enhance the efficacy of AI for Science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration is aimed at creating new models imbued with robust prior knowledge, such as simulated behaviors defined by fundamental equations and conservation laws. Our approach involves the close collaboration of experts from A*STAR IAIC, who bring extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

The Postdoctoral Researcher will be co-leading the development of a novel open-source accelerator communication library (OCCL). This communication library should support point-to-point as well as collective communication calls. It should allow communication initiated by the host, as well as communication initiated within accelerator device kernels. It should support a wide range of devices and network substrates, such as NVLINK and next-generation ethernet networks.

Some key components in the work are:

• Co-lead the design and development of an open-source accelerator communication library (OCCL) supporting point-to-point and collective communication for distributed AI workloads.


• Design and implement communication primitives and algorithms optimised for machine learning training and inference workloads, supporting communication initiated by both host applications and accelerator device kernels.


• Investigate and optimise communication patterns for large-scale deep learning workloads, including techniques for improving efficiency through communication scheduling and overlap with computation.


• Ensure the communication library is configurable and portable across diverse hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet and related networking frameworks.


• Evaluate and optimise the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness and usability across heterogeneous computing architectures.


• Collaborate with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.


• Maintain effective communication and collaboration with supervisors, research staff and external partners distributed across Singapore and Zurich, ensuring alignment of research objectives and deliverables.


• Prepare and publish key scientific outputs, including papers documenting research outcomes and system developments.


• Contribute to the co-supervision of MSc and BSc students working on related research topics.

Selection Criteria

• PhD in Computer Science or any related field.


• The successful candidates should have a strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems. Excellent knowledge of C and C++ is required.


• A good working knowledge of the design principles of existing high-level communication libraries, such as MPI and NCCL.


• Experience and/or knowledge of modern communication and network technologies used in HPC and ML environments, such as UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.


• Prior knowledge of communication optimisation for large-scale ML training and inference workloads is essential. A candidate with existing publications in distributed ML systems, HPC systems or related are highly desirable.


• Since the aim of this project is to build a software product, not only a prototype, prior experience in managing and carrying out larger software engineering projects is required.


• The team working on this subject is distributed across Singapore and Zurich, thus a successful candidate must both be able to work independently and communicate effectively with remote colleagues.


• Applicants need to be proficient in written and spoken English.

Why SEC is your employer of choice?

• Accredited with 5 Tripartite Standards by Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.


• A diverse workplace with 32 nationalities, offering ample opportunities for mutual learning.


• Positive and inclusive working environment.


• 25 days of annual leave for fixed-term contracts.


• 1 day of Birthday Leave.


• Committed to being a supportive employer as you prioritize your physical and mental wellness.


• Comprehensive healthcare insurance coverage.


• Flexible hybrid work arrangement (up to 2 days per week from home).


• Accredited with NS mark certification.

Curious? So are we.

The Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated on their merits and qualifications, without regards to gender, race, age or religion.

Work location: 1 Create Way, CREATE Tower, Singapore 138602 (NUS University Town)

Contract Duration: 2 Years

Remuneration will be commensurate with the candidate's qualifications and relevant experience.

Further information about The Singapore-ETH Centre can be found on our website: https://sec.ethz.ch/

For further information, please contact: Mrs Iva Kabosch (ETH Zurich) at [e-mail]