Postdoctoral Researcher (OCCL)

Immigration Policy Lab; Singapore; On-site; SGD 60,000 - 90,000; Full time

Benefits offered by this job

TAFEP accreditation

Birthday leave

Dental benefits

Healthcare insurance

Flexible hybrid work

Networking opportunities

NS mark certification

Job summary

The Postdoctoral Researcher at Singapore-ETH Centre will co-lead the development of a novel open-source accelerator communication library (OCCL) for distributed AI workloads, enabling point-to-point and collective communication across host and accelerator device kernels.

The role involves implementing optimized communication primitives, ensuring portability across UCX, NVLINK, InfiniBand and related networks, collaborating with researchers in Singapore and Zurich, and publishing results while

Qualifications

• PhD in Computer Science or related field.
• Strong background in parallel computing, compilers, performance optimization, and HPC systems.
• Excellent knowledge of C and C++.
• Experience with MPI, NCCL, UCX, NVLINK, InfiniBand, UltraEthernet.

Responsibilities

• Co-lead the design and development of OCCL for distributed AI workloads.
• Implement communication primitives and algorithms for training and inference.
• Optimize communication patterns for large-scale DL workloads.
• Ensure portability across hardware and networks (UCX, NVLINK, InfiniBand, UltraEthernet).
• Evaluate performance on GPU clusters and HPC systems.
• Collaborate with researchers at partner institutions in Singapore and Zurich.
• Publish scientific outputs and papers.
• Contribute to supervising MSc and BSc students.

Skills

C/C++ programming

Parallel computing

HPC systems

MPI/NCCL

UCX/NVLINK/InfiniBand

English communication

Software engineering leadership

Education

PhD in Computer Science

Tools

MPI

NCCL

UCX

NVLINK

InfiniBand

UltraEthernet

Job description

Postdoctoral Researcher (OCCL)

100%, Singapore, fixed-term

The Singapore-ETH Centre was established in 2010 by ETH Zurich - The Swiss Federal Institute of Technology and Singapore's National Research Foundation (NRF), as part of the NRF's CREATE campus. As ETH Zurich's only research centre outside of Switzerland, the centre has strengthened the research capacity of ETH Zurich to develop sustainable solutions to global challenges in Switzerland, Singapore and the surrounding regions.

Set in Asia, in a rapidly urbanising region, the Singapore-ETH Centre aims to provide practical solutions to some of the most pressing challenges on urban sustainability, resilience and health through its programmes: Future Cities Lab Global (FCL Global) and Future Health Technologies (FHT).

The centre serves as an intellectual hub for research, bringing together principal investigators and researchers from diverse disciplines and backgrounds. To promote the exchange of ideas and expertise, our researchers actively collaborate with universities and research institutes and engage with industry and government agencies to translate knowledge to practical solutions to real-world problems.

Project background

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the "graph of thoughts". We are firmly in the "Age of Computation", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid growth in computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country's researchers to advance in this most important field.

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the "graph of thoughts". We are firmly in the "Age of Computation", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid grow thin computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country's researchers to advance in this most important field.

Our collaborative efforts with the A*STAR Institute of Advanced Intelligence and Computing (ASTAR IAIC) and other partners in Singapore, such as NTU, could be pivotal in this field, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while accelerating scientific productivity. To enhance the efficacy of AI for Science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration is aimed at creating new models imbued with robust prior knowledge, such as simulated behaviors defined by fundamental equations and conservation laws. Our approach involves the close collaboration of experts from A*STAR IAIC, who bring extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

Job description

The Postdoctoral Researcher will be co-leading the development of a novel open-source accelerator communication library (OCCL). This communication library should support point-to-point as well as collective communication calls. It should allow communication initiated by the host, as well as communication initiated within accelerator device kernels. It should support a wide range of devices and network substrates, such as NVLINK and next-generation ethernet networks.

Some key components in the work are:

• Co-lead the design and development of an open-source accelerator communication library (OCCL) supporting point-to-point and collective communication for distributed AI workloads.
• Design and implement communication primitives and algorithms optimised for machine learning training and inference workloads, supporting communication initiated by both host applications and accelerator device kernels.
• Investigate and optimise communication patterns for large-scale deep learning workloads, including techniques for improving efficiency through communication scheduling and overlap with computation.
• Ensure the communication library is configurable and portable across diverse hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet and related networking frameworks.
• Evaluate and optimise the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness and usability across heterogeneous computing architectures.
• Collaborate with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.
• Maintain effective communication and collaboration with supervisors, research staff and external partners distributed across Singapore and Zurich, ensuring alignment of research objectives and deliverables.
• Prepare and publish key scientific outputs, including papers documenting research outcomes and system developments.
• Contribute to the co-supervision of MSc and BSc students working on related research topics.

Profile

You will have:

• PhD in Computer Science or any related field.
• The successful candidates should have a strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems. Excellent knowledge of C and C++ is required.
• A good working knowledge of the design principles of existing high-level communication libraries, such as MPI and NCCL.
• Experience and/or knowledge of modern communication and network technologies used in HPC and ML environments, such as UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.
• Prior knowledge of communication optimisation for large-scale ML training and inference workloads is essential. A candidate with existing publications in distributed ML systems, HPC systems or related are highly desirable.
• Since the aim of this project is to build a software product, not only a prototype, prior experience in managing and carrying out larger software engineering projects is required.
• The team working on this subject is distributed across Singapore and Zurich, thus a successful candidate must both be able to work independently and communicate effectively with remote colleagues.
• Applicants need to be proficient in written and spoken English.

Workplace

Workplace

We offer

• Accredited with 5 Tripartite Standards by Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.
• A diverse workplace with 32 nationalities, offering ample opportunities for mutual learning.
• Positive and inclusive working environment.
• 25 days of annual leave for fixed-term contracts.
• 1 day of Birthday Leave.
• Annual dental benefits.
• Committed to being a supportive employer as you prioritize your physical and mental wellness.
• Comprehensive healthcare insurance coverage.
• Flexible hybrid work arrangement (up to 2 days per week from home).
• Abundant networking opportunities across various disciplines.
• Accredited with NS mark certification.

The Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated on their merits and qualifications, without regards to gender, race, age or religion.

chevron_right Working, teaching and research at Singapore-ETH Centre

We value diversity and sustainability

In line with our values, ETH Zurich encourages an inclusive culture. We promote equality of opportunity, value diversity and nurture a working and learning environment in which the rights and dignity of all our staff and students are respected. Visit our Equal Opportunities and Diversity website to find out how we ensure a fair and open environment that allows everyone to grow and flourish. Sustainability is a core value for us – we are consistently working towards a climate-neutral future.

Work location: 1 Create Way, CREATE Tower, Singapore 138602 (NUS University Town)

Contract Duration: 2 Years

Remuneration will be commensurate with the candidate's qualifications and relevant experience.