Postdoctoral Researcher (OCCL)

ETH SINGAPORE SEC LTD.; Singapore; On-site; SGD 70,000 - 110,000; Full time

Benefits offered by this job

Flexible hybrid work

25 days annual leave

Birthday leave

Dental benefits

Healthcare insurance

NS mark certification

Networking opportunities

Job summary

Singapore-ETH Centre in Singapore invites a Postdoctoral Researcher to co-lead the development of OCCL, an open-source accelerator communication library for distributed AI workloads, supporting host and kernel initiated communication across NVLINK and InfiniBand networks.

You will design primitives for ML training, optimize patterns, ensure portability across HPC substrates, collaborate with researchers in Singapore and Zurich, publish results, and supervise MSc/BSc students.

Qualifications

• PhD in Computer Science or related field.
• Strong background in parallel computing, compilers, performance optimisation and HPC.
• Excellent knowledge of C and C++.
• Knowledge of MPI and NCCL design principles.
• Experience with UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.
• Experience or publications in distributed ML systems or HPC.
• Experience managing large software engineering projects.
• Ability to work independently and communicate with remote colleagues.
• Proficiency in written and spoken English.

Responsibilities

• Co-lead design and development of OCCL for distributed AI workloads.
• Implement communication primitives optimised for ML training and inference.
• Investigate and optimise communication patterns for large-scale DL.
• Ensure portability across diverse hardware and networks (UCX, NVLINK, InfiniBand, UltraEthernet).
• Evaluate and optimise OCCL on GPU clusters and large systems.
• Collaborate with researchers in Singapore and Zurich to integrate OCCL with FastTrackAI.
• Maintain communication with supervisors and partners to align objectives.
• Publish key scientific outputs and papers.
• Co-supervise MSc and BSc students on related topics.

Skills

Parallel computing

Compilers

Performance optimisation

High-performance computing

English proficiency

Education

PhD in Computer Science or related field

Tools

MPI

NCCL

UCX

NVLINK

InfiniBand

UltraEthernet

Job description

Project background

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the "graph of thoughts". We are firmly in the "Age of Computation", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid growth in computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the "graph of thoughts". We are firmly in the "Age of Computation", where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid grow thin computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country’s researchers to advance in this most important field.

Our collaborative efforts with the A*STAR Institute of Advanced Intelligence and Computing (ASTAR IAIC) and other partners in Singapore, such as NTU, could be pivotal in this field, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while accelerating scientific productivity. To enhance the efficacy of AI for Science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration is aimed at creating new models imbued with robust prior knowledge, such as simulated behaviors defined by fundamental equations and conservation laws. Our approach involves the close collaboration of experts from A*STAR IAIC, who bring extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

Job description

The Postdoctoral Researcher will be co-leading the development of a novel open-source accelerator communication library (OCCL). This communication library should support point-to-point as well as collective communication calls. It should allow communication initiated by the host, as well as communication initiated within accelerator device kernels. It should support a wide range of devices and network substrates, such as NVLINK and next-generation ethernet networks.

Some key components in the work are:

• Co-lead the design and development of an open-source accelerator communication library (OCCL) supporting point-to-point and collective communication for distributed AI workloads.
• Design and implement communication primitives and algorithms optimised for machine learning training and inference workloads, supporting communication initiated by both host applications and accelerator device kernels.
• Investigate and optimise communication patterns for large-scale deep learning workloads, including techniques for improving efficiency through communication scheduling and overlap with computation.
• Ensure the communication library is configurable and portable across diverse hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet and related networking frameworks.
• Evaluate and optimise the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness and usability across heterogeneous computing architectures.
• Collaborate with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.
• Maintain effective communication and collaboration with supervisors, research staff and external partners distributed across Singapore and Zurich, ensuring alignment of research objectives and deliverables.
• Prepare and publish key scientific outputs, including papers documenting research outcomes and system developments.
• Contribute to the co-supervision of MSc and BSc students working on related research topics.

Selection Criteria

• PhD in Computer Science or any related field.
• The successful candidates should have a strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems. Excellent knowledge of C and C++ is required.
• A good working knowledge of the design principles of existing high-level communication libraries, such as MPI and NCCL.
• Experience and/or knowledge of modern communication and network technologies used in HPC and ML environments, such as UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.
• Prior knowledge of communication optimisation for large-scale ML training and inference workloads is essential. A candidate with existing publications in distributed ML systems, HPC systems or related are highly desirable.
• Since the aim of this project is to build a software product, not only a prototype, prior experience in managing and carrying out larger software engineering projects is required.
• The team working on this subject is distributed across Singapore and Zurich, thus a successful candidate must both be able to work independently and communicate effectively with remote colleagues.
• Applicants need to be proficient in written and spoken English.

Why SEC is your employer of choice?

• Accredited with 5 Tripartite Standards by Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.
• A diverse workplace with 32 nationalities, offering ample opportunities for mutual learning.
• Positive and inclusive working environment.
• 25 days of annual leave for fixed-term contracts.
• 1 day of Birthday Leave.
• Annual dental benefits.
• Committed to being a supportive employer as you prioritize your physical and mental wellness.
• Comprehensive healthcare insurance coverage.
• Flexible hybrid work arrangement (up to 2 days per week from home).
• Abundant networking opportunities across various disciplines.
• Accredited with NS mark certification.

Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated on their merits and qualifications, without regards to gender, race, age or religion.

Work location: 1 Create Way, CREATE Tower, Singapore 138602 (NUS University Town)
Contract Duration: 2 Years

Remuneration will be commensurate with the candidate's qualifications and relevant experience.

Further information about The Singapore-ETH Centre can be found on our website: https://sec.ethz.ch/
For further information, please contact: Mrs Iva Kabosch (ETH Zurich) at [e-mail]