Post-Doctoral Research Visit F/M Dynamic Parallelization of Sparse Codes for High-Performance Computing and Machine Learning

1000scholars; Lyon; On-site; EUR 38,000 - 52,000; Full time

Job summary

1000scholars seeks a postdoctoral fellow at the Laboratoire de lInformatique du Parallélisme, ENS Lyon, France, under the PEPR NumPEx program, to address runtime code optimization for sparse computing in HPC and machine learning. The project aims to develop compiler and runtime algorithms to delay specialization of dense code until the sparse structure is known, enabling self-specializing code for sparse inputs.

Validation will use Florida sparse matrix data and ML tasks.

Qualifications

• Notions in compilers, parallelism, parallel architectures, parallel runtimes.
• Experience with C++.
• Languages: English, French.
• Excellent relational skills.

Responsibilities

• Propose code optimizations and data structures for scaling sparse propagation.
• Address runtime scheduling by relying on existing parallel runtimes.
• Validate the complete approach on scientific benchmarks using sparse tensors and ML applications.
• Publish reports and articles.

Skills

Compilers

Parallelism

Parallel runtimes

C++

English

French

Algorithms

Job description

Context

Within the framework of the French prioritary research program for Exascale computing in France (PEPR NumPEx), we are hiring a postdoctoral fellow to address runtime code optimization for sparse computing.

This research targets both High-Performance Computing and Machine Learning.

This postdoctoral fellowship will be held at Laboratoire de l'Informatique du Parallélisme at Ecole Normale Supérieure de Lyon, France.

Assignment

Most kernels of interest in machine learning and high-performance computing manipulate sparse tensors. Sparse codes are highly irregular and make use of array indirections and dynamic control which jeopardize static automatic parallelization algorithms.

The overall objective of this postdoctoral fellowship is to investigate compiler and runtime algorithms to delay the specialization of the dense code at runtime when the sparse structure is known. From a dense specification, we seek to compile a code able to specialize itself on the sparse input data.

The specialization will involve a set of sub-computations, which are expected to be achievable by standard linear algebra routines (e.g. gemm), using state-of-the art linear algebra libraries.

Several issues must be investigated: How to specialize the code? In particular, how propagate efficiently the sparsity along the computation flow? How to detect library kernels on the specialized code? How to enforce a proper scheduling for the parallel runtime?

Points 1 and 2 have been partially addressed by a PhD student.

The postdoctoral fellow will:

• Propose code optimizations and data structures for scaling sparse propagation (point 1)
• Address runtime scheduling (point 3) by relying on existing parallel runtimes
• Validate the complete approach (points 1, 2 and 3) on scientific benchmarks by using sparse tensors from the Florida sparse matrix collection as well as machine learning applications.

Main activities

• Research
• Software development
• Experimentation
• Report & article writing

Skills

Technical skills and level required :

• Notions in compilers, parallelism, parallel architectures, parallel runtimes
• Experience with C++
• Languages : English, French
• Excellent relational skills.