[PhD] Deploying AI Workloads in Real-Time Systems

HiPEAC; Toulouse; Hybrid; EUR 17,000 - 20,000; Full time

Job summary

HiPEAC and CNRS invite applications for a 36-month PhD thesis in Toulouse, exploring deploying AI workloads in real-time edge systems. You will study real-time scheduling, design space exploration, hardware resource partitioning, and runtime-adaptive neural networks to meet timing, performance, and energy constraints on AI accelerators.

The project is a collaboration among LAAS-CNRS, LIRIS-CNRS, Inria Rennes, and TUM, hosted by LAAS-CNRS in Toulouse, with co-supervision by senior researchers.

Qualifications

• Candidates must hold a graduate degree (or equivalent) in Computer Science or Mathematics.

Responsibilities

• Design space exploration: Develop a toolbox to optimize the compression levels of neural networks under timing and performance constraints.
• Hardware resource partitioning and configuration: Determine the optimal partitioning of available hard-ware resources, find the best topologies of processing elements (e.g., pipeline depth), and assign schedulingparameters (e.g., priorities and preemption points).
• Real-time scheduling: Develop scheduling policies tailored to AI models and AI hardware accelerators.
• Runtime-adaptive neural networks: Develop neural network models with runtime control of their execution, enabling the workload to adapt to current system utilization, available parallelism, or energy constraints.

Education

Graduate degree in Computer Science or Mathematics

Job description

Context. Deploying large AI models on edge devices presents several challenges due to inherent hardware constraints and the rapidly increasing size of AI models. Embedded platforms typically offer limited computational resources and storage capacity, with less flexibility than cloud computing infrastructures. In recent years, numerous hardware solutions specifically designed for edge AI have emerged. These domain-specific architectures exploit parallelism, optimize memory transfers, and support reduced-precision computation. At the software level, specialized techniques aim to reduce model size so that models can fit within the available on-chip memory by reducing the number of parameters and their bit-width representation, through approaches such as model compression, pruning, and quantization.

Real-time scheduling for AI workloads. Nonetheless, hardware acceleration and software optimization alone cannot guarantee real-time performance. Resource management and scheduling are required to decide, at each instant, which neural network should execute on which processing unit. Traditionally, these approaches focus primarily on meeting timing objectives, whereas edge AI requires consideration of several additional aspects: accuracy, energy consumption, and, when necessary, thermal constraints. Satisfying these constraints requires accurately characterizing the AI workload (i.e., the task model) in terms of its parallelism level, memory footprint, and precision. Consequently, scheduling policies should not only account for these additional constraints but also be designed with regard to theunderlying hardware capabilities (e.g., large number of processing units, limited on-chip memory capacity, and high context-switching overhead).

Objectives. Real-time systems are composed of multiple tasks, each with specific timing constraints (e.g., deadlines and execution frequency), execution requirements, and criticality levels. In edge AI applications, each real-time task can run a neural network inference. The goal of this thesis is to propose a framework for integrating AI models intoreal-time and edge AI systems while meeting specific timing and performance constraints on modern AI hardware accelerators. To this end, the following objectives will be addressed:

• Design space exploration: Develop a toolbox to optimize the compression levels of neural networks under timing and performance constraints.
• Hardware resource partitioning and configuration: Determine the optimal partitioning of available hard-ware resources, find the best topologies of processing elements (e.g., pipeline depth), and assign schedulingparameters (e.g., priorities and preemption points).
• Real-time scheduling: Develop scheduling policies tailored to AI models and AI hardware accelerators.
• Runtime-adaptive neural networks: Develop neural network models with runtime control of their execution, enabling the workload to adapt to current system utilization, available parallelism, or energy constraints.

Research group. The project is a collaboration among the Laboratory for Analysis and Architecture of Systems (LAAS-CNRS), the Laboratory for Computer Science in Images and Information Systems (LIRIS-CNRS), Inria Center at Rennes University, and the Technical University of Munich (TUM). The host laboratory is the LAAS-CNRS in Toulouse, France. The thesis will co-supervised by Dr. Tomasz Kloda (LAAS-CNRS), Prof. Stefan Duffner(LIRIS-CNRS), Prof. Angeliki Kritikakou (Inria Rennes) and Dr. Binqi Sun (TUM).

Qualifications. Candidates must hold a graduate degree (or equivalent) in Computer Science or Mathematics.

Duration. 36 months. The position is available immediately.

The Centre National de la Recherche Scientifique (CNRS) is a public research organization in France, fostering interdisciplinary collaboration to address social and economic challenges through innovative research.

Metadata

Topics: Accelerators, Artificial intelligence, Cyber-physical systems, Design Space Exploration, Edge computing, Embedded Systems, Neural networks, Optimization, Parallel computing, Resource management / Scheduling

[PhD] Deploying AI Workloads in Real-Time Systems

Full-time

CNRS - The French National Centre for Scientific Research

Toulouse, FR

Context. Deploying large AI models on edge devices presents several challenges due to inherent hardware constraints and the rapidly increasing size of AI models. Embedded platforms typically offer limited computational resources and storage capacity, with less flexibility than cloud computing infrastructures. In recent years, numerous hardware solutions specifically designed for edge AI have emerged. These domain-specific architectures exploit parallelism, optimize memory transfers, and support reduced-precision computation. At the software level, specialized techniques aim to reduce model size so that models can fit within the available on-chip memory by reducing the number of parameters and their bit-width representation, through approaches such as model compression, pruning, and quantization.

Real-time scheduling for AI workloads. Nonetheless, hardware acceleration and software optimization alone cannot guarantee real-time performance. Resource management and scheduling are required to decide, at each instant, which neural network should execute on which processing unit. Traditionally, these approaches focus primarily on meeting timing objectives, whereas edge AI requires consideration of several additional aspects: accuracy, energy consumption, and, when necessary, thermal constraints. Satisfying these constraints requires accurately characterizing the AI workload (i.e., the task model) in terms of its parallelism level, memory footprint, and precision. Consequently, scheduling policies should not only account for these additional constraints but also be designed with regard to theunderlying hardware capabilities (e.g., large number of processing units, limited on-chip memory capacity, and high context-switching overhead).

Objectives. Real-time systems are composed of multiple tasks, each with specific timing constraints (e.g., deadlines and execution frequency), execution requirements, and criticality levels. In edge AI applications, each real-time task can run a neural network inference. The goal of this thesis is to propose a framework for integrating AI models intoreal-time and edge AI systems while meeting specific timing and performance constraints on modern AI hardware accelerators. To this end, the following objectives will be addressed:

• Design space exploration: Develop a toolbox to optimize the compression levels of neural networks under timing and performance constraints.
• Hardware resource partitioning and configuration: Determine the optimal partitioning of available hard-ware resources, find the best topologies of processing elements (e.g., pipeline depth), and assign schedulingparameters (e.g., priorities and preemption points).
• Real-time scheduling: Develop scheduling policies tailored to AI models and AI hardware accelerators.
• Runtime-adaptive neural networks: Develop neural network models with runtime control of their execution, enabling the workload to adapt to current system utilization, available parallelism, or energy constraints.

Research group. The project is a collaboration among the Laboratory for Analysis and Architecture of Systems (LAAS-CNRS), the Laboratory for Computer Science in Images and Information Systems (LIRIS-CNRS), Inria Center at Rennes University, and the Technical University of Munich (TUM). The host laboratory is the LAAS-CNRS in Toulouse, France. The thesis will co-supervised by Dr. Tomasz Kloda (LAAS-CNRS), Prof. Stefan Duffner(LIRIS-CNRS), Prof. Angeliki Kritikakou (Inria Rennes) and Dr. Binqi Sun (TUM).

Qualifications. Candidates must hold a graduate degree (or equivalent) in Computer Science or Mathematics.

Duration. 36 months. The position is available immediately.

The HiPEAC project has received funding from the European Union's Horizon Europe research and innovation funding programme under grant agreement number 101296676. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.