Doctoral Student in Neuromotor Interfaces for Dexterous Robot Teleoperation (multimodal egocentric vision + EMG)

ETH Zürich; Zürich; Sur place; CHF 50 000 - 60 000; Plein temps

Résumé du poste

ETH Zurich’s Sensing, Interaction & Perception Lab invites applications for a fully funded PhD position at its Zurich campus. The project combines wearable EMG at the wrist or forearm with egocentric vision to decode motor intent and understand hand-object interactions.

You will develop signal-processing and machine-learning methods for multimodal sensing, addressing real-time inference and robust performance in interactive settings, with applications in Mixed Reality and robotics.

Qualifications

• Excellent master's degree (MSc., M.Eng. or equivalent) in CS/EE/Robotics or related field.
• Strong background in ML, signals and time-series data.
• Experience implementing and evaluating computational methods for real-time sensing.

Responsabilités

• Develop signal-processing and machine-learning methods for multichannel EMG.
• Develop egocentric computer-vision methods for hands and objects.
• Develop multimodal learning and reasoning combining EMG with egocentric vision.
• Investigate reasoning over objects, actions, and task state for dexterous teleoperation.

Connaissances

Machine learning

Signal processing

Time-series data

Programming

Real-time systems

Formation

Master's degree in Computer Science, Electrical Engineering, Robotics, or related field

Outils

Python

TensorFlow/PyTorch

ROS

OpenCV

Description du poste

The Sensing, Interaction & Perception Lab invites applications for a fully funded PhD position at ETH Zürich at the intersection of robotics,wearable sensing, signal processing, machine learning, and human-computer interaction.

The goal of this PhD is to develop neuromotor interfaces for dexterous robot teleoperation. The primary sensing modality will be surface electromyography (sEMG) measured at the wrist or forearm, with the aim of decoding subtle hand and finger activity, continuous movement, and motor intent in real time.

A secondary component of the project will investigate egocentric vision as a complementary sensing modality. While EMG provides information about human motor intent, egocentric cameras can provide context about the surrounding scene, manipulated objects, hand-object interactions, and task state. Combining these modalities creates opportunities for interfaces that understand both what the user intends to do and what is happening in the environment.

The PhD will focus on the computational and sensing methods required to make such systems robust, generalizable, and effective in real interactive settings, including multimodal learning and reasoning across neuromotor and egocentric visual signals, while exploring applications in Mixed Reality and other interactive systems.

Note: This is not a position in biomedical engineering and there is no health focus.

Project background

Wearable EMG provides a direct and unobtrusive way to sense muscle activity underlying hand and finger movements. This creates opportunities for interfaces that can recognize subtle actions, continuously estimate movement, and infer motor intent before or without large observable movements.

Making such interfaces work reliably outside controlled settings remains a substantial research problem. EMG varies across users and recording sessions and is sensitive to electrode placement, contact conditions, movement, and other sources of noise. This motivates research in signal processing, temporal machine learning, representation learning, adaptation, and real-time inference.

A second challenge is that neuromotor signals alone provide limited information about whatthe user is interacting with and why a particular movement is occurring. The project will therefore combine EMG with egocentric vision to capture objects, hands, contacts, affordances, and task state. This creates a multimodal research problem: learning representations that combine neuromotor and visual information and developing methods for reasoning about human intent, hand-object interaction, and the state of an ongoing manipulation task.

For robotic teleoperation, these multimodal signals can support systems that jointly reason about what action the user intends, which object or target the action refers to, and how that intent should be translated to a robot with a different embodiment. Relevant problems include dexterous manipulation, shared autonomy, multimodal intent inference, and reasoning over sequences of human and robot actions.

Job description

• Develop signal-processing and machine-learning methods for multichannel EMG, including discrete and continuous decoding of hand and finger activity
• Develop egocentric computer-vision methods for understanding hands, objects, hand-object interactions, contacts, affordances, and task state
• Develop multimodal learning and reasoning methods that combine EMG with egocentric vision to infer motor intent and interaction context
• Investigate reasoning over objects, actions, interaction sequences, and task state to resolve ambiguous motor signals and anticipate intended actions
• Apply methods todexterous robot teleoperation, including mapping human motor intent to robot manipulators and dexterous hands

optionally:

• Prototype wearable sensing systems, including electrode configurations, signal acquisition, embedded processing, and hardware and software integration
• Develop and evaluate applications in robotics, Mixed Reality, and other interactive systems

and as in each PhD

• Disseminate your findings in publications at top-tier venues and open-source research results and tools where appropriate
• Present research findings at academic conferences, workshops, and seminars

Profile

ETH requirements:

• written and spoken fluency in English
• an excellent master's degree (MSc., M.Eng. or equivalent) in Computer Science, Electrical Engineering, Robotics, or related field

You bring:

• A strong foundation in machine learning, including classification and regression, model evaluation, representation learning, and generalization
• A solid understanding of signals and time-series data (e.g., sampling, frequency, phase, noise, spectral representations, and filtering)
• Strong programming skills and experience implementing and quantitatively evaluating computational methods
• An interest in building real-time sensing and interactive systems, including experimentation with physical sensors

A strong background in one or more of the following areas is particularly beneficial:

• Electrical engineering and signal processing: EMG or other electrophysiological signals, sensor systems, embedded systems, electronics, or wearable sensing
• Robotics: manipulation, dexterous manipulation, teleoperation, robot learning, shared autonomy, or human-robot interaction
• Computer Vision: egocentric vision, hand and object pose estimation, hand-object interaction, contact and affordance estimation, video understanding, or action recognition
• Machine learning: temporal modeling, representation learning, multimodal learning, multimodal reasoning, domain adaptation, or learning from noisy sensor data
• Interactive systems: HCI, Mixed Reality, wearable computing, or real-time input systems

We do not expect applicants to already be experts across all of these areas.

We offer

We offer an exciting research environment and team to study in and work with. You will have the opportunity to develop complete research systems spanning wearable sensing, egocentric perception, machine learning, and robotic interaction.

Beyond the lab, ETH Zurich has several internationally recognized research groups in robotics, machine learning, computer vision, interactive systems, and Mixed Reality. In our research, we frequently collaborate with other groups and departments as well as institutions and companies in Switzerland and abroad.

During your PhD, you will have the opportunity to contribute to and collaborate with the ETH AI Center and engage in ETHAR, ETH's Research Hub for Augmented Reality in collaboration with Google XR.

The position provides access to infrastructure for wearable and embedded prototyping, egocentric sensing, interactive systems, and robotic manipulation. We support publication and presentation at leading international conferences and journals and encourage intellectual independence and technically ambitious research.

Working, teaching and research at ETH Zurich

We value diversity and sustainability

In line with our values, ETH Zurich encourages an inclusive culture. We promote equality of opportunity, value diversity and nurture a working and learning environment in which the rights and dignity of all our staff and students are respected. Visit our Equal Opportunities and Diversity website to find out how we ensure a fair and open environment that allows everyone to grow and flourish. Sustainability is a core value for us - we are consistently working towards a climate-neutral future.

About ETH Zürich

ETH Zurich is one of the world’s leading universities specialising inscience and technology. We are renowned for our excellent education,cutting-edge fundamental research and direct transfer of new knowledgeinto society. Over 30,000 people from more than 120 countries find ouruniversity to be a place that promotes independent thinking and an environment that inspires excellence. Located in the heart of Europe,yet forging connections all over the world, we work together to develop solutions for the global challenges of today and tomorrow.