Speech AI & LLM Research Engineer

Commotion; New York (NY); Hybrid; USD 120,000 - 190,000; Full time

Job summary

Commotion invites an AI/ML Engineer to advance LLM-backed speech systems, focusing on fine-tuning, audio-language models, and speech-to-speech pipelines. The role targets research into new architectures, training methodologies, and agentic learning behaviors, in a fast-paced environment in New York.

The candidate will work on turn-taking, VAD, and real-time voice synthesis, collaborating with the Research Director to chart long-term AI research roadmaps.

Qualifications

• 2+ years in speech, multimodal, or LLM research.
• Bachelor's or master's degree in CS/AI or related field.
• Strong Python and scientific computing skills; experience in JupyterHub environments.
• Deep understanding of LLM architectures, transformers, and multimodal embeddings.
• Experience with speech modeling pipelines (ASR/TTS/speech-to-speech).
• Knowledge of turn-taking, VAD, prosody modeling, and real-time voice synthesis.
• Familiarity with self-supervised learning, contrastive learning, and ART.
• Dataset curation, experimental design, and model evaluation.

Responsibilities

• Research and develop direct speech-to-speech pipelines using LLM backbones and audio encoders/decoders.
• Model and evaluate turn-taking, latency, and VAD for real-time conversational AI.
• Explore ART and self-learning loops for continual improvement of speech models.
• Design memory-augmented multimodal architectures for persistent recall and context understanding.
• Collaborate with Research Director to define long-term AI research roadmaps.

Skills

Python

LLM research

Speech modeling

Transformers

Multimodal embeddings

JupyterHub

Agentic Reinforcement

Data curation

Education

BSc/MS in CS/AI

Tools

Hugging Face

PyTorch

LangChain

Pipecat

Agno

Job description

Commotion invites an AI/ML Engineer to advance LLM-backed speech systems, focusing on fine-tuning, audio-language models, and speech-to-speech pipelines. The role targets research into new architectures, training methodologies, and agentic learning behaviors, in a fast-paced environment in New York.

The candidate will work on turn-taking, VAD, and real-time voice synthesis, collaborating with the Research Director to chart long-term AI research roadmaps.