Senior Deep Learning Scientist, Multimodal Agentic RL

Bot Jobs; Myrtle Point (OR); On-site; USD 184,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA in the United States is seeking researchers to develop, train, and deploy large language models for agentic systems that combine audio-visual reasoning, tool usage, and document understanding.

You will advance post-training methods, such as instruction tuning, preference optimization, and RLHF/RLVR/MOPD, to improve multimodal agents for complex tasks across digital and physical environments.

Qualifications

• Master's degree (or PhD) in CS/AI or applied math with 8+ years of experience.
• Strong Python and PyTorch for scalable model development.
• Deep learning fundamentals, Transformers, and MoE architectures.
• Foundational RL algorithms and reward design.
• Experience with post-training multimodal models and multimodal reasoning.

Responsibilities

• Apply research to develop, train, fine-tune, and deploy large language models for agentic systems across audio-visual reasoning, tool usage, and document understanding.
• Advance post-training alignment methods including instruction tuning and RLHF/RLVR/MOPD to improve multimodal agents.
• Lead collection, development, and benchmarking of multimodal datasets; ensure high-quality evaluation of accuracy, safety, and task completion.
• Manage model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines.

Skills

Python

PyTorch

Transformers

MoE models

Reinforcement learning

Experiment tracking

Dataset versioning

Multimodal reasoning

Education

Master's/PhD in CS/AI/Applied Math

Job description

What you'll be doing:

Apply fundamental and applied research to develop, train, fine-tune, and deploy large language models for agentic systems encompassing audio-visual reasoning, tool usage, and document understanding. Advance post-training and alignment methods including instruction tuning, preference optimization, and RLHF/RLVR/MOPD to improve multimodal agents for complex use cases. Research and develop agentic reasoning and grounded perception capabilities, focusing on planning, tool execution, and long-horizon task completion across digital and physical environments. Lead the collection, development, and benchmarking of multimodal datasets, ensuring high-quality evaluation of model accuracy, safety, and task completion success.

What we need to see:

• Master's degree (or equivalent experience) or PhD in Computer Science, AI, or Applied Math with 8+ years of relevant work experience.
• Excellent programming skills in Python with strong fundamentals in scalable model development and deep learning frameworks like PyTorch.
• Strong knowledge of ML/DL techniques and modern foundation model architectures, including Transformers and mixture-of-experts models.
• Foundational understanding of reinforcement learning algorithms and implementation, including MDPs, policies, and reward design.
• Hands-on experience in post-training multimodal models for omni-modality (audio-visual) reasoning, full-duplex voice chat, and human-AI interaction.
• Proven ability to manage model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines.

Ways to stand out from the crowd:

• Strong record of publications in top-tier AI and machine learning venues such as NeurIPS, ICML, ICLR, or CVPR.
• Validated experience training and deploying multimodal foundation models using large-scale distributed infrastructure.
• Experience applying deep reinforcement learning techniques to train multimodal agents in complex simulation or gaming environments.
• Background in audio/speech AI, especially audio language models or audio generation.
• Background in building embodied AI systems that integrate multimodal perception with backend action-fulfillment and long-horizon planning.

Salary and benefits:

With highly competitive salaries and a comprehensive benefits package, NVIDIA is considered one of the industry's most desirable employers. As you plan your future, see what we can offer you and your family at https://www.nvidiabenefits.com/. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until October 11, 2026.