Senior Software Engineer, Robotics Foundation Models

Weingart Foundation; Santa Clara (CA); On-site; USD 148,000 - 288,000; Full time

Job summary

A leading technology company is seeking a robotics foundation model expert to drive innovation in humanoid robots. The role involves conducting research, developing algorithms, and collaborating with teams to enhance robotic capabilities. Ideal candidates will have a strong educational background and experience in AI and deep learning frameworks.

Benefits

Competitive salary package

Equity and benefits

Diverse work environment

Qualifications

• 5+ years of algorithm development/research experience.
• Experience with vision-language models and foundation models.
• Excellent communication and collaboration skills.

Responsibilities

• Conduct applied research and design innovative algorithms.
• Deploy algorithms on real humanoid robots.
• Collaborate across engineering and research teams.

Skills

Algorithm Development

Deep Learning

Computer Vision

Natural Language Processing

Collaboration Skills

Education

Masters in Computer Science, Robotics, or related field

Tools

TensorFlow

PyTorch

C++

Python

Job description

We are in the midst of a revolution where artificial intelligence is helping us accelerate all aspects of our lives. NVIDIA's mission is to accelerate this revolution. By combining NVIDIA's strengths in simulation (Omniverse), world models (Cosmos), foundation models (GR00T), and compute, we strive to bring general intelligence to humanoid robots. The Isaac team is looking for a robotics foundation model expert to help with this revolution. In this role, you will be working on imbuing humanoid robots with natural language intelligence, semantic understanding, and geometric awareness. Come join us push the boundaries of robotics and enable the next generation of robotics!

What you'll be doing:

• Conduct applied research and design innovative algorithms in the space of geometric computer vision and image/video/vision-language/vision-centric foundation models.
• Deploy algorithms on real humanoid robots to gauge sim-to-real transfer of models.
• Work hand-in-hand across multiple engineering and research teams to enable foundation models on humanoid robots.
• Constantly learn and explore unfamiliar technologies.

What We Need To See:

• Masters in Computer Science, Robotics, Deep Learning, or other related fields (or equivalent experience).
• 5+ years of algorithm development/research experience relevant work/research experience in one or many of the following areas: Vision-language models, Foundation Models, 3D-LLMs, Video generative models and diffusion algorithms, or Action-based transformers.
• Experience in training vision-foundation models including 3D-VLMs, ViT, LLaVA, CLIP, Diffusion, and VLMs.
• Hands-on experience with deep learning frameworks (e.g., TensorFlow, PyTorch) and proficiency in modern software development practices (version control, testing, CI/CD).
• Hands-on experience with running algorithms on real-world robotic systems.
• Excellent communication skills and the ability to collaborate efficiently in a cross-functional, distributed team environment.
• Proficient in C++ and Python.
• You have a thirst and ability to learn and adapt to new technologies.

Ways To Stand Out From The Crowd:

• Ph.D. in Computer Science, Robotics, Deep Learning, or other related fields.
• Publications in top-tier AI conferences or contributions to open-source projects.
• Experience with vision-language-action models.

With a competitive salary package and benefits, NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us. Are you a creative and autonomous GenAI Engineer, who loves challenges? Do you have a genuine passion for advancing the state of AI & machine learning across a variety of industries? If so, we want to hear from you.

The base salary range is 148,000 USD - 287,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits . NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.