Staff Research Engineer, Autonomy VLM

Rivian; Sacramento (CA); On-site; USD 265,000 - 331,000; Full time

Benefits offered by this job

401(k) Plan

Employee Stock Purchase Program

Job summary

Rivian is seeking a Staff Research Engineer to lead the VLM strategy within the Autonomy stack, focusing on training, shipping, and optimizing VLM models and extending multi-modal capabilities for new use cases.

You will own the end-to-end lifecycle of VLM model delivery, from data acquisition to deployment and feedback loops, collaborating with core Autonomy teams to translate requirements into robust ML entregables.

Qualifications

• 5+ years of professional experience scaling ML solutions.
• Hands-on VLM model training or fine-tuning using LoRA/QLoRA and RL alignment.
• Large-scale data mining techniques for long-tail distributions and rare cases.
• Zero-shot or few-shot classification with open vocabulary models.
• Strong Python and understanding of perception pipelines, benchmarking tools, and infrastructure.

Responsibilities

• Define, drive and execute the VLM model delivery roadmap including training, deployment and extension to multi-modalities.
• Design and deliver models for automated data mining, long-tail distributions, rare-edge-case detection and anomaly detection at scale.
• Establish rigorous evaluation/monitoring benchmarks and diagnose system-level anomalies.
• Collaborate with Autonomy teams to translate vehicle feature requirements into ML deliverables.
• Define system requirements and guide cross-functional alignment and trade-offs.

Skills

VLM training

Data mining

Zero-shot / few-shot

Python

System engineering

Performance optimization

Education

BS/MS/PhD in CS/Robotics/EE

Tools

TensorRT

Job description

About Rivian

Rivianis on a mission to keep the world adventurous forever. This goes for the emissions-free Electric Adventure Vehicles we build, and the curious, courageous souls we seek to attract.As a company, we constantly challenge what’s possible, never simply accepting what has always been done. We reframe old problems, seek new solutions and operate comfortably in areas that are unknown. Our backgrounds are diverse, but our team shares a love of the outdoors and a desire to protect it for future generations.

Role Summary

Vision-Language Models (VLMs) are a foundational pillar of our Autonomy stack. In this StaffResearch Engineer role, you will play a key role in delivering the overarching VLM strategy,especially training, shipping, optimizing the VLM models, as well as extending tomulti-modalities and enabling new use cases, among others. In this role, you will also beresponsible to define and deliver VLM-driven solutions to solve some of autonomy's hardestchallenges, including automated data mining, handling long-tail distributions, rare edge-casedetection, and scene anomaly reasoning. As part of the model delivery, you will also own thewhole end-to-end lifecycle of VLM model delivery: data acquisition, metrics definition,benchmarking, model performance optimization, deployment, feedback loop.

Responsibilities

• Drive and deliver the VLM model strategy: Define, drive and execute the roadmap ofVLM model delivery, including training and delivering VLM models, optimization,deployment, as well as the extension to other multi-modalities.
• Accelerate data mining: Design and deliver VLM/LLM related models and strategiesthat power automated data mining, long-tail distributions, rare/edge case detection, andanomaly detection at scale, across multiple modalities (vision, lidar, text, etc).
• Iterate and optimize performance: Establish rigorous evaluation and monitoringbenchmarks. Identify and root-cause top-tier system anomalies, prioritizing high-impactoptimizations to continuously push the needle on performance.
• Cross-functional collaboration: Partner closely with core Autonomy teams(Perception, Planning, Calibration, Systems, etc) to translate vehicle featurerequirements into concrete ML deliverables.
• Influence trade-offs & requirements: Define system requirements and guidecross-functional efforts through technical trade-off decisions.

Qualifications

Education: BS, MS, or PhD in Computer Science, Robotics, Electrical Engineering, or ahighly related quantitative field.

• Experience: 5+ years of professional experience scaling ML solutions, with a strongfocus on the following:
  • VLM model training: Hands-on experience training or fine-tuning VLMs usingmodern parameter-efficient techniques (LoRA, QLoRA) and RL alignment.
  • Large-scale data mining: Proven track record developing VLM/LLM-relatedtechniques for data mining, long-tail distributions, rare cases, safety-criticalevents.
  • Zero/few-shot capabilities: Experience with open-vocabulary, zero-shot, orfew-shot classification models, particularly in long-tail scenarios.
  • System engineering: Strong proficiency in Python alongside a solidunderstanding of modern Perception pipelines, benchmarking tools, andinfrastructure.
  • Execution: Demonstrated ability to root-cause complex issues across adistributed, cross-functional stack in a fast-paced environment.

Preferred Qualifications

• Experience applying VLMs within the Autonomous Vehicle domain.
• Experience with Auto Prompt Optimization (APO) and automated prompt engineeringtechniques.
• Experience with spatial grounding in 2D and/or 3D.
• Experience extending foundational models to extra modalities (e.g., LiDAR, Radar, IMU,ego-motion).
• Experience utilizing VLMs or Foundation Models for complex behavior reasoning andplanning.
• Experience with onboard edge deployment, cloud inference architectures, and balancingcompute/efficiency trade-offs.
• Experience with quantization techniques (PTQ, QAT) and high-performance inferenceengines like TensorRT.

Pay Disclosure

The listed base salary range for this role is $265,000 - $331,300 for San Francisco Bay Area based applicants. This is the lowest to highest salary we in good faith believe we would pay for this role at the time of this posting. An employee’s position within the salary range will be based on several factors including, but not limited to, specific competencies, relevant education, qualifications, certifications, experience, skills, geographic location, shift, and organizational needs.

We offer a comprehensive package of benefits for full-time and part-time employees, their spouse or domestic partner, and children up to age 26, including but not limited to paid vacation, paid sick leave, and a competitive portfolio of insurance benefits including life, medical, dental, vision, short-term disability insurance, and long-term disability insurance to eligible employees.

You may also have the opportunity to participate in Rivian’s 401(k) Plan and Employee Stock Purchase Program if you meet certain eligibility requirements. Full-time employee coverage is effective on their first day of employment. Part-time employee coverage is effective the first of the month following 90 days of employment. More information about benefits is available at rivianbenefits.com.

Equal Opportunity

Rivian is an equal opportunity employer and complies with all applicable federal, state, and local fair employment practices laws. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, ancestry, sex, sexual orientation, gender, gender expression, gender identity, genetic information or characteristics, physical or mental disability, marital/domestic partner status, age, military/veteran status, medical condition, or any other characteristic protected by law.

Rivian is committed to ensuring that our hiring process is accessible for persons with disabilities. If you have a disability or limitation, such as those covered by the Americans with Disabilities Act, that requires accommodations to assist you in the search and application process, please email us at .

Candidate Data Privacy

Rivian may collect, use and disclose your personal information or personal data (within the meaning of the applicable data protection laws) when you apply for employment and/or participate in our recruitment processes (“Candidate Personal Data”). This data includes contact, demographic, communications, educational, professional, employment, social media/website, network/device, recruiting system usage/interaction, security and preference information. Rivian may use your Candidate Personal Data for the purposes of (i) tracking interactions with our recruiting system; (ii) carrying out, analyzing and improving our application and recruitment process, including assessing you and your application and conducting employment, background and reference checks; (iii) establishing an employment relationship or entering into an employment contract with you; (iv) complying with our legal, regulatory and corporate governance obligations; (v) recordkeeping; (vi) ensuring network and information security and preventing fraud; and (vii) as otherwise required or permitted by applicable law.

Rivian may share your Candidate Personal Data with (i) internal personnel who have a need to know such information in order to perform their duties, including individuals on our People Team, Finance, Legal, and the team(s) with the position(s) for which you are applying; (ii) Rivian affiliates; and (iii) Rivian’s service providers, including providers of background checks, staffing services, and cloud services.

Rivian may transfer or store internationally your Candidate Personal Data, including to or in the United States, Canada, the United Kingdom, and the European Union and in the cloud, and this data may be subject to the laws and accessible to the courts, law enforcement and national security authorities of such jurisdictions. Please note that we are currently not accepting applications from third party application services.