Research Intern - Deep Learning

Pony.ai; Fremont (CA); On-site; USD 78,000 - 112,000; Full time

Job summary

Pony.ai in Fremont, CA is seeking a graduate student to advance autonomous vehicle AI through large-scale foundation models. You will frame real-world problems as ML tasks and develop real-time capable models for driving systems.

You will design multi-modal perception models, train and deploy deep learning solutions, and contribute to data pipelines using large-scale datasets. This rolling, onsite role starts in Fremont.

Qualifications

• Currently pursuing a Masters or PhD in Computer Science, Machine Learning, Robotics, or similar field.
• Strong background in deep learning with experience in model design, training and evaluation.
• Experience with deep learning research and tools.
• Proficiency in software design and development using Python and C++.
• Experience working with large-scale datasets, data preprocessing, and pipeline management.

Responsibilities

• Work with experts in self-driving vehicles to design and develop large-scale foundation models trained on real-world data.
• Frame open-ended ML problems into well-defined tasks and scale them into data pipelines for real-time car applications.
• Develop and deploy deep learning models, including vision-language models and LLMs.
• Design multi-modality perception models focusing on 3D object detection, tracking, and scene understanding.
• Optimize models to run robustly under tight runtime constraints.

Skills

Python

C++

Deep learning

ML research

Data preprocessing

Pipeline management

CUDA

TensorRT

GPU

Education

Masters or PhD in CS/ML/Robotics

Tools

CUDA

TensorRT

Job description

Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai's leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural "XB100" 2023 list of the world's top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.

Responsibility

• Work with experts in the field of self-driving vehicles on designing and developing large-scale foundation models trained on vast amounts of real world data
• Frame the open-ended real-world problems into well-defined ML problems; develop and apply cutting-edge ML approaches (deep learning, reinforcement learning, imitation learning, etc) to these problems; scale them to data pipelines; and streamline them to run in real-time on the cars
• Develop and deploy deep learning models, including vision language models (VLMs) and Large Language Models (LLMs)
• Design and implement multi-modality and multi-task perception models focusing on 3D object detection and tracking, segmentation, semantics understanding, video understanding, scene understanding, traffic control, or trajectory prediction, etc
• Optimize deep learning models to run robustly under tight run-time constraints

Requirements

• Currently pursuing a Masters or PhD program in Computer Science, Machine Learning, Robotics, or similar field
• Strong background in deep learning, with experience in model design, training and evaluation
• Experience with deep learning research and tools
• Proficiency in software design and development using Python and C++
• Experience working with large-scale datasets, data preprocessing, and pipeline management

Preferred Experience

• Publications on top-tier conferences like CVPR/ICCV/ECCV/ICLR/ICML/NeurIPS/ICLR/AAAI
• Experience in applying ML/DL for behavior prediction, imitation learning, motion planning
• Experience in deploying deep learning algorithms for real time applications, with limited computing resources
• Experience in convex optimization, computational geometry or linear algebra
• Experience in GPU/CUDA/TensorRT
• Previous internships involving large-scale deep learning models and systems
• Preferred graduate before Dec 2026

Note

• This position is rolling based and it can start any time
• This position is fully onsite in Fremont, at least 3 months

Compensation

• Master: $7000/month
• PhD: $10,000/month

Please click here for our privacy disclosure.