Research Intern - Deep Learning

Pony.ai; Fremont (CA); On-site; USD 78,000 - 112,000; Full time

Benefits offered by this job

Onsite in Fremont

Job summary

Pony.ai is seeking a highly motivated ML researcher to design and deploy large-scale foundation models for autonomous driving. You will work with experts to frame real-world problems into well-defined ML tasks, apply state-of-the-art approaches, and scale them into robust data pipelines that run in real-time on vehicles.

The role requires a Master’s or PhD in CS/ML/Robotics, strong deep learning background, and proficiency in Python and C++.

Qualifications

• Masters or PhD in Computer Science, ML, Robotics or related field.
• Strong background in deep learning, model design, training and evaluation.
• Experience with deep learning research and tools.
• Proficiency in Python and C++.
• Experience with large-scale datasets, data preprocessing, and pipeline management.

Responsibilities

• Design and develop large-scale foundation models trained on real-world data.
• Frame real-world problems into well-defined ML problems; apply cutting-edge ML approaches to these problems; scale data pipelines; run in real-time on cars.
• Develop and deploy deep learning models, including vision-language models and LLMs.
• Design and implement multi-modality and multi-task perception models focusing on 3D object detection, tracking, segmentation, semantics, video, scene understanding, traffic control, or trajectory prediction.
• Optimize deep learning models to run under tight run-time constraints.

Skills

Deep learning

Python

C++

Data pipelines

Education

Master's or PhD in CS/ML/Robotics

Tools

PyTorch

TensorFlow

CUDA

TensorRT

Job description

Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai’s leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural “XB100” 2023 list of the world’s top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.

Responsibility

• Work with experts in the field of self-driving vehicles on designing and developing large-scale foundation models trained on vast amounts of real world data.
• Frame the open-ended real-world problems into well-defined ML problems; develop and apply cutting-edge ML approaches (deep learning, reinforcement learning, imitation learning, etc) to these problems; scale them to data pipelines; and streamline them to run in real-time on the cars.
• Develop and deploy deep learning models, including vision language models (VLMs) and Large Language Models (LLMs)
• Design and implement multi-modality and multi-task perception models focusing on 3D object detection and tracking, segmentation, semantics understanding, video understanding, scene understanding, traffic control, or trajectory prediction, etc.
• Optimize deep learning models to run robustly under tight run-time constraints.

Requirements

• Currently pursuing a Masters or PhD program in Computer Science, Machine Learning, Robotics, or similar field
• Strong background in deep learning, with experience in model design, training and evaluation.
• Experience with deep learning research and tools.
• Proficiency in software design and development using Python and C++.
• Experience working with large-scale datasets, data preprocessing, and pipeline management.

Preferred Experience

• Publications on top-tier conferences like CVPR/ICCV/ECCV/ICLR/ICML/NeurIPS/ICLR/AAAI
• Experience in applying ML/DL for behavior prediction, imitation learning, motion planning.
• Experience in deploying deep learning algorithms for real time applications, with limited computing resources.
• Experience in convex optimization, computational geometry or linear algebra.
• Experience in GPU/CUDA/TensorRT
• Previous internships involving large-scale deep learning models and systems

Note

This position is rolling based and it can start any time.

This position is fully onsite in Fremont, at least 3 months.

Compensation

• Master: $7000/month
• PhD: $10,000/month