Senior/Staff Machine Learning Engineer, Perception

Agtonomy; South San Francisco (CA); On-site; USD 200,000 - 280,000; Full time

Benefits offered by this job

Stock Options

Unlimited PTO

401k Plan

Job summary

Agtonomy seeks an ML engineer to create perception systems that give autonomous machines human-like awareness in rugged environments. You will transform noisy camera and LiDAR data into robust 3D scene understanding for safe operation on heavy equipment worldwide.

You will push beyond bounding boxes toward dense representations, deploy models on embedded hardware, and contribute to production-grade software with a strong emphasis on performance and safety.

Qualifications

• MS/PhD or equivalent with strong research background in computer vision and perception systems.
• Experience building and deploying perception models (detection, segmentation, BEV, 3D scene understanding).
• Fluency adapting and distilling large pre-trained vision models.
• Experience with multi-sensor integration (camera, LiDAR, radar) and sensor fusion.
• Proficient Python development for production real-time systems.

Responsibilities

• Develop real-time perception models for open-world obstacle and terrain understanding.
• Build multi-modal fusion that combines camera and LiDAR into a unified 3D/BEV representation, robust to occlusions and sensor degradation.
• Optimize models for low-latency inference on resource-constrained hardware.
• Design auto-labeling pipelines leveraging foundation models and teacher-student distillation.
• Design data and evaluation pipelines for large multi-sensor datasets with strong visualization tooling.
• Analyze performance metrics and iterate to improve accuracy and efficiency of perception subsystems.

Skills

Python

PyTorch

TensorFlow

OpenCV

Multi-sensor fusion

CUDA

Real-time inference

Education

MS/PhD in Computer Science, AI, or related field

6+ years of industry experience in vision-based perception systems

Tools

TensorRT

CUDA

Job description

About Us

At Agtonomy, we're not just building tech-we're transforming how vital industries get work done. Our Physical AI and fleet services turn heavy machinery into intelligent, autonomous systems that tackle the toughest challenges in agriculture, turf, and beyond. Partnering with industry-leading equipment manufacturers, we're creating a future where labor shortages, environmental strain, and inefficiencies are relics of the past. Our team is a tight-knit group of bold thinkers—engineers, innovators, and industry experts—who thrive on turning audacious ideas into reality. If you want to shape the future of industries that matter, this is your shot.

About the Role

We're looking for a skilled ML engineer to build the perception systems that give our autonomous machines human-like awareness in rugged, unstructured environments. You'll develop computer vision and machine learning systems that turns noisy camera and LiDAR data into robust 3D scene understanding - enabling heavy equipment to operate safely through dust, glare, occlusion, and whatever messy conditions a working site throws at it.

The field is moving past bounding-box detection and hand-tuned tracking toward learned, dense scene representations, foundation-model-driven data engines, and uncertainty-aware perception. You'll be at the center of that shift. This role is hands-on: you'll write production-grade software, distill and optimize models for embedded hardware, and validate your work on real machines at operating around the world.

What You'll Do

• Develop real-time perception models for open-world obstacle and terrain understanding.
• Build multi-modal fusion that combines camera and LiDAR into a unified 3D/BEV representation, robust to occlusions, sensor degradation, and GNSS outages.
• Optimize models for low-latency inference on resource-constrained hardware, balancing accuracy and performance.
• Design auto-labeling pipelines that leverage foundation models and teacher-student distillation to scale labeling and close the loop from real-world field interventions.
• Design data and evaluation pipelines that curate large multi-sensor datasets and surface failures fast, with strong visualization and debugging tooling.
• Analyze performance metrics and iterate on algorithms to improve accuracy and efficiency of various perception subsystems.

What You'll Bring

• A MS/PhD in Computer Science, AI, or a related field, or 6+ years of industry experience building vision-based perception systems.
• Deep expertise developing and deploying modern perception models: detection, segmentation, mono/stereo/metric depth, BEV/occupancy, sensor fusion, and 3D scene understanding.
• Fluency adapting, fine-tuning, and distilling large pre-trained vision and vision-language models.
• Strong grounding in multi-sensor integration (camera, LiDAR, radar): calibration, spatiotemporal sync, and cross-modal fusion.
• Experience handling large datasets efficiently and organizing them for labeling, training and evaluation.
• Fluency in Python with PyTorch/TensorFlow/OpenCV and the ability to write efficient, production-ready code for real-time systems.
• Proven ability to design experiments, analyze metrics (mAP, IoU, latency/throughput, and calibration/ECE), and optimize to meet stringent real-world performance and safety requirements.
• An eagerness to get your hands dirty and agility in a fast-moving, collaborative, small team environment with lots of ownership.

What Makes You a Strong Fit

• Experience architecting multi-sensor ML systems from scratch.
• Experience building auto-labeling / data-engine flywheels at scale.
• Experience with compute-constrained pipelines including optimizing models to balance the accuracy vs. performance tradeoff, leveraging TensorRT, model quantization, etc.
• Familiarity with emerging predictive world models for anticipation, anomaly detection, or closed-loop simulation, and adjacent policy paradigms such as Vision-Language-Action (VLA) and World-Action (WAM) models.
• Experience with compute-constrained deployment: TensorRT, model quantization, and custom CUDA operations.
• Publications at top-tier perception/robotics venues (CVPR, ICRA, CoRL, RSS, etc.).
• Passion for how we feed, build, move, and maintain the world.

$200,000 - $280,000 a year

The US base salary range for this full-time position is $200,000 to $280,000 + equity + benefits + unlimited PTO

The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location, internal equity, and additional factors, including, but not limited to, job-related skills, experience, and relevant education or specialty training. Your recruiter can share more about the specific salary range during the hiring process.

Benefits

• 100% covered medical, dental, and vision for the employee (partner, children, or family is additional)
• Commuter Benefits
• Flexible Spending Account (FSA or HSA)
• Life Insurance
• Short- and Long-Term Disability
• 401k Plan
• Stock Options
• Collaborative work environment working alongside passionate mission-driven team!