Machine Learning Engineer (Video Foundation Models & Agentic AI)

Maxinsights; Santa Clara (CA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health insurance

Vision care

Dental coverage

401(k)

Paid holidays

PTO (Paid Time Off)

Sick leave

Job summary

Maxinsights is building Max VLM, a next-generation video foundation model designed to understand human activities and object interactions from first-person video. We seek a talented ML engineer to design, train, and optimize video-language models for real-world applications.

You will build scalable ML pipelines, analyze failures, and collaborate with product and engineering to deliver production-ready capabilities. This is a hands-on, fast-paced role in a startup environment.

Qualifications

• Bachelor's, Master's, or PhD in Computer Science, AI, ML, or related field, or equivalent practical experience.
• Strong understanding of deep learning and hands-on experience with model training, fine-tuning, or inference.
• Proficiency in Python and deep learning frameworks such as PyTorch.
• Familiarity with modern LLMs, multimodal models, or computer vision models.
• Strong problem-solving skills and the ability to independently implement, debug, and optimize ML systems.
• Ability to work in a fast-paced environment, take ownership, and deliver practical solutions.

Responsibilities

• Design, train, fine-tune, and optimize video-language foundation models for real-world applications.
• Build and improve large-scale ML pipelines covering data processing, training, evaluation, and inference.
• Analyze model failures, identify performance bottlenecks, and drive continuous improvements.
• Develop efficient model inference and deployment solutions balancing quality, latency, and cost.
• Collaborate with product and engineering teams to translate business requirements into technical solutions.
• Explore and evaluate emerging techniques in multimodal learning and agentic AI for practical problems.
• Take ownership of ML projects from problem definition to deployment and ongoing improvement.

Skills

Python

Deep learning

Problem solving

Fast-paced

Education

Bachelor's/Master's/PhD in CS/AI/ML or equivalent

Tools

PyTorch

LLMs

Vision models

Job description

Job Description:

About the Team

We are building Max VLM, a next-generation video foundation model designed to understand human activities, object interactions, and the physical world through first-person video.

Our mission is to develop intelligent video understanding systems that can accurately interpret complex real-world activities and turn continuous visual observations into structured, actionable information.

As a fast-moving startup team, we work across the entire machine learning lifecycle, from data collection and model training to evaluation, optimization, and production deployment. We combine state-of-the-art multimodal AI technologies with real-world data and practical engineering to build scalable, reliable, and efficient AI systems.

Project Overview & Technical Challenges

Despite rapid advances in multimodal large language models, understanding real-world video remains a challenging problem. Existing systems often struggle with fine-grained activity recognition, temporal reasoning, long-video comprehension, and computational efficiency.

Our team focuses on building high-performance video foundation models and bringing them into real-world applications. Key technical directions include:

1 - Video Foundation Models & Multimodal Understanding

Develop and improve video-language models capable of understanding complex human activities and physical-world interactions.

• Fine-grained action recognition, temporal grounding, and video understanding.

• Long-video comprehension, temporal reasoning, and event-level understanding.

• Multimodal model fine-tuning, post-training, and performance optimization.

• High-quality training data generation, automated labeling, and data selection.

• Model evaluation, error analysis, and data-driven iteration to continuously improve production performance.

2 - Efficient Model Training & Deployment

Build scalable and efficient machine learning systems that support real-world product requirements.

• Large-scale model training, fine-tuning, and distributed training optimization.

• Efficient video processing, and inference acceleration.
  Model compression, quantization, and optimization of latency, throughput, and computational cost.

• Production model serving, monitoring, and continuous performance improvement.

• End-to-end optimization of model quality, reliability, and deployment efficiency.

3 - Agentic Video Understanding & Advanced Post-Training

Explore advanced AI techniques to improve video understanding and reasoning capabilities.

• Develop agentic video understanding systems with autonomous tool use and multi-step reasoning.

• Explore reinforcement learning and advanced post-training techniques to improve model performance.

• Apply emerging AI research to real-world applications and product improvements.

Key Responsibilities

• Design, train, fine-tune, and optimize video-language foundation models for real-world applications.

• Build and improve large-scale ML pipelines covering data processing, training, evaluation, and inference.

• Analyze model failures, identify performance bottlenecks, and drive continuous improvements through data and algorithmic iteration.

• Develop efficient model inference and deployment solutions that balance quality, latency, and computational cost.

• Collaborate closely with product and engineering teams to translate business requirements into technical solutions and deliver production-ready ML capabilities.

• Explore and evaluate emerging techniques in multimodal learning, reinforcement learning, and agentic AI, applying promising advances to practical problems.

• Take ownership of ML projects from problem definition and experimentation through deployment and ongoing improvement.

Qualifications

Minimum Qualifications

• Bachelor's, Master's, or PhD degree in Computer Science, Artificial Intelligence, Machine Learning, or a related field, or equivalent practical experience.

• Strong understanding of deep learning and hands-on experience with model training, fine-tuning, or inference.

• Proficiency in Python and deep learning frameworks such as PyTorch.

• Familiarity with modern LLMs, multimodal models, or computer vision models.

• Strong problem-solving skills and the ability to independently implement, debug, and optimize machine learning systems.

• Ability to work in a fast-paced environment, take ownership, and deliver practical solutions to complex problems.

Preferred Qualifications

• Strong research capabilities, demonstrated through publications at top-tier conferences (e.g., NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV) or other impactful research contributions.

• Experience with video understanding, vision-language models, or multimodal foundation models.

• Experience with LLM/VLM post-training, including SFT, reinforcement learning, or reasoning-oriented training.

• Experience with distributed training, efficient inference, or production-scale ML systems.

• Familiarity with agentic AI, tool-augmented reasoning, or multimodal reasoning.

• Proven ability to translate novel research ideas into practical implementations and measurable improvements.

Why Join Us?

• Real-World Impact: Build video AI systems that directly power real-world products and reach users through production deployment.

• End-to-End Ownership: Work across the complete ML lifecycle, from data and model development to optimization and deployment, with the opportunity to own meaningful technical initiatives.

• Challenging MIL Problems: Solve practical and technically challenging problems in video understanding, large-scale model training, and efficient inference.

• Frontier AI Exploration: Stay close to advances in multimodal foundation models, reasoning, reinforcement learning, and agentic AI, with opportunities to apply new ideas to real-world problems.

• Startup Environment: Work in a small, fast-moving team that values ownership, technical excellence, rapid iteration, and measurable impact.

Default Benefits:

• Health insurance

• Vision care

• Dental coverage

• 401(k)

• Paid holidays

• PTO (Paid Time Off)

• Sick leave