Product Manager - AI SDK / Model-Serving Product Manager - US

Majestic Labs ai

Los Altos (CA)

On-site

USD 170,000 - 250,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Majestic Labs is building the next generation of AI infrastructure. We are seeking an experienced software product manager to define and deliver the software stack that makes our system seamless for developers and cloud teams.

You will own the compiler roadmap, runtime engines, and framework integrations that convert standard deep learning models into optimized execution paths on the Ignite APU. The role blends strategic planning with hands-on leadership in a fast-moving startup, focusing on

Qualifications

  • Advanced degree required in CS/EE/Math/Physics.
  • 5–8 years in product management or software systems engineering shipping compiler technologies or developer platform tools.
  • Expertise in CUDA, Triton, and HPC libraries.
  • Understanding of inference serving and latency vs. throughput tradeoffs.
  • Experience with HPC scale-out infrastructures.

Responsibilities

  • Translate customer, engineering, and business requirements into PRDs, technical specs, and feature definitions.
  • Develop detailed product schedules, milestone plans, and release timelines while coordinating dependencies across multiple engineering teams.
  • Compiler Pipeline and Toolchain Roadmap: Define requirements for the software and AI model compilation toolchain.
  • Framework Integration and Interoperability: Lead partnerships and integrations with ML frameworks and runtimes, ensuring native support for PyTorch, JAX, vLLM, SGLang, and Hugging Face Optimum without customer code changes.
  • High-performance Kernel Optimization Roadmap: Define performance roadmap for low-level operator libraries, including BLAS and OpenAI Triton kernels.
  • Scale-Out and Clustering Orchestration: Define requirements for clustering and multi-node configurations across SLURM, Ray, LMCache, etc.
  • Developer Platform Usability: Champion developer experience and minimize manual memory management or bespoke compiler hacks.

Skills

Compiler technologies
Framework integrations
High-performance computing
CUDA & Triton
Dev platform tooling

Education

MS or PhD in CS/EE/Math/Physics

Tools

CUDA
Triton
MLIR/LLVM
SLURM/Ray

Job description

We're a fast-moving, US-Israeli AI startup building next-generation infrastructure for the world's most demanding AI workloads. Our mission is to accelerate the future of intelligence and make it ubiquitously accessible by delivering full stack custom AI servers optimized for large-scale inference and training.

Backed by top-tier investors and led by industry veterans, we're scaling rapidly. We seek forward-thinking engineers and operators who thrive in a collaborative environment and excel at solving complex challenges from first principles. If you want to build infrastructure that fundamentally changes what the world can accomplish with AI, come join us!

About The Position

We are building the next generation of AI infrastructure—from custom silicon to the servers and systems that power the world's largest AI models. Our vertically integrated approach enables us to optimize every layer of the AI compute stack, delivering breakthrough performance, efficiency, and scalability.

We're looking for an experienced software product manager to define and deliver the software stack that makes our system seamless for developers and cloud infrastructure teams. You'll own the compiler roadmap, runtime engines, and framework integrations that take standard deep learning model representations and turn them into highly optimized execution paths on the Ignite APU.

The goal is simple to state and hard to achieve: customers should be able to bring the models and tools they already use and get excellent inference performance without having to think about the hardware underneath.

Responsibilities
  • Translate customer, engineering, and business requirements into detailed Product Requirements Documents (PRDs), technical specifications, and feature definitions.
  • Develop detailed product schedules, milestone plans, and release timelines while managing dependencies across multiple engineering organizations.
  • Compiler Pipeline and Toolchain Roadmap: Define requirements for the software and AI model compilation toolchain
  • Framework Integration and Interoperability: Lead product partnerships and technical integrations with major machine learning frameworks and runtimes, ensuring native support for PyTorch, JAX, vLLM, SGLang, and Hugging Face Optimum without requiring customer code modifications.
  • High-performance Kernel Optimization Roadmap: Define performance optimization roadmap for low-level operator libraries, including BLAS, custom OpenAI Triton kernels
  • Scale-Out and Clustering Orchestration: Define requirements for clustering and multi-node scale-out configurations, ensuring seamless execution across scale-out setups utilizing SLURM, Ray, LMCache, etc.
  • Developer Platform Usability: Act as the primary advocate for developer experience, monitoring how software compiles and executes in customer sandboxes, and proactively removing the need for manual memory management scripts, custom compiler hacks, or API wrappers.
Requirements
  • Education: Master of Science (M.Sc.) or Doctor of Philosophy (Ph.D.) in Computer Science, Electrical Engineering, Mathematics, or Physics is a strict, non-negotiable requirement.
  • AI Software Stack Expertise: Minimum of 5 to 8 years of experience in product management or software systems engineering, with a proven track record of shipping compiler technologies, deep learning frameworks, or developer platform tools.
  • Parallel Programming Fluency: Deep theoretical and practical understanding of CUDA, Triton, high-performance C++ DSP libraries, and multi-threaded parallel computation paradigms.
  • Understanding of inference serving (vLLM, TensorRT-LLM, Triton Inference Server, SGLang) and what matters including batching, KV cache management, and latency versus throughput tradeoffs.
  • Scale-Out Infrastructure Knowledge: Demonstrated familiarity with HPC scale-out infrastructures, distributed memory architectures, and networking protocols designed for high-performance clustering.
  • Prior experience at accelerated software, cloud, or startup organizations building complex AI compute systems, such as NVIDIA, Meta, Google, AWS, SambaNova, Cerebras, Groq, or Lightning AI.
Preferred Qualifications
  • MBA or equivalent experience in technology commercialization or product strategy.
  • Active contributions to open-source AI infrastructure projects, compilers (MLIR/LLVM), or performance optimization libraries.
  • Direct experience working inside a frontier AI lab or foundation model organization (e.g., OpenAI, Anthropic, xAI, Google DeepMind) developing large-scale distributed systems.

We seek smart, curious, self-driven individuals who enjoy creative problem-solving and continuous learning. Even if you don't meet all requirements, we'd love to hear from you if you have the mindset to tackle complex challenges.

Majestic Labs is an equal opportunity employer deeply committed to diversity of background and thought. If you are ready to dare greatly, learn from mistakes, engage in vigorous debates, and help create a step-function shift in human technical capability, we want to meet you!

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Manager - AI SW Infrastructure (AI Software Stack / Compiler / Platform) - US
Product Manager - AI SW Infrastructure (AI Software Stack / Compiler / Platform) - US

Majestic Labs ai • Los Altos (CA)

On-site
USD 150,000 - 210,000
AI Kernel Engineer
AI Kernel Engineer

Majestic Labs ai • Los Altos (CA)

On-site
USD 140,000 - 210,000
Compiler Engineer (PyTorch / Triton / LLVM / MLIR)
Compiler Engineer (PyTorch / Triton / LLVM / MLIR)

Majestic Labs ai • Los Altos (CA)

On-site
USD 150,000 - 230,000
Senior Product Manager, AI Infra & Compiler Stack
Senior Product Manager, AI Infra & Compiler Stack

Majestic Labs ai • Los Altos (CA)

On-site
USD 150,000 - 210,000
Senior Software Engineer, Infrastructure Software for AI (Centralized AI Data Centers & Distrib[...]
Senior Software Engineer, Infrastructure Software for AI (Centralized AI Data Centers & Distrib[...]

Intelliswift • Sunnyvale (CA)

On-site
USD 120,000 - 150,000
Competitive salary
Health insurance
Flexible work hours
Technical Product Manager - South Bay
Technical Product Manager - South Bay

Major Scale • Mountain View (CA)

On-site
USD 185,000 - 250,000
LLM Inference Engineer
LLM Inference Engineer

Majestic Labs ai • Los Altos (CA)

On-site
USD 210,000 - 350,000
AI Models, Product Manager
AI Models, Product Manager

Cerebras • Sunnyvale (CA)

Hybrid
USD 120,000 - 160,000
Opportunity to work on breakthrough AI technology
Diverse and inclusive work environment
Job stability in a startup setting
Senior Technical Product Manager - Token Factory
Senior Technical Product Manager - Token Factory

Nebius Group • United States

Hybrid
USD 120,000 - 180,000
Competitive compensation
Career growth
Flexibility and ownership
+3
Senior Product Manager, AI Infrastructure (Managed Services)
Senior Product Manager, AI Infrastructure (Managed Services)

Lightning AI • New York (NY)

Hybrid
USD 160,000 - 325,000
Health coverage
Equity
401(k) + pension
+7