LLM Inference Optimization Leader

Modular; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health benefits

401k with match

Tax advice

Home office stipend

Unlimited PTO

Flexible hours

Job summary

Modular is seeking a senior leader to head a high‑impact team driving LLM inference on Modular Cloud. You will partner across GTM, Product, and Engineering to redefine inference performance, shaping the product direction and building scalable systems that meet demand.

You’ll own the technical direction to achieve Pareto‑optimal performance across GPUs and ASICs, translating customer workloads into actionable engineering roadmaps while growing and mentoring a top-tier team.

Qualifications

• 5+ years in distributed systems or performance engineering.
• Hands-on GPU kernel programming experience.
• Experience leading or managing engineering teams.
• Ability to translate customer/product needs into engineering direction.
• Familiarity with modern LLM architectures and inference optimization techniques.

Responsibilities

• Lead a high impact team partnering with GTM, Product, and Engineering.
• Shape product direction of Modular Cloud and build systems for scalable performance.
• Own technical direction to achieve state-of-the-art LLM performance on Modular Cloud.
• Collaborate with GTM to tailor inference to customer workloads and drive full-stack optimizations.
• Translate customer engagements into a clear technical roadmap for engineering teams.
• Grow and develop a high impact team and foster an environment for engineers to excel.
• Represent the team externally through blog posts on LLM inference optimization.

Skills

GPU kernel programming

Inference engine internals

Distributed inference architectures

Cloud native ecosystems

LLM architectures

Team leadership

Tools

Kubernetes

Job description

Modular is seeking a senior leader to head a high‑impact team driving LLM inference on Modular Cloud. You will partner across GTM, Product, and Engineering to redefine inference performance, shaping the product direction and building scalable systems that meet demand.

You’ll own the technical direction to achieve Pareto‑optimal performance across GPUs and ASICs, translating customer workloads into actionable engineering roadmaps while growing and mentoring a top-tier team.