Remote LLM Inference Optimization Architect

Modular; United States; Hybrid; USD 198,000 - 286,000; Full time

Benefits offered by this job

Stock options

Health insurance

401k matching

Flexible PTO

Team onsite events

Job summary

Modular is seeking a senior engineer to build an optimization platform that pushes LLM inference performance to state-of-the-art levels across GPUs and ASICs. You will shape Modular Cloud's technical direction and partner with GTM to tailor optimizations for customer workloads.

The role emphasizes collaboration with engineering, product, and customers to deliver measurable gains. Your contributions will include publishing insights on novel optimization approaches and helping scale the platform

Qualifications

• 5+ years of experience in distributed systems or performance engineering.
• Proven track record of building durable, reusable software tools adopted across teams.
• Strong communication and technical leadership skills with good judgment on tradeoffs.
• Creative, curious, collaborative, and aligned with our culture.

Responsibilities

• Build the optimization platform that drives inference performance of LLMs on Modular Cloud across GPUs and ASICs.
• Define the technical direction for Modular Cloud to maintain Pareto frontier as landscapes evolve.
• Collaborate with GTM and engineering to tailor LLM inference optimizations to customer use cases.
• Publish blog posts sharing innovative approaches to LLM inference optimization.

Job description

Modular is seeking a senior engineer to build an optimization platform that pushes LLM inference performance to state-of-the-art levels across GPUs and ASICs. You will shape Modular Cloud's technical direction and partner with GTM to tailor optimizations for customer workloads.

The role emphasizes collaboration with engineering, product, and customers to deliver measurable gains. Your contributions will include publishing insights on novel optimization approaches and helping scale the platform