Inference Systems Engineer - Scalable AI Platform

Modular, a Qualcomm company; United States; Hybrid; USD 148,000 - 270,000; Full time

Benefits offered by this job

Healthcare coverage

Retirement plan

Employee stock purchase

Paid time off

Learning and development

Job summary

Modular, a Qualcomm company, is building the Modular AI platform to simplify development and deployment of models. We seek ML developers who can drive architecture in a fast-paced, collaborative environment, turning research into scalable production features.

Based in the United States, you will join a hybrid team in Los Altos, CA, working on distributed systems from metal to cloud. The role offers strong compensation, equity, and a culture of growth and impact.

Qualifications

• 5+ years of systems programming experience focusing on performance and distributed architectures.
• Strong API and abstraction design with ability to defend proposals.
• Proficient in Python, Rust, Mojo; deep understanding of software systems matters more than language syntax.
• Ability to diagnose pathological performance bottlenecks and translate into solutions.
• Proven ownership from implementation through production support.
• Active interest in research literature and building meaningful tools.

Responsibilities

• Lead cross-team architecture decisions on concurrency, data movement, and system boundaries.
• Develop flexible interfaces for open-source model developers and contributors.
• Design modular systems with clear contracts and testability for robust development.

Skills

Systems programming

API design

Python

Rust

Mojo

Performance optimization

Distributed systems

Tools

SGLang

vLLM

TRT-LLM

GPU programming

Job description

Modular, a Qualcomm company, is building the Modular AI platform to simplify development and deployment of models. We seek ML developers who can drive architecture in a fast-paced, collaborative environment, turning research into scalable production features.

Based in the United States, you will join a hybrid team in Los Altos, CA, working on distributed systems from metal to cloud. The role offers strong compensation, equity, and a culture of growth and impact.