Staff Engineer, Inference Runtime — Performance & Scale

Menlo Ventures; New York (NY); Hybrid; USD 405,000 - 485,000; Full time

Job summary

Menlo Ventures is looking for a Staff Engineer to lead technical efforts in their Inference organization. This senior role involves setting the technical direction for the inference runtime, ensuring it scales efficiently across various platforms.

The ideal candidate will have a strong background in systems engineering and experience with ML infrastructure, optimizing performance in distributed systems. The role is based in New York, NY, with a hybrid work policy.

Qualifications

• Deep background in systems engineering or ML infrastructure.
• Experienced with GPU, TPU or Trainium ecosystems.
• Significant software engineering experience.

Responsibilities

• Set technical direction for the inference serving stack.
• Own and evolve the accelerator-agnostic runtime.
• Drive efficient accelerator usage across computing platforms.

Skills

Systems engineering

Performance profiling

Distributed systems

Technical alignment

Communication skills

Education

Bachelor’s degree or equivalent

Tools

Rust

Python

Kubernetes

Job description

Menlo Ventures is looking for a Staff Engineer to lead technical efforts in their Inference organization. This senior role involves setting the technical direction for the inference runtime, ensuring it scales efficiently across various platforms.

The ideal candidate will have a strong background in systems engineering and experience with ML infrastructure, optimizing performance in distributed systems. The role is based in New York, NY, with a hybrid work policy.