Staff Engineer - Foundation Model Serving & Inference

Menlo Ventures; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

A leading tech enterprise in San Francisco is seeking a Staff Engineer to shape their foundation model API product. You will design and build systems that ensure efficient performance on high-throughput, low-latency GPU workloads. The ideal candidate will have experience with operational sensitive systems but does not need prior AI experience. Collaboration with various teams is essential to improve product offerings and architectural decisions.

Qualifications

• No prior ML or AI experience is necessary.
• Strong engineering skills required.
• Ability to work in a collaborative environment.

Responsibilities

• Design and build systems for high-throughput, low-latency inference.
• Influence architectural direction for AI model serving.
• Collaborate across various teams to enhance product experience.

Skills

Experience with high scale operational sensitive systems

Interest in building LLM APIs

Experience in customer facing APIs

Job description

A leading tech enterprise in San Francisco is seeking a Staff Engineer to shape their foundation model API product. You will design and build systems that ensure efficient performance on high-throughput, low-latency GPU workloads. The ideal candidate will have experience with operational sensitive systems but does not need prior AI experience. Collaboration with various teams is essential to improve product offerings and architectural decisions.