Staff Engineer – Foundation Model Serving & APIs

RoShay Services; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

RoShay Services in San Francisco is seeking a Staff Engineer responsible for designing and implementing core systems and APIs for Foundation Model Serving. You will drive high-throughput, low-latency inference on GPU workloads and shape the platform architecture.

Collaborate on system design, algorithm development, and scalable APIs to enable reliable large-scale model deployment. This role rewards strong communicators who can translate complex tradeoffs into actionable plans and lead

Qualifications

• 10+ years of experience in distributed systems and system design.
• Strong background in algorithms and system architecture.
• Excellent communication and strategic thinking abilities.

Responsibilities

• Develop and optimize systems to enable high-throughput, low-latency inference on GPU workloads.
• Design scalable and efficient APIs to support foundation model deployment.
• Collaborate on system architecture and algorithm development to enhance performance.

Skills

Distributed systems

System design

Algorithms

Communication skills

Job description

Role: Staff Engineer responsible for designing and implementing core systems and APIs for Foundation Model Serving.

Key Responsibilities

• Develop and optimize systems to enable high-throughput, low-latency inference on GPU workloads.


• Design scalable and efficient APIs to support foundation model deployment.


• Collaborate on system architecture and algorithm development to enhance performance.

Qualifications & Skills

• Over 10 years of experience in distributed systems and system design.


• Strong background in algorithms and system architecture.


• Excellent communication skills and strategic thinking abilities.