Backend Architect: Inference Gateway & Smart Routing

you.com; New York (NY); À distance; USD 150 000 - 210 000; Plein temps

Avantages offerts par ce poste

Remote-friendly culture

Competitive compensation

Résumé du poste

you.com is seeking an experienced backend engineer to own and scale the inference gateway that routes every model call. You will manage provider/model selection, fallback, and enforcement within a sub-millisecond budget while building an intelligence layer for selecting the right model.

This role emphasizes cost attribution, metering, and robust, scalable routing across hosted providers and internal models.

Qualifications

• Hands-on production experience with model serving stacks on relevant platforms.
• Strong backend generalist with Python and Kubernetes experience.
• Familiarity with tokenization, prompt-caching, and usage reporting.

Responsabilités

• Own and scale the inference gateway between model serving and products.
• Build routing, fallback, and model/provider selection with low latency.
• Implement cost attribution, metering, and billing for cross-provider traffic.
• Ensure graceful degradation and shadow-mode deployment for changes.

Connaissances

Python

Kubernetes

Backend engineering

Model serving

Latency optimization

Caching

Billing/cost optimization

Provider APIs

Streaming responses

Outils

vLLM

SGLang

Description du poste

you.com is seeking an experienced backend engineer to own and scale the inference gateway that routes every model call. You will manage provider/model selection, fallback, and enforcement within a sub-millisecond budget while building an intelligence layer for selecting the right model.

This role emphasizes cost attribution, metering, and robust, scalable routing across hosted providers and internal models.