Senior Software Engineer - AI Gateway Platform NEW

The Briefed; San Francisco (CA); On-site; USD 196,000 - 294,000; Full time

Benefits offered by this job

Market-leading compensation

Equity compensation

Work-from-home setup budget

Conferences & growth

Job summary

Vercel is seeking a Senior Software Engineer on the AI Gateway Team to design, enhance, and scale the AI Gateway platform—a unified API layer providing access to AI models from providers like OpenAI, Anthropic, and Google. In a high-concurrency role, you will build low-latency systems with automatic fallbacks and unified billing/analytics.

You will work with TypeScript and backend distributed architectures to ensure production reliability and scalable integrations, collaborating with product

Qualifications

• 5+ years of professional software engineering experience building backend systems, APIs, or cloud platforms.

Responsibilities

• AI Gateway Platform Architecture: design, implement, and scale unified API endpoints, rate limiting, caching, and dynamic failover.
• Low-Latency High-Throughput Engineering: write modular TypeScript/JavaScript for high-volume AI inference with low latency.
• Provider & Model Integrations: integrate new LLM providers and models into a single developer-agnostic API.
• Open-Source & Developer Ecosystem: contribute to open-source tooling, including the Vercel AI SDK and ecosystem packages.
• Analytics & Platform Features: build features from telemetry, including unified billing metrics and usage analytics.
• Production Reliability & Operations: ensure high availability with automatic fallbacks and circuit breaking.

Skills

TypeScript

Node.js

High concurrency

AI/ML integration

Open-source

Remote collaboration

Job description

As a Senior Software Engineer on the AI Gateway Team, you will design, enhance, and scale Vercel's AI Gateway platform- a unified API layer providing seamless access to hundreds of AI models across major providers like OpenAI, Anthropic, and Google. In this high-concurrency role, you will build low-latency systems handling intelligent failover routing, rate limiting, and unified billing/analytics across multi-provider LLM infrastructure. You will work with TypeScript and backend distributed architectures to guarantee production reliability and automatic fallback mechanisms during upstream provider outages.

Key Responsibilities

• AI Gateway Platform Architecture: Contribute to the design, execution, and scaling of Vercel's AI Gateway platform-implementing unified API endpoints, rate limiting algorithms, caching layers, and dynamic failover mechanisms.
• Low-Latency High-Throughput Engineering: Write clean, modular, and well-tested TypeScript/JavaScript code, maintaining low-latency response times for high-volume AI inference requests.
• Provider & Model Integrations: Partner with product managers, AI researchers, and cloud infrastructure teams to integrate emerging LLM providers and models into a single developer-agnostic API.
• Open-Source & Developer Ecosystem: Actively contribute to open-source developer tooling (including the Vercel AI SDK and related ecosystem packages), adhering to Vercel's ethos of open developer collaboration.
• Analytics & Platform Features: Build platform features based on user telemetry-such as unified billing metrics, provider-agnostic authentication flows, and detailed usage analytics.
• Production Reliability & Operations: Ensure high-availability standards (automatic fallbacks during provider downtime, circuit breaking, and resilience under high request concurrency).

Requirements

Technical Systems & Engineering Depth

• Experience Track: 5+ years of professional software engineering experience building backend systems, APIs, or cloud platforms.
• TypeScript & Node.js Mastery: Strong proficiency in JavaScript/TypeScript paired with extensive experience developing scalable backend services and APIs.
• High-Throughput Infrastructure: Proven experience working with distributed systems, high-concurrency architectures, rate-limiting frameworks, distributed caching, or API gateways.
• AI/ML Integration Experience: Hands-on experience integrating AI/ML APIs, LLM providers, vector stores, or building AI-powered developer tooling.
• Open-Source Contribution: Demonstrated track record of participating in or contributing to open-source software projects.
• Collaboration & Remote Work: Excellent communication abilities with proven success collaborating across distributed, remote engineering teams.

Preferred Qualifications

• Direct experience building middleware or API proxies handling high-volume LLM inference workloads.
• Active maintainer or contributor status within the Vercel AI SDK, Next.js, or broader Web ecosystem.
• Familiarity with modern serverless platforms, edge compute networks, and cloud deployment pipelines.

Total Rewards & Benefits

• Compensation & Equity: Market-leading base salary ($196,000-$294,000 USD SF range) + generous Vercel equity compensation + potential variable performance bonuses.
• Comprehensive Healthcare: Fully inclusive healthcare insurance covering medical, dental, and vision for employees and dependents.
• Work-From-Home Support: Dedicated work-from-home setup budget + company-provided hardware and tech gear.
• Professional Growth: Mentorship, conference sponsorship, and learning stipends to attend industry events and build technical skills.
• Time Off: Flexible Time Off (FTO) policy to relax and recharge.