Full-stack Software Engineer III/Senior, AI Gateway

ngrokinc; United States; Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Health insurance

401(k) match

Generous vacation

Parental leave

Development budget

Home office stipend

Regular performance reviews

Catered lunches when onsite

Job summary

ngrokinc in San Francisco is seeking a senior full-stack engineer to own the AI-aware layer of our production network gateway. The role focuses on building and operating request-path systems that classify, govern, and measure traffic for AI workloads, with a strong emphasis on latency, safety, and predictable behavior on the hot path.

This hybrid role requires several years of production software experience, expertise in Go, Rust, C++, or Java, and frontend skills with TypeScript, React, HTML,

Qualifications

• Several years of professional software engineering experience, including ownership of systems that run in production.
• Strong proficiency in a statically typed, compiled language such as Go, Rust, C++, or Java, with hands-on Go experience valued.
• Frontend engineering experience with TypeScript, React, HTML, and CSS.
• Practical exposure to AI or large language model workloads and comfort with edge-case systems.
• A track record of caring about developer experience, abstractions, and observability rather than only plumbing.

Responsibilities

• Design, build, and evolve gateway components that classify and handle AI traffic in real time, with an emphasis on latency, safety, and predictable behavior on the hot path.
• Implement and own traffic policy objects tailored to AI workloads, including rate limits, token usage caps, and access rules that prevent runaway cost, misuse, or accidental data exposure.
• Build and maintain the metering systems that accurately count requests, tokens, and related metadata so customers can understand what their AI systems are consuming.
• Surface trustworthy, customer-facing signals about what was allowed, blocked, or rate-limited, and which policy applied, so support questions can be answered from telemetry rather than guesswork.
• Collaborate with adjacent gateway, data, and platform teams to keep policy enforcement, usage telemetry, and downstream billing signals consistent end to end.
• Partner with product and design to define AI-specific gateway primitives that hide provider quirks and feel intentional to operators.

Skills

Go

Rust

C++

Java

TypeScript

React

HTML

CSS

Observability

Telemetry

Tools

API gateways

Job description

Role overview

A senior-level full-stack engineering position is available on a team that owns the AI-aware layer of a production network gateway. The work centers on building and operating the request-path systems that classify, govern, and measure traffic bound for AI and large language model workloads. This is a hybrid role based in San Francisco, with two days of expected in-office collaboration each week.

Responsibilities

• Design, build, and evolve gateway components that classify and handle AI traffic in real time, with an emphasis on latency, safety, and predictable behavior on the hot path.
• Implement and own traffic policy objects tailored to AI workloads, including rate limits, token usage caps, and access rules that prevent runaway cost, misuse, or accidental data exposure.
• Build and maintain the metering systems that accurately count requests, tokens, and related metadata so customers can understand what their AI systems are consuming.
• Surface trustworthy, customer-facing signals about what was allowed, blocked, or rate-limited, and which policy applied, so support questions can be answered from telemetry rather than guesswork.
• Collaborate with adjacent gateway, data, and platform teams to keep policy enforcement, usage telemetry, and downstream billing signals consistent end to end.
• Partner with product and design to define AI-specific gateway primitives that hide provider quirks and feel intentional to operators.

Requirements

• Several years of professional software engineering experience, including ownership of systems that run in production.
• Strong proficiency in a statically typed, compiled language such as Go, Rust, C++, or Java, with hands-on Go experience valued.
• Frontend engineering experience with TypeScript, React, HTML, and CSS.
• Practical exposure to AI or large language model workloads and comfort with the edge cases those systems introduce.
• A track record of caring about developer experience, abstractions, and observability rather than only plumbing.
• Must be based in the United States with permanent authorization to work; visa sponsorship is not available for this role.

Nice to have

• Background building AI platforms, inference infrastructure, or model-routing systems.
• Experience with API gateways that have product-level opinions, not just proxying.
• Work on usage limits, quotas, or billing-adjacent systems.
• Building customer-facing observability or audit tooling.

Benefits and work setup

• Hybrid schedule with two in-office days per week at a San Francisco headquarters.
• Full premium coverage on base health, dental, and vision plans for employees, with dependents covered at fifty percent, plus mental health support.
• 401(k) retirement plan with a meaningful employer match.
• Open, flexible vacation policy with manager encouragement to actually use it.
• Parental leave of up to sixteen weeks for a birthing parent and up to eight weeks for other new parents, including through adoption or fostering.
• Annual professional development budget and a recurring home office stipend.
• Co-working space stipend for remote work and regular catered lunches when onsite.
• Twice-yearly company offsets and bi-annual performance reviews with explicit compensation benchmarking.
• Mix of salary and equity, with published bands that vary by location and seniority.