Full-stack Software Engineer III/Senior, AI Gateway

ngrokinc; United States; Hybrid; USD 180,000 - 230,000; Full time

Benefits offered by this job

Hybrid schedule two in-office days per

Premium health coverage

401(k) with employer match

Flexible vacation policy

Parental leave (16 weeks birthing, 8+)

Professional development budget

Home office stipend

Co-working space stipend

Catered lunches when onsite

Performance reviews with benchmarking

Salary and equity bands by location

Job summary

ngrok is seeking a senior full-stack engineer to own the AI-aware layer of a production gateway. You will design and operate request-path systems that classify, govern, and measure AI traffic with a focus on latency, safety, and observable behavior on the hot path.

You will collaborate across gateway, data, and platform teams, implement policy objects for AI workloads, and surface telemetry for support and billing. This hybrid role requires SF in-office presence two days per week.

Qualifications

• Several years of professional software engineering experience with ownership of systems that run in production.
• Strong proficiency in statically typed, compiled languages (Go, Rust, C++, Java).
• Frontend experience with TypeScript, React, HTML, and CSS.
• Practical exposure to AI or large language model workloads and related edge cases.

Responsibilities

• Design, build, and evolve gateway components that classify and handle AI traffic in real time, with emphasis on latency, safety, and predictable behavior on the hot path.
• Implement and own traffic policy objects tailored to AI workloads, including rate limits and access rules.
• Build and maintain metering systems that count requests, tokens, and metadata for customers to understand usage.
• Surface signals about what was allowed, blocked, or rate-limited to support telemetry-driven questions.
• Collaborate with gateway, data, and platform teams to keep policy enforcement, usage telemetry, and downstream billing signals consistent end to end.
• Partner with product and design to define AI-specific gateway primitives that feel intentional to operators.

Skills

Production ownership

Observability

Latency optimization

Tools

Go

Rust

C++

Java

TypeScript

React

HTML

CSS

Job description

Role overview

A senior-level full-stack engineering position is available on a team that owns the AI-aware layer of a production network gateway. The work centers on building and operating the request-path systems that classify, govern, and measure traffic bound for AI and large language model workloads. This is a hybrid role based in San Francisco, with two days of expected in-office collaboration each week.

Responsibilities

• Design, build, and evolve gateway components that classify and handle AI traffic in real time, with an emphasis on latency, safety, and predictable behavior on the hot path.
• Implement and own traffic policy objects tailored to AI workloads, including rate limits, token usage caps, and access rules that prevent runaway cost, misuse, or accidental data exposure.
• Build and maintain the metering systems that accurately count requests, tokens, and related metadata so customers can understand what their AI systems are consuming.
• Surface trustworthy, customer-facing signals about what was allowed, blocked, or rate-limited, and which policy applied, so support questions can be answered from telemetry rather than guesswork.
• Collaborate with adjacent gateway, data, and platform teams to keep policy enforcement, usage telemetry, and downstream billing signals consistent end to end.
• Partner with product and design to define AI-specific gateway primitives that hide provider quirks and feel intentional to operators.

Requirements

• Several years of professional software engineering experience, including ownership of systems that run in production.
• Strong proficiency in a statically typed, compiled language such as Go, Rust, C++, or Java, with hands-on Go experience valued.
• Frontend engineering experience with TypeScript, React, HTML, and CSS.
• Practical exposure to AI or large language model workloads and comfort with the edge cases those systems introduce.
• A track record of caring about developer experience, abstractions, and observability rather than only plumbing.
• Must be based in the United States with permanent authorization to work; visa sponsorship is not available for this role.

Nice to have

• Background building AI platforms, inference infrastructure, or model-routing systems.
• Experience with API gateways that have product-level opinions, not just proxying.
• Work on usage limits, quotas, or billing-adjacent systems.
• Building customer-facing observability or audit tooling.

Benefits and work setup

• Hybrid schedule with two in-office days per week at a San Francisco headquarters.
• Full premium coverage on base health, dental, and vision plans for employees, with dependents covered at fifty percent, plus mental health support.
• 401(k) retirement plan with a meaningful employer match.
• Open, flexible vacation policy with manager encouragement to actually use it.
• Parental leave of up to sixteen weeks for a birthing parent and up to eight weeks for other new parents, including through adoption or fostering.
• Annual professional development budget and a recurring home office stipend.
• Co-working space stipend for remote work and regular catered lunches when onsite.
• Twice-yearly company offsets and bi-annual performance reviews with explicit compensation benchmarking.
• Mix of salary and equity, with published bands that vary by location and seniority.