AI Gateway Engineer: Scalable API & LLM Routing

Kong

Turbigo

Remote

EUR 70,000 - 120,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Kong is hiring a Software Engineer for the AI Gateway Team to help build and scale API management for AI/LLM workloads, focusing on multi-LLM routing and semantic caching. You will join a globally distributed engineering crew and implement features, troubleshoot issues, and optimize performance.

You'll contribute to low-level programming in Golang or Rust, work on Linux-based systems, and help extend Kong's AI platform with robust, scalable solutions. Remote collaboration is expected.

Qualifications

  • 3+ years of server-side programming experience.
  • Hands-on experience with Golang or Rust.
  • Experience building scalable, distributed systems.
  • Low-level Linux troubleshooting and profiling.
  • APIs on L7/L4 transports and protocols experience.
  • Comfortable working with a distributed remote team.
  • Strong ownership and documentation skills.
  • Familiarity with AI/LLM concepts.

Responsibilities

  • Build and maintain API management software for AI/LLM workloads.
  • Implement features including proxying, load balancing, and database support.
  • Troubleshoot bugs, tune performance, and optimize at low level.
  • Bridge Kong with deployment platforms and tooling.
  • Perform any tasks needed by the manager.

Skills

Golang
Rust
Distributed systems
Linux networking
Performance tuning
API management
Documentation
Remote collaboration

Tools

NGINX/OpenResty
LuaJIT
Envoy
Istio

Job description

Kong is hiring a Software Engineer for the AI Gateway Team to help build and scale API management for AI/LLM workloads, focusing on multi-LLM routing and semantic caching. You will join a globally distributed engineering crew and implement features, troubleshoot issues, and optimize performance.

You'll contribute to low-level programming in Golang or Rust, work on Linux-based systems, and help extend Kong's AI platform with robust, scalable solutions. Remote collaboration is expected.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Gateway Engineer: Scalable API for LLMs
AI Gateway Engineer: Scalable API for LLMs

Kong • Toscana

On-site
EUR 55,000 - 90,000
Equity
Bonus
Health, dental, and vision insurance
+1
AI Gateway Engineer - Build Scalable APIs & LLMs
AI Gateway Engineer - Build Scalable APIs & LLMs

Kong • Lombardia

Remote
EUR 65,000 - 110,000
AI Gateway Engineer - Scalable API & LLM Stack
AI Gateway Engineer - Scalable API & LLM Stack

Kong • Liguria

On-site
EUR 55,000 - 75,000
Equity
Bonus
Health insurance
+3
AI Gateway Engineer: Multi-LLM Routing & Distributed
AI Gateway Engineer: Multi-LLM Routing & Distributed

Kong • Milano

Hybrid
EUR 45,000 - 65,000
AI Gateway Engineer: Scalable API Stack (Golang/Rust)
AI Gateway Engineer: Scalable API Stack (Golang/Rust)

Kong • Toscana

On-site
EUR 55,000 - 75,000
Equity
Bonus
Health, dental, and vision insurance
+1
AI Gateway Engineer: Build Scalable LLM API Stack
AI Gateway Engineer: Build Scalable LLM API Stack

Kong • Chieti

On-site
EUR 60,000 - 90,000
Equity
Bonus
Health, dental, and vision insurance
+1
AI Gateway Engineer — Scale AI/LLM API Stack (Equity + Bonus)
AI Gateway Engineer — Scale AI/LLM API Stack (Equity + Bonus)

Kong • Belluno

On-site
EUR 60,000 - 90,000
Equity
Bonus
Health, dental, and vision insurance
+1
AI Gateway Engineer (Golang/Rust) – Equity & Bonus
AI Gateway Engineer (Golang/Rust) – Equity & Bonus

Kong • Sicilia

On-site
EUR 60,000 - 90,000
Equity
Bonus
Health insurance
+3
AI Gateway Software Engineer (Go/Rust) - Equity
AI Gateway Software Engineer (Go/Rust) - Equity

Kong • Trentino-Alto Adige

On-site
EUR 65,000 - 85,000
Equity
Bonus
Health, dental, and vision insurance
+1
Software Engineer AI Gateway
Software Engineer AI Gateway

Kong • Lombardia

Remote
EUR 65,000 - 110,000