Remote Software Engineer — AI Gateway & LLM Routing

Kong; United States; Remote; USD 72,000 - 89,000; Full time

Job summary

Kong Inc. is seeking a Software Engineer for the AI Gateway Team to help build the Kong AI gateway, enabling global governance of AI and LLM-based innovations. You will contribute to multi-LLM routing, semantic caching, MCP, and related core features.

Ideal candidates have 3+ years in server-side development, experience with Golang or Rust, and strong Linux performance and distributed systems skills. This role supports a remotely distributed team and emphasizes ownership and clear communication.

Qualifications

• 3+ years of experience in server-side applications and components.
• Hands-on experience with Golang or Rust.
• Experience building and debugging highly scalable distributed systems.
• Solid low-level Linux troubleshooting and profiling skills.

Responsibilities

• Build and maintain API management software for AI/LLMs and its underlying stack.
• Develop new features including proxying, load balancing, and database support for Kong AI Gateway.
• Troubleshoot bugs, optimize performance, and conduct low-level optimizations.
• Bridge Kong with third-party deployment platforms and tooling.
• Execute additional tasks as directed by management.

Skills

Golang

Rust

Distributed systems

APIs

Linux

Tools

NGINX/OpenResty

LuaJIT

Envoy

Istio

Job description

Kong Inc. is seeking a Software Engineer for the AI Gateway Team to help build the Kong AI gateway, enabling global governance of AI and LLM-based innovations. You will contribute to multi-LLM routing, semantic caching, MCP, and related core features.

Ideal candidates have 3+ years in server-side development, experience with Golang or Rust, and strong Linux performance and distributed systems skills. This role supports a remotely distributed team and emphasizes ownership and clear communication.