AI Gateway Engineer: Scale LLM Routing & API

Kong Inc.; Lombardia; Hybrid; EUR 64,000 - 79,000; Full time

Job summary

Kong Inc. is seeking a Software Engineer for the AI Gateway Team to advance the Kong AI gateway, enabling secure governance of AI and LLM-based innovation worldwide.

You will work on core features like multi-LLM routing, semantic caching, MCP, and more, building scalable, distributed servers with a strong Linux foundation. The role emphasizes ownership, collaboration with a remotely distributed team, and clear documentation.

Qualifications

• 3+ years of programming in server-side applications and components.
• Hands-on experience with either Golang or Rust.
• Experience building and debugging highly scalable, distributed systems.
• Solid Linux systems troubleshooting and profiling skills.
• Knowledge of Linux networking stack and APIs on L7/L4 transports is a must.
• Experience working with distributed teams.
• Strong ownership and documentation skills.
• Familiarity with AI/LLM concepts is a plus.

Responsibilities

• Build and maintain API management software for AI/LLMs and its underlying stack.
• Implement new features including proxying, load balancing, and database support for Kong AI Gateway.
• Troubleshoot bugs, tune performance, and perform low-level optimizations.
• Bridge Kong with third-party deployment platforms and tooling.
• Handle additional tasks as assigned by management.

Skills

Go

Rust

Distributed systems

Linux troubleshooting

Tools

NGINX/OpenResty

Envoy

Kong/LuaJIT

Job description

Kong Inc. is seeking a Software Engineer for the AI Gateway Team to advance the Kong AI gateway, enabling secure governance of AI and LLM-based innovation worldwide.

You will work on core features like multi-LLM routing, semantic caching, MCP, and more, building scalable, distributed servers with a strong Linux foundation. The role emphasizes ownership, collaboration with a remotely distributed team, and clear documentation.