AI Gateway Engineer: Scale LLM APIs (Golang/Rust)

Kong; Lecce; On-site; EUR 45,000 - 65,000; Full time

Benefits offered by this job

Equity

Bonus

Health insurance

Dental insurance

Vision insurance

Paid holidays

PTO

Job summary

Kong is looking for a Software Engineer 2 to join the AI Gateway engineering team in Lecce, Puglia. You will help build and evolve Kong's API management stack for AI and LLM workloads, shaping features like multi-LLM routing, semantic caching, and high-scale components.

You’ll work on proxying, load balancing, and platform integrations, partnering with a distributed team to deliver reliable, low-latency capabilities.

Qualifications

• 3+ years of server-side programming.
• Hands-on experience with Golang or Rust.
• Proven ability to build and debug highly scalable, distributed, reliable systems.
• Strong Linux systems troubleshooting and profiling skills.
• Experience with distributed systems and Linux networking on L7/L4.
• Comfortable working in a globally distributed team.
• Strong ownership and communication skills (documentation written by engineers).
• Familiarity with AI/LLM concepts.
• Clear written and verbal communication.
• Collaboration in a distributed team.

Responsibilities

• Build and maintain API management software for AI/LLM stacks.
• Develop features such as proxying, load balancing, and database support for Kong AI Gateway.
• Troubleshoot bugs, tune performance, and perform low-level optimizations.
• Bridge Kong with third-party deployment platforms and tooling.
• Handle additional tasks as assigned by management.

Skills

Server-side programming

Golang

Rust

Distributed systems

Linux networking

Linux systems programming

Troubleshooting

Ownership

Communication skills

AI/LLM concepts

Job description

Kong is looking for a Software Engineer 2 to join the AI Gateway engineering team in Lecce, Puglia. You will help build and evolve Kong's API management stack for AI and LLM workloads, shaping features like multi-LLM routing, semantic caching, and high-scale components.

You’ll work on proxying, load balancing, and platform integrations, partnering with a distributed team to deliver reliable, low-latency capabilities.