Senior Software Engineer, AI Gateway

Kong; Lombardia; On-site; EUR 90,000 - 140,000; Full time

Job summary

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. We are seeking a senior backend engineer to design and own core components of our AI Gateway, ensuring 99.99% uptime and low-latency AI traffic handling.

You will implement features like prompt engineering, caching, and observability in Golang and/or Python, influence architecture, and collaborate with a distributed team across cloud environments. Mentoring and full lifecycle ownership are key aspects of the role.

Qualifications

• 5+ years of backend development experience (Golang or Python).
• Experience with AI/ML infra and LLMs.
• Strong foundations in distributed systems and API design.
• Cloud-native experience (Kubernetes, Docker, AWS/GCP/Azure).
• Familiarity with API gateways or service mesh is a plus.

Responsibilities

• Design and own core components of the AI Gateway ensuring high uptime and low latency for critical AI traffic.
• Lead implementation of features like prompt engineering, caching, and observability in Golang/Python.
• Influence technical roadmap and mitigate debt to enable long-term scalability.
• Architect integrations with various LLMs and AI platforms for API management.
• Mentor junior engineers and participate in design reviews.

Skills

Golang

Python

LangChain

LlamaIndex

Distributed systems

API design

Cloud-native

Tools

Kubernetes

Docker

AWS/Azure/GCP

Job description

Are you ready to unlock intelligence? If you don’t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.

The Mission

The AI Gateway team is at the forefront of Kong's evolution into the Agentic Era, building the critical infrastructure that connects enterprises to the world of generative AI. We are architecting a highly performant, secure, and observable gateway that provides developers with an unparalleled 'Developer Delight' experience for AI workloads. Your work will directly empower our customers to innovate faster by securely integrating LLMs and AI services into their applications at massive scale.

What You’ll Do

• Design and own core components of our AI Gateway, ensuring 99.99% uptime and low-latency performance for critical AI traffic.
• Lead the implementation of features like prompt engineering, caching, and observability for AI services, writing clean, maintainable Golang and/or Python code.
• Influence the technical roadmap and architecture, proactively identifying and mitigating technical debt to ensure long-term scalability.
• Architect robust solutions for integrating with various Large Language Models (LLMs) and AI platforms, shaping the future of API management for AI.
• Mentor junior engineers, fostering a culture of technical excellence, high-ownership, and rapid iteration through code reviews and design discussions.
• Drive the full development lifecycle, from ideation and design to deployment and operation within a modern Kubernetes environment.
• Collaborate with product managers and cross-functional teams to translate complex customer requirements into elegant and powerful technical solutions.

What You’ll Bring

The Toolkit

• 5+ years of experience in backend development, preferably with Golang and/or Python.
• Experience with AI/ML infrastructure, LLMs, or frameworks like LangChain or LlamaIndex.
• Deep understanding of distributed systems, microservices architecture, and API design principles.
• Proven experience designing and operating services in a cloud-native environment (Kubernetes, Docker, AWS/GCP/Azure).
• Familiarity with API gateways, service mesh, or related infrastructure technologies.
• Strong computer science fundamentals (data structures, algorithms, concurrency).

The Kong DNA

• Ownership: You are a self-starter who takes initiative and sees features through from conception to production.
• Urgency: You thrive in a fast-paced environment, balancing quality with the need to deliver value to customers quickly.
• Collaboration: You excel at working with a globally distributed team, communicating clearly and effectively to achieve shared goals.

Bonus Points

• Contributions to open-source projects.
• Experience with performance tuning and optimization of high-throughput systems.
• Knowledge of Lua/OpenResty.

About Kong

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Trusted by the Fortune 500® and AI-native startups alike, Kong’s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic — on any model, any cloud. For more information, visit www.konghq.com.