Senior AI ADC Architect: Intelligent Routing & Load Balancing

Insight Global; San Jose (CA); Hybrid; USD 200,000 - 220,000; Full time

Job summary

Insight Global seeks a Senior Staff Engineer - AI ADC to join a global technology client, focusing on AI-driven traffic routing, intelligent load balancing, and secure, high-performance application delivery.

You will lead architecture and prototyping across data plane and control plane, collaborate with product and AI strategy teams, and help scale AI workloads with LLM integration, latency-aware routing, and observability.

Qualifications

• MS or PhD in CS, Networking, or a related field.
• Deep knowledge of L4–L7 protocols (TCP/TLS/HTTP/2/3, QUIC) and GSLB strategies.
• Strong coding skills in C/C++ for data plane and Go/Python for control/ML; proficiency in performance profiling.

Responsibilities

• Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI traffic.
• Design and implement LLM-aware load balancing with semantic, token, latency, and model-level insights.
• Enhance ADC and GSLB platforms to support AI workloads (token-aware rate limiting, latency-based routing, model endpoint health checks).
• Integrate with vector databases, model registries, and ML observability systems.
• Drive system-wide architecture: data plane, control plane, configuration, and distributed state management.

Skills

C/C++

Go/Python

AI/ML systems

Networking protocols

Education

MS/PhD in CS/Networking

Tools

perf

eBPF

flamegraphs

VTune

Job description

Insight Global seeks a Senior Staff Engineer - AI ADC to join a global technology client, focusing on AI-driven traffic routing, intelligent load balancing, and secure, high-performance application delivery.

You will lead architecture and prototyping across data plane and control plane, collaborate with product and AI strategy teams, and help scale AI workloads with LLM integration, latency-aware routing, and observability.