Senior Staff Engineer

Insight Global; San Jose (CA); Hybrid; USD 180,000 - 195,000; Full time

Job summary

Insight Global is assisting a global technology client in San Jose, CA, seeking a Senior Staff Engineer - AI ADC to lead architecture for AI gateway components, AI-aware load balancing, and AI workload enhancements. The role requires driving AI strategy with product and platform teams, in a permanent hybrid on-site arrangement.

The position offers a base salary range of $180,000–$195,000 and emphasizes architecture, prototyping, and execution across data and control planes to support enterprise

Qualifications

• Extensive architecture leadership in ADC/SLB domains.
• Experience with AI/LLM systems and intelligent routing.
• Familiarity with vector databases and model registries.

Responsibilities

• Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.
• Design and implement LLM-aware load balancing with token, latency, and model-level insights.
• Develop enhancements for ADC and GSLB platforms to support AI workloads including AI model endpoint discovery and health checks.
• Contribute to Layer-8 context-aware routing, adaptive traffic shaping, and AI-driven anomaly detection.
• Drive system-wide architecture across data plane, control plane, configuration, and distributed state management.

Job description

A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations. This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference‑aware load balancing, and Layer‑8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution—working closely with product management, platform teams, and AI strategy groups.

Key Responsibilities Include:

• Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.
• Design and implement LLM‑aware load balancing, incorporating semantic, token, latency, and model‑level insights into traffic decisions.
• Develop enhancements for ADC and GSLB platforms to support AI workloads, including: Token‑aware rate limiting Inference latency‑based routing AI model endpoint discovery and health checks Integration with vector databases, model registries, and ML observability systems
• Contribute to advanced features including Layer‑8 context‑aware routing, adaptive traffic shaping, and AI‑driven anomaly detection.
• Drive system‑wide architecture: data plane, control plane, configuration, and distributed state management.

This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $180,000-$195,000 base salary.