Senior Staff AI Engineer - Intelligent Routing & AI ADC

Insight Global; San Jose (CA); Hybrid; USD 180,000 - 195,000; Full time

Job summary

Insight Global is hiring a Senior Staff Engineer - AI ADC to lead the architecture and delivery of AI‑driven application delivery components. The role blends ADC/SLB expertise with practical AI/LLM system experience to enable intelligent routing and inference‑aware load balancing across complex enterprise environments.

The candidate will drive architecture, design, prototyping, and execution, collaborating with product management, platform teams, and AI strategy groups in a hybrid on‑site

Qualifications

• Advanced degree in Computer Science, Networking, or related field (MS/PhD preferred).
• Extensive experience with ADCs, SLB, and GSLB architectures for AI workloads.
• Proven ability to drive architecture, design, prototyping, and execution at senior staff level.
• Strong programming skills in C/C++ (data plane) and Go/Python (control/ML).
• Experience with AI/ML systems, including model inference pipelines and LLM integrations.

Responsibilities

• Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI traffic.
• Design and implement LLM‑aware load balancing incorporating semantic, token, latency, and model insights.
• Develop enhancements for ADC and GSLB platforms to support AI workloads, including token‑aware rate limiting and latency‑based routing.
• Integrate with vector databases, model registries, and ML observability systems.
• Drive system‑wide architecture: data plane, control plane, configuration, and distributed state management.

Skills

ADC architecture

SLB / DNS‑GSLB

AI/LLM systems

architecture leadership

prototyping & execution

traffic engineering

latency optimization

multi-model orchestration

Education

MS/PhD preferred in CS or Networking

Tools

eBPF

perf

VTune

flamegraphs

vector databases

model registries

ML observability systems

Job description

Insight Global is hiring a Senior Staff Engineer - AI ADC to lead the architecture and delivery of AI‑driven application delivery components. The role blends ADC/SLB expertise with practical AI/LLM system experience to enable intelligent routing and inference‑aware load balancing across complex enterprise environments.

The candidate will drive architecture, design, prototyping, and execution, collaborating with product management, platform teams, and AI strategy groups in a hybrid on‑site