AI Infrastructure & Solutions Architecture Lead

Rafay Systems; United States; Hybrid; USD 190,000 - 280,000; Full time

Benefits offered by this job

Stock options

Job summary

Rafay Systems seeks a Principal Solutions Architect to enable enterprise customers/Neo clouds in deploying, operating, and scaling AI/ML workloads on its GPU Platform-as-a-Service offering. This hybrid leadership role combines hands-on architecture with building and growing a team of Solutions Architects.

You will guide AI infrastructure strategy, collaborate with platform engineering and data science teams, and drive production-ready GPU/ML deployments on Kubernetes.

Qualifications

• 8+ years in infrastructure, platform, or solutions engineering.
• 3+ years AI/ML infrastructure or MLOps.
• 2+ years direct people-management experience.
• Demonstrated ability to lead and grow a technical team while remaining hands-on with customers.
• Deep Kubernetes expertise including cluster lifecycle and RBAC.
• Hands-on experience with NVIDIA GPU infrastructure (H100/H200/B200).
• Proficiency with distributed training concepts (NCCL, tensor parallelism).
• Experience with LLM inference serving and optimization.
• Familiarity with GPU Operator, MIG, SR-IOV, and network fabrics.
• Strong scripting and automation skills (Python, Bash, Go).
• Ability to communicate complex technical concepts to diverse audiences, including executive stakeholders.
• Experience with AWS, Azure, or GCP platforms.
• Familiarity with monitoring tools like Prometheus, Grafana, and OpenTelemetry.
• Understanding of GPU-based workloads and model serving.
• Proven troubleshooting capabilities for infrastructure issues.
• Excellent communication, coaching, and customer-facing skills.

Responsibilities

• Recruit, hire, and onboard Solutions Architects as the team scales.
• Directly manage a team of Solutions Architects, including workload allocation, coaching, and day-to-day support.
• Set individual and team goals; conduct regular 1:1s and performance reviews.
• Own career development planning for direct reports, including skills growth, promotion readiness, and succession planning.
• Foster an inclusive, high-performing team culture aligned with Rafay's values.
• Manage team capacity, prioritization, and staffing against customer and project demand.
• Partner with sales, engineering, and executive leadership on hiring plans and team structure.
• Mentor and upskill both direct reports and junior team members across the broader organization.
• Design comprehensive AI/ML platform architectures covering inference, training, and data pipelines.
• Develop reference architectures for GPU cluster deployment and LLM serving infrastructure.
• Evaluate inference serving frameworks including vLLM, TGI, and Triton.
• Advise on GPU fabric topology options for distributed training scenarios.
• Design observability strategies using DCGM, OpenTelemetry, and eBPF.
• Translate infrastructure requirements into actionable platform designs.
• Deliver technical presentations, workshops, and proof-of-concept engagements.
• Serve as trusted advisor on AI infrastructure strategy, cost optimization, and scaling.
• Partner with customer stakeholders to understand workload requirements.
• Architect networking, identity management, observability, and security integrations.
• Monitor and troubleshoot production environments for GPU utilization and cluster health.
• Lead root cause analysis for complex customer issues.
• Document reference architectures and implementation best practices.

Skills

Infra/Platform experience

AI/ML infra

People management

Kubernetes expertise

NVIDIA GPUs

Distributed training

LLM serving

Prometheus

Grafana

Cloud platforms

Scripting languages

Communication

Tools

NVIDIA GPU Operator

Slurm

PyTorch

TensorFlow

Run:AI

Job description

Rafay Systems seeks a Principal Solutions Architect to enable enterprise customers/Neo clouds in deploying, operating, and scaling AI/ML workloads on its GPU Platform-as-a-Service offering. This hybrid leadership role combines hands-on architecture with building and growing a team of Solutions Architects.

You will guide AI infrastructure strategy, collaborate with platform engineering and data science teams, and drive production-ready GPU/ML deployments on Kubernetes.