MTS: GPU Cluster Benchmarking & Performance

S27a; San Francisco, New York (CA, NY); Hybrid; USD 140,000 - 210,000; Full time

Benefits offered by this job

Generous PTO

Office stipend

Competitive healthcare (medical,Dental

Conference support

Job summary

SemiAnalysis is seeking a Senior MTS to build the next generation of ClusterMAX benchmarks and run evaluations across GPU clusters, focusing on storage IO, bandwidth, and fault-tolerance.

You will collaborate with executives and engineers across neoclouds, hyperscalers, and chip vendors to extend our TCO and goodput methodologies, and to author research that shapes AI infrastructure decisions.

Qualifications

• Hands-on experience operating GPU clusters: Slurm and/or Kubernetes, InfiniBand/RoCE fabrics, distributed storage.
• Strong Python and shell scripting; comfort building benchmark harnesses and CI pipelines.
• Understanding of distributed training/inference failure modes — node failures, checkpointing, blast radius, recovery.
• Experience similar to our Technical Consultant profile is a plus: due diligence, TCO analysis, and client-facing technical communication.
• Security mindset: multi-tenant isolation, bare-metal vs virtualized trade-offs.

Responsibilities

• Leading development of the next generation of ClusterMAX benchmarks: storage IO and bandwidth, NCCL/RCCL collectives, fault-tolerance and goodput measurement, multi-tenant security and isolation testing.
• Deploying to and evaluating dozens of GPU clusters across hyperscalers and neoclouds (GB300/GB200 NVL72, B300/B200, H200, MI355X, TPUv7)
• Extending our TCO and goodput methodology into automated, reproducible tests
• Working with executives and engineers at 200+ neoclouds, hyperscalers, and chip vendors
• Authoring technical research analyzing benchmark results, reliability, security, and ease of use, with direct authorship recognition

Skills

GPU cluster operations

Slurm

Kubernetes

InfiniBand/RoCE fabrics

Distributed storage

Python

Shell scripting

Benchmark harnesses & CI pipelines

Understanding distributed training/inf

Client-facing technical communication

Tools

CI pipelines tooling

Benchmarking tools

Job description

SemiAnalysis is seeking a Senior MTS to build the next generation of ClusterMAX benchmarks and run evaluations across GPU clusters, focusing on storage IO, bandwidth, and fault-tolerance.

You will collaborate with executives and engineers across neoclouds, hyperscalers, and chip vendors to extend our TCO and goodput methodologies, and to author research that shapes AI infrastructure decisions.