Staff Engineer, AI Benchmarking & Inference Systems

S27a; San Francisco, New York (CA, NY); Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Generous PTO

Office stipend

Competitive healthcare (medical,Dental

Vision benefits

Conference support

Job summary

SemiAnalysis is seeking a highly motivated Member of Technical Staff to join our growing engineering team. You will contribute to inference benchmarks, system modeling, and the preparation of technical newsletters with potential authorship recognition.

We hire individuals at all experience levels and offer competitive compensation. As part of the process, you’ll complete a paid coding challenge reflecting daily tasks and collaborate with a global team across multiple AI and hardware domains.

Qualifications

• Proactive, self-motivated, and capable of working independently in a global team.
• Experience with ML frameworks such as PyTorch or JAX through professional experience, personal projects, or personal Substack blogs.
• Solid understanding of at least 1 of the following: transformer architecture, LLM parallelism strategies, and/or CUDA parallel programming.

Responsibilities

• Conduct training and inference benchmarks across AI accelerators (H100, Mi300X, TPUs, Trainium2) using PyTorch/JAX.
• Author detailed reports analyzing benchmark results, hardware performance, scalability, and efficiency.
• Develop system modelling using Python and NCCL for AI compute clusters from single-GPU to large scales.
• Establish partnerships with cloud providers and AI hardware makers (AMD, NVIDIA, etc.).
• Stay current on trends by attending NeurIPS, MLSys, GTC and related conferences.

Skills

PyTorch

JAX

Python

Transformer architecture

LLM parallelism

Tools

NCCL

CUDA

vLLM

SGLang

Job description

SemiAnalysis is seeking a highly motivated Member of Technical Staff to join our growing engineering team. You will contribute to inference benchmarks, system modeling, and the preparation of technical newsletters with potential authorship recognition.

We hire individuals at all experience levels and offer competitive compensation. As part of the process, you’ll complete a paid coding challenge reflecting daily tasks and collaborate with a global team across multiple AI and hardware domains.