AI Inference Benchmark Architect

Neurophos; Austin, Sunnyvale (TX, CA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health insurance

Unlimited PTO

401k matching

Stock options

Voluntary benefits

Personalized benefits

Job summary

Neurophos is seeking a Performance Engineer to own benchmarking numbers for the T100 optical inference accelerator. You will define measurement methodology and generate end-to-end results across workloads, with full harness, plots, logs, and assumptions for reproducibility.

You will compare RTL models against measured GPUs, manage cloud or lab environments, and report time-to-first-token, latency, and energy metrics while maintaining a benchmark suite and governance for external claims.

Qualifications

• BS or MS in Computer Engineering, Electrical Engineering, Computer Science, or equivalent practical experience.
• 5+ years in GPU performance engineering, accelerator benchmarking, HPC performance, or ML systems measurement.
• Experience building or operating benchmark harnesses with real GPUs or accelerators.
• Hands-on with roofline or limiter analysis and analytical performance modeling.
• GPU performance analysis with Nsight tools for HBM vs compute and batching.
• Familiar with LLM inference stacks and MoE concepts.
• Proficient in Python for harnesses, parsing, and plots; Linux comfort.
• Cloud GPU ops on AWS/GCP/Azure, containers, drivers, and quotas.

Responsibilities

• Own performance and energy metrics across fidelities and hardware.
• Produce numbers for workloads across fidelities: roofline, models, RTL sim, real GPUs.
• Keep workload definitions constant across fidelities (model, sequence length, batch, precision).
• Bring up inference workloads from Hugging Face, PyTorch, papers, and vendor stacks.
• Measure competing GPUs end-to-end; manage cloud or lab accounts, images, drivers, runs.
• Report TTFT, ITL, tokens/sec, tokens/sec per watt, and energy metrics with instrumentation.
• Document discrepancies between RTL, models, and measured results with configs/logs.

Skills

GPU performance

Benchmark harness

Roofline analysis

Nsight Systems

LLM inference stacks

Python scripting

Linux proficiency

Cloud GPU ops

Education

BS/MS in CE/EE/CS

Tools

Hugging Face

vLLM

SGLang

TensorRT-LLM

CUDA

Triton

PyTorch

Job description

Neurophos is seeking a Performance Engineer to own benchmarking numbers for the T100 optical inference accelerator. You will define measurement methodology and generate end-to-end results across workloads, with full harness, plots, logs, and assumptions for reproducibility.

You will compare RTL models against measured GPUs, manage cloud or lab environments, and report time-to-first-token, latency, and energy metrics while maintaining a benchmark suite and governance for external claims.