ML Performance Engineer: Low-Level Systems & GPUs

Trading Interview; New York (NY); On-site; USD 170,000 - 210,000; Full time

Job summary

Jane Street is seeking an engineer with expertise in low-level systems programming and optimisation to join our ML team. The role focuses on optimizing model performance for training and inference, with attention to scalable training, low-latency real-time inference, and high-throughput research workloads.

You will engage with CUDA, GPU memory hierarchies, and distributed systems, ensuring throughput aligns with real throughput expectations.

Qualifications

• Understanding of modern ML techniques and toolsets.
• Experience debugging a training run's performance end to end.
• Low-level GPU knowledge including PTX, SASS, warps, Tensor Cores.
• Familiar with CUDA graph launch, memory hierarchy and synchronization.
• Experience with GPU networking concepts (Infiniband, NVLink, RoCE).

Responsibilities

• Optimize performance of ML models for training and inference.
• Improve throughput in research and real-time systems; reduce latency.
• Work across storage, networking and host/GPU levels for end-to-end efficiency.
• Debug performance bottlenecks using CUDA tools and profiling suites.

Skills

ML techniques

Performance debugging

GPU architecture

Tools

CUDA GDB

NSight Systems

NSight Compute

PTX

SASS

Tensor Cores

Triton

CUTLASS

CUB

Thrust

cuDNN

cuBLAS

Infiniband

NVLink

Job description

Jane Street is seeking an engineer with expertise in low-level systems programming and optimisation to join our ML team. The role focuses on optimizing model performance for training and inference, with attention to scalable training, low-latency real-time inference, and high-throughput research workloads.

You will engage with CUDA, GPU memory hierarchies, and distributed systems, ensuring throughput aligns with real throughput expectations.