GPU-Accelerated ML Training Performance Engineer

Blackrock Neurotech; Salt Lake City (UT); On-site; USD 120,000 - 190,000; Full time

Job summary

Blackrock Neurotech, based in Salt Lake City, UT, seeks an ML Training Performance Engineer to optimize large-scale model training across GPUs and clusters. You’ll work hands-on across Python, C++, and GPU kernels while collaborating with researchers and IT teams to scale compute efficiently.

You will own performance baselines, profiling, and tooling, ensuring numerical correctness and scientific intent as models grow in complexity and data. Occasional on-site work is required.

Qualifications

• Demonstrated experience improving the performance of substantial deep learning training workloads with measurable gains

Responsibilities

• Own training performance across single-GPU, multi-GPU, and multi-node workloads with reproducible baselines
• Profile the full training path to identify bottlenecks and impact
• Optimize tensor layouts, memory, and precision to fit larger models
• Write and validate custom GPU kernels using CUDA, Triton, or HIP
• Improve Python scripts, training configs, batching, and optimizers
• Design distributed training strategies based on model structure and memory limits
• Collaborate with researchers on hardware-aware changes and convergence impact
• Coordinate with infra teams on GPU selection, cloud configs, and containers

Skills

Deep learning optimization

GPU kernel development

Profiling & benchmarking

Python & C++ programming

PyTorch knowledge

Distributed training

Linux GPU environments

Big data / HPC concepts

Education

Bachelor’s degree in CS/Engineering or equivalent

Tools

CUDA

HIP/ROCm

Triton

TensorRT

Job description

Blackrock Neurotech, based in Salt Lake City, UT, seeks an ML Training Performance Engineer to optimize large-scale model training across GPUs and clusters. You’ll work hands-on across Python, C++, and GPU kernels while collaborating with researchers and IT teams to scale compute efficiently.

You will own performance baselines, profiling, and tooling, ensuring numerical correctness and scientific intent as models grow in complexity and data. Occasional on-site work is required.