GPU Performance Engineer for Transformer Models

Luma; Seattle (WA); On-site; USD 195,000 - 395,000; Full time

Job summary

Luma seeks a high-impact engineer to optimize multimodal transformer workloads on GPUs and CPUs, writing high-performance CUDA and Triton kernels. You will push for maximum throughput, low latency, and scalable deployments across multi-node systems, while improving profiling and automation tools.

You will own kernel fusion, tensor-core Utilization, and architectures for distributed inference/training. Deep transformer knowledge and strong PyTorch skills are essential for success.

Qualifications

• Expert-level Triton/CUDA programming and GPU optimization.
• Strong PyTorch skills, including kernel development and custom operations.
• Proficiency with profiling tools (NVIDIA Nsight, torch profiler, custom tooling).
• Deep understanding of transformer architectures and attention mechanisms.

Responsibilities

• Profile and optimize GPU/CPU/accelerator code for maximum utilization and minimal latency.
• Write high-performance PyTorch, Triton, and CUDA, dropping to custom operations when needed.
• Develop fused kernels and leverage tensor cores and modern hardware features across platforms.
• Optimize model architectures and implementations for distributed multi-node production deployment.
• Build performance monitoring and analysis tools and automation.
• Research and implement cutting-edge optimization techniques for transformer models.

Skills

Transformer architectures

Attention mechanisms

Profiling and optimization

GPU optimization

Kernel development

Tools

Triton

CUDA

PyTorch

Nsight Profiling

Job description

Luma seeks a high-impact engineer to optimize multimodal transformer workloads on GPUs and CPUs, writing high-performance CUDA and Triton kernels. You will push for maximum throughput, low latency, and scalable deployments across multi-node systems, while improving profiling and automation tools.

You will own kernel fusion, tensor-core Utilization, and architectures for distributed inference/training. Deep transformer knowledge and strong PyTorch skills are essential for success.