GPU Transformer Performance Engineer (Triton/CUDA)

Luma AI; United States; Remote; USD 180,000 - 280,000; Full time

Job summary

Luma AI seeks an expert GPU optimization engineer to accelerate multimodal models. You will profile and optimize GPU/CPU/accelerator code for maximal utilization and minimal latency, writing high-performance PyTorch, Triton, and CUDA kernels and developing fused kernels for tensor cores.

Drive distributed multi-node deployments, build monitoring tooling, and apply state-of-the-art transformer optimization techniques.

Qualifications

• Expert-level Triton/CUDA programming and GPU optimization.
• Strong PyTorch skills, including kernel development and custom operations.
• Proficiency with profiling tools (NVIDIA Nsight, torch profiler, custom tooling).
• Deep understanding of transformer architectures and attention mechanisms.

Responsibilities

• Profile and optimize GPU/CPU/accelerator code for maximum utilization and minimal latency.
• Write high-performance PyTorch, Triton, and CUDA, dropping to custom operations when needed.
• Develop fused kernels and leverage tensor cores and modern hardware features across platforms.
• Optimize model architectures and implementations for distributed multi-node production deployment.
• Build performance monitoring and analysis tools and automation.
• Research and implement cutting-edge optimization techniques for transformer models.

Job description

Luma AI seeks an expert GPU optimization engineer to accelerate multimodal models. You will profile and optimize GPU/CPU/accelerator code for maximal utilization and minimal latency, writing high-performance PyTorch, Triton, and CUDA kernels and developing fused kernels for tensor cores.

Drive distributed multi-node deployments, build monitoring tooling, and apply state-of-the-art transformer optimization techniques.