Senior ML Engineer - GPU Inference & Large-Scale AI Cloud

Nebius; United States; Remote; USD 180,000 - 250,000; Full time

Job summary

Nebius is building a high-performance inference and fine-tuning platform in a leading GPU cloud. We aim to push foundation models to hardware limits with maximum throughput, minimal latency, and optimised cost-per-token across thousands of GPUs.

Join Token Factory within Nebius Cloud to work on inference optimization, low-precision training, and novel architectures, contributing to open-source and in-house components. A strong ML/engineering background is essential.

Qualifications

• Strong foundations in machine learning and transformer architecture.
• Experience profiling GPU workloads with profiling tools.
• Proficiency in Python and modern deep learning frameworks.

Responsibilities

• Develop and optimize inference and fine-tuning pipelines at scale.
• Profile and improve GPU utilization, latency, and throughput.

Skills

Transformer architectures

Gpu profiling

Python

CI/CD

Deep learning frameworks

Communication

Tools

Nsight

PyTorch

Triton

CUDA

Job description

Nebius is building a high-performance inference and fine-tuning platform in a leading GPU cloud. We aim to push foundation models to hardware limits with maximum throughput, minimal latency, and optimised cost-per-token across thousands of GPUs.

Join Token Factory within Nebius Cloud to work on inference optimization, low-precision training, and novel architectures, contributing to open-source and in-house components. A strong ML/engineering background is essential.