Systems Software Engineer - Data Center Performance

NVIDIA Corporation; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

NVIDIA продолжает поиск опытного инженера по системной производительности для команды Applied Systems Engineering Performance. Ваша работа объединит эксперименты и разработку ПО для изучения спросов на новых платформах и масштабирования в дата-центрах.

Основные задачи включают анализ узких мест, сбор и интерпретацию данных, разработку инструментов на Python и сотрудничество с командами по оборудованию, драйверам и приложениям для достижения воспроизводимых улучшений в производительности.

Qualifications

• Опыт работы с profiling и tracing инструментами: Nsight Systems, Nsight Compute, Linux perf, eBPF- инструменты.
• Практический опыт с RDMA и MPI или NCCL, анализ производительности через PCIe, NVLink, InfiniBand или Ethernet.
• Опыт с Slurm, контейнерными рантаймами, cgroups; влияние размещения задач на производительность.
• Опыт улучшения распределённого обучения, inference, или научных приложений с помощью масштабирования (strong/weak).
• Базовая зарплата: определят по локации, опыту и факторам.

Responsibilities

• Характеризовать производительность узлов, стоек и кластера с помощью микро-бенчмарков и реальных приложений.
• Измерять пропускную способность, задержку и масштабируемость; выявлять узкие места вычислений, памяти и передачи.
• Выявлять проблемы Linux в планировании CPU, NUMA, affinity, прерываниях и конкуренции ресурсов.
• Оценивать влияние на использование GPU и эффективность приложений.
• Изучать распределённую нагрузку, взаимодействие между вычислениями и коммуникацией.
• Изучать размещение GPU/ NIC, структуру interconnect и сетевой трафик на производительность.
• Использовать профильные данные, трассировки и счетчики; работать с командами для исправления.
• Разрабатывать Python-утилиты для автоматического запуска бенчмарков, сбора данных и анализа.
• Устанавливать повторяемые базовые линии, оценивать вариацию запуска и регрессию на разных конфигурациях.
• Определять и завершать валидацию производительности для новых платформ и развертываний.

Skills

Python

C/C++

CUDA

Linux performance

HPC performance tooling

MPI/NCCL

Slurm

Automation

Education

BS or MS in Computer Science, Engineering, Mathematics, Physics

Tools

Nsight Systems

Nsight Compute

Linux perf

eBPF-based tools

ReFrame

Job description

Описание: NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world. We are hiring a proficient engineer to join our Applied Systems Engineering Performance team. This role integrates comprehensive experimentation and software engineering to explore how emerging platforms respond under demanding HPC and AI workloads. You will reveal performance restrictions, establish trustworthy measurements, and provide collaborating teams with the insights they need to improve new platforms. Your work will connect innovations in individual technologies with the challenge of harmonizing their performance at datacenter scale.

Задачи:

• Characterize node, rack, and cluster performance using microbenchmarks and representative applications.
• Measure throughput, latency, and scaling efficiency to identify compute, memory, communication, and storage bottlenecks.
• Identify Linux performance problems related to CPU scheduling, NUMA locality, CPU and memory affinity, interrupt handling, and resource contention.
• Assess their impact on GPU usage and application efficiency.
• Examine distributed workload scaling, considering collective communication, synchronization delays, slow nodes, and the interplay between computation and communication.
• Assess how GPU and NIC positioning, interconnect structure, and network traffic impact performance.
• Combine application profiles, system traces, hardware counters, and cluster telemetry to isolate root causes.
• Develop focused reproductions and collaborate with hardware, firmware, driver, library, and application teams to implement and verify fixes.
• Develop and maintain Python tools for automated benchmark execution, data collection, and performance analysis.
• Set up repeatable baselines, assess run-to-run variability, and spot regressions across hardware configurations and software releases.
• Define and complete performance validation for new platforms and cluster deployments.
• Identify underperforming nodes, links, or storage paths, and verify that corrective actions restore encouraged workload performance.

What we need to see:

Extensive practical experience in Linux performance analysis, including knowledge of processes and threads, CPU and memory architecture, NUMA, I/O, and networking. Experience investigating performance in multi-node HPC or GPU-accelerated systems, distinguishing local resource bottlenecks from communication overhead, load imbalance, and shared-resource contention. Deep expertise in at least one relevant area: GPU application performance, Linux systems performance, high-speed networking and collective communication, parallel storage, or HPC performance tooling. Proficiency in Python to build maintainable automation and analysis tools. Experience in developing and debugging performance-sensitive code in C, C++ and CUDA. Ability to build controlled experiments, interpret profiling and counter data, and demonstrate reproducible improvements in application performance. Independently driving technical efforts across hardware and software boundaries, communicating findings clearly, and working with component owners to resolve issues. BS or MS in Computer Science, Engineering, Mathematics, Physics (or equivalent experience), with 8+ years of applicable experience.

Требования:

Experience working with profiling and tracing tools like Nsight Systems, Nsight Compute, Linux perf, or eBPF-based tools. Practical experience with RDMA and MPI or NCCL, and analyzing performance across PCIe, NVLink, InfiniBand, or Ethernet interconnects. Experience with Slurm, container runtimes, and cgroups, including understanding how workload placement, resource isolation, and CPU/GPU/NIC affinity impact performance. Experience improving distributed training, inference, or scientific applications by conducting strong- and weak-scaling studies, or building automated performance regression frameworks using tools such as ReFrame. Your base salary will be determined based on your location, experience, and other objective job-related factors.

Salary ranges:

• Poland: 292,500 PLN – 507,000 PLN for Level 4; and 375,000 PLN – 650,000 PLN for Level 5.
• France: 87,750 EUR – 152,100 EUR for Level 4; and 114,000 EUR – 197,600 EUR for Level 5.
• Germany: 112,200 EUR – 194,480 EUR for Level 4; and 141,900 EUR – 245,960 EUR for Level 5.