Senior Software Engineer, LLM Inference & Performance

NVIDIA AI; Town of Santa Clara (NY); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Equity

Health Insurance

Job summary

NVIDIA AI is seeking a Senior Software Engineer to contribute features and optimizations to upstream inference engines and enhance runtime capabilities for high-throughput, low-latency inference. You will profile and improve performance across layers and collaborate with teams to translate production requirements into effective solutions.

The role requires 5+ years of production software experience with a focus on performance and reliability, and strong skills in Python, C++/CUDA, and LLM

Qualifications

• Candidates should have over 5 years of experience in building production software with a focus on performance and reliability improvements.
• Strong programming skills in Python and C++/CUDA, along with experience in LLM inference stacks and profiling, are essential.

Responsibilities

• Contribute features and optimizations to upstream inference engines and improve runtime capabilities for high-throughput, low-latency inference.
• Profile and enhance performance across various layers and collaborate with teams to translate production requirements into effective solutions.

Skills

Python

C++/CUDA

LLM Inference

Profiling

Performance Optimization

Distributed Systems

Concurrency

Benchmarking

Open-source Contributions

Data Analysis

Resource Utilization

Collaboration

Debugging

Job description

NVIDIA AI is seeking a Senior Software Engineer to contribute features and optimizations to upstream inference engines and enhance runtime capabilities for high-throughput, low-latency inference. You will profile and improve performance across layers and collaborate with teams to translate production requirements into effective solutions.

The role requires 5+ years of production software experience with a focus on performance and reliability, and strong skills in Python, C++/CUDA, and LLM