Senior LLM Infra Engineer — HPC, Benchmarking & DevEx

Baseten; United States; Remote; USD 180,000 - 300,000; Full time

Job summary

Baseten is seeking a Senior Software Engineer to join a high-impact team at the crossroads of HPC and Large Language Model engineering. You will help define the roadmap, drive key technical decisions, and own the future of this work.

You will build automated benchmarks, GPU-enabled dev environments, and open-source tools like InferenceMAX and genai-bench, while profiling performance and optimizing the model runtimes stack.

Qualifications

• Mid-senior level with high-leverage impact.
• Strong communication skills to drive cross-team efforts.
• Ability to navigate vague requirements and mentor other engineers.

Responsibilities

• Benchmarking: Evaluate, run and automate LLM quality benchmarks and custom performance suites.
• DevEx Improvement: Develop GPU-enabled development environments for model experimentation.
• Tool Development: Build/open-source tools like InferenceMAX and genai-bench for evaluation and benchmarking.
• System Profiling: Use profilers to collect performance profiles and identify bottlenecks.
• Monitoring Observability: Develop real-time dashboards and alerts for system health and runtime performance.
• Continuous Integration: Automate performance testing via CI/CD pipelines and release workflow automation.
• Optimization Automation: Build tools to identify the Pareto frontier for latency, cost, and quality.

Skills

Systems hardware

Automation mindset

Mathematical curiosity

Strong communication

Mentoring

NVIDIA Nsight Systems

PyTorch Profiler

py-spy

Tools

NVIDIA Nsight Systems

PyTorch Profiler

py-spy

Job description

Baseten is seeking a Senior Software Engineer to join a high-impact team at the crossroads of HPC and Large Language Model engineering. You will help define the roadmap, drive key technical decisions, and own the future of this work.

You will build automated benchmarks, GPU-enabled dev environments, and open-source tools like InferenceMAX and genai-bench, while profiling performance and optimizing the model runtimes stack.