Senior Software Engineer, LLM Performance Tooling

BaseTen; San Francisco, New York (CA, NY); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity

Health plans for dependents

Flexible PTO (Winter Break)

Parental leave

Fertility stipend

401(k)

Job summary

Baseten is seeking a mid-to-senior Software Engineer to advance HPC and LLM infrastructure. You will define roadmaps, own critical tooling, and drive performance optimizations across GPU clusters for production-scale AI workloads.

You will build internal development environments, automate benchmarking, and contribute to open-source projects, shaping Baseten's platform for cutting-edge AI deployments.

Qualifications

• A love for systems & hardware: GPU memory subsystems, InfiniBand, data movement in clusters.
• An automation mindset: scripting repetitive tasks, stress-testing, and breaking points.
• Mathematical curiosity: understanding Transformers, FLOPs, memory requirements.
• Technical toolkit: proficiency with Python and NVIDIA software stack; knowledge of C++ is a plus.

Responsibilities

• Benchmarking: run and automate LLM quality benchmarks (GSM8K, MMLU) and workload-specific tests.
• DevEx improvements: build GPU-enabled development environments for fast model experimentation.
• Tool development: contribute to open-source tools for evaluation and benchmarking.
• System profiling: use profilers to identify bottlenecks in compute and networking stacks.
• Monitoring & observability: create real-time dashboards and alerts for system health and startup times.
• CI: automate performance tests and release workflows for the runtime stack.
• Optimization automation: discover Pareto-optimal configurations for latency, cost, and quality.

Skills

Python

NVIDIA software stack

C++ familiarity

Job description

Baseten is seeking a mid-to-senior Software Engineer to advance HPC and LLM infrastructure. You will define roadmaps, own critical tooling, and drive performance optimizations across GPU clusters for production-scale AI workloads.

You will build internal development environments, automate benchmarking, and contribute to open-source projects, shaping Baseten's platform for cutting-edge AI deployments.