Senior Principal ML Inference Systems Architect

Ampere Computing LLC; Durham (NC); Hybrid; USD 225,000 - 339,000; Full time

Job summary

Ampere Computing LLC in Durham, North Carolina, seeks a Sr. Principal Machine Learning Software Engineer to lead the architecture and development of high‑performance AI inference‑serving software for Ampere accelerators.

You will work across model serving, frameworks, compilers, runtimes, and optimized kernels to enable efficient deployment of modern AI workloads, including generative AI and large language models, solving complex systems and performance challenges.

Qualifications

• Bachelor’s degree or higher in a technical field with extensive experience.
• Production inference runtimes and high-performance AI systems experience.
• Proficiency with C++ and systems-level performance optimization.
• Experience with profiling, benchmarking, and low-level debugging.
• Ability to lead cross-functional technical initiatives and communicate with stakeholders.

Responsibilities

• Lead architecture and development of high-performance AI inference-serving software.
• Drive end-to-end serving performance, including throughput and latency.
• Develop capabilities such as batching, scheduling, model loading, and memory planning.
• Collaborate with model-serving, framework, compiler, runtime, kernel, and hardware teams.
• Mentor engineers and lead technical decisions across initiatives.
• Partner with product, field, and customer teams to prioritize capabilities.

Education

Bachelor’s degree in Computer Science, Computer Engineering, Mathematics, Electrical Engineering, or a related technical field

Master’s degree with 8+ years

Ph.D. with 5+ years

Tools

vLLM

llama.cpp

TensorRT/TensorRT-LLM

ONNX Runtime

OpenVINO

Triton Inference Server

TVM

XLA

MLIR

CUDA

SYCL

OpenCL

MPI

Job description

Ampere Computing LLC in Durham, North Carolina, seeks a Sr. Principal Machine Learning Software Engineer to lead the architecture and development of high‑performance AI inference‑serving software for Ampere accelerators.

You will work across model serving, frameworks, compilers, runtimes, and optimized kernels to enable efficient deployment of modern AI workloads, including generative AI and large language models, solving complex systems and performance challenges.