Senior LLM Inference Architect — Edge, Data Center, Remote

Cerence AI; United States; Hybrid; USD 185,000 - 280,000; Full time

Benefits offered by this job

Annual bonus opportunity

Insurance coverage (medical, dental, vision, life, and disability)

Paid time off

Job summary

Cerence AI is seeking a Senior Principal Software Engineer to lead optimization of Large Language Model inference pipelines across data centers, edge, and embedded platforms. The role requires proven experience in optimizing ML inference performance, with a focus on CUDA and latency improvements.

The ideal candidate will have hands-on experience in deploying models and possess a deep understanding of GPU architecture. Benefits include a competitive salary ranging from $185,000 to $280,000, annual bonuses, and insurance coverage.

Qualifications

• Proven experience optimizing ML inference performance in production.
• Hands-on experience with CUDA and low-level performance tuning.
• Ability to solve deployment challenges on edge or embedded targets.

Responsibilities

• Optimize and deploy high-performance LLM inference pipelines.
• Own inference runtimes across various platforms.
• Drive latency and throughput improvements.

Skills

Optimizing ML inference performance

GPU architecture understanding

CUDA experience

Latency optimization

Quantisation techniques

Tools

vLLM

TensorRT-LLM

llama.cpp

QAIRT

Job description

Cerence AI is seeking a Senior Principal Software Engineer to lead optimization of Large Language Model inference pipelines across data centers, edge, and embedded platforms. The role requires proven experience in optimizing ML inference performance, with a focus on CUDA and latency improvements.

The ideal candidate will have hands-on experience in deploying models and possess a deep understanding of GPU architecture. Benefits include a competitive salary ranging from $185,000 to $280,000, annual bonuses, and insurance coverage.