Senior ML Serving Engineer for LLMs & Inference

Alldus; San Jose (CA); On-site; USD 180,000 - 220,000; Full time

Job summary

A tech company in AI/ML is seeking a Senior Software Engineer specializing in ML Serving to build robust infrastructure for ML models. The ideal candidate has 5+ years of experience in software engineering, with a focus on ML serving. Proficiency in Python and knowledge of various serving frameworks are essential. This full-time role is located in San Jose, California and offers a competitive salary.

Qualifications

• 5+ years of software engineering experience with an ML serving focus.
• Proven experience deploying large language models in production.
• Strong programming skills in Python, Go, or C++.

Responsibilities

• Design and build ML serving infrastructure.
• Optimize inference pipelines for efficiency.
• Integrate models into varied customer environments.

Skills

ML serving

Python

Cloud platforms (AWS, GCP, Azure)

Distributed systems

Container orchestration (Kubernetes, Docker)

Tools

TensorFlow Serving

TorchServe

Triton Inference Server

BentoML

Ray Serve

Job description

A tech company in AI/ML is seeking a Senior Software Engineer specializing in ML Serving to build robust infrastructure for ML models. The ideal candidate has 5+ years of experience in software engineering, with a focus on ML serving. Proficiency in Python and knowledge of various serving frameworks are essential. This full-time role is located in San Jose, California and offers a competitive salary.