Senior AI Inference Engineer — Scale & Production

Spectro Cloud; San Jose (CA); Presencial; USD 140 000 - 210 000; Tempo integral

Resumo da oferta

Spectro Cloud in San Jose, CA, seeks a senior engineer to own AI inference at scale. You will build production systems to serve models across data centers and edge sites, ensuring speed, reliability, and efficiency.

Bring new models into service, improve performance under real demand, and set testing and release standards. Strong collaboration across teams is required, with Kubernetes experience highly desired.

Qualificações

• Hands-on experience running AI or LLM inference in production.
• Working understanding of inference engines, batching, and memory management.
• Experience improving performance, reliability, and efficiency of production systems.
• Experience with distributed systems and multi-GPU workloads.
• Experience designing benchmarks and release criteria.
• Bachelor's degree in Computer Science or related field.
• 4+ years of software development experience, or 2+ years with a Master's degree.
• Strong Python and hands-on PyTorch.
• Golang for backend services and APIs.
• Working knowledge of Kubernetes.
• Experience with modern GPU platforms, drivers, and monitoring tools.
• Understanding of high-speed networking and interconnects.
• Experience with cloud-native applications on AWS, Azure, or GCP.
• Comfort in Linux environments and with common command line tools.
• Experience with automated testing.
• Familiarity with AI-assisted development tools.
• Clear written communication in English.
• Experience with model optimization, quantization, and acceleration.
• GPU scheduling in Kubernetes
• Experience with monitoring and observability tools
• Kubernetes certification such as CKA or CKAD

Responsabilidades

• Build and operate AI inference services in production, with engines such as vLLM, SGLang, or TensorRT-LLM.
• Bring new models into service, from first test to full rollout.
• Improve throughput, response time, and memory use across the stack.
• Find the root cause of performance and reliability issues, then fix them.
• Design benchmarks and release criteria for every model we ship.
• Set reliability targets and meet them in production.
• Scale inference across many GPUs and nodes.
• Run inference on modern GPU platforms, including the latest accelerators.
• Automate deployment and operations on Kubernetes.
• Write runbooks, tuning guides, and clear technical documentation

Conhecimentos

Python

PyTorch

Kubernetes

Golang

Distributed systems

Multi-GPU

Model optimization

Formação académica

Bachelor's degree in Computer Science or related field

Ferramentas

CUDA/TensorRT

Linux

Descrição da oferta de emprego

Spectro Cloud in San Jose, CA, seeks a senior engineer to own AI inference at scale. You will build production systems to serve models across data centers and edge sites, ensuring speed, reliability, and efficiency.

Bring new models into service, improve performance under real demand, and set testing and release standards. Strong collaboration across teams is required, with Kubernetes experience highly desired.