AI Inference Solutions Developer

Bellatrix Systems LLC; San Francisco (CA); Remote; USD 180,000 - 260,000; Full time

Job summary

Bellatrix Systems LLC seeks an AI Inference Solutions Developer to design and optimize a scalable LLM inference platform across GPUs and AI processors. You will focus on disaggregated inference, KV cache management, and cache-aware routing in a distributed, Kubernetes-based environment.

The role demands deep expertise in distributed systems, RDMA networking, accelerator programming, and performance engineering to deliver high-throughput, low-latency inference across multi-node clusters.

Qualifications

• Deep hands-on experience with vLLM or SGLang internals and scheduling, KV cache management.
• Understanding of prefill/decode separation and multi-tier caching architectures.
• Production experience with NCCL or equivalent technologies and multi-node inference deployments.
• Strong Python development with production-quality Go/C++/Rust.

Responsibilities

• Design, build, and optimize disaggregated LLM inference serving across heterogeneous accelerators.
• Develop multi-tier KV cache management, including connector integrations and storage policies.
• Create cache-aware routing with prefix-aware scoring and SLO-based admission.
• Architect accelerator/engine adapters, build conformance tests, and deploy on Kubernetes in multi-node setups.
• Benchmark, profile, and validate throughput, cache hit rates, tenant isolation, and failover.
• Troubleshoot accelerator and network issues; contribute improvements to open-source projects.

Skills

vLLM/SGLang

Disaggregated Inference

Multi-Node GPU

Python

Go/C++/Rust

RDMA Networking

Kubernetes

Accelerator Programming

Distributed Systems

Performance Profiling

Cache-Aware Routing

Agentic Development

CI/CD/Testing

Technical Communication

Tools

CUDA/ROCm

Job description

Position: AI Inference Solutions Developer

Location: San Francisco Bay Area or Remote

Duration: Long-term

Job Summary:

We are seeking a highly skilled AI Inference Solutions Developer to design, build, and optimize a scalable, hardware-agnostic LLM inference platform across heterogeneous accelerators, including GPUs and other AI processors. The role focuses on disaggregated inference, KV cache management, cache-aware routing, and high-performance multi-node inference using open-source engines such as vLLM or SGLang. The ideal candidate will have deep expertise in distributed systems, RDMA networking, Kubernetes, accelerator programming, and performance optimization. This is a hands-on engineering role involving architecture, development, production deployment, and performance validation.

Must-Have Skills:

• LLM Inference Engines: Deep hands-on experience with vLLM or SGLang internals, including scheduling, KV cache management, prefix caching, and connector APIs.
• Disaggregated Inference: Understanding of prefill/decode separation, KV cache transfer, and multi-tier caching architectures.
• Multi-Node Accelerator Workloads: Production experience with NCCL or equivalent technologies, tensor parallelism, expert parallelism, and kernel/network-level debugging.
• Programming: Strong Python development skills, along with production-quality experience in Go, C++, or Rust.
• RDMA Networking: Hands-on experience with RoCEv2 or InfiniBand networking.
• Kubernetes: Strong systems-level knowledge of Kubernetes and multi-node inference deployments.
• Accelerator Programming: Experience with CUDA, ROCm, Triton, or vendor-specific accelerator SDKs, including understanding differences between accelerator architectures.
• Distributed Systems: Strong knowledge of stateful distributed systems, caching, scheduling, and scalable service architecture.
• Performance Engineering: Experience with profiling, benchmarking, bandwidth optimization, latency analysis, and performance troubleshooting.
• Cache-Aware Routing: Knowledge of prefix-aware routing, cache-aware request scheduling, and service-level objective (SLO) admission.
• Agentic Development: Regular hands-on use of AI coding tools such as Claude Code, with strong code review and debugging capabilities.
• Software Engineering Practices: Strong testing, CI/CD, observability, and clean, versioned API development.
• Technical Communication: Ability to analyze performance measurements, make architecture decisions, and document technical findings clearly.

Responsibilities:

• Design, develop, and optimize disaggregated LLM inference serving using vLLM or similar open-source engines, including prefill/decode pools, expert parallelism, and KV cache transfers.
• Build multi-tier KV cache management solutions, including connector integrations, cache offloading, storage policies, and shared storage admission.
• Develop cache-aware routing capabilities with prefix-aware scoring, hash consistency, and SLO-based request admission using open-source routing frameworks.
• Design accelerator and engine adapter interfaces, build conformance tests, and deploy scalable multi-node inference workloads on Kubernetes.
• Optimize inference performance through benchmarking, profiling, and validation of throughput, cache hit rates, tenant isolation, and failover.
• Troubleshoot accelerator and network-level issues, contribute improvements to open-source projects, and support production deployment and infrastructure lifecycle operations.