Inference Infra Architect for Multimodal ML Systems

Elorian AI; San Francisco (CA); On-site; USD 200,000 - 400,000; Full time

Benefits offered by this job

Health, dental, and vision benefits

Unlimited PTO

Parental leave

Job summary

Elorian AI in Palo Alto, CA, seeks an infrastructure engineer to design, optimize, and scale systems that serve large multimodal models. You will drive faster, cheaper, and more reliable inference, enabling researchers to push model capabilities while reducing bottlenecks.

You will own infra for deployment and evaluation at scale, collaborating with researchers to enable high-performance inference across architectures, focusing on low latency, high throughput, and robust observability.

Qualifications

• 3+ years building low-latency inference serving systems for large models.
• Knowledge of quantization, batching, speculative decoding, KV cache management.
• Experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang.
• Experience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline).
• Experience with autoscaling and load balancing for production ML services.
• Track record of GPU cost optimization at scale.

Responsibilities

• Build low-latency, high-throughput inference serving systems for large multimodal models.
• Design latency, throughput, and efficiency improvements (quantization, batching, speculative decoding, KV cache).
• Optimize codebase and GPU fleet to maximize FLOPs, bandwidth, and memory usage.
• Implement multi-GPU and multi-node model parallelism for serving (tensor or pipeline).
• Build autoscaling and load balancing for production ML services.
• Establish reliability, observability, and reproducibility across the inference stack.
• Collaborate with researchers to enable high-performance inference for novel architectures.

Skills

Low-latency serving

Inference optimization

Multi-GPU parallelism

Autoscaling

GPU cost optimization

Serving frameworks

Tools

vLLM

TensorRT-LLM

Triton

SGLang

Job description

Elorian AI in Palo Alto, CA, seeks an infrastructure engineer to design, optimize, and scale systems that serve large multimodal models. You will drive faster, cheaper, and more reliable inference, enabling researchers to push model capabilities while reducing bottlenecks.

You will own infra for deployment and evaluation at scale, collaborating with researchers to enable high-performance inference across architectures, focusing on low latency, high throughput, and robust observability.