Founding LLM Inference Engineer

Acceler8 Talent; Palo Alto (CA); On-site; USD 180,000 - 320,000; Full time

Job summary

Acceler8 Talent is partnering with an AI infrastructure startup in Mountain View, CA to hire a Founding Engineer focused on LLM infrastructure and inference for next-generation AI serving systems.

You will own architecture, build scalable distributed inference, optimize performance, and collaborate with the founding team to shape core technology. Senior and exceptional junior engineers are encouraged to apply.

Qualifications

• Hands-on experience building, implementing, or optimizing LLM inference or model-serving infrastructure.
• Strong understanding of distributed systems and multi-machine execution.
• Ability to identify performance bottlenecks and implement measurable improvements in latency, throughput, or hardware utilization.
• For founding-level candidates, demonstrated ability to independently design and build substantial systems.

Responsibilities

• Design, build, and optimize scalable infrastructure for hosting and serving large language models across multiple machines.
• Own architecture and implementation of distributed LLM inference systems, with significant technical autonomy.
• Develop infrastructure supporting agentic AI workloads and evolving inference requirements.
• Analyze system performance across compute, memory, and distributed execution.
• Investigate bottlenecks affecting inference latency, throughput, hardware utilization, and overall efficiency.
• Optimize inference serving strategies, including batching, scheduling, caching, and parallel execution.
• Develop performance measurement and benchmarking methodologies for representative LLM workloads.
• Evaluate and improve KV cache management, memory utilization, and inference execution efficiency.
• Collaborate with performance and infrastructure engineers to translate workload-level optimizations into system-wide improvements.
• Work directly with the founding team to influence architecture, engineering decisions, and development of the serving platform.

Skills

LLM inference

Distributed systems

Performance optimization

Software engineering fundamentals

Architecture decisions

Troubleshooting complex systems

Independent design

Tools

vLLM

SGLang

TensorRT-LLM

llm-d

DeepSpeed

Megatron

FSDP

Job description

Acceler8 Talent is partnering with an AI infrastructure startup in Mountain View, CA to hire a Founding Engineer focused on LLM infrastructure and inference for next-generation AI serving systems.

You will own architecture, build scalable distributed inference, optimize performance, and collaborate with the founding team to shape core technology. Senior and exceptional junior engineers are encouraged to apply.