Founding LLM Inference Engineer (Onsite)

Acceler8 Talent; California; On-site; USD 210,000 - 320,000; Full time

Benefits offered by this job

Vision insurance

Medical insurance

Dental insurance

401(k)

Job summary

Acceler8 Talent is seeking a Founding Engineer to design and build scalable LLM infrastructure and inference systems in Mountain View, CA. You will own architecture and implementation, collaborating directly with the founding team to shape the core technology from day one.

This hands-on role focuses on distributed serving, performance optimization, and system-level engineering across compute, memory, and network boundaries.

Qualifications

• Hands-on experience building or optimizing LLM inference infrastructure.
• Strong understanding of distributed systems and multi-machine execution.
• Ability to identify performance bottlenecks and implement measurable improvements.

Responsibilities

• Design, build, and optimize scalable infrastructure for hosting and serving large language models across multiple machines.
• Own architecture and implementation of distributed LLM inference systems with technical autonomy.
• Develop infrastructure supporting agentic AI workloads and evolving inference requirements.
• Analyze system performance across compute, memory, and distributed execution.
• Investigate bottlenecks affecting inference latency, throughput, hardware utilization, and efficiency.
• Optimize inference serving strategies including batching, scheduling, caching, and parallel execution.
• Develop benchmarking methodologies for representative LLM workloads.
• Collaborate with performance and infra engineers to translate workload optimizations into system-wide improvements.
• Work with founding team to influence architecture and platform development.

Skills

LLM inference engineering

Distributed systems

Performance optimization

Multi-machine execution

Debugging complex systems

Architectural design

Model serving frameworks

Education

BS in Computer Science or equivalent

Tools

vLLM

TensorRT-LLM

DeepSpeed

Megatron

Job description

Acceler8 Talent is seeking a Founding Engineer to design and build scalable LLM infrastructure and inference systems in Mountain View, CA. You will own architecture and implementation, collaborating directly with the founding team to shape the core technology from day one.

This hands-on role focuses on distributed serving, performance optimization, and system-level engineering across compute, memory, and network boundaries.