Inference Engineer (Fully Remote)

Undisclosed; United States; Remote; USD 200,000 - 300,000; Full time

Job summary

Undisclosed, an early-stage AI infrastructure startup, seeks a systems/infrastructure engineer to optimize large-language-model inference. You will focus on latency and throughput, including batching and quantization, and write and tune kernels using CUDA and Triton while deploying on heterogeneous GPU clusters.

Ideal candidates have 1–6 years in systems engineering, comfort with CUDA/Triton or GPU programming, and experience with cluster operations.

Qualifications

• 1-6 years in systems/infrastructure engineering.
• Familiarity with CUDA, Triton, or GPU programming and cluster operations.

Responsibilities

• Optimize inference systems for latency and throughput, including batching and quantization.
• Write and tune kernels using CUDA and Triton; deploy heterogeneous clusters.

Skills

GPU programming

Tools

CUDA

Triton

Job description

• Optimize inference systems for latency and throughput, including batching and quantization.
• Write and tune kernels using CUDA and Triton; deploy heterogeneous clusters.

What we're looking for:

• 1-6 years in systems/infrastructure engineering, strong new grads considered.
• Familiarity with CUDA, Triton, or GPU programming and cluster operations.

Compensation & location:

• $200k-$300k annually, visa sponsorship available.
• Remote within the US; SF as primary location.

They are an early-stage AI infrastructure startup making large language model inference faster and more cost-efficient. The small team focuses on low-level performance engineering and AI-assisted workflows, with a strong emphasis on technical fundamentals.