Inference Engineer

Collision; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Collision AI is seeking an Inference Engineer to build and own the inference servers that run our frontier cybersecurity models, both on our infrastructure and in customer environments.

You will optimize batching, memory management, GPU utilization, and coordinate with research to productionize new models, while ensuring reliable deployments on client networks.

Qualifications

• Experience building model-serving infrastructure or other performance-sensitive distributed systems.
• Strong Python software engineering with C/C++ and/or Rust.
• Understanding of LLM inference, batching, KV-cache, and GPU bottlenecks.
• Experience profiling systems and shipping measurable performance improvements.
• Work with inference runtimes such as vLLM, SGLang, or TensorRT-LLM, incl. extensions/open-source contributions.
• Experience with CUDA or Triton and multi-GPU inference.

Responsibilities

• Own inference servers across our infrastructure and customer on-prem environments.
• Improve latency, throughput, and hardware utilization while preserving model quality.
• Profile workloads, identify bottlenecks, and ship reproducible benchmarks and tooling.
• Support deployment with reliable upgrade/rollback paths for client environments.
• Collaborate with research to bring new models into production.

Skills

Python

C/C++

Rust

LLM inference

Distributed systems

Profiling

Inference runtimes

CUDA

Tools

CUDA

Triton

vLLM

TensorRT

Job description

Collision AI is building frontier cybersecurity models to help defend our customers. As an Inference Engineer, you’ll build and own the inference servers that run our models, both on our infrastructure and in customer on-prem environments.

You’ll work across model runtimes and the serving layer: request scheduling, batching, caching, GPU memory management, and multi-GPU execution. You’ll profile real workloads, identify bottlenecks, and improve latency, throughput, and hardware utilization while preserving model quality. You’ll work closely with the research team to bring new models into production and support the multi-step workloads our security agents depend on.

Customer deployments are a core part of the role. You’ll make the serving stack straightforward to install, observe, update, and recover on customer-owned hardware, including environments with limited network access. You’ll build reproducible deployment tooling, sensible defaults, and reliable upgrade and rollback paths, so the same system can run well in our own fleet and inside a customer’s network.

Required qualifications

• Experience building model-serving infrastructure or other performance-sensitive distributed systems.
• Strong software engineering skills in Python, with experience in C++, Rust, or a comparable systems language.
• An understanding of LLM inference, including batching, KV-cache management, and GPU compute and memory bottlenecks.
• Experience profiling systems, designing reproducible benchmarks, and shipping measurable improvements to performance or reliability.
• Work on inference runtimes such as vLLM, SGLang, or TensorRT-LLM, including extensions or open-source contributions.
• Experience with CUDA or Triton, quantization, speculative decoding, or multi-GPU inference.