Senior ML Inference Systems Engineer

castaigroupinc; Stati Uniti; Remoto; USD 180.000 - 280.000; Tempo pieno

Vantaggi offerti da questo lavoro

Remote-first work

equity options

a learning budget

equipment support

extra days off

time allocated for personal developmen

Descrizione del lavoro

castaigroupinc is seeking a senior engineer to improve the performance and economics of large-scale language-model inference across the serving stack.

You will own from model loading to caching and distributed execution, balancing latency, throughput, and cost while guiding optimization priorities.

This role blends hands-on systems work with setting strategy, working remotely in a remote-first environment.

Competenze

• Minimum five years building production machine-learning systems.
• Strong production Python skills and experience with vLLM, SGLang, or TensorRT-LLM.
• Understanding of GPU performance and quantization effects on quality, not only compression.
• Experience with multi-GPU or multi-node systems, parallelism, and collective communication.
• Ability to instrument systems and distinguish repeatable improvements from benchmark artifacts.

Mansioni

• Tune serving engines, batching, prefill, decoding, and scheduling to raise throughput across GPU types.
• Profile time-to-first-token and token-generation latency, identify bottlenecks, and optimize compute/memory/storage/network paths.
• Improve KV-cache usage through prefix caching, paging, tiering, eviction policies, and quantization.
• Shorten model startup by improving weight-loading paths, compilation caches, and shared-filesystem reads.
• Evaluate quantization on real workloads, balancing model quality with memory and performance gains.
• Design and improve multi-GPU and multi-node inference, including placement and separation of prefill and decode; establish reproducible benchmarks and explain technical recommendations.

Conoscenze

Production ML systems

Production Python

GPU performance profiling

Multi-GPU / multi-node

System instrumentation

Strumenti

vLLM

SGLang

TensorRT-LLM

Direct I/O

Shared filesystem

Descrizione del lavoro

castaigroupinc is seeking a senior engineer to improve the performance and economics of large-scale language-model inference across the serving stack.

You will own from model loading to caching and distributed execution, balancing latency, throughput, and cost while guiding optimization priorities.

This role blends hands-on systems work with setting strategy, working remotely in a remote-first environment.