ML Inference Systems Engineer — Production-Scale

Gimlet Labs, Inc.; San Francisco (CA); Hybride; USD 150 000 - 390 000; Plein temps

Avantages offerts par ce poste

Equity

Résumé du poste

Gimlet Labs, Inc. is hiring a Member of Technical Staff focused on ML systems in San Francisco, on-site. You will build inference systems that run models end-to-end in production, focusing on batching, scheduling, memory placement, and cross-hardware optimization.

You will collaborate with teams across compilers, kernels, networking, and distributed systems to improve latency, throughput and efficiency of production workloads while enabling new models and accelerators to run at scale.

Qualifications

• Bachelor's degree or equivalent experience in a relevant field.
• Experience building or operating ML inference or model serving systems.
• Ability to reason about latency, throughput and memory under load.
• Proficient in Python and C++ software development.

Responsabilités

• Build and operate end-to-end inference systems in production.
• Optimize batching, scheduling, concurrency, and resource utilization.
• Advance KV cache management and memory placement strategies.
• Collaborate with compiler, kernel, networking, and distributed systems teams.

Connaissances

Strong software engineering

ML inference systems

Performance & memory reasoning

Python & C++ development

Formation

Bachelor's degree in a relevant field

Outils

TensorRT-LLM

vLLM

Custom model serving

Description du poste

Gimlet Labs, Inc. is hiring a Member of Technical Staff focused on ML systems in San Francisco, on-site. You will build inference systems that run models end-to-end in production, focusing on batching, scheduling, memory placement, and cross-hardware optimization.

You will collaborate with teams across compilers, kernels, networking, and distributed systems to improve latency, throughput and efficiency of production workloads while enabling new models and accelerators to run at scale.