Inference Systems Engineer: Low-Latency, On-Device

Genesis; San Carlos (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Genesis seeks an experienced ML infrastructure engineer to build low-latency on-device inference pipelines and scale GPU-accelerated workloads in production. The role emphasizes real-time performance, robust monitoring, and a system-level mindset for efficient hardware-software orchestration.

You will optimize CUDA and Triton kernels, tune memory and scheduling, and collaborate across teams to maintain throughput without compromising latency.

Qualifications

• 8+ years in distributed systems, ML infrastructure, or high-performance serving.
• Production-grade Python with a strong background in C++, Rust or Go.
• Low-level performance mastery: CUDA, Triton, kernel optimization, quantization, memory scheduling.

Responsibilities

• Build low-latency inference pipelines for on-device deployment and real-time control loops.
• Design and optimize distributed inference systems on GPU clusters for high throughput.
• Implement efficient low-level code (CUDA, Triton, custom kernels) and integrate into high-level frameworks.
• Optimize workloads for throughput and latency including caching and memory management.
• Develop monitoring and debugging tools to ensure reliability and rapid diagnosis of regressions.

Skills

Distributed systems

ML infrastructure

High-performance serving

Python

C++

Rust

Go

CUDA

Triton

Kernel optimization

Memory management

System tuning

Tools

N/A

Job description

Genesis seeks an experienced ML infrastructure engineer to build low-latency on-device inference pipelines and scale GPU-accelerated workloads in production. The role emphasizes real-time performance, robust monitoring, and a system-level mindset for efficient hardware-software orchestration.

You will optimize CUDA and Triton kernels, tune memory and scheduling, and collaborate across teams to maintain throughput without compromising latency.