Senior Inference Systems Engineer - Low-Latency & On-Device

Genesis AI; United States; On-site; USD 180,000 - 240,000; Full time

Job summary

Genesis AI is seeking a senior engineer to build and optimize real-time inference pipelines for on-device and cluster deployment. You will design GPU-accelerated systems, implement CUDA and Triton kernels, and ensure efficient memory and compute scheduling across heterogeneous stacks.

You’ll work on throughput- and latency-optimized workloads, with monitoring and debugging tools to diagnose regressions rapidly.

Qualifications

• 8+ years in distributed systems, ML infrastructure, or high-performance serving.
• Production-grade Python with strong C++/Rust/Go systems background.
• Deep CUDA, Triton, kernel optimization, memory management expertise.
• Experience scaling inference workloads for clusters and on-device deployments.
• System-level mindset for hardware–software tuning and efficiency.

Responsibilities

• Build low-latency inference pipelines for on-device deployment.
• Design and optimize distributed inference systems on GPU clusters.
• Implement low-level code (CUDA, Triton, custom kernels) into high-level frameworks.
• Optimize workloads for throughput and latency (batching, quantization, caching).
• Develop monitoring and debugging tools for reliability and fast regression diagnosis.

Skills

Distributed systems

ML infrastructure

Python

C++/Rust/Go

CUDA

Triton

Kernel optimization

Quantization

Memory scheduling

Tools

CUDA

Triton

Job description

Genesis AI is seeking a senior engineer to build and optimize real-time inference pipelines for on-device and cluster deployment. You will design GPU-accelerated systems, implement CUDA and Triton kernels, and ensure efficient memory and compute scheduling across heterogeneous stacks.

You’ll work on throughput- and latency-optimized workloads, with monitoring and debugging tools to diagnose regressions rapidly.