Low-Latency AI Inference Engineer

OP Recruiting; Chicago (IL); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health benefits

Job summary

OP Recruiting in New York, NY is seeking an AI Research Engineer to advance deep learning inference with a focus on ultra-low latency and scalable deployment. You will bridge ML research and hardware acceleration, building low-latency inference systems that handle continuous global data streams to inform core business decisions.

The role emphasizes optimizing kernels, collaborating with researchers on neural architectures, and accelerating inference across diverse hardware platforms for

Qualifications

• Hands-on experience engineering production-grade deep learning systems in complex domains.
• Strong lower-level engineering foundation with custom compute kernels.
• Proficiency with framework compilation internals and low-level runtimes such as PyTorch/JAX/XLA/CUDA Graphs.

Responsibilities

• Drive performance optimizations for large-scale model execution, including custom kernel creation and data streaming pipelines.
• Collaborate with researchers to co-design neural architectures for ultra-real-time execution.
• Develop low-level primitives and custom ops to maximize compute throughput on modern hardware.
• Evaluate and deploy hardware acceleration technologies, from accelerators to specialized silicon.
• Identify and address bottlenecks in ultra-low-latency deep learning inference.

Skills

Deep learning systems

CUDA kernels

PyTorch internals

Hardware acceleration

Tools

CUDA

JAX

XLA

CuTe DSLs

Job description

OP Recruiting in New York, NY is seeking an AI Research Engineer to advance deep learning inference with a focus on ultra-low latency and scalable deployment. You will bridge ML research and hardware acceleration, building low-latency inference systems that handle continuous global data streams to inform core business decisions.

The role emphasizes optimizing kernels, collaborating with researchers on neural architectures, and accelerating inference across diverse hardware platforms for