Low-Latency ML Inference Engineer (GPU/FPGA)

Tower Research Capital; New York (NY); Hybrid; USD 200,000 - 300,000; Full time

Benefits offered by this job

Generous paid time off policies

Hybrid working opportunities

In-office wellness experiences

Job summary

Tower Research Capital is hiring for a Core Engineering role focused on building and optimizing ML inference pipelines that push microsecond latency limits. Lead evaluation of CPUs, GPUs, and FPGAs, optimize memory hierarchies, and collaborate with cross-functional teams to deploy low-latency trading workloads.

Required are 2+ years in latency-sensitive DL inference, deep knowledge of PyTorch/JAX, Python/C++, and GPU kernel development.

Qualifications

• 2+ years optimizing deep learning inference in latency-sensitive or high-throughput environments.
• Deep expertise in lower-level ML framework development with PyTorch/JAX plus strong Python/C++ skills.
• Experience in custom GPU kernel development and advanced optimization tooling and profiling.
• Strong knowledge of GPU microarchitecture and memory hierarchy optimization.

Responsibilities

• Lead evaluation of diverse inference platforms across CPUs, GPUs, and FPGAs.
• Analyze and optimize execution across memory hierarchies to maximize performance.
• Collaborate with Infrastructure to design latency-critical inference strategies.
• Develop highly optimized GPU kernels and integrate performance libraries.
• Implement model reduction techniques for low-latency, real-time workloads.
• Collaborate with ML researchers and engineers to deploy target solutions.

Skills

Python

C++

PyTorch/JAX

GPU optimization

Kernel development

Mixed-precision computing

Data-driven evaluation

Tools

Triton

TensorRT

ONNX

IREE

HLS4ML

cuBLAS

CUTLASS

Job description

Tower Research Capital is hiring for a Core Engineering role focused on building and optimizing ML inference pipelines that push microsecond latency limits. Lead evaluation of CPUs, GPUs, and FPGAs, optimize memory hierarchies, and collaborate with cross-functional teams to deploy low-latency trading workloads.

Required are 2+ years in latency-sensitive DL inference, deep knowledge of PyTorch/JAX, Python/C++, and GPU kernel development.