Machine Learning Engineer- Inference Optimization | Experienced Hire

Priorslab

Bala Cynwyd (PA)

On-site

USD 150,000 - 210,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Priorslab in Bala Cynwyd, PA is seeking a Machine Learning Engineer focused on fast inference to build, tune, and productionize high-performance model serving systems. You will collaborate with researchers and engineers to identify bottlenecks, optimize transformer-style architectures, and evaluate inference runtimes.

The role involves GPUs, memory layouts, and end-to-end latency constraints in production systems.

Qualifications

  • Experience deploying ML inference workloads in production or production-like environments.
  • Proficiency in Python and a systems language (C/C++, Rust, or Go).
  • Solid understanding of ML frameworks such as PyTorch, with profiling knowledge.
  • Ability to reason about latency, throughput, batching, memory use, GPU utilization, and reliability.

Responsibilities

  • Design, build, and optimize low-latency inference systems for production workloads.
  • Profile inference pipelines across execution, runtime config, batching, memory, IO, and networking.
  • Evaluate and tune inference runtime systems and deployment strategies.
  • Improve latency and GPU utilization for production workloads.
  • Develop benchmarking/profiling tools to compare variants and hardware targets.
  • Debug GPU memory, compute saturation, kernel behavior, CPU/GPU coordination, data movement.

Skills

Python
Java
C/C++
Rust
Go
PyTorch
Performance profiling
Latency optimization

Education

Bachelor's degree in CS/CE or related field

Tools

CUDA
Triton
Kubernetes
GPU clusters

Job description

Overview

We are looking for a Machine Learning Engineer focused on low-latency inference optimization to help build, tune, and productionize high-performance model serving systems. This role sits at the intersection of machine learning, systems engineering, and GPU performance. You will work on inference workloads where latency, throughput, reliability, and hardware efficiency all matter, and where a deep understanding of modern inference runtimes can meaningfully improve production outcomes.

You will work closely with quantitative researchers and engineers to understand model structure, identify inference bottlenecks, and turn research ideas into efficient production systems. The work may involve other types of models, but focuses on transformer-style architectures, and structured inference workloads. You will evaluate and tune frameworks and related serving or compilation systems, while also reasoning about GPU execution, memory layout, batching strategies, precision tradeoffs, and end-to-end latency.

What you'll do
  • Design, build, and optimize low-latency inference systems for production machine learning workloads.
  • Profile model inference pipelines across model execution, runtime configuration, batching, memory movement, serialization, networking, and I/O.
  • Evaluate, integrate, and tune inference runtime systems.
  • Improve latency, throughput, GPU utilization, for production inference workloads.
  • Build and support benchmarking and profiling tools to compare model variants, hardware targets, runtime configurations, and deployment strategies.
  • Debug performance issues involving GPU memory, compute saturation, kernel behavior, CPU/GPU coordination, data movement, and serving-layer overhead.
  • Help shape model and system design choices so that research models are efficient to deploy under real latency constraints.
  • Where necessary, collaborate with lower-level systems or GPU specialists on custom operators, kernel-level optimization, or hardware-specific performance work.

What we're looking for

  • Experience deploying, optimizing, or operating machine learning inference workloads in production or production-like environments.
  • Programming experience in Python, Java, C# etc. and at least one systems language such as C, C++, Rust, or Go
  • Solid understanding of modern ML frameworks such as PyTorch, including model execution, export, tracing, compilation, and performance profiling.
  • Ability to reason about latency, throughput, batching, memory use, GPU utilization, and reliability under real workloads.
  • Strong practical judgment around tradeoffs between model quality, latency, throughput, implementation complexity, and maintainability.
Preferred qualifications
  • Experience optimizing inference for latency-sensitive or high-throughput applications.
  • Experience with model optimization techniques such as quantization, pruning, distillation, operator fusion, graph lowering, custom operators, or model compilation.
  • Exposure to CUDA, Triton language, ROCm, PTX, CuTe, CUTLASS, FlashInfer, or similar low-level GPU programming tools.
  • Experience running inference workloads on Kubernetes or GPU clusters, including scheduling, autoscaling, observability, and resource management.
  • Background in mathematics, physics, computer science, engineering, statistics, quantitative finance, or another technical field.
  • Demonstrated ability to improve real-world inference performance beyond a baseline framework implementation.

If you're a recruiting agency and want to partner with us, please reach out to recruiting@sig.com. Any resume or referral submitted in the absence of a signed agreement will not be eligible for an agency fee.

  • Experience deploying, optimizing, or operating machine learning inference workloads in production or production-like environments.
  • Programming experience in Python, Java, C# etc. and at least one systems language such as C, C++, Rust, or Go
  • Solid understanding of modern ML frameworks such as PyTorch, including model execution, export, tracing, compilation, and performance profiling.
  • Ability to reason about latency, throughput, batching, memory use, GPU utilization, and reliability under real workloads.
  • Strong practical judgment around tradeoffs between model quality, latency, throughput, implementation complexity, and maintainability.
Preferred qualifications
  • Experience optimizing inference for latency-sensitive or high-throughput applications.
  • Experience with model optimization techniques such as quantization, pruning, distillation, operator fusion, graph lowering, custom operators, or model compilation.
  • Exposure to CUDA, Triton language, ROCm, PTX, CuTe, CUTLASS, FlashInfer, or similar low-level GPU programming tools.
  • Experience running inference workloads on Kubernetes or GPU clusters, including scheduling, autoscaling, observability, and resource management.
  • Background in mathematics, physics, computer science, engineering, statistics, quantitative finance, or another technical field.
  • Demonstrated ability to improve real-world inference performance beyond a baseline framework implementation.

If you're a recruiting agency and want to partner with us, please reach out to recruiting@sig.com. Any resume or referral submitted in the absence of a signed agreement will not be eligible for an agency fee.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Machine Learning Engineer- Inference Optimization | Experienced Hire
Machine Learning Engineer- Inference Optimization | Experienced Hire

Susquehanna International Group, LLP • Bala Cynwyd (PA)

On-site
USD 150,000 - 230,000
Member of Technical Staff - ML Systems & Inference
Member of Technical Staff - ML Systems & Inference

Gimlet Labs • San Francisco (CA)

On-site
USD 120,000 - 160,000
Senior Software Engineer - Model Performance
Senior Software Engineer - Model Performance

inference.net • San Francisco (CA)

Hybrid
USD 220,000 - 320,000
Equity in a high-growth startup
Comprehensive benefits
Machine Learning Engineer, LLM Inference Optimization
Machine Learning Engineer, LLM Inference Optimization

GMI Cloud, Inc • San Francisco (CA)

On-site
USD 180,000 - 280,000
Senior Software Engineer - Model Performance
Senior Software Engineer - Model Performance

Inference • San Francisco (CA)

Hybrid
USD 220,000 - 320,000
Competitive compensation
Equity in a high-growth startup
Comprehensive benefits
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • New York (NY)

On-site
USD 220,000 - 485,000
AI Researcher — Inference Optimization
AI Researcher — Inference Optimization

Featherlessai • United States

Remote
USD 140,000 - 210,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • San Francisco (CA)

On-site
USD 140,000 - 210,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • Palo Alto (CA)

On-site
USD 220,000 - 485,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

FDE Times • San Francisco (CA)

On-site
USD 140,000 - 240,000