AI Inference Engineer (GPU/Rust/CUDA)

Perplexity

New York (NY)

On-site

USD 220,000 - 485,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Perplexity in New York is seeking an experienced engineer to join our team and own ML inference workloads, building a Rust/Python-based serving runtime and CUDA kernel work that scales to production. You will work with transformer-based models, enabling efficient weight loading, scheduling, and KV-cache management in our in-house infrastructure.

You will read research papers, implement kernels, and diagnose production incidents in a fast-moving environment, collaborating across languages and

Qualifications

  • 3+ years of professional software engineering experience with meaningful work on ML inference or high-performance systems.
  • Familiarity with at least one deep learning framework (PyTorch, JAX, TensorFlow).
  • Understanding of GPU architectures (memory hierarchy, warp scheduling, tensor cores).
  • Understanding of common LLM architectures and inference optimization techniques (e.g. quantization, speculative decoding, prefill-decode disaggregation).

Responsibilities

  • New models support for transformer-based retrieval, text-generation, and multimodal models in the inference infrastructure.
  • Port CUDA kernels to CuTe DSL and migrate GPU kernels for portability.
  • Develop a Rust-native serving runtime to handle growing traffic and reduce Python pains.
  • Profile and optimize performance across network, batching, and GPU kernel interleaving.
  • Build dashboards, alerts, and automated remediation to catch regressions.

Skills

ML inference
High-performance systems
Distributed systems
Rust
Python
CUDA

Tools

PyTorch
JAX
TensorFlow

Job description

Perplexity in New York is seeking an experienced engineer to join our team and own ML inference workloads, building a Rust/Python-based serving runtime and CUDA kernel work that scales to production. You will work with transformer-based models, enabling efficient weight loading, scheduling, and KV-cache management in our in-house infrastructure.

You will read research papers, implement kernels, and diagnose production incidents in a fast-moving environment, collaborating across languages and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Inference Engineer: GPU Performance & Rust Stack
AI Inference Engineer: GPU Performance & Rust Stack

Perplexity • San Francisco (CA)

On-site
USD 140,000 - 210,000
AI Inference Engineer — High-Performance GPU Systems
AI Inference Engineer — High-Performance GPU Systems

Perplexity • Palo Alto (CA)

On-site
USD 220,000 - 485,000
AI Inference Engineer for GPU-Driven LLMs
AI Inference Engineer for GPU-Driven LLMs

Fdetimes • San Francisco (CA)

On-site
USD 140,000 - 240,000
Rust-powered ML Systems Engineer - GPU Inference
Rust-powered ML Systems Engineer - GPU Inference

Pivots Hiring • San Mateo (CA)

On-site
USD 190,000 - 260,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

FDE Times • San Francisco (CA)

On-site
USD 140,000 - 240,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • San Francisco (CA)

On-site
USD 140,000 - 210,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • Palo Alto (CA)

On-site
USD 220,000 - 485,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • New York (NY)

On-site
USD 220,000 - 485,000
AI Inference Kernel Optimization Engineer
AI Inference Kernel Optimization Engineer

Vidman AI • United States

Remote
USD 140,000 - 210,000
Senior Inference Engine Engineer - GPU Kernel & ML Runtime
Senior Inference Engine Engineer - GPU Kernel & ML Runtime

FriendliAI Inc. • San Francisco (CA)

On-site
USD 190,000 - 270,000
Flexible hours
Lunch/dinner provided
Collaborative environment
+4