AI Inference Engineer — High-Performance GPU Systems

Perplexity; Palo Alto (CA); On-site; USD 220,000 - 485,000; Full time

Job summary

Perplexity is building and running the inference engine behind every Perplexity query, deploying dozens of model architectures at scale with tight latency and cost budgets. You will join a Rust/Python/CUDA stack to advance the serving runtime, GPU kernels, and inference performance.

We seek someone with deep GPU programming experience, production distributed systems know-how, and the ability to read papers, implement kernels, and debug incidents rapidly in a fast-moving environment.

Qualifications

• 3+ years of professional software engineering in ML inference or high-performance systems.
• Familiarity with at least one deep learning framework (PyTorch, JAX, TensorFlow).
• Understanding of GPU architectures (memory hierarchy, warp scheduling, tensor cores).
• Understanding of common LLM architectures and inference optimization techniques (quantization, speculative decoding, prefill-decode disaggregation).

Responsibilities

• Develop new models and support transformer-based retrieval, text-generation, and multimodal models in inference infrastructure.
• Migrate GPU kernels to CuTe DSL for portability and performance.
• Build Rust-native serving runtime to handle growing traffic and reduce Python pains.
• Profile and fix bottlenecks from network to batching and GPU kernel interleaving.
• Develop dashboards, alerts, and remediation for production incidents.

Skills

GPU programming

LLM architectures

Distributed systems

Rust Python CUDA

End-to-end ownership

Self-directed

Tools

Nsight Compute

CUDA-GDB

PTX/SASS

Kubernetes

Job description

Perplexity is building and running the inference engine behind every Perplexity query, deploying dozens of model architectures at scale with tight latency and cost budgets. You will join a Rust/Python/CUDA stack to advance the serving runtime, GPU kernels, and inference performance.

We seek someone with deep GPU programming experience, production distributed systems know-how, and the ability to read papers, implement kernels, and debug incidents rapidly in a fast-moving environment.