AI Inference Engineer: GPU Performance & Rust Stack

Perplexity; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

Perplexity is building and running the inference engine behind every Perplexity query, deploying dozens of model architectures at scale with tight latency and cost budgets. We are seeking another engineer to join our team to advance transformer-based retrieval, text generation, and multimodal models in production.

You will work on a Rust-native serving runtime, CUDA/CuTe kernel migration, and ongoing performance optimizations to support growing traffic while maintaining reliability.

Qualifications

• 3+ years of professional software engineering experience with meaningful work on ML inference or high-performance systems.

Responsibilities

• Develop new model support for transformer-based retrieval, text-generation, and multimodal models in inference infrastructure.
• Migrate GPU kernels to CuTe DSL and ensure portability to future hardware.
• Build Rust-native serving runtime to handle growing traffic and reduce Python-related pains.
• Profile, optimize, and interleave GPU kernels for performance across end-to-end pipelines.
• Improve reliability with dashboards, alerts, and automated remediation for production incidents.

Skills

GPU programming

Performance tuning

Distributed systems

Cross-language development

Tools

CUDA

CuTe DSL

Triton

CUTLASS

Job description

Perplexity is building and running the inference engine behind every Perplexity query, deploying dozens of model architectures at scale with tight latency and cost budgets. We are seeking another engineer to join our team to advance transformer-based retrieval, text generation, and multimodal models in production.

You will work on a Rust-native serving runtime, CUDA/CuTe kernel migration, and ongoing performance optimizations to support growing traffic while maintaining reliability.