AI Inference Engineer for GPU-Driven LLMs

Fdetimes

San Francisco (CA)

On-site

USD 140,000 - 240,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Fdetimes is seeking an experienced software engineer to join the inference team building the backbone of Perplexity query processing. You will work across Rust, Python, CUDA, and CuTe DSL to deploy scalable, low-latency models and explore new architectures.

You will contribute to model support, kernel migrations, and a robust serving runtime while improving performance, reliability, and observability in production workloads.

Qualifications

  • 3+ years of professional software engineering experience with ML inference or high-performance systems.
  • Familiarity with at least one deep learning framework (PyTorch, JAX, TensorFlow).
  • Understanding of GPU architectures (memory hierarchy, warp scheduling, tensor cores).
  • Understanding of common LLM architectures and inference optimization techniques (e.g. quantization, speculative decoding, prefill-decode disaggregation).

Responsibilities

  • Work on new model support for transformer-based retrieval, text-generation, and multimodal models in inference infrastructure.
  • Migrate GPU kernels to CuTe DSL for portability and performance.
  • Develop Rust-based serving runtime to reduce Python-related bottlenecks and support increasing traffic.
  • Profile and fix bottlenecks across network ingress, batching, and GPU kernel interleaving.
  • Build dashboards, alerts, and automated remediation to catch regressions and triage production incidents.

Skills

GPU programming
LLM architectures
Distributed systems
Rust
Python

Tools

CUDA
CuTe DSL
Kubernetes

Job description

Fdetimes is seeking an experienced software engineer to join the inference team building the backbone of Perplexity query processing. You will work across Rust, Python, CUDA, and CuTe DSL to deploy scalable, low-latency models and explore new architectures.

You will contribute to model support, kernel migrations, and a robust serving runtime while improving performance, reliability, and observability in production workloads.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Inference Engineer — High-Performance GPU Systems
AI Inference Engineer — High-Performance GPU Systems

Perplexity • Palo Alto (CA)

On-site
USD 220,000 - 485,000
AI Inference Engineer: GPU Performance & Rust Stack
AI Inference Engineer: GPU Performance & Rust Stack

Perplexity • San Francisco (CA)

On-site
USD 140,000 - 210,000
AI Inference Engineer (GPU/Rust/CUDA)
AI Inference Engineer (GPU/Rust/CUDA)

Perplexity • New York (NY)

On-site
USD 220,000 - 485,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • San Francisco (CA)

On-site
USD 140,000 - 210,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

FDE Times • San Francisco (CA)

On-site
USD 140,000 - 240,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • New York (NY)

On-site
USD 220,000 - 485,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Perplexity • Palo Alto (CA)

On-site
USD 220,000 - 485,000
AI Inference Kernel Optimization Engineer
AI Inference Kernel Optimization Engineer

Vidman AI • United States

Remote
USD 140,000 - 210,000
Senior Inference Engine Engineer - GPU Kernel & ML Runtime
Senior Inference Engine Engineer - GPU Kernel & ML Runtime

FriendliAI Inc. • San Francisco (CA)

On-site
USD 190,000 - 270,000
Flexible hours
Lunch/dinner provided
Collaborative environment
+4
Inference Engineer
Inference Engineer

Collision • San Francisco (CA)

On-site
USD 150,000 - 230,000