Senior LLM Storage Systems Engineer & Researcher

ByteDance; Seattle (WA); On-site; USD 202,160 - 368,220; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) matching

Paid parental leave

Life insurance

Job summary

ByteDance in Seattle is seeking a Senior Research Engineer/Scientist for Storage for LLM to design and maintain a high-performance KV cache layer for LLM inference across GPUs and nodes. This role focuses on reducing latency, increasing throughput, and lowering cost for transformer‑based model serving.

You will collaborate with inference and serving teams, optimize eviction policies, memory management, and explore open‑source KV stores or GPU‑aware caching techniques.

Qualifications

• PhD or equivalent research background required.
• Deep understanding of transformer internals and KV caching effects on decoding.
• Experience with distributed systems and low-latency serving.

Responsibilities

• Design and implement a distributed KV cache system for LLM inference across GPUs/nodes.
• Optimize low-latency access, eviction, and memory management for caching.
• Collaborate with inference/serving teams to integrate with pipelines and model parallelism.
• Develop cache consistency protocols for multi-tenant environments.
• Explore GPU-aware caching layers and open-source KV stores; iterate on caching algorithms.

Skills

Transformer internals

Distributed systems

Low-latency serving

C++/Rust/Go/CUDA

Education

PhD in Computer Science or related field

Tools

CUDA

TensorRT

Triton Inference Server

NCCL

Job description

ByteDance in Seattle is seeking a Senior Research Engineer/Scientist for Storage for LLM to design and maintain a high-performance KV cache layer for LLM inference across GPUs and nodes. This role focuses on reducing latency, increasing throughput, and lowering cost for transformer‑based model serving.

You will collaborate with inference and serving teams, optimize eviction policies, memory management, and explore open‑source KV stores or GPU‑aware caching techniques.