LLM Storage Systems Research Engineer

ByteDance; San Jose (CA); On-site; USD 156,000 - 387,600; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) savings plan with company match

Paid parental leave

Short-term and long-term disability coverage

Life insurance

Job summary

ByteDance is seeking a Research Engineer / Scientist for their San Jose location. The role involves designing and optimizing distributed KV caching systems for transformer-based LLMs. Candidates must have a PhD in a relevant field and expertise in distributed systems and KV caching.

The position offers a competitive salary and benefits, including medical and dental insurance, 401(k), and personal time off. Join a team dedicated to innovative machine learning solutions.

Qualifications

• PhD in Computer Science, Applied Mathematics, Electrical Engineering, or a related field.
• Strong understanding of transformer-based model internals and KV caching.
• Experience with distributed systems and memory management.

Responsibilities

• Design and implement a distributed KV cache system for transformer-based LLMs.
• Collaborate with teams to integrate the cache with token streaming.
• Monitor system performance and iterate on caching algorithms.

Skills

Transformer-based model internals

Distributed systems

Memory management

Low-latency serving

C++

CUDA

Education

PhD in Computer Science or related field

Tools

CUDA-aware networking

NVIDIA GPUs

Job description

ByteDance is seeking a Research Engineer / Scientist for their San Jose location. The role involves designing and optimizing distributed KV caching systems for transformer-based LLMs. Candidates must have a PhD in a relevant field and expertise in distributed systems and KV caching.

The position offers a competitive salary and benefits, including medical and dental insurance, 401(k), and personal time off. Join a team dedicated to innovative machine learning solutions.