LLM/VLM Inference Optimization Research Engineer

Bytedance; San Jose (CA); On-site; USD 244,800 - 450,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with company match

Paid parental leave

Holiday and paid time off

Job summary

ByteDance Seed Infra in San Jose is seeking a Research Engineer to design, develop, and optimize high-performance inference systems for large-scale LLMs and VLMs. You will work on inference engines, serving frameworks, and end-to-end deployment pipelines, aiming to reduce latency and increase throughput.

Ideal candidates will have strong C/C++ and Python skills, hands-on ML framework experience (PyTorch/TensorFlow), and background in GPU-accelerated optimization.

Qualifications

• Bachelor's degree or above in Computer Science, Electrical Engineering, Software Engineering, or a related field.
• Strong proficiency in C/C++ and Python; solid foundations in algorithms, data structures, and systems programming; familiarity with containerization and server-side debugging.
• Hands-on experience with at least one mainstream machine learning framework (e.g., PyTorch, TensorFlow).
• Experience deploying or optimizing LLM/VLM inference at production scale, with demonstrated impact on latency, throughput, or serving cost.
• Familiarity with GPU architecture and experience optimizing compute-intensive operators (e.g., FlashAttention, GEMM, GEMV, Conv2D).

Responsibilities

• Design, develop, and optimize high-performance inference systems for large-scale LLMs and VLMs, covering inference engines, serving frameworks, and end-to-end deployment pipelines.
• Build state-of-the-art model inference engines through advanced performance optimization techniques such as compiler-level optimizations, parallel computing, graph fusion, efficient CUDA kernel development, low-precision computation, streaming inference, speculative decoding, and high-concurrency request optimization.
• Collaborate with other research teams to identify performance bottlenecks, conduct performance analysis, and optimize large models; contribute to model toolchains and the broader technical ecosystem.

Skills

C/C++

Python

Algorithms

Data structures

Systems programming

Containerization

PyTorch/TensorFlow

CUDA

GPU architectures

Education

Bachelor's degree or above in CS/EE/SE

Tools

TensorRT

Triton

CUTLASS

CUDA/OpenCL

Job description

ByteDance Seed Infra in San Jose is seeking a Research Engineer to design, develop, and optimize high-performance inference systems for large-scale LLMs and VLMs. You will work on inference engines, serving frameworks, and end-to-end deployment pipelines, aiming to reduce latency and increase throughput.

Ideal candidates will have strong C/C++ and Python skills, hands-on ML framework experience (PyTorch/TensorFlow), and background in GPU-accelerated optimization.