High-Performance AI Inference Framework Engineer

ByteDance; San Jose (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

ByteDance’s Data AML team is seeking talented individuals to design and implement high-performance model inference services for large-parameter AI models. You will work on scalable, enterprise-level systems, focusing on inference engine scheduling, monitoring, canary releases, and integration across TikTok, Douyin, and other ByteDance platforms.

The role requires strong C/C++ programming skills, experience with high-concurrency distributed services, and a passion for optimizing performance.

Qualifications

• Individuals who are completing or have recently completed a Bachelor's/Master's degree in Software Development, Computer Science, Computer Engineering, or a related technical discipline.

Responsibilities

• Architect and implement high-performance, scalable enterprise-level inference services for large-parameter AI models.
• Research and optimize core modules of the inference framework (scheduling, monitoring, canary release).
• Track latest inference technologies and propose architecture improvements for high-concurrency scenarios.

Skills

C/C++ programming

Linux basics

Data structures & algorithms

Multi-threading

Distributed services

Problem solving

Cross-team collaboration

Documentation

Education

Bachelor's or Master's in Software Development / Computer Science / Computer Engineering

Tools

Redis

RocksDB

BRPC

GRPC

GPUs

Job description

ByteDance’s Data AML team is seeking talented individuals to design and implement high-performance model inference services for large-parameter AI models. You will work on scalable, enterprise-level systems, focusing on inference engine scheduling, monitoring, canary releases, and integration across TikTok, Douyin, and other ByteDance platforms.

The role requires strong C/C++ programming skills, experience with high-concurrency distributed services, and a passion for optimizing performance.