Director, AI Memory & Large-Scale Inference

Samsung Semiconductor; San Jose (CA); Hybrid; USD 219,000 - 351,000; Full time

Benefits offered by this job

4+ weeks PTO

Fertility care stipend

Medical travel support

Onsite Café & gym

Job summary

Samsung Semiconductor seeks a Principal Engineer to own full-stack AI memory solutions for inference at fleet scale. You will design tiered memory architectures, from HBM to NVMe, and drive performance, latency, and cost improvements across production inference stacks.

You will mentor engineers, define reference architectures, and speak with customers and partners at the deepest technical level. Remote/hybrid options available, onsite preferred in San Jose.

Qualifications

• BS in Computer/Electrical Engineering or Computer Science; 10+ years in systems engineering.
• MS in Computer/Electrical Engineering or Computer Science preferred; 8+ years experience.

Responsibilities

• Define engineering requirements for tiered memory systems for inference at fleet scale.
• Design and validate memory architectures, including HBM, DRAM, CXL, and NVMe tiers.
• Lead architecture reviews and mentor engineers across the memory-stack domain.
• Represent the company in CTO-level discussions and with customers on memory for AI.

Skills

Memory systems

AI inference

Performance engineering

NUMA topology

PCIe topology

Open-source inference

Memory bandwidth

Cache policies

GPGPU profiling

Education

BS in Computer/Electrical Engineering or Computer Science

MS in Computer/Electrical Engineering or Computer Science (preferred)

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Samsung Semiconductor seeks a Principal Engineer to own full-stack AI memory solutions for inference at fleet scale. You will design tiered memory architectures, from HBM to NVMe, and drive performance, latency, and cost improvements across production inference stacks.

You will mentor engineers, define reference architectures, and speak with customers and partners at the deepest technical level. Remote/hybrid options available, onsite preferred in San Jose.