Senior AI Memory Architect & Inference Lead

samsungsemiconductor; San Jose (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Samsung Semiconductor in San Jose is seeking a Hands-on Principal Engineer to own the requirement for memory-driven AI inference at scale. You will bridge model architecture with memory-system design, spanning GPU HBM, host DRAM, and disaggregated memory tiers, and deliver production-ready memory solutions.

You will lead memory-footprint analyses, latency and throughput optimization, and benchmark methodologies for frontier models like Llama/Qwen-class families.

Qualifications

• In-depth knowledge of AI model architectures and memory systems.
• Experience with production-scale inference stacks and benchmarking.
• Strong systems-level thinking and memory-policy design.

Responsibilities

• Define memory-system requirements for large-scale AI inference.
• Lead engineering to optimize HBM, DRAM, and NVMe memory usage.
• Collaborate with model teams on memory-aware workloads.
• Establish benchmarking methodologies for memory systems.

Skills

Model architectures

Memory-system design

Production inference

MoE experts

GPU memory

Tools

SGLang HiCache

llama.cpp engines

TensorRT-LLM

NVIDIA Dynamo

Job description

Samsung Semiconductor in San Jose is seeking a Hands-on Principal Engineer to own the requirement for memory-driven AI inference at scale. You will bridge model architecture with memory-system design, spanning GPU HBM, host DRAM, and disaggregated memory tiers, and deliver production-ready memory solutions.

You will lead memory-footprint analyses, latency and throughput optimization, and benchmark methodologies for frontier models like Llama/Qwen-class families.