AI Systems Intern: LLM Inference & SoC Hardware

Samsung

Folsom (CA)

On-site

USD 39,000 - 94,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Relocation Support
Housing Stipend
Volunteer/community giving

Job summary

Samsung Semiconductor is seeking a hands-on intern to help develop and optimize next-generation SoC/SSD architectures. You will work on LLM inference services, RTL design integration, and validation on a GPU-accelerated stack.

The role is onsite in San Jose, CA, reporting to a Senior Director and collaborating with storage, FPGA, and firmware engineers. The position emphasizes practical experimentation, data-driven bottleneck analysis, and cross-team communication.

Qualifications

  • Pursuing Bachelor's, Master's or PhD in CS/CE/EE with at least one academic quarter/semester remaining.
  • Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux.
  • Understanding LLM serving internals: KV cache, batching, latency metrics, and throughput.
  • Experience benchmarking and profiling system performance with scripts (Python, pandas) to explain bottlenecks.
  • Familiarity with KV cache offloading, RDMA/NVMe-oF storage, and containers (Docker/Kubernetes).
  • You are inclusive, curious, collaborative and innovative.

Responsibilities

  • Deploy and operate an LLM inference service (vLLM with Llama-3.3-70B) on 4 × NVIDIA H200 GPUs and benchmark latency/throughput.
  • Build an automated benchmark harness for long-context, multi-turn and agent-style traces.
  • Configure KV cache offloading to NVMe-oF storage and validate end-to-end data paths with FPGA/SoC teams.
  • Run daily experiment sweeps across configurations and keep results reproducible in Git.
  • Analyze GPU/network/SSD bottlenecks with Python and present findings in team meetings.
  • Summarize storage requirements (bandwidth, IOPS, latency) for future SSD controllers.
  • Collaborate across FPGA, SoC and firmware teams to debug data paths and validate KV cache correctness.
  • Complete other responsibilities as assigned.

Skills

LLM inference with Python
PyTorch
Linux
Benchmarking
Profiling
KV cache offloading
RDMA/NVMeoF
Docker/Kubernetes
Collaboration

Education

Bachelor/Master/PhD in CS/CE/EE

Tools

vLLM
SGLang
Hugging Face TGI

Job description

Samsung Semiconductor is seeking a hands-on intern to help develop and optimize next-generation SoC/SSD architectures. You will work on LLM inference services, RTL design integration, and validation on a GPU-accelerated stack.

The role is onsite in San Jose, CA, reporting to a Senior Director and collaborating with storage, FPGA, and firmware engineers. The position emphasizes practical experimentation, data-driven bottleneck analysis, and cross-team communication.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Systems Engineer: LLM Inference on GPU, Flexible
AI Systems Engineer: LLM Inference on GPU, Flexible

Samsung Electronics America • Folsom (CA)

On-site
USD 39,000 - 94,000
Relocation stipend
Housing stipend
Volunteer matching
+2
LLM Inference Intern: GPU, KV Cache & SoC
LLM Inference Intern: GPU, KV Cache & SoC

Linuxconfig • Folsom (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Give Back
+3
AI Systems Architecture Intern — Memory & LLM Research
AI Systems Architecture Intern — Memory & LLM Research

Samsung Electronics America • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Give Back
+1
AI Systems Performance Intern — Benchmark & Profiling
AI Systems Performance Intern — Benchmark & Profiling

Samsung Electronics America • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Give Back programs
+2
R&D Internship: GPU, SoC & AI Acceleration
R&D Internship: GPU, SoC & AI Acceleration

Samsung Austin Semiconductor, LLC. • Austin (TX)

On-site
USD 53,000 - 88,000
Medical coverage
Dental coverage
Vision coverage
+7
ML/AI Engineer Intern — Hardware & LLM Systems
ML/AI Engineer Intern — Hardware & LLM Systems

llmda Inc. • Santa Clara (CA)

Hybrid
USD 41,000 - 55,000
Medical, dental, vision insurance
Gym on-site
401(k) Retirement Savings Plan
AI Systems Architecture Research Intern
AI Systems Architecture Research Intern

samsungsemiconductor • San Jose (CA)

On-site
USD 55,000 - 83,000
AI Systems Performance Intern: Memory & Benchmarking
AI Systems Performance Intern: Memory & Benchmarking

samsungsemiconductor • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Charitable giving match
+3
Intern, AI Application Engineer
Intern, AI Application Engineer

Linuxconfig • Folsom (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Give Back
+3
Memory & Storage Systems Intern for RL ML
Memory & Storage Systems Intern for RL ML

Samsung • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation support
Housing stipend
Charitable giving match
+1