LLM Inference Intern: GPU, KV Cache & SoC

Linuxconfig

Folsom (CA)

On-site

USD 39,000 - 94,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Relocation Support
Housing Stipend
Give Back
Prioritize Emotional Wellness
Stay Fit
Embrace Flexibility

Job summary

Samsung Semiconductor in San Jose, CA, is offering a summer internship to work on cutting-edge SoC storage and memory projects. You will deploy and operate an LLM inference service on a multi-GPU server and build a reproducible benchmark harness to measure latency and throughput.

The role requires hands-on experience with Python and PyTorch on NVIDIA GPUs under Linux and familiarity with vLLM, SGLang, or Hugging Face TGI.

Qualifications

  • Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux, using a serving framework such as vLLM, SGLang or Hugging Face TGI.
  • Understanding of LLM serving internals: KV cache, prefix caching, batching and parallelism, and latency metrics TTFT, TPOT and throughput.
  • Experience benchmarking and profiling system performance, and analyzing results with scripts (e.g., Python, pandas, plotting) to explain bottlenecks.

Responsibilities

  • Deploy and operate an LLM inference service (vLLM serving Llama-3.3-70B) on a 4 × NVIDIA H200 GPU server, with mentorship and collaboration with storage architecture.
  • Build an automated benchmark harness that replays long-context, multi-turn traces and records latency and throughput.
  • Configure KV cache offloading to network-attached storage over NVMe-oF and RDMA, initially on a BlueField-3 storage server.

Skills

Python
PyTorch
Linux
NVIDIA GPUs
LLM serving

Education

Pursuing Bachelor’s/Master’s/PhD in CS/CE/EE
At least 1 academic quarter/semester remaining

Tools

vLLM
SGLang
Hugging Face TGI
Docker/Kubernetes

Job description

Samsung Semiconductor in San Jose, CA, is offering a summer internship to work on cutting-edge SoC storage and memory projects. You will deploy and operate an LLM inference service on a multi-GPU server and build a reproducible benchmark harness to measure latency and throughput.

The role requires hands-on experience with Python and PyTorch on NVIDIA GPUs under Linux and familiarity with vLLM, SGLang, or Hugging Face TGI.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Systems Engineer: LLM Inference on GPU, Flexible
AI Systems Engineer: LLM Inference on GPU, Flexible

Samsung Electronics America • Folsom (CA)

On-site
USD 39,000 - 94,000
Relocation stipend
Housing stipend
Volunteer matching
+2
AI Systems Intern: LLM Inference & SoC Hardware
AI Systems Intern: LLM Inference & SoC Hardware

Samsung • Folsom (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Volunteer/community giving
LLM Inference Engineer - GPU Systems & Optimization
LLM Inference Engineer - GPU Systems & Optimization

GMI Cloud, Inc • San Francisco (CA)

On-site
USD 180,000 - 280,000
Senior LLM Inference: GPU Kernel Optimization
Senior LLM Inference: GPU Kernel Optimization

NVIDIA Corporation • Austin (TX)

On-site
USD 184,000 - 287,500
Equity
Benefits
AI Systems Performance Intern: Memory & Benchmarking
AI Systems Performance Intern: Memory & Benchmarking

samsungsemiconductor • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Charitable giving match
+3
LLM Inference KV Cache Engineer (Distributed)
LLM Inference KV Cache Engineer (Distributed)

Bytedance • Seattle (WA)

On-site
USD 154,000 - 301,000
Member of Technical Staff - Inference
Member of Technical Staff - Inference

Sail Research • San Francisco (CA)

On-site
USD 120,000 - 160,000
Meals provided
Studio Display for every employee
Member of Technical Staff - Inference
Member of Technical Staff - Inference

Sail • San Francisco (CA)

On-site
USD 180,000 - 240,000
Principal Software Engineer, Inference
Principal Software Engineer, Inference

Hewlett Packard Enterprise • Fort Collins (CO)

Hybrid
USD 180,000 - 250,000
Health and wellbeing benefits
Professional development programs
Flexible work arrangements
AI Systems Performance Intern — Benchmark & Profiling
AI Systems Performance Intern — Benchmark & Profiling

Samsung Electronics America • San Jose (CA)

On-site
USD 39,000 - 94,000
Relocation Support
Housing Stipend
Give Back programs
+2