Senior Staff Engineer, AI Software

Samsung Semiconductor; San Jose (CA); Hybrid; USD 189,000 - 301,000; Full time

Benefits offered by this job

Medical/Dental/Vision

401k

Wellness apps

Paid time off

Flexible work hours

Job summary

Samsung Semiconductor in San Jose, CA is seeking a Lead Engineer to optimize AI model inference performance by overcoming memory bottlenecks. You will collaborate with hardware teams to influence architecture and develop benchmarks using frameworks like PyTorch.

The ideal candidate brings significant industry experience in AI frameworks, a deep understanding of AI infrastructure, and has skills in LLM architectures. Comprehensive benefits including medical plans, wellness resources, and paid time off are offered.

Qualifications

• Experience in developing AI software for GPUs or accelerators.
• Understanding of memory wall problem affecting AI performance.
• Familiarity with agentic AI architecture.

Responsibilities

• Lead the co-design of software and hardware solutions.
• Analyze AI workloads for performance optimization.
• Collaborate with hardware teams on architecture.

Skills

High-performance AI framework software development

AI infrastructure and software stack

LLM model architectures

PyTorch

Linux development environment

Memory architecture

Education

PhD with 10+ years, or Master's with 13+ years, or Bachelor's with 15+ years of industry experience

Tools

GitHub

Jira

vLLM

Job description

Location

Daily onsite presence at our San Jose, CA office / U.S. headquarters in alignment with our Flexible Work policy.

Artificial General Intelligence (AGI) Computing Lab

The AGI Computing Lab is dedicated to solving the complex system-level challenges posed by the growing demands of future AI/ML workloads. Our team is committed to designing and developing scalable platforms that can effectively handle the computational and memory requirements of these workloads while minimizing energy consumption and maximizing performance. We collaborate closely with both hardware and software engineers to identify and address the unique challenges posed by AI/ML workloads and to explore new computing abstractions that can provide a better balance between the hardware and software components of our systems. Additionally, we continuously conduct research and development in emerging technologies and trends across memory, computing, interconnect, and AI/ML, ensuring that our platforms are always equipped to handle the most demanding workloads of the future. By working together as a dedicated and passionate team, we aim to revolutionize the way AI/ML applications are deployed and executed, ultimately contributing to the advancement of AGI in an affordable and sustainable manner.

What You’ll Do

• Lead the co-design of software and hardware solutions that optimize AI model inference performance, with a focus on overcoming memory bottlenecks.
• Analyze and optimize LLM and agentic AI workloads across the full software stack, identifying opportunities for hardware‑aware acceleration.
• Profile and characterize model execution to expose memory wall limitations and guide architectural decisions for HBM and memory‑centric compute.
• Collaborate with hardware teams to influence memory architecture, acceleration strategies, and compute placement based on real workload behavior.
• Develop, optimize, and benchmark inference and serving solutions using frameworks such as PyTorch and vLLM.
• Define best practices and provide technical mentorship across software–hardware co‑design efforts.

What You Bring

• Bachelor’s with 15+ years, or Master’s with 13+ years, or PhD with 10+ years of industry experience.
• Strong experience writing high‑performance AI framework software development for GPUs or other accelerators.
• Strong, end‑to‑end understanding of the AI infrastructure and AI software stack, from model definition through deployment and serving.
• Solid understanding of LLM model architectures and workflows, including modern transformer‑based designs.
• Solid understanding of agentic AI architecture and workflows.
• Hands‑on expertise with the PyTorch framework.
• Practical experience with vLLM for high‑throughput model inference and serving.
• Solid understanding of the memory wall problem and its impact on AI system performance.
• Strong knowledge of memory architecture, including High Bandwidth Memory (HBM), and familiarity with memory‑centric acceleration and compute approaches.
• Proficiency working in a Linux development environment.
• Solid command of development tooling, including agentic coding, GitHub and Jira.

What We Offer

Base Pay Range: $189,000 - $301,000 USD.

Pay within this range varies by work location and may also depend on job‑related knowledge, skills, and experience. We also offer incentive opportunities that reward employees based on individual and company performance.

Benefits centered around the wellbeing of employees and their loved ones: Medical/Dental/Vision/401k, inclusive rewards plan, wellness apps, and confidential therapy sessions. We provide paid time off, holidays, sick leave, flexibility, fitness resources, and support for family care.

We also support community giving through charitable giving match and frequent opportunities to get involved.

Equal Opportunity Employment Policy

Samsung Semiconductor takes pride in being an equal opportunity workplace dedicated to fostering an environment where all individuals feel valued and empowered to excel, regardless of race, religion, color, age, disability, sex, gender identity, sexual orientation, ancestry, genetic information, marital status, national origin, political affiliation, or veteran status. When selecting team members, we prioritize talent and qualities such as humility, kindness, and dedication. We extend comprehensive accommodations throughout our recruiting processes for candidates with disabilities, long‑term conditions, neurodivergent individuals, or those requiring pregnancy‑related support. All candidates scheduled for an interview will receive guidance on requesting accommodations.

Recruiting Agency Policy

We do not accept unsolicited resumes. Only authorized recruitment agencies with a current and valid agreement with Samsung Semiconductor, Inc. are permitted to submit resumes for any job openings.

Applicant AI Use Policy

At Samsung Semiconductor, we support innovation and technology. To ensure a fair and authentic assessment, we prohibit the use of generative AI tools to misrepresent a candidate’s true skills and qualifications. Permitted uses are limited to basic preparation, grammar, and research, but all submitted content and interview responses must reflect the candidate’s genuine abilities and experience. Violation of this policy may result in immediate disqualification from the hiring process.

Trade Secret Notice

By submitting an application, you agree not to disclose to Samsung—or encourage Samsung to use—any confidential or proprietary information (including trade secrets) belonging to a current or former employer or other entity.

Applicant Privacy Policy

https://semiconductor.samsung.com/about-us/careers/us/privacy/