Sr. Inference Optimization Engineer (local / edge runtime)

Intel Corporation; Santa Clara (CA); Hybrid; USD 195,000 - 361,000; Full time

Job summary

Intel Corporation is seeking a software engineer to make models fast on hardware people own, optimizing inference engines for edge environments. You will work with llama.cpp and vLLM, tuning KV cache, batching, and quantization while reducing CPU overhead and startup costs.

You’ll benchmark across hardware tiers and contribute patches to open‑source engines, aligning with Intel’s mission to improve AI safety, privacy, and efficiency on local devices.

Qualifications

• BS/MS in CS, EE, Math or related STEM field.
• 8+ years software development background.
• Strong in C++ and/or Python; comfortable reading systems‑level code.
• Experience with LLM inference (attention, KV cache, decoding).
• Experience profiling and optimizing real performance problems (CPU or GPU).
• Linux, build systems, and low‑level debugging expertise.
• Hands‑on with llama.cpp, vLLM, ggml, or similar engines.
• Experience with GPU / accelerator programming (Vulkan, CUDA, SYCL, Metal).
• Familiarity with quantization formats and their quality trade‑offs.
• Open‑source contributions to inference engines.

Responsibilities

• Profile and optimize local inference for latency, throughput, and memory on edge hardware.
• Tune KV cache, continuous batching, and scheduling for interactive workloads.
• Drive quantization strategy and validate quality impact with the Post‑Training team.
• Cut CPU overhead and improve engine startup, model load, and lifecycle (start / stop / health).
• Benchmark across hardware tiers and publish honest performance comparisons.
• Upstream fixes and patches to open‑source engines where it helps us.

Skills

C++

Python

LLM inference

Profiling & optimization

Linux

Low-level debugging

Systems programming

Vulkan/CUDA/SYCL/Metal

Quantization formats

Open-source contributions

Education

BS/MS in CS, EE, Math or related STEM field

Tools

llama.cpp

vLLM

ggml

Vulkan

CUDA

SYCL

Metal

Job description

Job Details: Job Description: Job Description

Our Mission

At Intel, our journey is to transform AI into something safer, more trustworthy, and respectful of human privacy by design. We believe transformative AI should have a positive impact on people—powerful in capability, yet honest about its limits and protective of the data and resources it touches. To get there, we build agentic AI that combines the best of local and cloud intelligence — private, affordable, and sustainable by design. Small, efficient models run directly on the user's machine (AI PC, edge, on-prem, and beyond), keeping data private and token costs low, while powerful cloud models handle the hardest work: planning, reasoning, and complex problem-solving. Today, neither approach can deliver this alone. Together, they give people real capability without compromise—data stays private, spend stays predictable, and energy use stays in check. We're building intelligence that scales without sacrificing trust, cost, or the planet—because the future of AI should belong to the people it serves.

Role Summary

Make models fast on the hardware people actually own. You optimize inference engines (llama.cpp, vLLM) for constrained local and edge environments — GPU/iGPUs, Vulkan backends — not datacenter H100 environment, mostly PC/edge. KV cache, batching, quantization, scheduling, and CPU-overhead reduction are your daily tools. This is the rare skill that makes a hybrid, low-cost agent product viable.

What you’ll do

1. Profile and optimize local inference (llama.cpp-vulkan and vLLM) for latency, throughput, and memory on edge hardware
2. Tune KV cache, continuous batching, and scheduling for interactive agent workloads
3. Drive quantization strategy (GGUF / AWQ / GPTQ) and validate quality impact with the Post‑Training team
4. Cut CPU overhead and improve engine startup, model load, and lifecycle (start / stop / health)
5. Benchmark across hardware tiers and publish honest performance comparisons
6. Upstream fixes and patches to open‑source engines where it helps us

What you’ll learn / grow into

• Curiosity is required.
• The internals of modern inference engines and where the milliseconds actually go
• Hardware‑aware optimization across iGPU / CPU paths (Vulkan, SYCL, oneAPI, CUDA where relevant)
• The quality‑vs‑speed‑vs‑memory trade space for small models
• Interest in local / edge AI and squeezing hardware

Qualifications

• Required Qualifications
• BS/MS in CS, EE, Math or related STEM field
• 8+ years software development background
• Strong in C++ and/or Python; comfortable reading systems‑level code
• Experience with LLM inference. (attention, KV cache, decoding)
• Experience profiling and optimizing real performance problems (CPU or GPU) and can prove the speedup
• Linux, build systems, and low‑level debugging expertise
• Preferred Qualifications
• Hands‑on with llama.cpp, vLLM, ggml, or similar engines
• Experience with GPU / accelerator programming (Vulkan, CUDA, SYCL, Metal) or SIMD / CPU kernels
• Familiarity with quantization formats and their quality trade‑offs
• Open‑source contributions to inference engines
• Requirements listed would be obtained through a combination of industry relevant job experience, internship experiences and or schoolwork/classes/research.

Benefits at Intel

Our total rewards package goes above and beyond just a paycheck. Whether you're looking to build your career, improve your health, or protect your wealth, we offer generous benefits to help you achieve your goals. Go to Intel Benefits | Intel Careers for details of benefits available to you. Intel reserves the right to modify, change or discontinue benefit plans at any time in its sole discretion.

Job Type

Shift: Shift 1 (United States of America)

Location

Primary Location: US, California, Santa Clara

Additional Locations: US, Arizona, Phoenix, US, California, Folsom, US, Oregon, Hillsboro

Posting Statement

All qualified applicants will receive consideration for employment without regard to race, color, religion, religious creed, sex, national origin, ancestry, age, physical or mental disability, medical condition, genetic information, military and veteran status, marital status, pregnancy, gender, gender expression, gender identity, sexual orientation, or any other characteristic protected by local law, regulation, or ordinance.

Position of Trust

N/A

Benefits

We offer a total compensation package that ranks among the best in the industry. It consists of competitive pay, stock bonuses, and benefit programs which include health, retirement, and vacation. Find out more about the benefits of working at Intel.

Annual Salary Range

$195,200.00-361,200.00 USD

Work Model for this Role

This role will be eligible for our hybrid work model which allows employees to split their time between working on‑site at their assigned Intel site and off‑site.

Job posting details (such as work model, location or time type) are subject to change.

Intel is committed to Responsible Business Alliance (RBA) compliance and ethical hiring practices. We do not charge any fees during our hiring process. Candidates should never be required to pay recruitment fees, medical examination fees, or any other charges as a condition of employment. If you are asked to pay any fees during our hiring process, please report this immediately to your recruiter. Intel’s official careers website. Find your next job and take on projects that shape tomorrow’s technology. Benefits Internships Life at Intel Locations Recruitment Process