Senior Edge Inference Optimization Engineer

Intel Corporation; Santa Clara (CA); Hybrid; USD 195,000 - 361,000; Full time

Job summary

Intel Corporation is seeking a software engineer to make models fast on hardware people own, optimizing inference engines for edge environments. You will work with llama.cpp and vLLM, tuning KV cache, batching, and quantization while reducing CPU overhead and startup costs.

You’ll benchmark across hardware tiers and contribute patches to open‑source engines, aligning with Intel’s mission to improve AI safety, privacy, and efficiency on local devices.

Qualifications

• BS/MS in CS, EE, Math or related STEM field.
• 8+ years software development background.
• Strong in C++ and/or Python; comfortable reading systems‑level code.
• Experience with LLM inference (attention, KV cache, decoding).
• Experience profiling and optimizing real performance problems (CPU or GPU).
• Linux, build systems, and low‑level debugging expertise.
• Hands‑on with llama.cpp, vLLM, ggml, or similar engines.
• Experience with GPU / accelerator programming (Vulkan, CUDA, SYCL, Metal).
• Familiarity with quantization formats and their quality trade‑offs.
• Open‑source contributions to inference engines.

Responsibilities

• Profile and optimize local inference for latency, throughput, and memory on edge hardware.
• Tune KV cache, continuous batching, and scheduling for interactive workloads.
• Drive quantization strategy and validate quality impact with the Post‑Training team.
• Cut CPU overhead and improve engine startup, model load, and lifecycle (start / stop / health).
• Benchmark across hardware tiers and publish honest performance comparisons.
• Upstream fixes and patches to open‑source engines where it helps us.

Skills

C++

Python

LLM inference

Profiling & optimization

Linux

Low-level debugging

Systems programming

Vulkan/CUDA/SYCL/Metal

Quantization formats

Open-source contributions

Education

BS/MS in CS, EE, Math or related STEM field

Tools

llama.cpp

vLLM

ggml

Vulkan

CUDA

SYCL

Metal

Job description

Intel Corporation is seeking a software engineer to make models fast on hardware people own, optimizing inference engines for edge environments. You will work with llama.cpp and vLLM, tuning KV cache, batching, and quantization while reducing CPU overhead and startup costs.

You’ll benchmark across hardware tiers and contribute patches to open‑source engines, aligning with Intel’s mission to improve AI safety, privacy, and efficiency on local devices.