AI Inference Engineer

Quadric Inc.

Burlingame (CA)

On-site

USD 180,000 - 260,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Health Care Plan (Medical, Dental & Vision)

Retirement Plan (401k, IRA)

Life Insurance (Basic, Voluntary & AD&D)

Paid Time Off (Vacation, Sick & Public Holidays)

Family Leave (Maternity, Paternity)

Short Term & Long Term Disability

Training & Development

Work From Home

Free Food & Snacks

Stock Option Plan

Job summary

A leading technology company in California is seeking an AI Inference Engineer to bridge AI models with unique platforms. Key responsibilities include model optimization, deployment, and performance profiling. Candidates should have a Bachelor’s or Master’s degree, 5+ years' experience in AI frameworks, and proficiency in C/C++ and Python. Competitive benefits included, such as health care, retirement plans, and work from home options.

Qualifications

5+ years of experience in AI/LLM model inference and deployment frameworks/tools.
Experience with model quantization (PTQ, QAT) and tools.
Proficiency in C/C++ and Python.
Experience with model inference performance profiling and accuracy measures.

Responsibilities

Quantize, prune and convert models for deployment.
Port models to Quadric platform using Quadric toolchain.
Optimize inference deployment for latency, speed.
Benchmark and profile model performance and accuracy.
Develop tools to scale and speed up the deployment.
Provide technical support and documents to customers and developers.

Skills

AI model algorithms

System architecture

AI toolchains/frameworks

Problem solving

Debugging

Communication

Education

Bachelor’s or Master’s in Computer Science or Electrical Engineering

Tools

onnxruntime

Pytorch

vLLM

huggingface-transformer

neural-compressor

llamacpp

Job description

Quadric has created an innovative general purpose neural processing unit (GPNPU) architecture. Quadric's co-optimized software and hardware is targeted to run neural network (NN) inference workloads in a wide variety of edge and endpoint devices, ranging from battery operated smart-sensor systems to high-performance automotive or autonomous vehicle systems. Unlike other NPUs or neural network accelerators in the industry today that can only accelerate a portion of a machine learning graph, the Quadric GPNPU executes both NN graph code and conventional C++ DSP and control code.

Role:

The AI Inference Engineer in Quadric is the key bridge between the world of AI/LLM models and Quadric unique platforms. The AI Inference Engineer at Quadric will [1] port AI models to Quadric platform; [2] optimize the model deployment for efficient inference; [3] profile and benchmark the model performance. This senior technical role demands deep knowledge of AI model algorithms, system architecture and AI toolchains/frameworks.

Responsibilities:

Quantize, prune and convert models for deployment
Port models to Quadric platform using Quadric toolchain
Optimize inference deployment for latency, speed
Benchmark and profile model performance and accuracy
Develop tools to scale and speed up the deployment
Make Improvement to SDK and runtimeProvide technical support and documents to customers and developer community

Requirements:

Bachelor’s or Master’s in Computer Science and/or Electric Engineering.
5+ years of experience in AI/LLM model inference and deployment frameworks/tools
experience with model quantization (PTQ, QAT) and tools
experience with model accuracy measuresexperience with model inference performance profiling
experience with at least one of the following frameworks: onnxruntime, Pytorch, vLLM, huggingface-transformer, neural-compressor, llamacpp
Proficiency in C/C++ and Python
Demonstrate good capability in problem solving, debug and communication