Edge LLM Engineer: On-Device Inference

Desay SV; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Desay SV is seeking an Embedded LLM Systems Engineer to design and optimise on-device LLM inference for embedded, mobile and edge devices. You will work on inference engines, operator development and graph optimisation across multiple backends.

The role requires hands-on experience with modern C++, Python, and model-quantisation techniques, plus familiarity with CUDA, MediaPipe and related technologies for efficient deployment.

Qualifications

• Three years of experience in on-device inference, AI infrastructure, embedded systems engineering, or related area.
• Bachelor’s degree or above in CS/EE/Math or equivalent practical experience.
• Proficiency in English to read technical docs and discuss with teams.

Responsibilities

• Design, develop and optimise LLM inference engines for embedded, mobile and edge devices.
• Work on operator development, graph optimisation, memory management and multi-backend adaptation.
• Develop solutions using frameworks such as llama.cpp, TensorRT-LLM, MNN, ONNX Runtime or comparable technologies.
• Research and apply quantisation techniques such as INT4, INT8 and FP16.
• Work with technologies such as NEON/SVE, Vulkan Compute, OpenCL or comparable platforms.
• Conduct training-inference consistency validation and support deployment across cloud and edge environments.
• Translate emerging AI capabilities into embedded product value.

Skills

On-device inference

Modern C++

Python

English proficiency

Education

Bachelor's degree in Computer Science

Bachelor's degree in Electrical/Electronic Engineering

Bachelor's degree in Mathematics

Tools

llama.cpp

TensorRT-LLM

MNN

ONNX Runtime

CUDA

MediaPipe

Job description

Desay SV is seeking an Embedded LLM Systems Engineer to design and optimise on-device LLM inference for embedded, mobile and edge devices. You will work on inference engines, operator development and graph optimisation across multiple backends.

The role requires hands-on experience with modern C++, Python, and model-quantisation techniques, plus familiarity with CUDA, MediaPipe and related technologies for efficient deployment.