In-Office LLM Inference Engineer — Quantization

ODS Serbia; Time; On-site; NOK 900,000 - 1,300,000; Full time

Job summary

Sobolev Research Center is seeking an ML Engineer to develop and optimize algorithms that accelerate large language model inference, directly impacting latency, cost efficiency, and scalability of production-grade AI systems.

You will explore and implement cutting-edge techniques such as speculative decoding, prompt compression, quantization, and generation optimisation to improve performance across deployment scenarios.

Qualifications

• Strong experience with deep learning frameworks (PyTorch or TensorFlow).
• Solid understanding of Transformer architectures and LLMs.
• Experience with model inference optimization.
• Strong Python skills.
• Understanding of GPU/CPU performance and memory bottlenecks.

Responsibilities

• LLM Quantization & Low-Precision Optimization: develop and apply quantization techniques to reduce model memory footprint and accelerate inference while preserving output quality.
• Speculative Decoding & Generation Acceleration: design algorithms to reduce decoding steps and improve generation speed.
• Prompt Compression & Context Optimization: reduce input context length without degrading output quality.

Skills

Transformer basics

Python programming

Model inference optimization

GPU/CPU performance

Tools

PyTorch

TensorFlow

ONNX Runtime

Hugging Face Transformers

TensorRT

DeepSpeed

BitsAndBytes

GPTQ

FlashAttention

xFormers

Job description

Sobolev Research Center is seeking an ML Engineer to develop and optimize algorithms that accelerate large language model inference, directly impacting latency, cost efficiency, and scalability of production-grade AI systems.

You will explore and implement cutting-edge techniques such as speculative decoding, prompt compression, quantization, and generation optimisation to improve performance across deployment scenarios.