Inference & Model Quality Engineer

HostYourAI; United States; Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Remote work

HQ Groningen

Team weeks abroad

Top of market salary

Job summary

HostYourAI is seeking an engineer to help run open language models efficiently on GPU infrastructure and to establish a consistent way to measure their quality. The role covers model serving, performance and cost tuning, evaluation, and improvements to an in-house Dutch model with results documented for users and regulatory transparency.

You will deploy and operate models on GPUs using serving frameworks such as vLLM, assess hardware fit and cost, benchmark performance, and build an evaluation

Qualifications

• Experience serving models on GPUs or fine-tuning language models.
• Knowledge of SFT, DPO, or LoRA and related model-serving tools.
• Proficiency with Python, Linux, containers, and ML libraries like Hugging Face or PyTorch.
• Fluent English and ability to work independently.

Responsibilities

• Deploy and operate models on GPUs using serving frameworks such as vLLM.
• Assess hardware fit and serving cost through VRAM estimates, quantization, and related trade-offs.
• Benchmark throughput and latency, then improve performance and cost per token.
• Build a shared evaluation harness covering quality, Dutch-language ability, grounding, tool use, and speed.
• Create datasets, run fine-tuning experiments, and compare new model versions against prior ones.
• Publish model cards and benchmarks, including required AI Act transparency information.

Skills

GPU serving

Fine-tuning LM

Benchmarking

Python

Linux

Containers

Hugging Face

PyTorch

English fluency

Tools

vLLM

SGLang

LoRA

SFT

DPO

Job description

Role overview

Help run open language models efficiently on GPU infrastructure and establish a consistent way to measure their quality. The work spans model serving, performance and cost tuning, evaluation, and improvements to an in-house Dutch model, with results documented for users and regulatory transparency.

Responsibilities

• Deploy and operate models on GPUs using serving frameworks such as vLLM, with SGLang planned.
• Assess hardware fit and serving cost through VRAM estimates, quantization, and related trade-offs.
• Benchmark throughput and latency, then improve performance and cost per token.
• Build a shared evaluation harness covering quality, Dutch-language ability, grounding, tool use, and speed.
• Create datasets, run fine-tuning experiments, and compare new model versions against prior ones.
• Publish model cards and benchmarks, including required AI Act transparency information.

Requirements

• Experience serving models on GPUs, fine-tuning language models, or both.
• Familiarity with approaches such as SFT, DPO, or LoRA, or model-serving tools such as vLLM and SGLang.
• A measurement-led approach using benchmarks, evaluations, and statistics where appropriate.
• Python, Linux, containers, and common ML tools such as Hugging Face or PyTorch.
• Fluent English, residence in the Netherlands, and the ability to work independently.

Nice to have

• Dutch language skills, especially for judging the quality of Dutch model outputs.

Benefits and work setup

• Remote work, with a headquarters in Groningen.
• Competitive salary, described as top of market.
• Team weeks abroad twice a year and an early role with meaningful ownership.