Senior AI Platform Engineer – On-Prem GPU & LLM Infra

Bewerberportal; Town of Vienna (WI); Hybrid; USD 67,000 - 82,000; Full time

Benefits offered by this job

Health insurance

Paid time off

Retirement plans

Job summary

AI:AT AI FACTORY AUSTRIA in Vienna invites senior software engineers to join the AI Factory Austria supercomputing infrastructure project. You will design and operate scalable LLM inference and fine-tuning services on our on-premise GPU/HPC stack, building an automated B2B AI compute platform.

Requirements include 5-7 years in software/platform engineering, deep Kubernetes experience, Python (Go a plus), ML lifecycle tooling (MLflow/Kubeflow), and eligibility to work in Austria.

Qualifications

• Degree in computer science or a related field, or equivalent professional experience.
• 5-7 years of industry experience in software, platform, or infrastructure engineering, with at least 2 years building ML or AI platforms.
• Deep hands-on experience running Kubernetes in production and building scalable, distributed systems and APIs.
• Hands-on experience with LLM and model serving and inference optimization (e.g. vLLM, Triton).
• Strong programming skills in Python; experience with Go or another backend language is a plus.
• Experience with the ML lifecycle: training and fine-tuning pipelines, experiment tracking, and model registries (e.g. MLflow, Kubeflow).
• Good understanding of GPU workloads, object storage, and Infrastructure as Code, GitOps, and CI/CD practices.
• You are a self-starter, can work independently with limited supervision, and communicate fluently in English.
• You should be eligible for working in Austria.

Responsibilities

• Build and operate scalable LLM inference and fine-tuning services for on-prem GPU/HPC infrastructure.
• Design and run automated B2B AI compute stacks for production workloads.
• Lead model serving, GPU scheduling, and self-service APIs for users.

Skills

Kubernetes

Python

LLM serving

Go or backend

CI/CD

GITOps

GPU workloads

ML lifecycle

Education

Degree in computer science or related field

Tools

Kubernetes

MLflow

Kubeflow

vLLM

Triton

Job description

AI:AT AI FACTORY AUSTRIA in Vienna invites senior software engineers to join the AI Factory Austria supercomputing infrastructure project. You will design and operate scalable LLM inference and fine-tuning services on our on-premise GPU/HPC stack, building an automated B2B AI compute platform.

Requirements include 5-7 years in software/platform engineering, deep Kubernetes experience, Python (Go a plus), ML lifecycle tooling (MLflow/Kubeflow), and eligibility to work in Austria.