Senior AI Inference Engineer - Self-Hosted AI

Matchbox; Sydney; Hybrid; AUD 140,000 - 180,000; Full time

Job summary

Firmus Technologies in Sydney, Australia seeks a Senior AI Engineer (Inference) to build and operate self-hosted AI inference services for internal and customer-facing products. You will define onboarding workflows, validate models, and deploy scalable endpoints across interactive generation, embeddings, and batch processing.

The role demands strong experience in AI inference, model serving, and distributed systems, with Python and C++/Go proficiency and familiarity with TensorRT-LLM, Triton,

Qualifications

• 5+ years in software/AI inference or ML systems.
• Experience building production model-serving platforms.
• Strong Python; proficiency in C++/Go for inference services.

Responsibilities

• Build self-hosted AI inference services for internal and external products.
• Define onboarding workflows for models including validation and deployment.
• Provision secure, scalable inference endpoints for various AI patterns (interactive, embeddings, batching).
• Develop reusable deployment templates, APIs, SDKs, and self-service workflows.
• Collaborate with Kubernetes/scheduler teams to define resource profiles and autoscaling policies.

Skills

AI inference

Model serving

Distributed systems

Python

Go/C++

Education

Bachelor's degree in CS or related

Tools

TensorRT-LLM

Triton Inference Server

CUDA

Job description

Firmus Technologies in Sydney, Australia seeks a Senior AI Engineer (Inference) to build and operate self-hosted AI inference services for internal and customer-facing products. You will define onboarding workflows, validate models, and deploy scalable endpoints across interactive generation, embeddings, and batch processing.

The role demands strong experience in AI inference, model serving, and distributed systems, with Python and C++/Go proficiency and familiarity with TensorRT-LLM, Triton,