Senior AI Inference Engineer — Self-Hosted Platform

Firmus Technologies; Sydney; On-site; AUD 150,000 - 230,000; Full time

Job summary

Firmus Technologies seeks a Senior AI Engineer (Inferencing) to build and optimize self-hosted AI inference services and endpoints for internal and external customers. You will lead model onboarding, runtime selection, and secure deployment, enabling AI factory workflows with high performance.

You will own benchmarking, observability, and capacity planning while collaborating across Product, Platform, and Security teams to deliver reliable, cost‑efficient inference at scale.

Qualifications

• 5+ years of software engineering experience, including 3+ years in AI inference, model serving, ML systems, high‑performance computing, distributed systems, or comparable environments.
• Proven track record building, operating, or improving production model‑serving platforms, inference APIs, or multi‑tenant AI systems.
• Hands‑on experience with one or more modern inference frameworks (e.g., TensorRT‑LLM, TensorRT, Triton Inference Server, etc.).
• Strong Python skills with working proficiency in C++ or Go for inference services and performance‑critical development.

Responsibilities

• Build, operate and improve self-hosted AI inference services for internal and customer-facing products.
• Define onboarding workflows for models, including validation, packaging, deployment and lifecycle management.
• Provision secure, scalable inference endpoints for interactive generation, embeddings, RAG, batching, and more.
• Develop reusable deployment templates, APIs, SDKs and self-service workflows for endpoint access and monitoring.
• Collaborate with Kubernetes and scheduler teams to define resource profiles, placement, autoscaling, and quotas.
• Establish automated performance and quality targets via benchmarking, profiling and observability.

Skills

Self-hosted inference

Model serving

Endpoint provisioning

Python

C++/Go

GPU performance

Generative AI

Kubernetes

Benchmarking

Tools

TensorRT-LLM

TensorRT

Triton Inference Server

CUDA

cuDNN

NCCL

Kubernetes tooling

Job description

Firmus Technologies seeks a Senior AI Engineer (Inferencing) to build and optimize self-hosted AI inference services and endpoints for internal and external customers. You will lead model onboarding, runtime selection, and secure deployment, enabling AI factory workflows with high performance.

You will own benchmarking, observability, and capacity planning while collaborating across Product, Platform, and Security teams to deliver reliable, cost‑efficient inference at scale.