Senior AI Inference Engineer - Self-Hosted, Scalable Endpoints

Firmus Technologies; Sydney; On-site; AUD 140,000 - 240,000; Full time

Job summary

Firmus Technologies, based in Australia, seeks a Senior AI Engineer (Inferencing) to build and enhance our self-hosted AI inference capabilities. You will deliver reliable, secure endpoints for internal products and external customers within the AI Factory framework.

You will own onboarding workflows for models, optimize performance, and collaborate with Kubernetes teams to ensure scalable, cost-aware deployments across our Asia Pacific operations.

Qualifications

• 5+ years of software engineering, with 3+ years in AI inference or model serving.
• Experience building production inference platforms and multi-tenant services.
• Hands-on with modern inference frameworks and GPU tooling.
• Strong Python; working proficiency in C++ or Go.
• Familiarity with Kubernetes, CI/CD, observability, and security.
• Understanding of RAG, embeddings, batching, and agentic workflows.

Responsibilities

• Build and improve self-hosted AI inference services for internal applications, customer-facing products, and future Inference-as-a-service offerings.
• Define onboarding workflows for models including validation, packaging, runtime selection, testing, release, and lifecycle management.
• Provision and manage secure, scalable inference endpoints for AI application patterns (interactive generation, RAG, embeddings, batching, tool calling).
• Develop reusable deployment templates, APIs, SDKs, configurations for users to request, configure, access, monitor, update and retire endpoints.
• Work with inference frameworks and toolkits (TensorRT-LLM, TensorRT, Triton) to optimize performance and observability.
• Collaborate with Kubernetes and scheduler teams to define endpoint resource profiles, placement, quotas, and autoscaling rules.
• Build benchmarking and qualification workflows with controlled experiments and load tests.
• Measure inference indicators (latency, tokens/second, QPS, memory, power) and ensure governance.

Skills

AI inference

Model serving

Python

C++

Distributed systems

GPU optimization

Observability

Security governance

Tools

TensorRT-LLM

TensorRT

SGLang

vLLM

Triton Inference Server

NVIDIA NIM

CUDA

cuDNN

NCCL

Job description

Firmus Technologies, based in Australia, seeks a Senior AI Engineer (Inferencing) to build and enhance our self-hosted AI inference capabilities. You will deliver reliable, secure endpoints for internal products and external customers within the AI Factory framework.

You will own onboarding workflows for models, optimize performance, and collaborate with Kubernetes teams to ensure scalable, cost-aware deployments across our Asia Pacific operations.