Senior AI Inference Engineer – Self-Hosted, Scalable

Sustainable Metal Cloud; Sydney; On-site; AUD 180,000 - 280,000; Full time

Job summary

Firmus Technologies is hiring a Senior AI Engineer (Inferencing) to build and improve our AI inference capabilities. You will deliver reliable, secure, and scalable endpoints for internal products, external customers, and future Inference‑as‑a‑service offerings.

The role requires model onboarding, deployment, runtime selection, performance benchmarking, observability, security, and lifecycle management within our AI‑factory environment.

Qualifications

• 5+ years of software engineering experience, including 3+ years in AI inference or ML systems.
• Hands-on experience with production model-serving platforms and multi-tenant AI systems.
• Strong Python skills and working proficiency in C++ or Go.
• Experience with Kubernetes, CI/CD, and observability in production environments.
• Familiarity with RAG, embeddings, batching, and latency/throughput trade-offs.

Responsibilities

• Build, operate and improve self-hosted AI inference services for internal and customer-facing products.
• Define onboarding workflows for models, including validation, packaging, and deployment.
• Provide scalable, secure, and observable inference endpoints for various AI tasks.
• Develop reusable deployment templates, APIs, SDKs, and self-serve workflows for endpoint management.
• Collaborate with product, security, and platform teams to ensure reliable operation.

Skills

Python

Go/C++

Distributed systems

Performance optimization

ML systems

Tools

TensorRT-LLM

TensorRT

SGLang

vLLM

Triton Inference Server

CUDA

cuDNN

NCCL

Job description

Firmus Technologies is hiring a Senior AI Engineer (Inferencing) to build and improve our AI inference capabilities. You will deliver reliable, secure, and scalable endpoints for internal products, external customers, and future Inference‑as‑a‑service offerings.

The role requires model onboarding, deployment, runtime selection, performance benchmarking, observability, security, and lifecycle management within our AI‑factory environment.