Senior Inference Backend Engineer (Systems)

Hume AI; Sydney; On-site; AUD 197,000 - 295,000; Full time

Job summary

Hume AI is seeking a systems-oriented engineer to own the path from trained model to production inference in a high-performance environment in New York City. You will handle graph export, engine compilation, runtime integration, and production verification, collaborating with researchers and backend engineers to scale inference across products.

The role emphasizes performance, numerical correctness, reliability, and accelerator efficiency, with autonomy over lifecycle from design to deployment

Qualifications

• Significant professional experience building server-side, infrastructure, distributed, or systems software.
• Strong Linux fundamentals and troubleshooting production systems.
• Professional experience with at least one systems-oriented language such as Rust, Go, C, or C++.
• Experience with distributed systems concepts such as load balancing, health checking, failure recovery, observability, and capacity management.
• Practical understanding of neural-network execution, including computation graphs, tensor shapes, data types, and accelerator execution.
• Experience profiling and optimizing production systems.
• Comfort working across languages and tooling, including Python for model export, validation, and integration workflows.
• Strong ownership, independent technical judgment, and clear written and verbal communication.
• The ability to use AI-assisted coding tools effectively while retaining the ability to explain, validate, debug, and modify the result independently.

Responsibilities

• Own the path from trained checkpoint to served request, including graph export, engine compilation, runtime integration, and serving configuration.
• Build and evolve Hume’s internal inference platform for serving multiple models across products and workloads.
• Design and operate serving infrastructure, including routing, load balancing, health checking, autoscaling, capacity management, and failure handling.
• Build reproducible, versioned inference artifacts and tooling for validation, deployment, promotion, and rollback.
• Build verification gates that catch numerical, behavioral, and performance regressions before production.
• Design and maintain internal client libraries and standardized serving contracts.
• Profile and optimize latency, throughput, memory usage, batching, scheduling, and accelerator utilization.
• Diagnose production issues across application, runtime, container, networking, driver, and hardware boundaries.
• Improve observability, resilience, testability, and operational safety across the inference stack.
• Write clear technical documentation for the systems and APIs you build.

Skills

Server-side engineering

Linux fundamentals

Rust/Go/C/C++

Distributed systems

Neural network execution

Profiling production systems

Python workflows

Ownership & communication

AI-assisted coding tools

Tools

Python

CI/CD

Containers

CUDA

ONNX

TensorRT

Job description

Hume AI is seeking a systems-oriented engineer to own the path from trained model to production inference in a high-performance environment in New York City. You will handle graph export, engine compilation, runtime integration, and production verification, collaborating with researchers and backend engineers to scale inference across products.

The role emphasizes performance, numerical correctness, reliability, and accelerator efficiency, with autonomy over lifecycle from design to deployment