Lead AI Inference Systems Engineer

Hume AI; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Hume AI in New York City is seeking a systems-oriented engineer to own the path from trained models to production inference. You will manage graph export, engine compilation, runtime integration, and serving contracts to ensure reliable, scalable inference.

You will work with research scientists, ML engineers, backend engineers, and the Data Plane team to deploy new models, optimize performance, and maintain observability and safety across the stack.

Qualifications

• Significant professional experience building server-side, infrastructure, distributed, or systems software.
• Strong Linux fundamentals and production troubleshooting.
• Experience with a systems-oriented language such as Rust, Go, C, or C++.
• Understanding of distributed systems concepts like load balancing, health checks, observability, and capacity management.
• Practical understanding of neural-network execution, including graphs and tensor shapes.
• Experience profiling and optimizing production systems.
• Proficiency with Python for model export, validation, and integration workflows.
• Strong ownership, independent judgment, and clear communication.

Responsibilities

• Own the path from trained checkpoint to served requests, including graph export and engine integration.
• Build and evolve the internal inference platform serving multiple models across products.
• Design and operate serving infrastructure including routing, load balancing, health checks, and autoscaling.
• Create versioned inference artifacts and tooling for validation, deployment, promotion, and rollback.
• Implement verification gates to detect numerical, behavioral, and performance regressions before production.
• Develop internal client libraries and standardized serving contracts.
• Profile and optimize latency, throughput, memory usage, batching, and accelerator utilization.
• Diagnose production issues across application, runtime, container, networking, driver, and hardware boundaries.
• Improve observability, resilience, testability, and safety of the inference stack.
• Write clear technical documentation for systems and APIs.

Skills

Systems software

Linux fundamentals

Rust/Go/C/C++

Distributed systems

Python for ML workflows

Ownership & communication

AI-assisted coding tools

Tools

CUDA

ONNX

TensorRT

Docker

Job description

Hume AI in New York City is seeking a systems-oriented engineer to own the path from trained models to production inference. You will manage graph export, engine compilation, runtime integration, and serving contracts to ensure reliable, scalable inference.

You will work with research scientists, ML engineers, backend engineers, and the Data Plane team to deploy new models, optimize performance, and maintain observability and safety across the stack.