Senior AI Inference Systems Engineer

Hume AI; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

Hume AI in New York City seeks a Senior Software Engineer for the Data Plane team to build and optimize the on-premises inference and workflow platform powering large-scale AI workloads. You will own end-to-end system design, drive performance, reliability, and resource efficiency, mentor teammates, collaborate with research and product, and document architectures for production readiness.

This role offers strong ownership, opportunities to shape high-impact systems, and close collaboration with

Qualifications

• Experience building server-side or systems software.
• Strong Linux fundamentals and production troubleshooting skills.
• Background in distributed systems, networking, or concurrency.
• Proficiency in Rust, C, or C++ for systems work.
• Ability to own and ship complex designs independently.

Responsibilities

• Design, build, and maintain data plane software for AI model execution and workflows.
• Develop capabilities like declarative workflow execution and tracing.
• Integrate models and runtimes while maintaining performance and reliability.
• Diagnose production issues across application, OS, containers, networks, and hardware.
• Profile and optimize latency, throughput, and resource usage.
• Improve observability, resilience, and safety of the platform.
• Collaborate with research and product teams to productionize capabilities.
• Write clear technical documentation for systems and features.

Skills

Server-side software

Linux fundamentals

Distributed systems

Networking

Filesystems

Concurrency

Resource management

Computer architecture

Rust/C/C++

Ownership & independence

Documentation

AI-assisted coding

Tools

PyTorch

CUDA

ONNX

TensorRT

Docker

CI/CD pipelines

Job description

Hume AI in New York City seeks a Senior Software Engineer for the Data Plane team to build and optimize the on-premises inference and workflow platform powering large-scale AI workloads. You will own end-to-end system design, drive performance, reliability, and resource efficiency, mentor teammates, collaborate with research and product, and document architectures for production readiness.

This role offers strong ownership, opportunities to shape high-impact systems, and close collaboration with