Senior Data Plane Engineer, On-Prem AI Inference

Hume AI, Inc.; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Hume AI, Inc. in New York City seeks a Senior Software Engineer for the Data Plane team to build an on-premises inference and workflow platform for large-scale AI workloads.

You will collaborate with backend engineers, researchers, ML engineers, product managers, and infrastructure engineers to deploy new models and ensure reliability, performance, and efficient resource usage across the system lifecycle.

Qualifications

• Significant professional experience building server-side or systems software.
• Strong Linux fundamentals and ability to troubleshoot production systems.
• Experience in distributed systems, networking, filesystems, concurrency, or resource management.

Responsibilities

• Design, build, and maintain the data plane software that executes AI models and workflows.
• Develop capabilities like declarative workflow execution and parallel processing.
• Integrate models and runtimes while maintaining performance and reliability.
• Diagnose production issues across OS, containers, networks, and hardware.
• Profile and optimize latency, throughput, and memory usage.
• Improve observability, resilience, and safety of the platform.
• Collaborate with research and product teams to productionize model capabilities.
• Write clear technical documentation for systems and features.

Skills

Server-side software

Linux fundamentals

Distributed systems

Networking

Concurrency

Ownership

C/C++/Rust

Documentation

Tools

PyTorch

CUDA

ONNX

TensorRT

CI/CD

Job description

Hume AI, Inc. in New York City seeks a Senior Software Engineer for the Data Plane team to build an on-premises inference and workflow platform for large-scale AI workloads.

You will collaborate with backend engineers, researchers, ML engineers, product managers, and infrastructure engineers to deploy new models and ensure reliability, performance, and efficient resource usage across the system lifecycle.