Remote AI Inference Engineer: Edge & Performance

ITRex Group; United States; Remote; USD 120,000 - 170,000; Full time

Benefits offered by this job

Remote flexibility

Competitive salary + medical benefits

Learning opportunities

Job summary

ITRex is seeking a strong C++ Engineer to deploy and optimize modern AI models for production, focusing on edge devices and high-performance inference.

You will integrate, profile, and optimize AI inference pipelines using llama.cpp, ggml, and related frameworks, collaborating with researchers to move models from research to production.

Remote-friendly role across the US; you’ll contribute to edge deployments, performance tuning, and memory management in distributed systems.

Qualifications

• 4+ years of professional experience with Modern C++ (C++17/20).
• Strong memory management, multithreading, profiling and perf optimization skills.
• Experience debugging memory leaks, fragmentation, OOM and concurrency issues.
• Experience Linux development environments and production deployments.

Responsibilities

• Deploy ML models to edge devices and optimize inference pipelines.
• Collaborate with researchers to move models from research to production.
• Integrate AI features into existing products and evolve AI capabilities in production environment.

Skills

Modern C++

Multithreading

Profiling

Linux

Memory management

OOP/low-level debugging

Tools

llama.cpp

ggml

ONNX Runtime

TensorRT

OpenVINO

MLC LLM

ExecuTorch

TVM

Job description

ITRex is seeking a strong C++ Engineer to deploy and optimize modern AI models for production, focusing on edge devices and high-performance inference.

You will integrate, profile, and optimize AI inference pipelines using llama.cpp, ggml, and related frameworks, collaborating with researchers to move models from research to production.

Remote-friendly role across the US; you’ll contribute to edge deployments, performance tuning, and memory management in distributed systems.