Edge AI Inference Engineer - On-Device ML (Llama/ONNX)

Tether.io; United States; Remote; USD 110,000 - 170,000; Full time

Job summary

Tether is hiring an AI engineer to work on the C++ layer powering local AI, porting and enhancing inference engines for edge devices. You will optimize loading, runtime efficiency, and cross-hardware performance to enable private on-device AI without cloud reliance.

Collaborate with researchers to move models from research to production, deploy ML models to edge devices, and integrate AI features into existing products with the latest ML advances.

Qualifications

• Experience with Llama.cpp and ggml inference engines.
• Understanding of deep learning concepts and model architectures.
• Experience with transformers and LLMs.
• Degree in Computer Science, AI, ML, or related field.

Responsibilities

• Work on deploying machine learning models to edge devices using the frameworks: llama.cpp, ggml, ONNX
• Collaborate closely with researchers to assist in coding, training and transitioning models from research to production environments
• Integrate AI features into existing products, enriching them with the latest advancements in machine learning

Skills

Llama.cpp

ggml

Transformers

LLMs

Education

CS/AI degree or related field

Tools

llama.cpp

ggml

ONNX

Job description

Tether is hiring an AI engineer to work on the C++ layer powering local AI, porting and enhancing inference engines for edge devices. You will optimize loading, runtime efficiency, and cross-hardware performance to enable private on-device AI without cloud reliance.

Collaborate with researchers to move models from research to production, deploy ML models to edge devices, and integrate AI features into existing products with the latest ML advances.