Backend Engineer — Model Inference

Best AI Tools Wiki; Toronto; Hybrid; CAD 110,000 - 190,000; Full time

Benefits offered by this job

Premium health and dental

Remote-first culture

Home office budget

Annual learning stipend

Flexible PTO

Job summary

Cohere is hiring a Backend Engineer to optimize model inference systems and build low-latency serving infrastructure that powers the enterprise API.

You will develop backend services in Go, Rust or C++, focusing on high-throughput performance, model serving, and inference optimization using gRPC/REST and microservices. This is a remote-friendly role within a well-funded AI startup.

The position offers premium health and dental, a home office budget, an annual learning stipend and flexible PTO.

Qualifications

• 5+ years of backend engineering experience.
• Proficiency in Go, Rust or C++.
• Experience with high-throughput, low-latency systems.
• Knowledge of model serving and inference optimization.
• Experience with gRPC, REST APIs, and microservices.

Responsibilities

• Build low-latency serving infrastructure for enterprise API.
• Develop backend services powering model inference and optimization.
• Improve throughput and latency of serving pipelines.
• Collaborate on system design and API interfaces.

Skills

Backend engineering

Go

Rust

C++

High-throughput systems

Model serving

gRPC

REST APIs

Microservices

Job description

Cohere is hiring a Backend Engineer to optimize model inference systems. You will build low-latency serving infrastructure, implement batching strategies, and develop the backend services that power Cohere's enterprise API.

Requirements

• 5+ years of backend engineering experience
• Strong proficiency in Go, Rust, or C++
• Experience with high-throughput, low-latency systems
• Knowledge of model serving and inference optimization
• Experience with gRPC, REST APIs, and microservices

Nice to Have

• Experience with vLLM, TensorRT-LLM, or similar
• Familiarity with GPU memory management
• Experience with continuous batching techniques

Equity in a well-funded AI startup

Premium health and dental

Remote-first culture

Home office budget

Annual learning stipend

Flexible PTO

Skills

Go Rust Model Serving Backend Engineering gRPC Inference Optimization

This listing was posted on 2026-02-17 and is almost certainly closed. It is kept for reference — check the employer’s own careers page for current openings.

Vincony has all 400+ AI models in one place — compare responses, AI debate, Image/Video/Voice generator, and 20 more tools to help you learn and build with AI.