Embedded AI Engineer — On-Device Models & Edge Optimizations

Fdetimes; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Deepgram is seeking an Embedded AI Engineer to push models to edge devices, writing kernels for embedded CPUs/NPUs and optimizing quantization, memory layouts, and runtimes. You will partner with silicon vendors and Applied ML to land models on constrained hardware while maintaining latency and power targets.

You’ll work near the metal on edge platforms, using C/C++/Rust and vendor toolchains, contributing to low-latency, production-grade voice AI workloads across diverse hardware.

Qualifications

• Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices.
• Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
• Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation.
• Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
• A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance.
• Experience working close to the metal: bare-metal or RTOS environments, embedded Linux, or microcontroller and edge SoC development.
• Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly.

Responsibilities

• Write and optimize custom kernels and operators for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs.
• Own target-side optimization: collapse models onto device execution units through quantization and fusion.
• Integrate with vendor NPU/DSP toolchains and edge runtimes, extending them with custom operators.
• Deliver kernels and runtime components as reusable building blocks for Applied ML Engineers.
• Build performance-critical runtime code for embedded environments and RTOS targets.
• Establish per-platform benchmarking and validation for latency, power, and memory.
• Partner with silicon and platform vendors on SDK integration and low-level performance tuning.
• Feed hardware constraints back to model designers to ease landing on constrained targets.

Skills

C

C++

Rust

Embedded systems

Edge AI

Quantization

Performance optimization

Hardware knowledge

Tools

ONNX Runtime

TensorRT

TFLite

Job description

Deepgram is seeking an Embedded AI Engineer to push models to edge devices, writing kernels for embedded CPUs/NPUs and optimizing quantization, memory layouts, and runtimes. You will partner with silicon vendors and Applied ML to land models on constrained hardware while maintaining latency and power targets.

You’ll work near the metal on edge platforms, using C/C++/Rust and vendor toolchains, contributing to low-latency, production-grade voice AI workloads across diverse hardware.