Senior Tech Lead, On-Device AI Inference

Hark; San Jose, Northern (CA, KY); Hybrid; USD 300,000 - 500,000; Full time

Job summary

Hark is seeking a senior hardware/software leader to own how our models run on silicon. You will select accelerators, shape architectures within latency, memory, and power budgets, and build the low-level inference stack that enables millisecond responses on battery-powered devices.

You will lead a team of engineers, partner with silicon vendors, and drive efficient transformer execution on new accelerators from research to product.

Qualifications

• 8–12+ years in high-performance computing with production workloads on GPUs or accelerators.
• Deep understanding of attention, KV-cache, quantization, and memory bandwidth limits.
• You've designed or optimized inference engines, runtimes, or ML compilers and write kernels when needed.
• You've led teams on performance-critical software and set architectural direction.

Responsibilities

• Evaluate GPUs, NPUs, DSPs, and accelerators for on-device deployment and advocate for hardware decisions.
• Collaborate with foundation model and audio ML teams to meet deployment constraints before training.
• Build low-level execution layer, kernels, runtimes, and compiler paths for transformer workloads on target hardware.
• Partner with silicon vendors and internal teams to bring up accelerators and optimize transformer execution.
• Hire and lead engineers, setting the technical bar for the inference stack.

Skills

High-performance computing

Team leadership

ML inference

GPU/accelerator knowledge

Performance optimization

Tools

TensorRT

ONNX Runtime

TVM

MLIR

Job description

Hark is seeking a senior hardware/software leader to own how our models run on silicon. You will select accelerators, shape architectures within latency, memory, and power budgets, and build the low-level inference stack that enables millisecond responses on battery-powered devices.

You will lead a team of engineers, partner with silicon vendors, and drive efficient transformer execution on new accelerators from research to product.