Edge-Optimized AI Inference Kernel Engineer

Framework Ventures; United States; Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Job summary

Framework Ventures is seeking an experienced AI model serving engineer to advance inference pipelines across edge and cloud environments in the United States. You will design high‑throughput, low‑latency deployments for multi‑modal models and research novel serving strategies.

You will collaborate with cross‑functional teams to benchmark performance, optimize memory usage, and push the boundaries of real‑world AI capabilities on resource‑constrained devices.

Qualifications

• Degree in Computer Science or related field; PhD in NLP/ML preferred.
• Proven experience in low‑level kernel and inference optimization on mobile devices.
• Strong understanding of modern model serving architectures and inference optimization techniques.
• Experience writing GPU kernels for mobile devices and end‑to‑end inference pipelines.
• Experience with empirical research and evaluation frameworks for optimization.

Responsibilities

• Design and deploy state‑of‑the‑art model serving architectures with high throughput and low latency; optimize memory usage across diverse environments.
• Build, run, and monitor controlled inference tests in simulated and live production environments; track latency, throughput, and memory metrics.
• Identify and prepare high‑quality test datasets and scenarios for real‑world deployment challenges on low‑resource devices.
• Analyze computational efficiency and diagnose bottlenecks in the serving pipeline; optimize for scalability and reliability on constrained systems.
• Collaborate with cross‑functional teams to integrate optimized serving frameworks into edge/on‑device production pipelines; define success metrics.

Education

PhD in NLP / Machine Learning

BSc/ MSc in Computer Science

Tools

Metal Shading Language (MSL)

GPU kernels (mobile)

Inference optimization

Model serving architectures

Job description

Framework Ventures is seeking an experienced AI model serving engineer to advance inference pipelines across edge and cloud environments in the United States. You will design high‑throughput, low‑latency deployments for multi‑modal models and research novel serving strategies.

You will collaborate with cross‑functional teams to benchmark performance, optimize memory usage, and push the boundaries of real‑world AI capabilities on resource‑constrained devices.