Staff Engineer, LLM Evaluation & Production

Hark; San Jose (CA); On-site; USD 180,000 - 450,000; Full time

Job summary

Hark is engineering the next generation of AI experiences and seeks an engineer who sits at the intersection of model research and production systems. You will help define what “better” means for a model and build infrastructure that makes those proofs reliable and repeatable, including evaluation harnesses, data pipelines, and scalable training setups.

As a hands-on individual contributor, you’ll own the systems that judge model improvements and the environments that scale them, working closely

Qualifications

• 3+ years in software engineering with hands-on LLM training/evaluation or data infrastructure.
• Deep Python fluency and distributed-systems experience across a fleet of machines.
• Experience building evaluation infrastructure, not just running benchmarks.
• Working knowledge of SFT and RL for language models.
• Production instincts: tests, reproducibility, and researcher tooling.

Responsibilities

• Build and own end-to-end evaluation suites for LLM capabilities, including task specs and dashboards.
• Run the eval harness as a production service with sandboxed execution, tool calls, multi-turn tasks, deterministic replay, and caching.

Skills

Python

Distributed systems

Evaluation infrastructure

Reproducibility

LLM training

Tools

PyTorch

JAX

vLLM

SGLang

Job description

Hark is engineering the next generation of AI experiences and seeks an engineer who sits at the intersection of model research and production systems. You will help define what “better” means for a model and build infrastructure that makes those proofs reliable and repeatable, including evaluation harnesses, data pipelines, and scalable training setups.

As a hands-on individual contributor, you’ll own the systems that judge model improvements and the environments that scale them, working closely