Applied Scientist II, LLM Quality & Measurement (Shopping)

Socket.dev; Seattle (WA); On-site; USD 120,000 - 180,000; Full time

Job summary

Socket.dev in Seattle is seeking an engineer to own an LLM-based measurement pipeline for CX, extending the framework to the homepage and detail page and building the measurement from scratch. You will turn perception questions into measurable artifacts and create scalable tooling to support evaluation at scale.

The role focuses on reusable labeling pipelines, evaluation frameworks, and inference infrastructure that can be adopted by teams handling model judgments and ground truth at scale,

Job description

Socket.dev in Seattle is seeking an engineer to own an LLM-based measurement pipeline for CX, extending the framework to the homepage and detail page and building the measurement from scratch. You will turn perception questions into measurable artifacts and create scalable tooling to support evaluation at scale.

The role focuses on reusable labeling pipelines, evaluation frameworks, and inference infrastructure that can be adopted by teams handling model judgments and ground truth at scale,