AI Data Delivery Lead: Automation & QA Orchestrator

OpenKrill; USA; Remote; USD 120.000 - 180.000; Vollzeit

Zusammenfassung

OpenKrill is seeking an engineer to design and run AI data projects and to build the automation that powers quality management, project management, and data analytics. You will pair deterministic tooling with LLM agents, calibrate QA checks, and own end-to-end data collection and reporting workflows.

You’ll work with Claude and our enterprise agentic environment to implement task specs, data connections, and monitoring, while documenting runbooks for operational teams.

Qualifikationen

• Intermediate JavaScript and Python for production automation.
• Hands-on agent building in Claude and enterprise environment.
• LLM evaluation: prompt design, rubric calibration, human judgement checks.
• Data analytics experience in data collection or annotation.
• Experience managing projects with external vendors or contributors.

Aufgaben

• Build hybrid automation combining deterministic scripts with LLM agents.
• Set up and run agents in Claude and our enterprise agentic environment.
• Design QA workflows, rubrics, and scorecards; calibrate automated checks against human reviewers.
• Run data collection and annotation projects end to end with tooling for coordinators.
• Collect, clean, and analyze production and quality data and turn into actionable reporting.
• Verify everything by testing automation and building continuous auditing workflows.
• Document specs and runbooks to enable others to operate and extend the system.

Kenntnisse

JavaScript

Python

Claude agent building

LLM evaluation

Data analytics

Vendor management

Decision making

Tools

Claude

Jobbeschreibung

OpenKrill is seeking an engineer to design and run AI data projects and to build the automation that powers quality management, project management, and data analytics. You will pair deterministic tooling with LLM agents, calibrate QA checks, and own end-to-end data collection and reporting workflows.

You’ll work with Claude and our enterprise agentic environment to implement task specs, data connections, and monitoring, while documenting runbooks for operational teams.