AI Systems Product Lead: Quality, Evaluation & Delivery

Axe Recruiting; United States; Hybrid; USD 160.000 - 260.000; Vollzeit

Zusammenfassung

Axe Recruiting presents a deeply technical product leadership role focused on defining how an AI system should behave, be evaluated and improved. You will work at the intersection of user needs, model capabilities, and engineering support, requiring close collaboration with ML and engineering teams to deliver correct, reliable, and trusted AI-powered experiences.

The role demands hands-on experience with LLM-based products, strong CS fundamentals, and a track record of delivering technical

Qualifikationen

• Strong computer science fundamentals: algorithms, data structures and system design.
• Solid grasp of modern ML and production behavior of AI systems.
• Ability to read and review technical designs, understand architectures and work deeply with engineers.
• Sharp instincts for model limitations, hallucination, bias, drift, evaluation and failure modes.
• Substantial experience owning complex technical products end-to-end with senior engineers and ML teams.
• Hands-on experience with AI-powered products, especially LLM-based systems, including evaluation, experimentation and feedback loops.
• Ability to turn ambiguous problems into clear, measurable outcomes and executable plans.

Aufgaben

• Write end-to-end requirements for AI capabilities, covering model and system behavior and user experience.
• Translate evaluation results and constraints into clear product decisions.
• Decide trade-offs among quality, cost, latency, safety, reliability and UX.
• Collaborate with ML, backend and client engineers on design, evaluation and iteration.
• Build and refine evaluation framework with offline metrics, online experiments and human feedback.
• Set explicit quality bars and feedback loops for AI-powered experiences.
• Drive delivery with clear specs, prioritization and sound technical judgment.
• Identify failure modes and ensure system predictability and graceful recovery.

Kenntnisse

CS fundamentals

ML systems

Architectural reading

Design review with engineers

End-to-end product ownership

Hands-on with LLM-based products

Requirements to outcomes

Decision making under constraints

Jobbeschreibung

Axe Recruiting presents a deeply technical product leadership role focused on defining how an AI system should behave, be evaluated and improved. You will work at the intersection of user needs, model capabilities, and engineering support, requiring close collaboration with ML and engineering teams to deliver correct, reliable, and trusted AI-powered experiences.

The role demands hands-on experience with LLM-based products, strong CS fundamentals, and a track record of delivering technical