Remote Member of Technical Staff - AI Systems & Evaluation

SaidGig; New York (NY); Remote; USD 180,000 - 320,000; Full time

Job summary

SaidGig is seeking a highly analytical researcher to lead end-to-end ML evaluation initiatives in a fully remote, full-time capacity in the United States. You will design evaluation frameworks, build data systems, and work with cross-functional teams to translate research insights into deployed improvements.

Ideal candidates bring experience in reinforcement learning environments, complex evaluation design, and the ability to operate with ambiguity while owning outcomes.

Qualifications

• Strong judgment about research-signal quality and readiness for external use.
• Experience designing ML-oriented datasets, evaluation frameworks, and quality-assurance processes.
• Ability to turn messy real-world system behavior into structured research and evaluation opportunities.

Responsibilities

• Lead research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
• Design ML-focused data systems, including task definitions, annotation schemas, rubrics, incentives, and pipelines that support downstream model performance.
• Investigate model and system failures to identify root causes, edge cases, and improvement opportunities.
• Convert ambiguous real-world behavior into structured evaluation frameworks and new data categories.
• Partner with researchers, domain experts, and operators to calibrate quality early, strengthen research signal, and translate findings into deployed-system improvements.
• Iterate quickly on evaluations, datasets, and feedback loops to improve performance.
• Serve as a quality gate by challenging claims, pausing work, or changing scope when data integrity or signal strength is insufficient.
• Work with cross-functional and client-facing teams to communicate research progress through clear, evidence-based narratives.
• Identify gaps in data and evaluation coverage, then recommend where to invest, iterate, or stop based on impact and learnings.

Skills

Research design

ML evaluation

Data pipelines

Ambiguity tolerance

Communication

Stakeholder collaboration

Systems thinking

Tools

Python

ML frameworks

Job description

SaidGig is seeking a highly analytical researcher to lead end-to-end ML evaluation initiatives in a fully remote, full-time capacity in the United States. You will design evaluation frameworks, build data systems, and work with cross-functional teams to translate research insights into deployed improvements.

Ideal candidates bring experience in reinforcement learning environments, complex evaluation design, and the ability to operate with ambiguity while owning outcomes.