Member of Technical Staff, Frontier AI

SaidGig; New York (NY); Remote; USD 180,000 - 320,000; Full time

Job summary

SaidGig is seeking a highly analytical researcher to lead end-to-end ML evaluation initiatives in a fully remote, full-time capacity in the United States. You will design evaluation frameworks, build data systems, and work with cross-functional teams to translate research insights into deployed improvements.

Ideal candidates bring experience in reinforcement learning environments, complex evaluation design, and the ability to operate with ambiguity while owning outcomes.

Qualifications

• Strong judgment about research-signal quality and readiness for external use.
• Experience designing ML-oriented datasets, evaluation frameworks, and quality-assurance processes.
• Ability to turn messy real-world system behavior into structured research and evaluation opportunities.

Responsibilities

• Lead research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
• Design ML-focused data systems, including task definitions, annotation schemas, rubrics, incentives, and pipelines that support downstream model performance.
• Investigate model and system failures to identify root causes, edge cases, and improvement opportunities.
• Convert ambiguous real-world behavior into structured evaluation frameworks and new data categories.
• Partner with researchers, domain experts, and operators to calibrate quality early, strengthen research signal, and translate findings into deployed-system improvements.
• Iterate quickly on evaluations, datasets, and feedback loops to improve performance.
• Serve as a quality gate by challenging claims, pausing work, or changing scope when data integrity or signal strength is insufficient.
• Work with cross-functional and client-facing teams to communicate research progress through clear, evidence-based narratives.
• Identify gaps in data and evaluation coverage, then recommend where to invest, iterate, or stop based on impact and learnings.

Skills

Research design

ML evaluation

Data pipelines

Ambiguity tolerance

Communication

Stakeholder collaboration

Systems thinking

Tools

Python

ML frameworks

Job description

Key Responsibilities

• Lead research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
• Design ML-focused data systems, including task definitions, annotation schemas, rubrics, incentives, and pipelines that support downstream model performance.
• Investigate model and system failures to identify root causes, edge cases, and improvement opportunities.
• Convert ambiguous real-world behavior into structured evaluation frameworks and new data categories.
• Partner with researchers, domain experts, and operators to calibrate quality early, strengthen research signal, and translate findings into deployed-system improvements.
• Iterate quickly on evaluations, datasets, and feedback loops to improve performance.
• Serve as a quality gate by challenging claims, pausing work, or changing scope when data integrity or signal strength is insufficient.
• Work with cross-functional and client-facing teams to communicate research progress through clear, evidence-based narratives.
• Identify gaps in data and evaluation coverage, then recommend where to invest, iterate, or stop based on impact and learnings.

Qualifications

• Strong judgment about research-signal quality and readiness for external use.
• Experience designing ML-oriented datasets, evaluation frameworks, and quality-assurance processes.
• Ability to turn messy real-world system behavior into structured research and evaluation opportunities.
• Comfort operating in ambiguity, with decisive ownership.
• Clear written and verbal communication for explaining tradeoffs, limitations, and signal strength to technical and non-technical stakeholders.
• Demonstrated ability to work directly with experts during project kickoff, calibration, and iteration.
• A systems-level mindset focused on end-to-end model or agent performance.

Preferred Experience

• Reinforcement learning environments, simulators, or feedback-driven training systems.
• Improving agentic AI systems or AI systems used in real-world workflows.
• Applied research or production work with direct impact on deployed systems.
• Evaluation design for complex or real-world tasks.
• Expert incentive design and engagement in high-stakes technical projects.

Work Terms

Full-time, remote role.

Compensation

Stated annual compensation range: $600, 000 to $2, 000, 000.Stated national base-salary range for this full-time position: $180, 000 to $320, 000 annually.