AI Model Evaluation Specialist

BAM Ventures; New York (NY); On-site; USD 100,000 - 130,000; Full time

Job summary

BAM Ventures is seeking an expert in AI model evaluation located in New York, NY. The candidate will design evaluation frameworks, conduct model assessments, and provide insights for improvements in AI models.

The ideal candidate holds a PhD or Masters in a relevant field and has over 5 years of experience in AI/ML model evaluation, particularly with statistical analysis and evaluation metrics. Excellent communication skills are essential for this role.

Qualifications

• 5+ years of experience in AI/ML model evaluation.
• Strong background in statistical analysis.
• Experience with evaluation frameworks and metrics.

Responsibilities

• Design and implement evaluation metrics for AI models.
• Conduct thorough testing of model performance across different scenarios.
• Analyze model outputs for bias, fairness, and accuracy.
• Collaborate with ML engineers to implement improvements.
• Document findings and recommendations.

Skills

Statistical analysis

AI/ML model evaluation

Evaluation frameworks and metrics

Communication skills

Education

PhD or Masters in Computer Science, ML, or related field

Job description

We are seeking an expert to evaluate and improve our AI models through comprehensive testing and analysis. You will be responsible for designing evaluation frameworks, conducting model assessments, and providing actionable insights for model improvement.

Key Responsibilities

• Design and implement evaluation metrics for AI models
• Conduct thorough testing of model performance across different scenarios
• Analyze model outputs for bias, fairness, and accuracy
• Collaborate with ML engineers to implement improvements
• Document findings and recommendations

Ideal Candidate

• PhD or Masters in Computer Science, ML, or related field
• 5+ years experience in AI/ML model evaluation
• Strong background in statistical analysis
• Experience with evaluation frameworks and metrics
• Excellent communication skills