Tooling Engineer Business Analysis AI Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 41,000 - 83,000; Part time

Benefits offered by this job

Remote work

Per-task payment

Job summary

OpenTrain AI, Inc. is seeking a Tooling Engineer focused on Business Analysis and AI evaluation. The role is remote and part-time, requiring a master’s degree and at least three years of experience.

Applicants will craft challenging prompts, test model responses, and provide detailed, evidence-based feedback to improve AI performance. You will evaluate language models used for business-analysis tasks, ensure factual accuracy, and identify weaknesses through rigorous testing and edge cases.

Qualifications

• Master's degree or higher in a relevant field.
• At least three years of professional, research, or teaching experience.
• Part-time remote contract requiring 20+ hours per week.
• Ability to write demanding prompts, test edge cases, and review AI responses.
• Provide evidence-based feedback and communicate findings clearly.

Responsibilities

• Write complex business-analysis prompts to test language-model capabilities.
• Review AI-generated responses for factual accuracy and reasoning quality.
• Identify factual errors, logical gaps, and knowledge gaps.
• Create difficult test cases to expose limits in model performance.

Skills

Business-analysis

Analytical thinking

Critical thinking

Education

Master's degree or higher in a relevant field

Job description

Tooling Engineer Business Analysis AI Evaluation

Evaluate large language models with business-analysis expertise by writing demanding prompts, testing edge cases, and reviewing AI responses. This remote contract role requires a relevant master's degree, three years of experience, and 20+ hours per week.

100% Remote

Contract, part-time

The Work

You will help evaluate and improve large language models used for business-analysis tasks. The work focuses on written prompts, AI-generated responses, test cases, and detailed feedback.

You will use business-analysis knowledge and careful judgment to find factual and reasoning problems that may affect model reliability.

• Write complex, business-analysis prompts that test language-model capabilities.
• Review AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Find factual errors, logical inconsistencies, knowledge gaps, and other weaknesses.
• Create difficult test cases and edge cases that expose limits in model performance.
• Explain your findings clearly and provide evidence-based feedback for model improvement.

What It Pays And Takes

The role is a remote contract position for an individual contributor. It is marked as part-time and requires at least 20 hours each week.

You should be able to make careful judgments, communicate precisely in writing, and challenge assumptions while spotting subtle factual or reasoning errors.

• Pay: Not provided in the listing.
• Schedule: Part-time, with a requirement of 20+ hours per week.
• Education: A master's degree or higher in a relevant field.
• Experience: At least three years of professional, research, or teaching experience.
• Core skills: Business-analysis knowledge, strong analytical and critical-thinking skills, and exceptional attention to detail.
• Helpful background: Experience with large language models, generative AI, prompt engineering, AI evaluation, model testing, difficult prompts, edge cases, or stress-testing AI models.
• Create demanding questions, review AI-generated responses, and explain errors using authoritative sources. This part-time contract pays $30 to $60 per hour equivalent, paid per task that meets the project specifications.
• Review AI conversations, identify missing context, and assess whether recommendations use enough information. This remote project runs for about three weeks, requires 20+ hours per week, and pays per task.
• Remote · Albania, Algeria, Angola +128 more
• Create research-backed questions and answers, test AI model responses, and improve evaluation content remotely for $20 to $60 per hour equivalent, paid per task.
• Create difficult, evidence-based question-and-answer materials for advanced AI model evaluation. Earn $30 to $60 per hour equivalent, paid per task that meets the project specifications, with fewer than 20 hours per week.

About AI Training Work

AI training is the human work behind systems that learn from examples and feedback. People write prompts, review model responses, and explain errors so AI systems can become more accurate and reliable.

People with strong subject knowledge are valuable because they can identify subtle mistakes and judge whether an answer is sound, relevant, and well reasoned.

Requirements

• Experience: Entry level