AI Scenario Writer & Evaluation Engineer (Project-Based)

Mindrift; Remote; USD 10,000 - 60,000; Part time

Job summary

A tech-focused consultancy is looking for software engineers for project-based AI opportunities. Contributors will create test cases, analyze agent logs, and define evaluation criteria. This role requires 3+ years of software development experience with a strong focus on Python. The position is part-time and offers flexible hours, with payments up to $80/hour based on expertise. Ideal candidates will also have experience with Git, Docker, and structured formats like JSON/YAML.

Qualifications

• 3+ years of software development experience with strong Python focus.
• Experience with Git and code repositories.
• Comfortable working with structured formats like JSON/YAML.

Responsibilities

• Create structured test cases simulating complex human workflows.
• Analyze agent logs and decision paths.
• Iterate on prompts and test cases for clarity.

Skills

Software development experience

Strong Python focus

Experience with Git

Understanding LLM limitations

Familiarity with Docker

Comfortable with JSON/YAML

English proficiency - B2

Job description

A tech-focused consultancy is looking for software engineers for project-based AI opportunities. Contributors will create test cases, analyze agent logs, and define evaluation criteria. This role requires 3+ years of software development experience with a strong focus on Python. The position is part-time and offers flexible hours, with payments up to $80/hour based on expertise. Ideal candidates will also have experience with Git, Docker, and structured formats like JSON/YAML.