Senior Software Engineer | $300/task

Codefeast; India; Remote; INR 689,000 - 2,066,000; Part time

Job summary

Codefeast is seeking an experienced machine learning/AI professional to design, review, and validate terminal-based tasks used to evaluate AI agents. You will develop realistic ML workflows, configure reproducible environments, and prepare datasets and evaluation inputs for rigorous testing.

Must have hands-on experience with Terminal-Bench 3.0 projects and a strong background in ML/AI, CV, NLP, or Data Science. Remote work with a flexible schedule is offered for a contractor role.

Qualifications

• Bachelor's degree from a university ranked among top 100 institutions in the country.
• Minimum 6 years of relevant ML/AI or related professional experience.
• Prior hands-on experience with Terminal-Bench 3.0 projects.
• Background in ML, AI, Computer Vision, NLP, Data Science, or Statistics.

Responsibilities

• Design multi-step tasks based on realistic ML and AI workflows.
• Configure reproducible computational environments with pinned dependencies.
• Prepare datasets, model configurations, and evaluation inputs.
• Implement reference solutions and automated validation tests.
• Review model outputs, evaluation metrics, and numerical correctness.
• Debug dependency conflicts, execution failures, and computational issues.
• Improve task reliability, reproducibility, and evaluation quality.
• Document assumptions, limitations, and expected outputs.

Skills

Machine Learning

AI

Data Processing

Experimentation

Model Evaluation

Terminal-Bench

Python

NLP

Education

Bachelor's degree

Job description

Experience: 6+ Years of Professional Machine Learning or AI Experience

Job Type: Contractor

Location: Remote

Required Profiles: LinkedIn profile required to apply

Role Overview

We are looking for experienced machine learning and AI professionals to develop, review, and validate terminal-based tasks used to evaluate AI agents. You’ll apply your expertise in model development, data processing, experimentation, and evaluation to create realistic and reproducible machine learning workflows.

Important: Prior hands-on experience working on Terminal-Bench 3.0 projects is mandatory.

Key Responsibilities

• Design multi-step tasks based on realistic machine learning and AI workflows.
• Configure reproducible computational environments with pinned dependencies.
• Prepare datasets, model configurations, and evaluation inputs.
• Implement reference solutions and automated validation tests.
• Review model outputs, evaluation metrics, and numerical correctness.
• Debug dependency conflicts, execution failures, and computational issues.
• Improve task reliability, reproducibility, and evaluation quality.
• Document assumptions, limitations, and expected outputs.

Key Requirements

• Bachelor's degree from a university ranked among the top 100 institutions in the country.
• Minimum 6 years of relevant machine learning, AI, or related professional experience.
• Prior hands-on experience with Terminal-Bench 3.0 projects.
• Relevant background in Machine Learning, AI, Computer Vision, NLP, Data Science, or Statistics.
• Practical experience implementing and validating computational workflows.
• Familiarity with automated testing and reproducible experimentation.
• Comfortable discussing technical workflows in an interview.
• Availability to work remotely on a flexible schedule.

Preferred Technical Skills

• Python and machine learning frameworks.
• Model training, inference, data preprocessing, and evaluation.
• NLP, LLMs, computer vision, or applied AI.
• MLOps, experiment tracking, and dependency management.
• AI agent evaluation, benchmark development, or automated grading.