Machine Learning Engineering Evaluator

OpenTrain AI, Inc.; United States; Remote; USD 138,000 - 207,000; Part time

Benefits offered by this job

Remote work

Flexible schedule

Portfolio building

Job summary

OpenTrain is seeking a Machine Learning Engineering Evaluator for a part-time, fully remote contractor role around 15 hours per week. You will develop and review ML models, pipelines, and evaluation systems using Python, PyTorch, and JAX, delivering rigorous technical assessments and documented trade-offs.

Flexible weekends may be available. The ideal candidate holds a Masters or PhD in a quantitative field, has practical ML experience, and can build reproducible workflows while explaining

Qualifications

• Advanced degree in a quantitative field; strong ML background.
• Proven ability to build reproducible ML workflows.
• Experience with multiple ML frameworks and inference tools.
• Ability to diagnose and explain performance trade-offs clearly.

Responsibilities

• Develop and validate ML models, training pipelines, and inference systems.
• Build reproducible workflows using Python and command-line tools.
• Analyze tensor operations, model architectures, and generation pipelines.
• Optimize latency, memory usage, and hardware utilization.
• Review AI-generated code and evaluate technical solutions.
• Design objective tests, benchmarks, and evaluation criteria.
• Document implementation choices, trade-offs, and failure modes.

Skills

Python proficiency

Debug ML systems

Explain trade-offs

Communication

Education

Master's or PhD in CS/ML

Tools

PyTorch

JAX

NumPy

Hugging Face

Job description

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. Create a free profile, discover projects that match your expertise, and build a lasting portfolio of work that demonstrates your contribution to cutting-edge AI.

• Build a credible AI training and evaluation portfolio
• Find flexible remote opportunities aligned with your technical skills

About AI Training Work

AI training is the human side of building modern artificial intelligence. Specialists develop, review, and evaluate examples, code, model outputs, and technical systems so AI models can become more accurate, reliable, efficient, and useful.

• Work directly on advanced machine learning and software evaluation
• Help assess whether AI-generated technical solutions meet objective standards
• Contribute to a fast-growing field at the forefront of technology

The Role

OpenTrain is recruiting a Machine Learning Engineering Evaluator to create, solve, review, and validate demanding machine learning engineering tasks for an AI training project. The work spans model development, training and inference systems, numerical computing, performance optimization, Python workflows, and technical evaluation.

You will assess whether implementations satisfy requirements for correctness, reproducibility, efficiency, and performance. You will also document technical decisions, trade-offs, limitations, and failure modes clearly.

This is a global, fully remote contractor role requiring approximately 15 hours per week. Scheduling is flexible, including the option to work weekends. The listed compensation is $100-$150 per hour, and the role is available to candidates in the listed countries.

• Employment type: Part-time contractor
• Time commitment: Approximately 15 hours per week
• Schedule: Flexible, with weekend work available
• Compensation: $100-$150 per hour
• Language: English
• Work arrangement: Fully remote

What You'll Do

You will develop and validate machine learning models, training pipelines, inference systems, and supporting infrastructure. The work combines hands-on implementation with rigorous technical review and objective evaluation.

• Implement model components, data pipelines, evaluation systems, and numerical methods
• Build reproducible workflows using Python and command-line tools
• Work with tensor operations, automatic differentiation, model architectures, tokenization, batching, and generation
• Optimize latency, throughput, memory usage, and hardware utilization
• Diagnose numerical instability, tensor errors, memory bottlenecks, distributed-system failures, and performance regressions
• Review AI-generated code and technical solutions
• Design objective tests, benchmarks, and verification criteria
• Document implementation choices, trade-offs, limitations, and failure modes

Required Qualifications

This role requires advanced technical training and meaningful professional or research experience in machine learning. The listed experience level is entry level, but the stated educational and technical requirements apply.

• Master's degree or PhD in computer science, machine learning, artificial intelligence, applied mathematics, statistics, engineering, or a closely related quantitative discipline
• Strong professional or research experience in machine learning
• Practical proficiency with Python
• Experience building reproducible technical workflows
• Meaningful experience with at least two relevant machine learning frameworks, libraries, or inference tools
• Strong understanding of model training, evaluation, numerical computation, or inference
• Ability to debug machine learning systems beyond surface-level API usage
• Ability to explain implementation decisions, performance trade-offs, and failure modes clearly

Relevant Technical Tools

Experience may include the tools listed below. Equivalent tools and substantial open-source or academic experience may also qualify.

• PyTorch
• JAX
• NumPy
• SciPy
• SGLang
• vLLM
• llama.cpp
• Hugging Face Transformers
• Hugging Face Tokenizers

Why Work With OpenTrain

OpenTrain helps people start and grow careers in AI training and data labeling, including specialized technical evaluation. A stronger profile lets you showcase credible experience, discover projects that fit your skills, and develop a long-term portfolio in a rapidly expanding industry.

• Work remotely with a flexible schedule
• Use your machine learning engineering expertise on AI development tasks
• Build a portfolio around model, code, and systems evaluation