Software and ML Engineer, LLM Systems Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 152,000 - 193,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Software and ML Engineer for LLM systems evaluation on a remote, part-time basis. You will craft technical scenarios, create reference solutions, and provide detailed feedback to evaluate AI models, focusing on GPU performance, model-serving systems, and quantization.

The role emphasizes defining what is correct and efficient work, rather than delivering production software, with an hourly pay of $125 and a flexible schedule.

Qualifications

• Bachelor's degree or higher in CS/CE/EE or related field.
• Hands-on GPU inference optimization, CUDA kernel development, model-serving systems, quantization, or optimizer design.
• Experience owning systems-engineering work at a tech/AI company.
• Strong written communication for technical explanations and feedback.

Responsibilities

• Design realistic scenarios and problem sets for LLM inference optimization, GPU kernel design, model-serving systems, and systems performance engineering.
• Write reference solutions that explain expert reasoning, implementation tradeoffs, and technical standards.
• Grade model outputs for correctness, performance, and architectural quality.
• Write technical explanations and feedback that distinguish acceptable answers from excellent ones.

Skills

GPU performance

Model serving

CUDA kernels

Quantization

Systems engineering

Education

Bachelor's degree in Computer Science / Computer Engineering / Electrical Engineering

Tools

CUDA

Job description

Software and ML Engineer, LLM Systems Evaluation

Evaluate advanced LLM inference and systems-engineering problems at $125 per hour on a flexible, part-time schedule. Create technical scenarios, reference solutions, and clear feedback for AI model evaluation.

Remote Hourly · $125/hr

$125/hr

Compensation

Contract, part-time

Engagement

Remote

Location

Open in 95 countries

United States + more

• Albania
• Algeria
• Antigua and Barbuda
• Armenia
• Australia
• Austria
• Bahamas
• Bahrain
• Belgium
• Benin
• Bhutan
• Bosnia and Herzegovina
• Botswana
• Brunei
• Bulgaria
• Canada
• Costa Rica
• Côte d’Ivoire
• Croatia
• Cyprus
• Czechia
• Denmark
• Ecuador
• El Salvador
• Estonia
• Ethiopia
• Finland
• France
• Gambia
• Germany
• Greece
• Guatemala
• Guyana
• Hong Kong
• Hungary
• Iceland
• India
• Indonesia
• Ireland
• Israel
• Italy
• Jamaica
• Jordan
• Kenya
• Kuwait
• Latvia
• Liechtenstein
• Lithuania
• Luxembourg
• Madagascar
• Malaysia
• Malta
• Mauritius
• Mexico
• Moldova
• Mongolia
• Morocco
• Mozambique
• Namibia
• Netherlands
• New Zealand
• North Macedonia
• Norway
• Oman
• Pakistan
• Panama
• Peru
• Philippines
• Poland
• Portugal
• Qatar
• Romania
• Rwanda
• Saint Lucia
• Senegal
• Serbia
• Singapore
• Slovakia
• Slovenia
• South Africa
• Spain
• Sri Lanka
• Sweden
• Switzerland
• Taiwan
• Tanzania
• Thailand
• Trinidad and Tobago
• Tunisia
• Türkiye
• United Arab Emirates
• United Kingdom
• United States
• Uzbekistan
• Vietnam

The work

The title and description of this role name different fields. Check the duties and qualifications below before you apply.

This contract role focuses on evaluating AI models on advanced LLM inference and systems-engineering problems. You will define what correct, efficient, and well-reasoned technical work looks like rather than deliver production software.

Your work will cover GPU performance, model serving, custom kernels, quantization, and related systems engineering. You will create technical content and assess model responses using your hands‑on expertise.

• Design realistic scenarios and problem sets for LLM inference optimization, GPU kernel design, model‑serving systems, and systems performance engineering.
• Write reference solutions that explain expert reasoning, implementation tradeoffs, and technical standards.
• Grade model outputs for correctness, performance, and architectural quality.
• Write technical explanations and feedback that distinguish acceptable answers from excellent ones.

What it pays and takes

This is a remote, part‑time contractor role with a flexible schedule.

• Pay: $125 per hour.
• Time: 20 or more hours per week.
• Location: Open to candidates in the countries listed for this role.
• Education: A bachelor's degree or higher in computer science, computer engineering, electrical engineering, or a related field.
• Experience: Hands‑on work with GPU inference optimization, custom CUDA kernel development, model‑serving systems, quantization, or optimizer design.
• Professional background: Experience owning systems‑engineering work at a recognized technology or AI company.
• Communication: Strong written communication for technical explanations, reference solutions, and model feedback.

Helpful technical background

The following experience is helpful for this work, but is not listed as required: SGLang, Mamba or Mamba2 architectures, IBM Granite‑family models, open-source inference or serving frameworks, distributed inference, continuous batching, and KV-cache optimization.

• SGLang or other open‑source inference and serving frameworks.
• Mamba or Mamba2 architectures.
• IBM Granite‑family models.
• Distributed inference and continuous batching.
• KV-cache optimization.

About AI training work

AI training is the human work behind modern AI systems, including writing examples, rating model responses, and checking technical outputs. People with deep specialist knowledge help set reliable standards and explain why one answer is stronger than another.