Remote LLM Systems Evaluation Engineer (Part-Time)

OpenTrain AI, Inc.; United States; Remote; USD 152,000 - 193,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Software and ML Engineer for LLM systems evaluation on a remote, part-time basis. You will craft technical scenarios, create reference solutions, and provide detailed feedback to evaluate AI models, focusing on GPU performance, model-serving systems, and quantization.

The role emphasizes defining what is correct and efficient work, rather than delivering production software, with an hourly pay of $125 and a flexible schedule.

Qualifications

• Bachelor's degree or higher in CS/CE/EE or related field.
• Hands-on GPU inference optimization, CUDA kernel development, model-serving systems, quantization, or optimizer design.
• Experience owning systems-engineering work at a tech/AI company.
• Strong written communication for technical explanations and feedback.

Responsibilities

• Design realistic scenarios and problem sets for LLM inference optimization, GPU kernel design, model-serving systems, and systems performance engineering.
• Write reference solutions that explain expert reasoning, implementation tradeoffs, and technical standards.
• Grade model outputs for correctness, performance, and architectural quality.
• Write technical explanations and feedback that distinguish acceptable answers from excellent ones.

Skills

GPU performance

Model serving

CUDA kernels

Quantization

Systems engineering

Education

Bachelor's degree in Computer Science / Computer Engineering / Electrical Engineering

Tools

CUDA

Job description

OpenTrain AI, Inc. is seeking a Software and ML Engineer for LLM systems evaluation on a remote, part-time basis. You will craft technical scenarios, create reference solutions, and provide detailed feedback to evaluate AI models, focusing on GPU performance, model-serving systems, and quantization.

The role emphasizes defining what is correct and efficient work, rather than delivering production software, with an hourly pay of $125 and a flexible schedule.