Senior Software Engineer – LLM Evaluation (Fully Remote)

Partner Company; United States; Remote; USD 83,000 - 165,000; Part time

Benefits offered by this job

Remote-friendly

Flexible hours

Contractor-friendly

Job summary

Partner Company in the United States seeks a Senior Software Engineer focused on LLM evaluation and AI-assisted software engineering. You will curate code samples and datasets for benchmarking and collaborate with researchers across evaluation strategies.

This fully remote, part-time independent contractor role offers flexible workload from 10 to 40 hours per week and the chance to shape evaluation workflows throughout the SDLC.

Qualifications

• 3+ years of professional software-engineering experience.
• Strong full-stack development and software architecture skills.
• Experience evaluating AI systems or LLMs is a plus.

Responsibilities

• Curate high-quality code examples and datasets for model training and benchmarking.
• Evaluate AI-generated code for correctness, maintainability, and scalability.
• Build agents and automated mechanisms to verify code quality across the SDLC.

Skills

LLM evaluation

Full-stack development

Software architecture

Job description

• Senior Software Engineer focused on LLM evaluation and AI-assisted software engineering.
• Collaborate with research and technical professionals on evaluation strategies.

Responsibilities:

• Curate high-quality code examples and datasets for model training and benchmarking.
• Evaluate AI-generated code for correctness, maintainability, and scalability.
• Build agents and automated mechanisms to verify code quality across the SDLC.

Requirements:

• 3+ years of professional software-engineering experience.
• Strong full-stack development and software architecture skills.
• Fully remote, part-time independent contractor engagement.
• Flexible workload from 10 to 40 hours per week.

Partner Company

This partner company specializes in evaluating large language models and improving AI systems through rigorous engineering benchmarks. It offers a remote, collaborative culture where engineers and researchers advance AI evaluation workflows together.