Senior Software Engineer, LLM Evaluation & Benchmark Remote

24-Mag Llc; New York (NY); Remote; USD 14,000 - 55,000; Part time

Job summary

24-MAG LLC invites experienced Senior Software Engineers to join a part-time, remote consulting engagement focused on evaluating AI-generated software, code benchmarking, and research-driven verification. You will curate code, craft reference solutions, and build automated assessment mechanisms across multiple languages and platforms.

The role requires deep software-engineering expertise, strong communication, and the ability to work with research teams across time zones.

Qualifications

• Several years of professional software-engineering experience.
• At least 2 years of continuous full-time experience at a top-tier product company.
• Strong full-stack development capabilities.
• Experience building scalable, production-grade software.
• Deep understanding of software architecture and system design.
• Strong knowledge of debugging and code-quality assessment.
• Experience reviewing and improving complex production code.
• Proficiency in one or more of Python, JavaScript, C/C++, Java, Rust, or Go.
• ReactJS experience is valuable for frontend-oriented assignments.
• Strong understanding of API design and production implementation.
• Familiarity with software monitoring and operational maintenance.
• Ability to reason across the complete software-engineering lifecycle.
• Strong analytical and problem-solving capabilities.
• Excellent written and verbal communication skills.
• Ability to produce clear and structured evaluation rationales.
• Comfortable collaborating with research and technical teams in a remote environment.

Responsibilities

• Code Curation & Solution Development: Curate high-quality code examples for model training and benchmarking.
• Develop precise reference solutions to software-engineering tasks.
• Correct and improve code across multiple programming languages.
• Work with Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go.
• Maintain strong standards for correctness, clarity, and maintainability.

Skills

Full-stack development

Production-grade software

Software architecture

Debugging & code-quality

API design

Remote collaboration

Python

JavaScript

C/C++

Java

Rust

Go

Tools

ReactJS

Job description

24-MAG LLC invites experienced Senior Software Engineers to join a part-time, remote consulting engagement focused on evaluating AI-generated software, code benchmarking, and research-driven verification. You will curate code, craft reference solutions, and build automated assessment mechanisms across multiple languages and platforms.

The role requires deep software-engineering expertise, strong communication, and the ability to work with research teams across time zones.