Senior Software Engineer – LLM Evaluation

Jobgether SRL; United States; A distancia; USD 55.000 - 96.000; A tiempo parcial

Ventajas ofrecidas por este puesto de trabajo

Fully remote

Part-time

Independent contractor

Flexible schedule

1-month project

International collaboration

No health benefits

Descripción de la vacante

Jobgether SRL is seeking an experienced software- engineering professional for a fully remote, part-time independent contractor role. You will curate datasets, evaluate AI-generated code, and develop evaluation frameworks across multiple languages and lifecycles.

The role emphasizes rigorous technical evaluation, cross-functional collaboration, and producing concise rationales while maintaining high standards of software quality. International remote engagement is supported.

Formación

• 3+ years of professional software-engineering experience.
• Strong full-stack development capabilities and production-grade software.
• Strong understanding of software architecture, system design, API design, and production implementation.
• Deep knowledge of software development, debugging, code review, and code-quality assessment.
• Demonstrated experience reviewing, troubleshooting, and improving complex software implementations.
• Proficiency in one or more relevant programming languages, including Python, JavaScript, Java, C++, Rust, or similar technologies.
• Experience with ReactJS, C, Go, or additional programming languages is valuable depending on project requirements.
• Familiarity with software monitoring, operational maintenance, and production reliability.
• Ability to reason across the full software-engineering lifecycle and assess technical decisions from development through ongoing operation.
• Strong analytical and problem-solving abilities, with a rigorous and detail-oriented approach to technical evaluation.
• Excellent written and verbal communication skills, including the ability to produce concise, well-structured evaluation rationales.
• Ability to distinguish between technically correct solutions and implementations that may introduce scalability, reliability, maintainability, or architectural concerns.
• Comfortable working independently and collaborating remotely with research, engineering, and other technical teams.
• Based in Canada, the United States, or an eligible Western European country.
• Willingness to complete a required AI video interview as part of the application process.

Responsabilidades

• - Curate high-quality software examples and datasets for model training, coding benchmarks, and technical evaluation.
• - Develop precise solutions to software-engineering tasks and correct or improve implementations across multiple programming languages.
• - Evaluate AI-generated code for technical correctness, maintainability, efficiency, scalability, reliability, and alignment with professional engineering standards.
• - Identify implementation weaknesses, recurring model-generated errors, and patterns that reveal gaps in AI coding capabilities.
• - Provide clear, structured rationales supporting technical evaluation decisions.
• - Build agents and automated mechanisms capable of assessing code quality and verifying software solutions consistently.
• - Develop reliable checks that enable reproducible evaluation across repeated engineering assignments.
• - Assess model capabilities across the complete software-development lifecycle, including prototyping, architecture, API design, implementation, experimentation, launch, monitoring, and maintenance.
• - Review AI reasoning and technical decisions against realistic production engineering expectations.
• - Collaborate with research and cross-functional technical teams to define evaluation strategies and improve coding-focused benchmarks.
• - Contribute to datasets used for training and benchmarking while maintaining rigorous standards for technical accuracy and quality.
• - Support iterative improvements to evaluation systems by comparing model performance against real-world software-engineering practices.
• - Complete all project work without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any third party.

Conocimientos

Full-stack

Software architecture

API design

Code review

Debugging

Python

JavaScript

Java

C++

Rust

Remote collaboration

Herramientas

ReactJS

C

Go

Descripción del empleo

Accountabilities:

- Curate high-quality software examples and datasets for model training, coding benchmarks, and technical evaluation.

- Develop precise solutions to software-engineering tasks and correct or improve implementations across multiple programming languages.

- Evaluate AI-generated code for technical correctness, maintainability, efficiency, scalability, reliability, and alignment with professional engineering standards.

- Identify implementation weaknesses, recurring model-generated errors, and patterns that reveal gaps in AI coding capabilities.

- Provide clear, structured rationales supporting technical evaluation decisions.

- Build agents and automated mechanisms capable of assessing code quality and verifying software solutions consistently.

- Develop reliable checks that enable reproducible evaluation across repeated engineering assignments.

- Assess model capabilities across the complete software-development lifecycle, including prototyping, architecture, API design, implementation, experimentation, launch, monitoring, and maintenance.

- Review AI reasoning and technical decisions against realistic production engineering expectations.

- Collaborate with research and cross-functional technical teams to define evaluation strategies and improve coding-focused benchmarks.

- Contribute to datasets used for training and benchmarking while maintaining rigorous standards for technical accuracy and quality.

- Support iterative improvements to evaluation systems by comparing model performance against real-world software-engineering practices.

- Complete all project work without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any third party.

Requirements:

- 3+ years of professional software-engineering experience.

- Strong full-stack development capabilities and experience building scalable, production-grade software.

- Strong understanding of software architecture, system design, API design, and production implementation.

- Deep knowledge of software development, debugging, code review, and code-quality assessment.

- Demonstrated experience reviewing, troubleshooting, and improving complex software implementations.

- Proficiency in one or more relevant programming languages, including Python, JavaScript, Java, C++, Rust, or similar technologies.

- Experience with ReactJS, C, Go, or additional programming languages is valuable depending on project requirements.

- Familiarity with software monitoring, operational maintenance, and production reliability.

- Ability to reason across the full software-engineering lifecycle and assess technical decisions from development through ongoing operation.

- Strong analytical and problem-solving abilities, with a rigorous and detail-oriented approach to technical evaluation.

- Excellent written and verbal communication skills, including the ability to produce concise, well-structured evaluation rationales.

- Ability to distinguish between technically correct solutions and implementations that may introduce scalability, reliability, maintainability, or architectural concerns.

- Comfortable working independently and collaborating remotely with research, engineering, and other technical teams.

- Based in Canada, the United States, or an eligible Western European country.

- Willingness to complete a required AI video interview as part of the application process.

Benefits:

• - Fully remote, part-time independent contractor engagement.
• - Flexible workload starting at 10 hours per week , with the potential to work up to 40 hours per week .
• - Approximately one-month initial project duration , with potential extension based on performance and project fit.
• - Opportunity to contribute directly to advanced LLM evaluation, coding benchmarks, and AI-assisted software-engineering research.
• - Exposure to cutting-edge AI evaluation workflows and research involving real-world software-development scenarios.
• - Opportunity to apply professional engineering expertise to improve how AI systems are evaluated across the software lifecycle.
• - Flexible consulting structure suited to experienced engineers seeking project-based work.
• - International remote collaboration with research and technical professionals.
• - No medical insurance or paid-leave benefits are included under the independent contractor arrangement.
• - Compensation is project-specific and was not specified in the available job materials.