Remote Senior Software Engineer - 34123

Turing; Toronto; Remote; CAD 90,000 - 130,000; Part time

Job summary

A rapidly growing AI company is seeking a Senior Software Engineer to join their LLM evaluation team. In this contract role, you'll analyze GitHub issues, set up code repositories, and evaluate how LLMs interact with real code. Your contribution will influence the future of AI-assisted software development, all while working remotely on cutting-edge projects with the potential for leadership responsibilities.

Benefits

Remote work environment

Opportunity to work on cutting-edge AI projects

20 hours/week commitment with some PST overlap

Contractor role

1 month contract duration

Qualifications

• Experience with well-maintained, widely-used repositories with 500+ stars required.
• Comfort with running, modifying, and testing projects locally.
• Experience in LLM research or evaluation projects is a plus.

Responsibilities

• Analyzing and triaging GitHub issues in trending open-source libraries.
• Setting up and configuring code repositories, including Dockerization.
• Evaluating unit test coverage and quality.

Skills

Proficiency with Git

Docker

Software pipeline setup

Navigating complex codebases

Open-source contribution

Job description

1 day ago Be among the first 25 applicants

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation

About the projects:

We are building LLM evaluation and training datasets to train LLMs to work on realistic software engineering problems. Our approach involves creating verifiable SWE tasks based on public repository histories in a synthetic manner with human-in-the-loop, expanding dataset coverage across different programming languages and difficulty levels.

About the Role:

We seek experienced software engineers (tech lead level) familiar with high-quality public GitHub repositories to contribute to this project. Experience with well-maintained, widely-used repositories with 500+ stars is required. The role includes hands-on software engineering tasks such as automating development environments, issue triaging, and evaluating test coverage and quality.

Why Join Us?

Turing is a rapidly growing AI company advancing AI system deployment. You will evaluate how LLMs interact with real code, influencing AI-assisted software development's future. This role combines practical software engineering with AI research.

Day-to-day responsibilities include:

• Analyzing and triaging GitHub issues in trending open-source libraries.
• Setting up and configuring code repositories, including Dockerization and environment setup.
• Evaluating unit test coverage and quality.
• Modifying and running codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborating with researchers to identify challenging repositories and issues for LLMs.
• Potentially leading a team of junior engineers.

Required Skills:

• Experience with well-maintained, widely-used repositories with 500+ stars.
• Proficiency with Git, Docker, and software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfort with running, modifying, and testing projects locally.
• Open-source contribution or evaluation experience is a plus.

Note: Experience with repositories having 500+ stars is mandatory.

Nice to Have:

• Participation in LLM research or evaluation projects.
• Experience with developer tools or automation agents.

Perks of Freelancing With Turing:

• Remote work environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.
• 20 hours/week commitment with some overlap with PST.
• Contractor role (no medical/paid leave).
• Contract duration: 1 month, starting next week.

Seniority level

• Mid-Senior level

Employment type

• Contract

Job function

• Engineering and Information Technology

Industries

• Software Development

Referrals increase your chances of interviewing at Turing by 2x.

Get notified about new Software Engineer jobs in Toronto, Ontario, Canada.