Remote Senior Software Engineer - 34123

Turing; San Francisco (CA); Remote; USD 200,000 - 250,000; Part time

Job summary

A leading AI company is seeking experienced software engineers to contribute to LLM evaluation and training datasets. This role involves hands-on tasks like analyzing GitHub issues, setting up code repositories, and collaborating with researchers. Join Turing to work on cutting-edge AI projects and influence the future of AI-assisted software development.

Benefits

Remote work environment

Opportunity to work on cutting-edge AI projects

Potential contract extension based on performance

Qualifications

• Experience with high-quality public GitHub repositories.
• Hands-on software engineering tasks including environment automation.
• Experience evaluating test coverage and quality.

Responsibilities

• Analyze and triage GitHub issues in trending open-source libraries.
• Set up and configure code repositories, including Dockerization.
• Evaluate unit test coverage and quality.

Skills

Familiarity with C++

Familiarity with Java

Familiarity with Go

Proficiency with Git

Proficiency with Docker

Ability to understand complex codebases

Comfort with testing real-world projects

Experience with open-source projects

Job description

We are building LLM evaluation and training datasets to train LLMs to work on realistic software engineering problems. One approach involves creating verifiable SWE tasks based on public repository histories, using a synthetic approach with human-in-the-loop, and expanding dataset coverage across various programming languages and difficulty levels.

About the Role:

We seek experienced software engineers (tech lead level) familiar with high-quality public GitHub repositories to contribute to this project. Candidates should have experience with well-maintained, widely-used repositories with 500+ stars. The role involves hands-on software engineering tasks such as environment automation, issue triaging, and evaluating test coverage and quality.

Why Join Us?

Turing is a rapidly growing AI company advancing the deployment of powerful AI systems. You will evaluate how LLMs interact with real code, influencing the future of AI-assisted software development. This position offers a unique blend of practical software engineering and AI research.

Day-to-Day Responsibilities:

1. Analyze and triage GitHub issues in trending open-source libraries.
2. Set up and configure code repositories, including Dockerization and environment setup.
3. Evaluate unit test coverage and quality.
4. Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
5. Collaborate with researchers to identify challenging repositories and issues for LLMs.
6. Potentially lead a team of junior engineers on projects.

Required Skills:

• Familiarity with other major languages (e.g., C++, Java, Go) is a plus.
• Proficiency with Git, Docker, and software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfort with running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is advantageous.

Nice to Have:

• Previous participation in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.

Perks of Freelancing With Turing:

• Remote work environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.
• Potential contract extension based on performance.
• Commitment: 20 hours/week, some overlap with PST.
• Employment: Contractor position (no medical/paid leave).
• Duration: 1 month, starting next week.

Seniority Level

• Entry level

Employment Type

• Contract

Job Function

• Engineering and Information Technology

Industry

• Software Development