Python Software Engineer

Jointaro; United States; Remote; USD 90,000 - 124,000; Part time

Job summary

Handshake is seeking motivated Software Engineers to evaluate Large Language Models (LLMs) in collaboration with top AI labs. As a Python Software Engineer, you will contribute to high-impact research collaborations focused on enhancing AI systems. This involves identifying gaps in modern LLMs and explaining how models fail, using Python to build and validate benchmarks.

Handshake projects are remote and part-time opportunities. Must be located in the US and have proper work authorization.

Qualifications

• Proficient Python for ML/AI tooling and benchmarks.
• Experience evaluating LLMs and explaining model failures.
• Ability to commit 15+ hours/week and work remotely from the US.

Responsibilities

• Develop and validate coding benchmarks by curating issues, solutions, and test suites from real-world repositories.
• Ensure comprehensive unit and integration tests for solution verification.
• Maintain consistency and scalability of benchmark task distribution.
• Provide structured feedback on solution quality and debug/optimize benchmark code.
• Document processes for reproducibility.

Skills

Python

Unit testing

Integration testing

Benchmarking

Debugging

Job description

Handshake is seeking motivated Software Engineers to evaluate Large Language Models (LLMs) in collaboration with top AI labs.

As a Python Software Engineer, you will contribute to high-impact research collaborations focused on enhancing AI systems. This involves identifying gaps in modern LLMs and explaining how models fail. You will work with coding benchmarks that reflect real-world development across diverse languages and domains using Python.

Handshake AI projects are remote and part-time opportunities. Must be located in the US and have proper work authorization (OPT and H-1B not supported).

Key responsibilities include developing and validating coding benchmarks by curating issues, solutions, and test suites from real-world repositories, ensuring comprehensive unit and integration tests for solution verification, and maintaining the consistency and scalability of benchmark task distribution. You'll also provide structured feedback on solution quality, debug and optimize benchmark code, and document processes for reproducibility.

Shape the future of AI while building skills and confidence to navigate an AI-driven job market. This role will leverage your engineering expertise in a flexible environment, but you must be able to commit 15+ hours/week. Pay starts at $65/hour, plus bonuses for task completion.

Handshake isn't offering any visa sponsorship support at this time.