Remote Part-Time Python AI Engineer — LLM Benchmarking

Jointaro; United States; On-site; USD 90,000 - 124,000; Full time

Job description

Handshake is seeking motivated Software Engineers to evaluate Large Language Models (LLMs) in collaboration with top AI labs.

As a Python Software Engineer, you will contribute to high-impact research collaborations focused on enhancing AI systems. This involves identifying gaps in modern LLMs and explaining how models fail. You will work with coding benchmarks that reflect real-world development across diverse languages and domains using Python.

Handshake AI projects are remote and part-time opportunities. Must be located in the US and have proper work authorization (OPT and H-1B not supported).

Key responsibilities include developing and validating coding benchmarks by curating issues, solutions, and test suites from real-world repositories, ensuring comprehensive unit and integration tests for solution verification, and maintaining the consistency and scalability of benchmark task distribution. You'll also provide structured feedback on solution quality, debug and optimize benchmark code, and document processes for reproducibility.

Shape the future of AI while building skills and confidence to navigate an AI-driven job market. This role will leverage your engineering expertise in a flexible environment, but you must be able to commit 15+ hours/week. Pay starts at $65/hour, plus bonuses for task completion.

Handshake isn't offering any visa sponsorship support at this time.