Job Details
- Pay: 50–100/hour
- Role Type: Contractor
- Location: Anywhere in the world, remote
- Estimated Time Commitment: Approximately 15 hours per task run
- Start Timeline: Within 24–48 hours of onboarding
Requirements
- Strong software engineering experience in one or more programming languages, including Python, Java, C++, Go, Rust, or TypeScript.
- Demonstrated experience developing backend systems, scalable services, APIs, or complex software applications.
- A clear track record of open-source contributions, preferably supported by a GitHub profile or equivalent public portfolio.
- Strong understanding of data structures, algorithms, software design principles, and debugging techniques.
- Practical experience with automated testing, code validation, and software quality assurance.
- Ability to identify and resolve complex bugs, implementation errors, and performance bottlenecks.
- Experience developing new features and improving existing codebases.
- Strong analytical thinking, problem‑solving, documentation, and technical communication skills.
- Ability to work independently, follow detailed technical specifications, and deliver reproducible, verifiable solutions.
- Interest in AI systems and understanding how software engineering expertise can support AI model training and evaluation.
Key Responsibilities
- Create realistic, technically challenging programming tasks involving bug fixes, feature development, code refactoring, and performance optimization.
- Build deterministic reinforcement learning environments with the necessary dependencies, test suites, configurations, and execution instructions.
- Develop reliable golden reference solutions that demonstrate the expected behavior and satisfy all task requirements.
- Design deterministic evaluators and test cases that accept valid implementations while rejecting incorrect, incomplete, or non‑compliant solutions.
- Investigate complex software defects, troubleshoot unexpected behavior, and resolve performance bottlenecks across diverse codebases.
- Implement robust and scalable features while following established software development principles and coding standards.
- Improve existing codebases by enhancing readability, maintainability, execution efficiency, and long‑term reliability.
- Review code contributions and technical submissions for correctness, clarity, performance, and adherence to best practices.
- Document problem statements, implementation decisions, debugging strategies, testing procedures, and solution reasoning to support high‑quality AI training and evaluation.
- Refine tasks and evaluation criteria based on observed model behavior and identified weaknesses.
Must‑Have Skills
- Software Engineering
- Open Source Contributions
- Backend Development and API Design
- Python, Java, C++, Go, Rust, or TypeScript
- Debugging and Troubleshooting
- Data Structures and Algorithms
- Automated Testing and Code Validation
- Code Review and Refactoring
- Performance Optimization
Good‑to‑Have Skills
- Experience with large‑scale or legacy codebases
- Reinforcement Learning Environment Development
- Deterministic Testing and Automated Evaluation
- CI/CD and Development Tooling
- Scalable Distributed Systems
- Performance Profiling and Optimization
- Complex Feature Implementation
- AI Model Evaluation and Coding Agent Assessment
- Public GitHub repositories or notable open‑source projects
Opportunity Snapshot
- Time Commitment: Approximately 15 hours per task run, depending on complexity and individual workflow.
- Flexible Schedule: Work with flexibility while meeting task requirements and weekly submission minimums.
Compensation
Payment is based on completed tasks that meet the specified quality and acceptance criteria.
Expected Availability
The selection process may move quickly, with positions often filled within approximately 48 hours. Successful candidates should be prepared to begin their first tasks within 24–48 hours after completing onboarding.