Senior Java Code Reviewer for AI Evaluation

OpenTrain AI; Hinoba-an; On-site; PHP 1,805,000 - 2,492,000; Part time

Job summary

OpenTrain AI is seeking a senior Java code reviewer to audit AI-generated Java code, validate compliance with prompts, and ensure performance and security standards. You will correct annotator evaluations and provide precise, actionable feedback within a structured QA workflow.

The role requires 7+ years in Java, deep JVM expertise, and excellent written English. This contractor position supports a distributed team and emphasizes quality and reproducibility in review processes.

Qualifications

• 7+ years of professional Java development, QA, or dedicated code-review experience.
• Deep expertise in Java 11–21, core APIs, JVM internals, and performance tuning.
• Advanced testing and debugging experience.
• Proven ability to analyze multithreaded and concurrent code.
• Strong secure-coding knowledge, including OWASP risks and permission flaws.
• Experience compiling and running code in sandbox or container environments.
• Familiarity with rubric-based scoring, checklist-driven reviews, Jira, Asana, or annotation tools.
• Excellent written English at B2 CEFR level or above.
• Ability to provide concise feedback and guide peers on best practices.

Responsibilities

• Review AI-generated Java code responses
• Audit and correct annotator evaluations
• Compile and run code in isolated sandbox or container environments
• Validate prompt compliance, functionality, security, and performance
• Provide concise, constructive written feedback

Skills

Java expertise

Code review

QA experience

English writing

Tools

JUnit 5

Maven/Gradle

Docker

Job description

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the leading platform for finding and building careers in AI training and data labeling, connecting skilled contributors with projects that shape how modern AI systems work.

Create a free OpenTrain account to build your profile and apply in minutes. This contractor position is available to candidates in India, the Philippines, and Vietnam.

• Contractor and part-time engagement
• 20+ hours per week
• Pay: $25 USD per hour
• English-language work
• Eligible locations: India, the Philippines, and Vietnam

About AI Training and Code Evaluation

AI training is the human side of building artificial intelligence. Specialists review examples, evaluate model responses, and provide structured feedback so AI systems can become more accurate, reliable, secure, and useful.

In this project, your Java expertise will help assess AI-generated programming responses. Your reviews will support the quality of datasets used to train next-generation AI models.

• Work on cutting-edge AI development
• Apply professional software engineering judgment to model outputs
• Help improve correctness, security, and performance in AI-generated code

The Role

OpenTrain AI is seeking a senior Java code reviewer with at least seven years of hands-on experience in modern Java, professional QA, or dedicated code-review roles. You will audit annotator evaluations of AI-generated Java code and determine whether each snippet satisfies its prompt, works correctly, follows secure-coding practices, and meets performance expectations.

The work requires clear written English and disciplined participation in a structured quality-assurance workflow. You will correct inaccurate ratings, provide targeted feedback, and ensure reviews align with the project quality rubric.

• Review AI-generated Java code responses
• Audit and correct annotator evaluations
• Compile and run code in isolated sandbox or container environments
• Validate prompt compliance, functionality, security, and performance
• Provide concise, constructive written feedback

What You'll Evaluate

You will use deep knowledge of modern Java and the JVM to assess implementation quality. Reviews may involve core APIs, standard-library usage, JVM behavior, concurrency, testing strategy, build configuration, and secure coding.

• Modern Java versions 11 through 21
• Standard library and core APIs
• JVM internals, garbage collection, JIT behavior, and profiling
• Thread safety, locks, CompletableFuture, and reactive streams
• JUnit 5, TestNG, Testcontainers, Mockito, and JaCoCo
• Maven, Gradle, Dockerized builds, and CI/CD pipelines
• GitHub or GitLab pull requests and code-review workflows
• OWASP Top 10 issues, deserialization attacks, injection, race conditions, and permission flaws

Requirements

This role is intended for an experienced Java professional who can independently validate code behavior and explain review decisions precisely. You should be comfortable working from rubrics, checklists, and structured annotation or ticketing workflows.

• 7+ years of professional Java development, QA, or dedicated code-review experience
• Deep expertise in Java 11 through 21, core APIs, JVM internals, and performance tuning
• Advanced testing and debugging experience
• Proven ability to analyze multithreaded and concurrent code
• Strong secure-coding knowledge, including OWASP risks and permission flaws
• Experience compiling and running code in sandbox or container environments
• Familiarity with rubric-based scoring, checklist-driven reviews, Jira, Asana, or annotation tools
• Excellent written English at B2 CEFR level or above
• Ability to provide concise feedback and guide peers on best practices

Preferred Experience

Experience with AI evaluation, reinforcement learning from human feedback pipelines, or AI and machine-learning data-labeling projects is welcome. It is listed as a nice-to-have rather than a core requirement.

• LLM evaluation experience
• RLHF pipeline experience
• Previous AI or machine-learning data-labeling work