Remote Java Engineer: AI Prompts & Code Evaluation

Crossing Hurdles; Netherlands (MO); Remote; USD 80,000 - 120,000; Full time

Job summary

A technology company is seeking a skilled software engineer to craft developer prompts and evaluate AI outputs. The role requires strong experience in software engineering and technical writing, along with a Bachelor's degree in a related field. The ideal candidate will work independently in a fully remote environment, contributing to the development of evaluation datasets and collaborating with research teams to ensure high-quality tasks.

Qualifications

• Strong experience in software engineering, technical research, or educational content development.
• Hands-on experience with debugging, testing, and validating code.
• Strong technical writing skills with high attention to detail.

Responsibilities

• Craft realistic, chat-style developer prompts covering various scenarios.
• Source real pull requests to create evaluation tasks.
• Write clear, accurate model responses demonstrating reasoning.

Skills

Software engineering expertise

Technical writing

Education

Bachelor’s degree in Software Engineering

Job description

• Craft realistic, chat-style developer prompts covering code review, debugging, error diagnosis, configuration, and related scenarios.
• Source and adapt real pull requests and engineering discussions to create authentic evaluation tasks.
• Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality.
• Evaluate AI outputs based on logic, clarity, and technical judgment rather than executable correctness.
• Apply software engineering expertise to ensure prompts and responses reflect real-world developer workflows.
• Contribute to structured evaluation datasets used to assess AI reasoning and communication quality.
• Collaborate asynchronously with research teams to refine task quality and consistency.

Requirements

• Strong experience in software engineering, technical research, or educational content development.
• Bachelor’s degree in Software Engineering, Computer Science, or a related field.
• Hands-on experience with debugging, testing, and validating code.
• Strong technical writing skills with high attention to detail.
• Ability to work independently in a fully remote, project-based environment.