AI Evaluation Analyst

AI Trainer Jobs; Deutschland; Remote; EUR 24.000 - 35.000; Vollzeit

Zusammenfassung

micro1 is seeking an AI Evaluation Analyst contractor to contribute to a customer project focused on advancing frontier language model capabilities. You will apply your expertise to help train next-generation AI systems, shaping how models learn, reason, and perform through high-quality input.

You will author detailed, multi-turn conversations and evaluation rubrics, test drafts against frontier models, and deliver artifacts with strict fidelity to project specs, operating independently with

Qualifikationen

• Native-level written English with clarity, structure, and detail
• Experience in data annotation, RLHF, SFT, evaluation or prompt engineering
• Knowledge of frontier LLM behaviors and failure patterns
• Ability to interpret and apply detailed specifications
• Strong analytical and writing-focused critical thinking
• Experience authoring evaluation items, rubrics, or analysis of outputs
• Background in research, editorial, or quality assurance is a plus

Aufgaben

• Author detailed, task-based multi-turn conversations and rubrics aligned with project specifications.
• Test and refine conversation drafts against frontier large language models, iterating to meet quality and difficulty requirements.
• Deliver comprehensive evaluation assets including transcripts, target behaviors, binary rubrics, and supporting evidence.
• Ensure strict fidelity to evolving project specs while maintaining high throughput and attention to detail.
• Validate and calibrate outputs with team leads and quality control as guidelines change.
• Work independently and consistently, meeting expected output rates for deliverable completion.

Kenntnisse

Frontier LLM behavior

Data Annotation

Written English clarity

Spec fidelity

Self-direction

Jobbeschreibung

AI Evaluation Analyst
$20 - $30/hour
pay
Required Skills
Working knowledge of frontier LLM behavior
Data Annotation
Written English clarity and structure
Spec fidelity at volume
Self-direction to a detailed spec
About micro1
micro1 is the leading AI data lab for training frontier models and evaluating AI agents. Experts contribute their diverse subject matter knowledge across domains such as finance, healthcare, STEM engineering, and more. micro1 transforms that real-world expertise into high-quality training data, evaluations, and feedback loops that improve how AI systems learn, reason, and perform.

Our platform identifies and vets top talent through an AI recruiter, enabling high-quality expert contributions at scale. We aim to enable 1 billion people to do meaningful work by applying their expertise to AI. As our global expert network grows, micro1 is building the human intelligence layer for frontier AI.

Role Title: AI Evaluation Analyst

Role Type: Contractor

Location: Remote

micro1 is engaging AI Evaluation Analysts to contribute to a customer’s project focused on advancing frontier language model capabilities. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. You will play a crucial role in producing evaluation and training data that directly influence how powerful AI models understand and interact. This is an exciting opportunity to operate at the forefront of AI, with a focus on written and verbal clarity, multi-turn conversation design, and deep analysis of model behavior.

Scope of Work

Author detailed, task-based multi-turn conversations and rubrics aligned with project specifications.
Test and refine conversation drafts against frontier large language models, iterating to meet quality and difficulty requirements.
Deliver comprehensive evaluation assets including transcripts, target behaviors, binary rubrics, and supporting evidence.
Ensure strict fidelity to evolving project specs while maintaining high throughput and attention to detail.
Validate and calibrate outputs with team leads and quality control as guidelines change.
Work independently and consistently, meeting expected output rates for deliverable completion.

Preferred Qualifications

Native-level written English with exceptional clarity, structure, and attention to detail.
Prior experience in data annotation, RLHF, SFT, evaluation, or prompt engineering for AI systems.
Working knowledge of frontier LLM behaviors and common model failure patterns.
Demonstrated ability to interpret and apply highly detailed specifications without supervision.
Strong critical thinking and analytical skills in writing-heavy or analysis-heavy domains.
Experience authoring evaluation items, rubrics, or conducting deep analysis of technology outputs.
Background in research, editorial, technical writing, or quality assurance is a plus.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.

Interested?
First name
Last name
Email
Phone number
Afghanistan
Åland Islands
Albania
Algeria
American Samoa
Andorra
Angola
Anguilla
Antigua and Barbuda
Argentina
Armenia
Aruba
Ascension Island
Australia
Austria
Azerbaijan
Bahamas
Bahrain
Bangladesh
Barbados
Belarus
Belgium
Belize
Benin
Bermuda
Bhutan
Bolivia
Bonaire, Sint Eustatius and Saba
Bosnia and Herzegovina
Botswana
Brazil
British Indian Ocean Territory
Brunei Darussalam
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Canada
Cape Verde
Cayman Islands
Central African Republic
Chad
Chile
China
Christmas Island
Cocos (Keeling) Islands
Colombia
Comoros
Congo
Congo, Democratic Republic of the
Cook Islands
Costa Rica
Cote d'Ivoire
Croatia
Cuba
Curaçao
Cyprus
Czech Republic
Denmark
Djibouti
Dominica
Dominican Republic
Ecuador
Egypt
El Salvador
Equatorial Guinea
Eritrea
Estonia
Ethiopia
Falkland Islands
Faroe Islands
Federated States of Micronesia
Fiji
Finland
France
French Guiana
French Polynesia
Gabon
Gambia
Georgia
Germany
Ghana
Gibraltar
Greece
Greenland
Grenada
Guadeloupe
Guam
Guatemala
Guernsey
Guinea
Guinea-Bissau
Guyana
Haiti
Holy See (Vatican City State)
Honduras
Hong Kong
Hungary
Iceland
India
Indonesia
Iran
Iraq
Ireland
Isle of Man
Israel
Italy
Jamaica
Japan
Jersey
Jordan
Kazakhstan
Kenya
Kiribati
Kosovo
Kuwait
Kyrgyzstan
Laos
Latvia
Lebanon
Lesotho
Liberia
Libya
Liechtenstein
Lithuania
Luxembourg
Macao
Madagascar
Malawi
Malaysia
Maldives
Mali
Malta
Marshall Islands
Martinique
Mauritania
Mauritius
Mayotte
Mexico
Moldova
Monaco
Mongolia
Montenegro
Montserrat
Morocco
Mozambique
Myanmar
Namibia
Nauru
Nepal
Netherlands
New Caledonia
New Zealand
Nicaragua
Niger
Nigeria
Niue
Norfolk Island
North Korea
North Macedonia
Northern Mariana Islands
Norway
Oman
Pakistan
Palau
Palestine
Panama
Papua New Guinea
Paraguay
Peru
Philippines
Poland
Portugal
Puerto Rico
Qatar
Reunion
Romania
Russia
Rwanda
Saint Barthélemy
Saint Helena
Saint Kitts and Nevis
Saint Lucia
Saint Martin (French Part)
Saint Pierre and Miquelon
Saint Vincent and the Grenadines
Samoa
San Marino
Sao Tome and Principe
Saudi Arabia
Senegal
Serbia
Seychelles
Sierra Leone
Singapore
Sint Maarten
Slovakia
Slovenia
Solomon Islands
Somalia
South Africa
South Korea
South Sudan
Spain
Sri Lanka
Sudan
Suriname
Svalbard and Jan Mayen
Swaziland
Sweden
Switzerland
Syria
Taiwan
Tajikistan
Tanzania
Thailand
Timor-Leste
Togo
Tokelau
Tonga
Trinidad and Tobago
Tristan da Cunha
Tunisia
Turkey
Turkmenistan
Turks and Caicos Islands
Tuvalu
Uganda
Ukraine
United Arab Emirates
United Kingdom
United States
Uruguay
Uzbekistan
Vanuatu
Venezuela
Vietnam
Virgin Islands, British
Virgin Islands, U.S.
Wallis and Futuna
Western Sahara
Yemen
Zambia
Zimbabwe
Linkedin profile URL
Upload your resume (in English)

Click to upload or drag & drop (.pdf)

Next

Please note that after completing the interview process, you will be considered for this and other roles that match your skills.

Have any questions? See FAQs

Refer and Earn
$100