LLM Evaluation Expert (Coding) - Part Time: 25-05149

Akraya, Inc.; Kennewick (WA); Remote; USD 60,000 - 80,000; Part time

Job summary

Akraya, Inc. is looking for a part-time LLM Evaluation Expert to join their Artificial General Intelligence Data Services team. This remote position involves evaluating and enhancing the coding abilities of large language models, ensuring efficient code that meets industry standards. Candidates should possess an advanced degree in Computer Science and have significant software development experience.

Qualifications

• 5+ years of software development experience.
• Proven ability to evaluate code in multiple languages including Python, PHP, Java, Ruby, JavaScript, TypeScript, C++, Go, Cypher, SQL.
• Experience in understanding user requests and evaluating model responses based on correctness, logic, and applicability.

Responsibilities

• Evaluate LLM-generated code across various programming languages.
• Select optimal coding solutions from LLM outputs, aligning with client needs.
• Develop benchmarks for high-quality AI-generated code.

Skills

LLM

Programming Languages

Generative AI

Code Generation

Code Evaluation

Education

Advanced degree in Computer Science

Job description

LLM Evaluation Expert (Coding) - Part Time: 25-05149

Get AI-powered advice on this job and more exclusive features.

Primary Skills: LLM (Expert), Programming Languages (Expert), Generative AI (Proficient), Code Generation (Proficient), Code Evaluation (Proficient)

Contract Type: W2

Duration: 6+ Months with possible extension or conversion to FTE

Location: Remote

Pay Range: $50.00 - $58.00 Per Hour

Job Summary

This role allows only 20 hours per week, with no flexibility on this. Join our Artificial General Intelligence (AGI) Data Services team as an LLM Evaluation Expert, where you'll assess and enhance our large language models' (LLMs) coding abilities, ensuring high-quality, efficient code output that meets industry standards. This is a remote position for 6 months, with potential for extension.

Key Responsibilities

• Evaluate LLM-generated code across various programming languages.
• Select optimal coding solutions from LLM outputs, aligning with client needs.
• Develop benchmarks for high-quality AI-generated code through demonstrations.
• Provide detailed feedback to improve LLM's understanding and output.
• Collaborate with the AI research team to enhance coding capabilities and stay updated with advancements.

Must-Have Skills

• Advanced degree in Computer Science with 5+ years of software development experience.
• Proven ability to evaluate code in multiple languages including Python, PHP, Java, Ruby, JavaScript, TypeScript, C++, Go, Cypher, SQL.
• Experience in understanding user requests and evaluating model responses based on correctness, logic, and applicability.

Industry Experience

Background in software engineering and AI systems, especially in code generation and natural language processing, is preferred.

About Akraya

Akraya is an award-winning IT staffing firm recognized for excellence and a thriving work environment, including Inc's Best Workplaces 2024 and other accolades.

Additional Details

• Seniority level: Mid-Senior level
• Employment type: Part-time
• Job function: Other
• Industry: Staffing and Recruiting