Senior Python AI Evaluator & SFT Lead (Remote)

Joblet-AI

Abu Dhabi

Remote

AED 240,000 - 320,000

Full time

44 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Joblet-AI is seeking a Senior Python Developer (Remote) to join a project with foundational LLM companies. You will design, develop and maintain high-quality Python code for training and optimizing AI models, conduct evaluations to benchmark performance and help improve model alignment with defined criteria.

You will lead SFT efforts, collaborate with researchers and annotators for RLHF, and continuously explore new tools and methodologies to enhance AI training processes.

Qualifications

  • 3 years of strong experience with Python programming language.
  • Industry experience and knowledge of code quality formatting and best practices of software development.
  • Experience with Python's testing ecosystem including unit, integration and property-based testing.
  • Knowledge of multi-threading and asynchronous programming in Python.
  • Ability to work with architectural patterns and refactor code without introducing regressions.
  • Strong debugging skills including fixing memory and concurrency issues.
  • Fluent in conversational and written English communication skills.

Responsibilities

  • Design, develop and maintain efficient high-quality Python code to train and optimize AI models.
  • Conduct evaluations (Evals) to benchmark model performance and analyze results for continuous improvement.
  • Evaluate and rank AI model responses to user queries across diverse domains ensuring alignment with predefined criteria.
  • Develop explanations and rationales for evaluations showcasing reasoning and technical expertise.
  • Lead efforts in Supervised Fine-Tuning (SFT) including creating and maintaining high-quality task-specific datasets.
  • Collaborate with researchers and annotators to execute RLHF and refine reward models.

Skills

Python programming
Debugging
Asynchronous programming
Code quality
English communication

Tools

Pytest or testing ecosystem
Python frameworks and libraries

Job description

Joblet-AI is seeking a Senior Python Developer (Remote) to join a project with foundational LLM companies. You will design, develop and maintain high-quality Python code for training and optimizing AI models, conduct evaluations to benchmark performance and help improve model alignment with defined criteria.

You will lead SFT efforts, collaborate with researchers and annotators for RLHF, and continuously explore new tools and methodologies to enhance AI training processes.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Senior Software Engineer for AI Code Evaluation
Remote Senior Software Engineer for AI Code Evaluation

Joblet-AI • Abu Dhabi

Remote
AED 506,000 - 759,000
Data Scientist - Python (Remote) Jobs Ai Fast Track available
Data Scientist - Python (Remote) Jobs Ai Fast Track available

HireHouse • United Arab Emirates

Remote
AED 120,000 - 180,000
Senior AI Engineer - LLM Agent Systems (Remote)
Senior AI Engineer - LLM Agent Systems (Remote)

Superbo • United Arab Emirates

Remote
AED 180,000 - 240,000
Senior Software Engineer (Remote)
Senior Software Engineer (Remote)

Joblet-AI • Abu Dhabi

Remote
AED 506,000 - 759,000
Remote Full-Stack Engineer (Contract) - AI Training Tasks
Remote Full-Stack Engineer (Contract) - AI Training Tasks

Joblet-AI • Abu Dhabi

Remote
AED 165,000 - 331,000
AI Developer (Fulltime - Remote)
AI Developer (Fulltime - Remote)

Confidential Company • Dubai

Remote
AED 180,000 - 300,000
Remote AI/ML Engineer for Production AI Agents
Remote AI/ML Engineer for Production AI Agents

Joblet-AI • Abu Dhabi

Remote
AED 477,000 - 588,000
Equity compensation
Performance-based bonuses
Health-insurance premium reimbursement
+3
Working Student: AI Engineering (Python & LLMs) – Remote UAE
Working Student: AI Engineering (Python & LLMs) – Remote UAE

AgentixLake GmbH • Abu Dhabi

Remote
AED 13,000 - 23,000
Remote AI Automation Architect (Production-Ready)
Remote AI Automation Architect (Production-Ready)

Jobsforhumanity • United Arab Emirates

Remote
AED 210,000 - 260,000
Full tools budget
Direct line to founder
Remote Python Data Scientist for AI Model Evaluation
Remote Python Data Scientist for AI Model Evaluation

HireHouse • United Arab Emirates

Remote
AED 120,000 - 180,000
Top locations