Python Developer-AI Evaluation

ZettaMine Labs Pvt. Ltd.; Canada; Remote; CAD 83,000 - 124,000; Full time

Benefits offered by this job

Fully remote

Cutting-edge AI projects

AI model training

Competitive hourly pay

Job summary

ZettaMine Labs Pvt. Ltd. is seeking a Senior Software Engineer – Python/TypeScript for a short-term AI Engineering and Evaluation project based in Canada.

This remote freelance role requires 4+ years of experience and focuses on building pipelines, evaluating data quality, and supporting AI training workflows. You’ll design evaluation rubrics, review data quality, and collaborate with researchers and engineers to scale qualitative judgments into robust processes.

Qualifications

• Experience building, maintaining, and improving software pipelines and infrastructure.
• Hands-on experience with preference data collection and rubric creation.
• Ability to review technical data and evaluate quality using defined criteria.
• Strong analytical, problem-solving, and technical judgment skills.
• Experience with large codebases, such as VS Code, is an advantage.
• Ability to explain why a technical solution is good, accurate, and effective.
• Strong written and verbal communication skills.

Responsibilities

• Design, develop, and improve evaluation rubrics and criteria for preference data.
• Review and label data while ensuring high standards of quality and accuracy.
• Identify subtle issues and inconsistencies in data and evaluation results.
• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.
• Analyze technical findings and prepare clear reports, summaries, and recommendations.
• Collaborate with researchers and engineers to convert qualitative judgments into scalable processes.
• Support AI training and evaluation workflows to improve the quality of AI model outputs.
• Communicate technical findings clearly and recommend improvements to project teams.

Skills

Analytical skills

Problem-solving

Technical judgment

Written communication

Verbal communication

Collaboration

Education

Bachelor’s or Master’s degree in a relevant field

Tools

VS Code

Job description

Freelancing Opportunity: We are hiring Senior Software Engineers – Python / TypeScript (USA) for a short-term AI Engineering and Evaluation project focused on improving AI models through high-quality training data, preference data evaluation, and engineering workflows.

Candidates must be based in Canada.

Role: Python Developer-AI Evaluation

Experience: 4+ Years

Location: Canada

Mode of Work: Remote

Engagement: Freelance / Project-Based

Mandatory Eligibility Criteria

Technical Qualification:

• Experience building, maintaining, and improving software pipelines and infrastructure.
• Hands-on experience with preference data collection and/or rubric creation.
• Ability to review technical data and evaluate quality using defined criteria.
• Strong analytical, problem-solving, and technical judgment skills.
• Experience working with large codebases, such as VS Code, is an advantage.
• Ability to explain why a technical solution or output is good, accurate, and effective.
• Strong written and verbal communication skills.

Educational Qualification:

• Bachelor’s or Master’s degree in a relevant field, including BCA, B.Tech, B.E., MCA, or M.Tech.

Experience:

• Candidates from Tier 1 companies are preferred.
• Candidates must not currently be working with Microsoft.

Availability:

• Minimum 4 hours per day and 20 hours per week.
• Minimum 4 hours of overlap with Pacific Standard Time (PST) working hours.
• Immediate availability preferred.
• Ability to commit to the full 3-month contract.

Others:

• An updated LinkedIn profile is mandatory for candidate review. Please include the LinkedIn profile URL in your resume.
• Ability to work independently in a fully remote environment.
• Strong attention to detail and commitment to high-quality deliverables.
• Ability to work in a fast-paced environment and deliver results within deadlines.
• Good communication and collaboration skills to work with researchers and engineering teams.

What You’ll Work On

• Design, develop, and improve evaluation rubrics and criteria for preference data.
• Review and label data while ensuring high standards of quality and accuracy.
• Identify subtle issues and inconsistencies in data and evaluation results.
• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.
• Analyze technical findings and prepare clear reports, summaries, and recommendations.
• Collaborate with researchers and engineers to convert qualitative judgments into scalable processes.
• Support AI training and evaluation workflows to improve the quality of AI model outputs.
• Communicate technical findings clearly and recommend improvements to project teams.

Evaluation Process

• AI Interview
• Technical and Cultural Interview – Approximately 30 minutes.

Perks of Freelancing

• Fully remote work environment.
• Opportunity to work on cutting-edge AI projects with leading AI companies.
• Opportunity to contribute to AI model training, evaluation, and engineering initiatives.
• Competitive hourly compensation based on candidate quality and interview feedback.