Python Developer-AI Evaluation

Linuxassociation; Canada; Remote; CAD 96,000 - 165,000; Full time

Benefits offered by this job

Fully remote work environment

Cutting-edge AI projects

Competitive hourly compensation

Job summary

ZettaMine Labs Pvt. Ltd. in Canada is seeking Senior Software Engineers for a remote, freelance Python/TypeScript evaluation project focused on improving AI models through high-quality training data and data rubric development.

You will design evaluation rubrics, build data pipelines, review data quality, and collaborate with researchers to translate qualitative judgments into scalable processes. Immediate availability and strong communication are valued.

Qualifications

• Experience building, maintaining, and improving software pipelines and infrastructure.
• Hands-on experience with preference data collection and/or rubric creation.
• Ability to review technical data and evaluate quality using defined criteria.
• Strong analytical, problem-solving, and technical judgment skills.
• Experience working with large codebases; VS Code is an advantage.
• Excellent written and verbal communication skills.

Responsibilities

• Design, develop, and improve evaluation rubrics and criteria for preference data.
• Review and label data ensuring high quality and accuracy.
• Identify subtle issues in data and evaluation results.
• Build and maintain pipelines/infrastructure for data generation, collection, and evaluation.
• Analyze findings and prepare clear reports and recommendations.
• Collaborate with researchers and engineers to scale qualitative judgments.
• Support AI training/evaluation workflows to improve model outputs.
• Communicate technical findings and suggest improvements to teams.

Skills

Software pipelines

Infrastructure

Preference data collection

Rubric creation

Data evaluation

Large codebases

Communication

Education

Bachelor's degree

Master's degree

Tools

VS Code

Job description

# Python Developer-AI EvaluationZettaMine Labs Pvt. Ltd. · , CABack to resultsPython Developer-AI EvaluationZettaMine Labs Pvt. Ltd., CA Posted October 29, 2026 Via JobLeads### Full Job DescriptionFreelancing Opportunity: We are hiring Senior Software Engineers – Python / TypeScript (USA) for a short-term AI Engineering and Evaluation project focused on improving AI models through high-quality training data, preference data evaluation, and engineering workflows.Candidates must be based in Canada.Role: Python Developer-AI EvaluationExperience: 4+ YearsLocation: CanadaMode of Work: RemoteEngagement: Freelance / Project-BasedMandatory Eligibility CriteriaTechnical Qualification:• Experience building, maintaining, and improving software pipelines and infrastructure.• Hands-on experience with preference data collection and/or rubric creation.• Ability to review technical data and evaluate quality using defined criteria.• Strong analytical, problem-solving, and technical judgment skills.• Experience working with large codebases, such as VS Code, is an advantage.• Ability to explain why a technical solution or output is good, accurate, and effective.• Strong written and verbal communication skills.Educational Qualification:• Bachelor’s or Master’s degree in a relevant field, including BCA, B.Tech, B.E., MCA, or M.Tech.Experience:• Candidates from Tier 1 companies are preferred.• Candidates must not currently be working with Microsoft.Availability:• Minimum 4 hours per day and 20 hours per week.• Minimum 4 hours of overlap with Pacific Standard Time (PST) working hours.• Immediate availability preferred.• Ability to commit to the full 3-month contract.Others:• An updated LinkedIn profile is mandatory for candidate review. Please include the LinkedIn profile URL in your resume.• Ability to work independently in a fully remote environment.• Strong attention to detail and commitment to high-quality deliverables.• Ability to work in a fast-paced environment and deliver results within deadlines.• Good communication and collaboration skills to work with researchers and engineering teams.What You’ll Work On• Design, develop, and improve evaluation rubrics and criteria for preference data.• Review and label data while ensuring high standards of quality and accuracy.• Identify subtle issues and inconsistencies in data and evaluation results.• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation.• Analyze technical findings and prepare clear reports, summaries, and recommendations.• Collaborate with researchers and engineers to convert qualitative judgments into scalable processes.• Support AI training and evaluation workflows to improve the quality of AI model outputs.• Communicate technical findings clearly and recommend improvements to project teams.Evaluation Process• AI Interview• Technical and Cultural Interview – Approximately 30 minutes.Perks of Freelancing• Fully remote work environment.• Opportunity to work on cutting-edge AI projects with leading AI companies.• Opportunity to contribute to AI model training, evaluation, and engineering initiatives.• Competitive hourly compensation based on candidate quality and interview feedback.