Data Engineering AI Evaluation Engineer

OpenTrain AI, Inc.; India; Remote; INR 1,981,000 - 3,302,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Data Engineering and Data Science AI Evaluation Engineer on a part-time contractor basis.

The role involves designing and validating data pipelines for benchmarking AI systems, processing structured and unstructured data, and writing Python code for local experiments. You will work with researchers and engineers to create realistic evaluation tasks and ensure reproducible results.

Qualifications

• At least 3 years of experience in data engineering, data science, or data-focused software engineering.
• Excellent spoken and written English.
• Strong proficiency in Python for data workflows.
• Ability to work with structured and unstructured data.

Responsibilities

• Design, build, and validate data pipelines for benchmarking and evaluation workflows.
• Process and analyze data, prepare features, and validate data science workflows.
• Write, run, and modify Python code for data processing and local experiments.
• Check data quality, transformations, and outputs for correctness and reproducibility.
• Create clean, documented, reusable workflows for benchmarking.
• Review code for quality and maintainability.
• Collaborate with researchers and engineers on real-world tasks.

Skills

Python

Data engineering

Data science

English proficiency

Code reading

Problem solving

Job description

The work

As a Data Engineering and Data Science AI Evaluation Engineer, you will create and check data workflows used to measure how well advanced AI systems perform. You will work with production-like structured and unstructured data, Python code, and real-world data problems.

You will also help design difficult, realistic data engineering and data science tasks for AI systems. The work includes local experiments, code reviews, and careful checks for correct and repeatable results.

• Support SWE Bench-style evaluation tasks with structured and unstructured datasets.
• Design, build, and validate data pipelines for benchmarking and evaluation workflows.
• Process and analyze data, prepare features, and validate data science workflows.
• Write, run, and modify Python code for data processing and local experiments.
• Check data quality, transformations, and outputs for correctness and reproducibility.
• Create clean, documented, reusable workflows suitable for benchmarking.
• Take part in code reviews focused on code quality and maintainability.
• Work with researchers and engineers to develop real-world data engineering and data science tasks for AI systems.

What it pays and takes

This is a part-time contractor role with a three-month contract. The contract length may be adjusted based on the engagement. The posting does not specify a pay rate.

The listing is marked entry level, but it requires at least three years of overall experience in data engineering, data science, or data-focused software engineering.

• Pay: Not specified in the posting.
• Schedule: At least 4 hours per day and at least 20 hours per week.
• Availability: At least 4 hours of overlap with Pacific Time, or PST.
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Brazil, and Mexico.
• Language: Excellent spoken and written English.
• Python: Strong proficiency for data engineering and data science workflows.
• Data experience: Data processing, analysis, model-related workflows, and work with structured and unstructured data.
• Technical ability: Understand, navigate, and modify complex real-world codebases.
• Code quality: Write readable, reusable, maintainable, and well-documented code.
• Problem solving: Experience with algorithmic or data-intensive problems.
• Helpful background: SWE Bench or similar benchmark projects, or experience designing AI model evaluation tasks.
• Contract type: Contractor and part time.

About AI training work

AI training work uses human-created data, code, and feedback to test and improve artificial intelligence systems. Experienced contributors are needed to check difficult technical tasks, identify errors, and help make model results more useful and reliable.