Remote AI Engineer (United States)

Rex.zone; United States; On-site; USD 100,000 - 140,000; Full time

Job summary

Rex.zone is seeking a Mid-Senior Remote AI Engineer to enhance model quality and reliability for LLM and multimodal systems. This fully remote position involves building evaluation harnesses, integrating RLHF workflows, and collaborating with data operations. The role requires experience in shipping AI/ML systems, strong Python skills, and the ability to improve training data quality across NLP and computer vision applications.

Qualifications

• Mid-Senior experience shipping AI/ML systems in production.
• Strong Python skills and ability to build reliable data/evaluation pipelines.
• Hands-on experience with model evaluation and prompt evaluation.

Responsibilities

• Build and maintain LLM evaluation pipelines.
• Integrate RLHF signals and feedback loops.
• Define and iterate on annotation guidelines compliance checks.

Skills

Python

AI/ML systems

Model evaluation

Experiment design

Data labeling

Quality assurance

Job description

Remote AI Jobs in the United States (Rex.zone)

Rex.zone is hiring a Mid-Senior Remote AI Engineer to improve model quality and reliability for LLM and multimodal systems. You will build evaluation harnesses, integrate RLHF workflows, and partner with data operations to raise training data quality across NLP and computer vision use cases.

Key Responsibilities

• Build and maintain LLM evaluation pipelines (rubric-based grading, pairwise preference tests)
• Integrate RLHF signals and feedback loops for safer, higher-quality outputs
• Define and iterate on annotation guidelines compliance checks and quality gates
• Develop prompt evaluation and regression suites to detect quality drops across releases
• Support NLP (e.g., named entity recognition) and CV (e.g., computer vision annotation) workflows
• Implement content safety labeling strategies and audit-friendly reporting
• Create dashboards/monitoring for training data quality, evaluation metrics, and production health
• Improve tooling and automation for QA evaluation, labeling workflows, and review operations

Required Qualifications

• Mid-Senior experience shipping AI/ML systems in production
• Strong Python and ability to build reliable data/evaluation pipelines
• Hands‑on experience with model evaluation, prompt evaluation, and experiment design
• Understanding of RLHF concepts and preference modeling workflows
• Ability to turn ambiguous quality issues into measurable metrics and engineering tasks
• Comfort collaborating across product, data operations, and QA

Remote Work and Location

This is a fully remote role aligned to United States hiring and payroll requirements.

How To Apply

Apply via Rex.zone and include a resume highlighting evaluation pipelines, RLHF, data labeling or QA evaluation, and production ML engineering experience.