AI Quality & Evaluation Engineer

Bask Health; New York (NY); Sur place; USD 120 000 - 190 000; Plein temps

Résumé du poste

Bask Health is seeking a Software Engineer, AI Quality & Evaluation to build systems for testing and improving AI-powered applications across development and production lifecycles.

You will develop internal tools, backend services, testing frameworks, and automation that support AI quality assurance across the platform, collaborating with AI and ML engineers to meet testing needs and translate requirements into reliable software capabilities.

Qualifications

• 3+ years of professional software engineering experience.
• Experience building and maintaining production backend services.
• Strong programming skills in TypeScript, JavaScript, Python, or comparable languages.
• Experience designing and developing APIs and service integrations.
• Familiarity with automated testing frameworks and software quality practices.
• Experience with CI/CD workflows.

Responsabilités

• Design, build, and maintain software infrastructure that supports AI quality and evaluation across Bask Health.
• Develop backend services that execute and manage automated AI evaluations.
• Build reusable testing tools for large language model applications.
• Create APIs that allow engineering teams to submit evaluation jobs and retrieve results.
• Develop systems for managing evaluation datasets, test cases, and expected outcomes.
• Build automated workflows for comparing prompts, models, and application configurations.
• Implement regression testing infrastructure for AI-powered features.
• Integrate AI evaluation checks into CI/CD pipelines.
• Develop services that record and organize evaluation results over time.
• Build tools for inspecting AI responses, failures, and test histories.

Connaissances

TypeScript

Python

JavaScript

APIs

Testing frameworks

CI/CD

Distributed systems

Cloud services

Git

SQL

Node.js

Outils

Git

SQL

Node.js

Description du poste

Bask Health is seeking a Software Engineer, AI Quality & Evaluation to build systems for testing and improving AI-powered applications across development and production lifecycles.

You will develop internal tools, backend services, testing frameworks, and automation that support AI quality assurance across the platform, collaborating with AI and ML engineers to meet testing needs and translate requirements into reliable software capabilities.