AI Engineering Lead: Evaluations & System Integration

Bain & Company, Inc.; Boston (MA); Presencial; USD 198.000 - 238.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Medical coverage

Dental coverage

Vision coverage

401(k) plan

Paid time off

Life insurance

Disability insurance

Descripción de la vacante

Bain & Company, Inc. is seeking an AI Engineering Lead to architect and own the evaluation framework for the Growth AI Performance System. You’ll oversee data ontology, agentic core integration, and product modules, ensuring high quality and reliability across the stack.

You will collaborate with Data Ontology, Agentic Core, and Platform teams, driving deployment readiness and client-ready evaluations while advancing responsible AI practices and scalable testing pipelines.

Formación

• Design and run evaluations for LLM-based and agentic AI systems across the delivery lifecycle.
• Build evaluation frameworks that influence what ships and end-to-end system quality.
• Experience with knowledge graphs, entity resolution, and semantic data.
• Strong Python and automation skills.

Responsabilidades

• Own and evolve the evaluation framework across the Growth AI platform.
• Design and maintain evaluation datasets and regression tests.
• Define deployment readiness criteria for all system layers.
• Build automated evaluation pipelines integrated with CI/CD.
• Coordinate evaluation work across distributed teams and report status.

Conocimientos

Python proficiency

CI/CD pipelines

Evaluation frameworks

Agentic AI systems

Statistical reasoning

Educación

Bachelor's degree in computer science, engineering, mathematics, or related field

Herramientas

Knowledge graph

Ontology evaluation

CI/CD tooling

Descripción del empleo

Bain & Company, Inc. is seeking an AI Engineering Lead to architect and own the evaluation framework for the Growth AI Performance System. You’ll oversee data ontology, agentic core integration, and product modules, ensuring high quality and reliability across the stack.

You will collaborate with Data Ontology, Agentic Core, and Platform teams, driving deployment readiness and client-ready evaluations while advancing responsible AI practices and scalable testing pipelines.