Senior GenAI Scientist - Evaluation & Agentic Memory Remote

Caseware International; Bogotá; Presencial; COP 120.000.000 - 200.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

100% remote work

Home office stipend

Competitive compensation

Flexible work options

Descripción de la vacante

Caseware International is hiring for a staff-to-senior level applied science role focused on evaluating and improving GenAI-powered audit and accounting tools. You will design experiments, build evaluation frameworks, and develop AI-driven capabilities behind our agent platforms.

The role emphasizes self-learning systems with strong collaboration across Security, Legal, and Domain SMEs. The position is fully remote, based in Colombia, with opportunities to influence technical direction and

Formación

• 6+ to 8+ years of professional experience in applied science, ML, research, or data-intensive engineering.
• 2+ years working on production GenAI or LLM-based systems (applications, agents, or tooling).
• Proven experience designing evaluation frameworks: metrics, benchmarks, scoring, and experiments.
• Ability to build and operate production tooling, ideally on AWS.
• Strong English communication and collaboration skills.
• Comfortable operating in fast-moving environments with ambiguity.

Responsabilidades

• Design and run experiments to measure and improve quality of LLM-based applications and agents.
• Own evaluation methodology: benchmarks, regression suites, thresholds for success, and drift monitoring.
• Build AI-based tools powering the end-to-end agent builder experience.
• Develop synthetic data generation and evaluation capabilities for internal teams and customers.
• Translate domain procedures into task-plus-verifier structures and validate memory promotion criteria.
• Contribute to self-learning systems that improve offline and online with guardrails for legal obligations.
• Influence technical direction via RFCs and design reviews; mentor other scientists and engineers.

Conocimientos

Applied science

Machine learning

Research

Data engineering

Experiment design

Educación

MS in quantitative field

PhD preferred

Herramientas

AWS

LangChain

LangSmith

OpenSearch

Textract

Descripción del empleo

Caseware International is hiring for a staff-to-senior level applied science role focused on evaluating and improving GenAI-powered audit and accounting tools. You will design experiments, build evaluation frameworks, and develop AI-driven capabilities behind our agent platforms.

The role emphasizes self-learning systems with strong collaboration across Security, Legal, and Domain SMEs. The position is fully remote, based in Colombia, with opportunities to influence technical direction and