Remote Senior Agentic AI Scientist

Caseware; Medellín; Presencial; COP 60.000.000 - 90.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Remote work 100%

Competitive compensation

Training budget

Mentorship

Descripción de la vacante

Caseware, a global fintech leader, seeks an experienced applied scientist to advance our GenAI-powered audit and accounting platform. You will design experiments, build evaluation frameworks, and own the metrics that define success for LLM-based agents.

You’ll shape memory, safety, and efficiency while collaborating with cross‑functional teams to deliver scalable, reliable tooling. Remote Colombia-based role offering mentorship, growth, and a culture of independence and innovation as Caseware

Formación

• 6+ to 8+ years of professional experience in applied science, machine learning, research, or data-intensive engineering.
• 2+ years working on production GenAI or LLM-based systems (applications, agents, or tooling/evaluation).
• Proven experience designing and building evaluation frameworks for ML/LLM systems.
• Strong experimentation mindset and ability to draw defensible conclusions from real-world data.

Responsabilidades

• Design and run experiments that measure and improve the quality of LLM-based applications and agents.
• Build out and own evaluation methodology including benchmarks, regression suites, scoring methods, and thresholds across task success, faithfulness, safety, latency, and cost.
• Build AI-based tools powering the end-to-end agent builder for Caseware teams and customers.
• Create synthetic data generation and eval-builder capabilities for internal teams and customers.
• Contribute to agentic memory research, defining promotion/demotion criteria and memory selection.

Conocimientos

Applied science

GenAI systems

Experiment design

Evaluation frameworks

Production tooling

AWS

English communication

Educación

MS/PhD preferred

Herramientas

LangChain

LangSmith

LangFuse

AWS

CDK

Terraform

Descripción del empleo

Caseware, a global fintech leader, seeks an experienced applied scientist to advance our GenAI-powered audit and accounting platform. You will design experiments, build evaluation frameworks, and own the metrics that define success for LLM-based agents.

You’ll shape memory, safety, and efficiency while collaborating with cross‑functional teams to deliver scalable, reliable tooling. Remote Colombia-based role offering mentorship, growth, and a culture of independence and innovation as Caseware