AI Engineer with Deep Data Science Understanding

emagine; København; On-site; DKK 800,000 - 1,200,000; Full time

Job summary

emagine is seeking a freelance AI Engineer with Deep Data Science understanding to contribute to the AI Skills Registry for clients in the financial sector. The role focuses on building reliable components for AI solutions across platforms, including AWS AgentCore-based agents and our enterprise AI solution.

The position is on-site in Denmark, full-time, and involves developing components, evaluating models, and ensuring robust evaluation practices in a regulated industry environment.

Qualifications

• Hands-on experience evaluating ML or LLM-based systems in production.
• Knowledge of evaluation methodology, including error analysis and rubric design.
• Strong Python and data analysis skills.
• Experience with cloud-native ML infrastructure, such as AWS Bedrock.
• Ability to work independently within a growing team.
• Experience in regulated industries such as financial services or healthcare.

Responsibilities

• Build components for the AgentCore and run evaluation frameworks for skills and tools builds.
• Ensure the reliability and accuracy of AI models.
• Operationalize the Evaluate and Iterate phase of the ADS to ensure robust evaluation practices across various agent builds.
• Define and track quality and cost-per-outcome metrics for production agents.
• Run structured error analysis on agent outputs to identify failure patterns and implement fixes.
• Develop innovative AI solutions within a dynamic team environment.

Skills

Python

Data analysis

Evaluation methodology

Error analysis

Independent work

Production ML/LLM experience

Tools

AWS Bedrock

AgentCore

Job description

emagine is looking for a freelance AI Engineer with Deep Data Science understanding for our partner within the financial sector.

The primary goal of the AI Engineer role is to contribute to the development of the AI Skills Registry focusing on building reliable components for AI solutions that operate effectively across various platforms.

This role focuses on building essential components that support both AWS AgentCore-based agents and our enterprise AI solution.

Main Responsibilities

• Build components for the AgentCore and/or Amazon Quick and run evaluation frameworks for skills and tools builds.
• Ensure the reliability and accuracy of AI models.
• Help operationalize the Evaluate and Iterate phase of the ADS to ensure robust evaluation practices across various agent builds.
• Define and track quality and cost-per-outcome metrics for production agents.
• Run structured error analysis on agent outputs to identify failure patterns and implement fixes.
• Focus on the development of innovative AI solutions within a dynamic team environment.

Key Requirements

• Hands-on experience evaluating ML or LLM-based systems in production.
• Working knowledge of evaluation methodology, including error analysis and rubric design.
• Strong Python and data analysis skills.
• Experience with cloud-native ML infrastructure, such as AWS Bedrock.
• Ability to work independently within a growing team.
• Experience in a regulated industry, such as financial services or healthcare.

Nice to Have

• Familiarity with AWS AgentCore, Strands Agents, or similar agent runtimes.
• Knowledge of GDPR or equivalent data protection regulations.

Other Details

This position is fulltime and requires working on-site.