Data Scientist - AI Engineer

Hub; København; Presencial; DKK 900.000 - 1.200.000; Jornada completa

Descripción de la vacante

Shine is building a Banking AI Foundation & Ops Efficiency team to scale AI-driven workflows with rigorous evaluation and compliant pipelines. We seek a Data Scientist to define evaluation metrics, curate golden datasets, and shape how models are tested across banking use cases.

You'll collaborate with software engineers to ship end-to-end systems, own privacy safeguards, and implement observability with Langfuse to keep agent behaviour transparent and auditable.

Formación

• Define and implement evaluation metrics tailored to what each use case needs to get right.
• Curate and maintain rom 'golden' datasets to compare real behaviour against the ideal result.
• Analyse historical data to determine what a given system's evals should be checking for in the first place.
• Monitor the AI/LLM-based components of our systems in production to catch regressions before they reach customers or compliance reviewers.
• Help build the agentic systems themselves, from single-purpose LLM agents to multi-agent systems with tool-calling and orchestration.
• Own privacy and compliance safeguards, ensuring sensitive data gets detected and redacted correctly and that every system is auditable end to end.
• Build and maintain observability for our agentic components using an LLM observability tool (Langfuse) to ensure agent behaviour stays traceable and transparent.
• Partner with the team's software engineers to take the project to production.
• Contribute to prioritising the automation opportunities across Banking that are worth pursuing, and help other teams adopt what we build.

Responsabilidades

• Define and implement evaluation metrics for our AI systems, tailored to what each use case needs to get right.
• Curate and maintain golden datasets to compare a system's real behaviour against the ideal result.
• Analyse historical data to determine what a given system's evals should be checking for in the first place.
• Monitor the AI/LLM-based components of our systems in production to catch regressions before they reach customers or compliance reviewers.
• Help build the agentic systems themselves, from single-purpose LLM agents to multi-agent systems with tool-calling and orchestration.
• Own privacy and compliance safeguards, ensuring sensitive data gets detected and redacted correctly and that every system is auditable end to end.
• Build and maintain observability for our agentic components using an LLM observability tool (Langfuse) to ensure agent behaviour stays traceable and transparent.
• Partner with the team's software engineers to take the project to production.
• Contribute to prioritising the automation opportunities across Banking that are worth pursuing, and help other teams adopt what we build.

Conocimientos

Evaluation metrics

Golden datasets

Historical data analysis

Production monitoring

Agentic systems

Privacy & compliance

Observability (Langfuse)

Collaboration with engineers

Automation prioritisation

Herramientas

Langfuse

Descripción del empleo

About Shine

Shine is the financial copilot for entrepreneurs and small business owners.

Founded by serial entrepreneurs Rico Andersen and Martin Hegelund, Shine is a leading European fintech unicorn on a mission to restore the joy of running a business, by ending wasted time on financial admin. Shine offers a connected solution for invoicing, accounting, payroll, business accounts, payments, and financing, meaning business owners can focus their energy on growing a healthy business, not held back by manual admin.

Part of something bigger

Today we're part of Cegid, a European leader in cloud software for finance and accounting. Together we're building Europe's leading financial copilot for small businesses and their accountants.

Shine already supports more than 400,000 small businesses. As part of Cegid, we now reach over one million small businesses and 15,000 accountants across Europe.

We're a multicultural team working from France, Germany, Denmark and the Netherlands, contributing to a wider European network that spans Spain, Portugal.

Your hiring experience matters

Just as we respect our customers' time, we respect yours. Your experience with Shine and Cegid should feel simple, transparent and genuinely supportive.

If this sounds like somewhere you want to grow, we'd love to hear from you.

The Banking AI Foundation & Ops Efficiency Team at Shine.

Our Banking AI Foundation & Ops Efficiency team accelerates operational efficiency across the Banking Domain by identifying and scaling high-impact, compliant AI-driven workflows.

We're a cross-functional team of Software/AI engineers and data scientists who build these systems end-to-end together.

Our work covers a wide range of AI-driven solutions depending on each team's specific needs and goals.

Evaluation, explainability, and traceability of these AI systems are a dedicated focus within our team, and the data scientists own that foundation across every project we ship.

Your role as a Data Scientist

We're looking for a Data Scientist/AI Engineer to contribute to the evaluation process of our system, as well as contribute to the orchestration design and AI safety foundation behind our AI-driven workflows.

AI solutions are scaling quickly across the Banking Domain, and each new one needs the same evaluation rigour and compliance safeguards we've already built into our existing systems, applied consistently rather than reinvented from scratch each time.

You will collaborate with the team to build and manage agentic systems, and be a key driver of the design of the evaluation process for these systems, with privacy and compliance safeguards in mind throughout the whole process. You will work together with your team to ship end-to-end systems.

About you

• Define and implement evaluation metrics for our AI systems, tailored to what each use case needs to get right.

• Curate and maintain "golden" datasets to compare a system's real behaviour against the ideal result.

• Analyse historical data to determine what a given system's evals should be checking for in the first place.

• Monitor the AI/LLM-based components of our systems in production to catch regressions before they reach customers or compliance reviewers.

• Help build the agentic systems themselves, from single-purpose LLM agents to multi-agent systems with tool-calling and orchestration.

• Own privacy and compliance safeguards, ensuring sensitive data gets detected and redacted correctly and that every system is auditable end to end.

• Build and maintain observability for our agentic components using an LLM observability tool (Langfuse) to ensure agent behaviour stays traceable and transparent.

• Partner with the team's software engineers to take the project to production.

• Contribute to prioritising the automation opportunities across Banking that are worth pursuing, and help other teams adopt what we build.

What success looks like

90 Days: Onboarded on the team's tooling and one live project. Shipped a first evaluation metric or golden dataset, and/or contributed to building or debugging an agent, with close support. Started a well-scoped task end to end. Relationships built with the team's engineers and senior teammates.

6 Months: Owns an evaluation pipeline and/or an agent-building or orchestration workstream for a live project end to end. Production issues within their scope are fixed independently rather than routed elsewhere. Design and evaluation trade-offs explained clearly to the team being supported. Contributing to a second project.

12 Months: Takes a loosely-specified problem, whether that's a new agent to build, a system to orchestrate, or a new eval strategy to define, from data analysis through implementation independently. Trusted as a second opinion on eval design and/or agent architecture by teammates. Contributing to prioritisation or adoption conversations across Banking and CX. Able to directly support a newer joiner on the team.

Equal Opportunity Employer

We follow the principle of equal treatment to consider all job applicants and do not discriminate based on their gender, sexual orientation, color, racial or ethnic origin, religion, disability, etc. as per applicable law.