AI Agent Evaluation Engineer for Simcenter Amesim

Siemens Digital Industries Software; Villeurbanne; On-site; EUR 12,000 - 15,000; Full time

Benefits offered by this job

Meal vouchers

Mentorship

Learning Hub access

Job summary

Siemens Digital Industries Software in Villeurbanne (near Lyon) offers a six-month internship to design and build an evaluation platform for MCP-based workflows and LLM usage, integrated with CI pipelines. You will collaborate with the Nexus team to analyze performance and automate result validation.

We seek a Master’s student in France with strong Python/Go skills, interest in AI, and experience with testing and CI/CD, ready to work in a bilingual team under mentorship. Starts January 2027.

Qualifications

• Enrolled in a Master’s-level engineering program in France (Bac +5 / MSc).
• Strong Python and Go development skills and interest in AI/LLMs.
• Knowledge of testing frameworks, performance analysis, and CI/CD practices.

Responsibilities

• Design and implement robust test suites (MLflow or equivalent) to measure the performance, quality, and relevance of AI agents and MCP-driven interactions with Simcenter Amesim.
• Develop a dedicated evaluation platform to automate the testing, monitoring, and benchmarking of AI workflows.
• Investigate and integrate LLM-based evaluation techniques to automatically validate generated results and assess workflow quality.
• Work in a rapidly evolving technological environment, requiring a high degree of adaptability.

Skills

Python

Go

MLflow

CI/CD

Testing frameworks

French English comms

Education

Master's in Engineering (Bac+5 / MSc)

Tools

Simcenter Amesim API

MLflow

CI/CD tools

Job description

Siemens Digital Industries Software in Villeurbanne (near Lyon) offers a six-month internship to design and build an evaluation platform for MCP-based workflows and LLM usage, integrated with CI pipelines. You will collaborate with the Nexus team to analyze performance and automate result validation.

We seek a Master’s student in France with strong Python/Go skills, interest in AI, and experience with testing and CI/CD, ready to work in a bilingual team under mentorship. Starts January 2027.