AI Research Engineer

Jobtailor; Berlin; Vor Ort; EUR 90.000 - 130.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a skilled researcher and software architect to design a multi-provider LLM council architecture and build robust evaluation tooling in a regulated domain.

You will implement deterministic replay, backtesting pipelines, and observability across large-scale inference workloads, aiming for high reliability and reproducibility.

Qualifikationen

• Master’s degree or PhD in Computer Science, Mathematics, Physics, Statistics, or a related quantitative field.
• Several years of experience developing production-ready software.
• Deep understanding of probability theory, statistics, and calibration.
• Comfortable with mathematical proofs.
• Deep experience in a technically demanding field where mathematics, scalability, and reliability all matter.
• Proficiency with modern LLMs and agent frameworks.
• High-velocity, team-oriented mindset.
• Academic publications strongly preferred.
• Open-source contributions to AI frameworks strongly preferred.
• Experience in a regulated domain (e.g., law, medicine, or finance) strongly preferred.
• Familiarity with modern workflow systems and uncertainty quantification strongly preferred.

Aufgaben

• Develop a multi-provider LLM council architecture with parallel models and voting for contentious questions.
• Implement deterministic replay for reproducible evaluations.
• Build an evaluation framework with stratified gold-standard datasets.
• Measure inter-rater reliability against human lawyers.
• Develop backtesting pipelines and confidence calibration.
• Research a mathematically rigorous approach to quantifying legal robustness.
• Produce original research contributions, conference papers, and open-source releases.
• Build production-ready orchestration infrastructure for tens of thousands of inferences per day across model families.
• Ensure complete observability and fault tolerance.

Kenntnisse

High-Velocity Mindset

Team-Oriented Mindset

Academic Publications

Open-Source Contributions

Experience in a Regulated Domain

Proficiency with LLMs

Ausbildung

Master’s or PhD in CS/Math/Physics/Statistics

Tools

LLM Frameworks

Workflow Systems

Inference Infrastructure

Jobbeschreibung

• Develop a multi-provider LLM council architecture in which multiple models work in parallel and vote on contentious legal questions
• Implement deterministic replay for reproducible evaluations
• Build an evaluation framework with stratified gold-standard datasets
• Measure inter-rater reliability against human lawyers
• Develop backtesting pipelines and confidence calibration
• Research a mathematically rigorous approach to quantifying legal robustness
• Produce original research contributions, conference papers, and open-source releases
• Build production-ready orchestration infrastructure for tens of thousands of inference calls per day across multiple model families
• Ensure complete observability and fault tolerance

Requirements

• Master’s degree or PhD in Computer Science, Mathematics, Physics, Statistics, or a related quantitative field
• Several years of experience developing production-ready software
• Deep understanding of probability theory, statistics, and calibration
• Comfortable with mathematical proofs
• Deep experience in a technically demanding field where mathematics, scalability, and reliability all matter
• Proficiency with modern LLMs and agent frameworks
• High-velocity, team-oriented mindset
• Academic publications strongly preferred
• Open-source contributions to AI frameworks strongly preferred
• Experience in a regulated domain (e.g., law, medicine, or finance) strongly preferred
• Familiarity with modern workflow systems and uncertainty quantification strongly preferred

Core Competencies

Demonstrates expertise in developing multi-provider LLM architectures and implementing evaluation frameworks, with a strong foundation in probability theory and statistics. Proven ability to produce original research and contribute to open-source AI frameworks in regulated domains.

Highest-signal resume keywords

• Master’s Degree Or PhD In Computer Science
• Deep Understanding Of Probability Theory
• Proficiency With Modern LLMS
• Experience In A Regulated Domain
• Production-Ready Software Development

Hard Skills

• Deterministic Replay Implementation
• Evaluation Framework Development
• Backtesting Pipelines
• Confidence Calibration
• Mathematical Proofs
• Inter-Rater Reliability Measurement
• Stratified Gold-Standard Datasets
• Quantifying Legal Robustness
• Observability And Fault Tolerance
• Scalability

Soft Skills

• High-Velocity Mindset
• Team-Oriented Mindset

Industry Keywords

• Legal Domain
• Regulated Domain
• Open-Source Contributions
• Academic Publications

Tools & Technologies

• LLM Frameworks
• Workflow Systems
• Inference Infrastructure