Junior AI/ML Engineer, Data and Evaluation

RedMimicry GmbH; Berlin; Vor Ort; EUR 70.000 - 100.000; Vollzeit

Zusammenfassung

RedMimicry GmbH in Berlin is building a capability to turn security logs into explainable, AI-powered threat analysis inside our platform.

As an AI/ML Engineer you will curate datasets, build ground truth, maintain evaluation pipelines, and run experiments with a strong focus on data quality and reproducibility. This fixed-term role runs to 31 October 2027 and supports Berlin office access or remote work in Germany.

Qualifikationen

• Strong practical work, research projects, or a relevant thesis can compensate for limited commercial experience.
• Degree in computer science, data science, machine learning, mathematics, cybersecurity, or related discipline, or equivalent practical experience.
• English required; German a plus.
• Experience with Python, Pandas, NumPy, PyTorch, or similar tooling.
• Ability to handle structured and unstructured data reliably.
• Familiarity with Git, automated tests, and reproducible development workflows.

Aufgaben

• Curate telemetry datasets from controlled engagements and test environments.
• Build ground truth via labeling guidelines and consistency checks.
• Maintain train/validation/test partitions along relevant dimensions.
• Automate benchmarks and maintain reproducible pipelines for local and CI runs.
• Run experiments with error analysis and identify recurring data-quality issues.
• Maintain data quality with provenance, deduplication, and versioning.
• Document experiments with records, plots, and technical summaries.

Kenntnisse

Python programming

Pandas/NumPy/PyTorch

Data processing

Git & reproducible workflows

Experiment tracking

Statistics basics

Ausbildung

Bachelor's or higher in CS/Data Science/ML/Math/Cybersecurity

Tools

Git

Jobbeschreibung

We are building a new capability that turns fragmented, noisy security logs into explainable, AI-powered threat analysis, delivered inside the RedMimicry platform.

As an AI/ML Engineer at RedMimicry, you will build the datasets, benchmarks, and evaluation infrastructure behind our applied AI/ML work on security telemetry.

The quality of this work depends on accurate ground truth, disciplined data handling, reproducible experiments, and systematic error analysis.

Your work will determine whether reported improvements are real and whether regressions are caught before deployment.

You will work closely with the Senior AI/ML Engineer and Offensive Security Engineer.

This role is suitable for an early-career or intermediate engineer with strong programming and data skills who wants to work on applied AI in a technically demanding cybersecurity environment.

This is a fixed-term position running until 31 October 2027.

Tasks

• Curate Security Datasets: Prepare telemetry from controlled engagements and test environments.
• Build Ground Truth: Create and maintain labelled examples, annotation guidelines, and consistency checks.
• Maintain Evaluation Partitions: Separate training, validation, and test data along relevant dimensions.
• Automate Benchmarks: Implement metrics and maintain reproducible benchmark and regression pipelines that run locally and in CI.
• Run Experiments and Error Analysis: Evaluate models and methods, and identify recurring failure modes and data-quality issues.
• Maintain Data Quality: Implement schema checks, provenance tracking, deduplication, validation, and dataset versioning.
• Document Experiments: Produce clear experiment records, plots, tables, and technical summaries that support engineering decisions.

Requirements

You do not need to meet every requirement to apply. Strong practical work, research projects, open-source contributions, or a relevant thesis can compensate for limited commercial experience.

Programming and Data Work

• Good Python programming skills
• Experience with Pandas, NumPy, PyTorch, or similar tooling
• Ability to process structured and unstructured data reliably
• Familiarity with Git, automated tests, and reproducible development workflows

Machine Learning and Evaluation

• Working understanding of supervised evaluation, train/test separation, and basic statistics
• Familiarity with language models, embeddings, retrieval, classification, or information extraction
• Experience with experiment tracking, benchmarking, or regression testing
• Ability to distinguish statistically meaningful results from anecdotal examples

Data Quality and Annotation

• Careful handling of labels, provenance, edge cases, and ambiguous examples
• Experience with dataset annotation or benchmark construction is a plus
• Ability to write clear annotation and evaluation guidelines

Cybersecurity Knowledge

• Security logs, operating-system telemetry, SIEM, EDR, network data, or incident-response experience is a plus
• Interest in attacker behaviour, detection, and evidence-based security analysis

Education

• Degree in computer science, data science, machine learning, mathematics, cybersecurity, or a related discipline, or equivalent practical experience

Languages

• English (required)
• German (a plus)

Benefits

• Work from anywhere in Germany, and use our Berlin office as often as you like
• 30 days of paid time off, and a quiet inbox while you are away
• Company-paid Deutschlandticket
• Annual budget for the courses and certifications you pick yourself
• Modern tooling and extensive use of AI
• Light process, clear communication, focus on what really matters

A close match is enough. What follows is short and transparent, a few conversations with the team and then your first week in Berlin. We are an equal opportunity employer and welcome applications from all backgrounds and genders. Questions about the role or the process are welcome at any point.