Senior ML Engineer – VLM

Jobtailor; Deutschland; Vor Ort; EUR 110.000 - 170.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a senior ML/data engineer to own the offline dataset pipeline, building cloud-based ETL that turns multi-sensor logs into VLM/VLA training data (3D/2D detection, tracking, segmentation, depth) and annotations.

You will drive open-vocabulary labeling with foundation models, generate language-grounded reasoning, curate rare scenarios, and partner with model teams to deliver a continuous dataset loop on scalable cloud infrastructure.

Qualifikationen

• Requires complex, high-impact work with minimal supervision.
• Influence across the organization; designs, maintains, and owns team technical solutions.
• Bachelor’s degree in Computer Science, Robotics, Electrical Engineering, or related field plus competences typically acquired through 6+ years of experience; OR Master’s Degree in a related technical field plus competences typically acquired through 3+ years of experience.
• Computer Vision & Deep Learning — model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation.

Aufgaben

• Own offline dataset pipeline — design, implement, test, and deploy Cloud-based pipelines that convert logged multi-sensor data into training datasets.
• Build VLM-assisted auto-labeling — develop open-vocabulary detection, dense captioning, semantic enrichment, and scene description generation.
• Generate reasoning-grounded labels — produce language-grounded reasoning and chain-of-causation style annotations, aligned to ego-motion and trajectories.
• Mine and curate the long tail — surface rare, difficult, high-uncertainty scenarios to improve downstream metrics.
• Close the data flywheel — define dataset schemas, quality metrics, and validation; track auto-labeling quality against model requirements.
• Partner with the end-to-end model team — co-define dataset specifications and delivery cadence; operationalize a continuous dataset delivery loop.
• Scale on cloud infrastructure — build distributed pipelines with columnar data formats and distributed compute, with version control and documentation.
• Lead and mentor — serve as project lead, guide engineers, run design reviews, set coding and annotation standards.
• Stay current — track advances in multimodal models and translate research into production data systems.

Kenntnisse

Python

Cloud computing

Data pipelines

Distributed ML

Computer Vision

Deep Learning

Team leadership

Ausbildung

Bachelor’s degree in Computer Science, Robotics, Electrical Engineering, or related field

Master’s degree in a related field

Tools

Databricks

Daft

Pandas

Jobbeschreibung

Responsibilities

• Own the offline dataset pipeline — design, implement, test, and deploy Cloud-based pipelines that convert logged multi-sensor data into VLM/VLA training datasets, spanning geometric labels (3D/2D detection, tracking, segmentation, depth) through semantic, scenario-level, and action/trajectory-grounded annotations.
• Build VLM-assisted auto-labeling — develop open-vocabulary detection, dense captioning, semantic enrichment, and scene/scenario description generation that move beyond closed-set bounding boxes, using foundation models to scale annotation and cut manual labeling cost.
• Generate reasoning-grounded labels — produce language-grounded reasoning and chain-of-causation style annotations, temporally aligned to ego-motion and trajectories, to support VLA training and explainable driving behavior.
• Mine and curate the long tail — surface rare, difficult, and high-uncertainty scenarios, and build curated datasets that measurably improve downstream VLM/VLA model metrics rather than simply adding volume.
• Close the data flywheel — define dataset schemas, quality metrics, and validation; track auto-labeling quality against model requirements; route model failures back into re-labeling and retraining loops.
• Partner with the end-to-end model team — co-define dataset specifications with VLM/VLA model developers, own the quality bar and delivery cadence, and operationalize a continuous dataset delivery loop into their training pipelines.
• Scale on cloud infrastructure — build distributed, reproducible pipelines using columnar data formats and distributed compute, with disciplined software practices, version control, and documentation.
• Lead and mentor — serve as project lead, guide less-experienced engineers, run design reviews, set coding and annotation standards, and drive alignment across team interfaces to the rest of the organization.
• Stay current — track the latest advances in multimodal models, auto-labeling, and end-to-end autonomous driving, and translate relevant research into production data systems.

Requirements

• Considered highly skilled and proficient in discipline; conducts complex, important work under minimal supervision and with wide latitude for independent judgment.
• Scope of Influence: Expected to drive alignment across team interfaces to the rest of the organization. Designs, maintains, and owns team technical solutions and drives consensus. Mentors and guides engineers within the group.
• Bachelor’s Degree in Computer Science, Robotics, Electrical Engineering, or related technical field plus competences typically acquired through 6+ years of experience; OR Master’s Degree in a related technical field plus competences typically acquired through 3+ years of experience.
• Computer Vision & Deep Learning — model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation.
• Multimodal / VLM experience — hands-on work with vision-language models, open-vocabulary or zero-shot recognition, dense captioning, or semantic embeddings / search applied to perception data.
• Model Data Curation — building targeted datasets that measurably improve downstream model performance; large-scale Parquet data processing (Databricks, Daft, Pandas, etc.).
• Distributed ML & data frameworks — PyTorch, Lightning, Ray, Spark, or equivalent for training and large-scale data processing.
• Scaled MLOps & Tooling — experiment tracking, model registry, MLflow / Weights & Biases, and ML metrics, evaluation, and quality.
• Development Tools & Eco-System (at scale) — strong Python software development, VDI and cloud-based development environments, CI systems (GitHub Actions), and Docker.