Data Engineer LLM Workflows

Apple; Hamburg; Vor Ort; EUR 60.000 - 80.000; Vollzeit

Zusammenfassung

A leading technology company in Hamburg is seeking a Data Engineer specializing in LLM workflows. You will work with large-scale datasets, designing tools for data collection and model evaluation. The ideal candidate has strong Python skills and experience with data processing, web development, and machine learning frameworks. This role offers an exciting opportunity to build and enhance tools that facilitate machine learning applications.

Qualifikationen

• Experience with data processing and evaluation of LLM workflows.
• Experience building data collection and evaluation for LLM-supported systems.
• Experience deploying web tools or applications on cloud platforms.

Aufgaben

• Work directly with large-scale datasets for LLM workflows.
• Design and build web-based tools to support data collection and model evaluation.
• Develop APIs and data pipelines for seamless workflows.

Kenntnisse

Data processing

Data evaluation

Web development

Machine learning fundamentals

Data visualization

Ausbildung

BS / MS in Computer Science, Data Engineering or related field

Tools

Apache Hive

AWS

Spark

Hadoop

Kafka

Jobbeschreibung

Make a difference. As a Data Engineer specializing in LLM Workflows on the Apps Engineering team you’ll work directly with large-scale datasets by exploring their characteristics, evaluating their quality and maintaining them throughout the ML development lifecycle. You’ll deep into data for LLM workflows to understand what makes datasets effective for model training and fine-tuning adapters. Beyond dataset work you’ll design and build web-based tools that support data collection, model evaluation, and result visualization through intuitive interfaces. You’ll develop APIs and data pipelines that connect data collection tools with ML training infrastructure, creating seamless workflows for model development and fine-tuning. This role requires strong Python skills for both data analysis and web development combined with curiosity about machine learning fundamentals and LLM workflows.

• Experience with data processing and evaluation of LLM workflows.
• Experience building data collection and evaluation for LLM-supported systems.
• Experience with developing web tools using JavaScript or TypeScript.
• Experience deploying web tools or applications on cloud platforms (AWS or GCP).
• Experience with machine learning fundamentals and frameworks (scikit-learn, PyTorch).
• BS / MS in Computer Science, Data Engineering or related technical field, or 3 years of equivalent work experience.
• Experience exploring and analyzing large-scale datasets for ML applications.
• Experience with dataset curation quality assessment and bias detection methodologies.
• Experience building data collection tools or working with crowd annotation platforms.
• Experience designing data augmentation pipelines.
• Experience with data visualization libraries and creating dashboards for dataset analysis and ML metrics.
• Experience with model development or LLM fine-tuning pipelines.
• Experience working with creators, audio/video production, or creative software applications.

Key Skills

Apache Hive, S3, Hadoop, Redshift, Spark, AWS, Apache Pig, NoSQL, Big Data, Data Warehouse, Kafka, Scala

Employment Type : Full Time

Experience : years

Vacancy : 1