Middle Data Engineer with ML skills

Madfish; Deutschland; Vor Ort; EUR 60.000 - 80.000; Vollzeit

Zusammenfassung

A technology company in Germany is seeking a mid-level Data Engineer with ML experience to deploy production models at scale. The role involves designing scalable ML pipelines and working closely with data scientists. Key qualifications include AWS experience, proficiency in Python, and experience with data engineering processes. This position offers an exciting opportunity to apply state-of-the-art models with significant revenue impact.

Qualifikationen

• Experience designing and implementing large-scale production workflows within AWS.
• Experience transitioning notebooks into production code.
• Strong proficiency in Python.

Aufgaben

• Design scalable ML pipelines that integrate into existing architecture.
• Take ownership of ML ops and data engineering deliverables.
• Support deployment of models into production environments.
• Build internal tools to showcase ML outputs.

Kenntnisse

Designing scalable ML pipelines

AWS experience

Python

Pipeline Orchestration

Tools

AWS (lambda, kafka, kinesis, EKS, step functions)

Databricks

Sagemaker

Airflow

MLFlow

DBT

Spark/PySpark

Jobbeschreibung

N-iX is looking for a mid-level Data Engineer with ML experience to deploy production ML models at a significant scale.

As part of the data licensing team, you will work closely with data science and data engineering colleagues to apply state-of-the-art models in production.

Projects will vary from deploying models that provide new metadata for the existing catalogue to developing novel techniques for asset retrieval. Given the magnitude of data and AI focus, this role presents unique and exciting opportunities that have an immediate revenue impact.

Our customer is a technology company that powers one of the world's largest two-sided marketplaces for high-quality photos, illustrations, videos and music used by individuals, businesses, marketing agencies and media organisations of all sizes.

Responsibilities

• Work with data scientists to design scalable ML pipelines that integrate into existing architecture.
• Under the guidance of a principal engineer, take ownership of all ML ops and data engineering deliverables for an assigned project.
• Support the deployment of models into production environments (batch and/or real-time), including containerization, inference orchestration, and integration with relevant tools and services.
• Build internal tools (e.g., Streamlit apps) to showcase ML outputs.

Requirements

Essential:

• Experience designing and implementing large-scale cost cost-effective production workflows within AWS (lambda, kafka, kinesis, EKS, step functions)
• Experience working with data scientists to transition notebooks (Databricks, Sagemaker) into production code.
• Python
• Pipeline Orchestration, ideally Airflow.

Nice to have:

• MLFlow
• DBT
• Frontend Experience
• Spark / PySpark