Data Engineer for AI (f/m/d)

Allianz Popular SL.; Frankfurt; Presencial; EUR 90.000 - 130.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Hybrid work model

Company pension/savings plans

Mobility solutions (Jobrad, Jobticket)

Career opportunities within Allianz

Learning & development

Volunteering time

Descripción de la vacante

Allianz Global Investors is seeking a Data Engineer for AI in Frankfurt to design, build and operate data products on the Data and AI Platform. You will enable self-service access, ensure data quality, and implement governance-by-design across AI/ML use cases, with Databricks and Foundry as core technologies.

You will collaborate with data scientists and ML engineers to deliver reusable pipelines, feature stores and GenAI data workflows while continuously evolving the platform’s capabilities and

Formación

• Minimum 3 years of practical experience building and operating data pipelines and data products at scale on Databricks.
• Strong understanding of data engineering concepts for AI/ML and data quality, contracts and reproducibility.
• Expert-level skills in Databricks, Spark, Python and SQL, with CI/CD and infrastructure-as-code on Azure.
• Diploma in computer science or a similar field.
• Good understanding of Lakehouse, Delta, medallion patterns and data product thinking in governed environments.
• Excellent analytical, planning and organizational skills; strong English communication.

Responsabilidades

• Develop, maintain and promote standardized data pipelines, templates and utilities for AI/ML datasets.
• Build and operate batch and streaming pipelines on Databricks for trusted AI/ML consumption.
• Establish controlled dataset evolution and enforce data quality and freshness.
• Design feature engineering pipelines and manage a feature store with versioning and docs.
• Collaborate on GenAI/RAG data pipelines and ingestion patterns, updates and embeddings.
• Enable systematic experiments and model retraining using feedback loops.
• Evolve platform capabilities for ingestion, transformation and serving with governance controls.
• Onboard new data sources with Lakeflow connectors and standardize ingestion.
• Provide L3 support for pipeline incidents and drive continuous platform improvements.

Conocimientos

Databricks

Spark

Python

SQL

CI/CD

Terraform

Azure

Data engineering concepts

AI/ML data modelling

Data quality

Governance

Documentation

English communication

Educación

Diploma in computer science or a similar field

Herramientas

Databricks

Spark

Python

SQL

Terraform

Azure

Descripción del empleo

In the role of Data Engineer for AI (f/m/d), you will design, build and operate data products and reusable data preparation components on the Data and AI Platform at Allianz Global Investors, built on Databricks and Foundry.

Your focus is to enable platform customers to reliably source, ingest, transform, validate and serve high-quality, compliant data for AI and ML use cases (including analytics and GenAI), so teams can consume data in a self-service manner. You will provide technical guidance and L3 support to delivery teams, define and promote best practices for data pipelines and data quality, and ensure adherence to security, privacy and regulatory requirements following a compliance-by-design approach.

In addition, you will continuously evolve the platform’s data engineering capabilities by integrating new features (e.g., ingestion patterns, transformation frameworks, governance controls and monitoring) and by delivering standardized, reusable pipelines and templates that scale across use cases. Databricks is the platform’s primary technology for data engineering, with Microsoft Foundry part of its wider scope.

This position will be based in Frankfurt

What you will do

• Reusable data preparation building blocks & ML‑ready pipelines: Develop, maintain and enhance standardized pipeline patterns, templates and utilities (ingestion, transformation, validation, enrichment) to deliver AI/ML‑ready datasets. Build robust pipelines for structured and unstructured data with reproducible, deterministic outputs
• Data pipeline engineering on Databricks: Build and operate reliable batch and streaming pipelines to ingest data from internal and external sources, curate datasets, and publish trusted data products for AI, ML and analytical consumption
• Curated data products (Bronze/Silver/Gold): Establish controlled dataset evolution, enforce data quality and freshness, and curate gold tables explicitly designed for analytical, ML and AI use cases.
• Feature engineering & feature store: Design, build and operate feature engineering pipelines and curate a ready‑to‑use feature store with proper versioning and lightweight documentation (ownership, purpose, inputs/outputs, SLAs)
• GenAI / RAG data & embedding pipelines: Collaborate with ML and AI Engineers to build and operate pipelines for GenAI/RAG systems, including new data ingestion, Bronze/Silver updates, chunking, embedding refresh and index updates to maintain knowledge base relevance
• Systematic experiments & model improvement loops: Enable systematic experimentation (offline evaluations, A/B tests) and support model retraining using user feedback (binary and non‑binary) to continuously improve models
• Platform capability evolution (Data for AI): Collaborate with the Data and AI Platform Architect to introduce and operationalize platform capabilities for ingestion, transformation and serving (Delta/Unity Catalog patterns, orchestration, monitoring), making them available securely and consistently
• New data source onboarding & Lakeflow connectors: Facilitate provisioning and rollout of Databricks Lakeflow connectors to integrate new data sources and standardize ingestion across domains
• Security, governance & compliance‑by‑design: Implement and enforce governance using Unity Catalog, including access control, lineage, retention, encryption and auditing, ensuring compliance with internal standards and regulations
• Data quality, validation & reliability: Establish automated data quality checks and validations (e.g., DQX), define SLAs/SLOs for freshness and reliability, and implement monitoring and observability
• Performance & cost optimization: Optimize Spark workloads, storage layouts and compute usage to ensure scalable, stable and cost‑efficient data pipelines
• Collaboration & enablement: Partner with Data Scientists, ML Engineers, AI CoE, DevOps and SecOps to define data requirements, promote reusable patterns, and enable self‑service through documentation, coaching and reviews
• Operations & support: Provide L3 support for pipeline and data product incidents, perform root‑cause analysis, and drive continuous improvements to platform stability and customer satisfaction
• Integration & data product exposure: Integrate Databricks with (Azure) cloud services, enterprise systems, external providers and SaaS platforms, and expose curated data products securely to downstream consumers (feature stores, model training/inference, BI)

What you bring

Required:

• Minimum 3 years of practical experience building and operating data pipelines and data products at scale on Databricks (batch and/or streaming), ideally in environments with strong governance requirements
• Strong understanding of data engineering concepts for AI/ML (data modelling, data quality, data contracts, feature engineering collaboration, reproducibility) and how data characteristics impact model outcomes
• Expert‑level technology skills in Databricks, Spark, Python and SQL, plus solid engineering practices such as CI/CD and infrastructure‑as‑code (e.g., Terraform) on Azure
• Diploma in computer science or a similar field
• Very good understanding of data architecture and platform patterns (Lakehouse concepts, Delta, medallion approaches, data product thinking) and how to operationalize them in a governed enterprise context
• Excellent analytical, planning, and organizational skills
• Good oral and written communication skills in English
• Ability to efficiently and effectively document and share knowledge and enable others

Preferred:

• Working experience in the financial industry, preferably in asset management
• International work experience
• Certification “Databricks Certified Data Engineer Professional” (or equivalent)
• Certification “Databricks Certified Data Engineer Associate” (or equivalent)
• Optional: Certification “Databricks Certified Generative AI Engineer Associate” (helpful for GenAI‑related data preparation patterns)
• Hands‑on exposure to Microsoft Foundry (Azure AI Foundry), or a strong willingness to build expertise in it

What we offer

• We empower our employees by ensuring flexible work arrangements that maintain a balance between performance, productivity, career development and personal priorities (e.g., hybrid model/ flexible working hours)
• Securing your future: Access to company pension/savings plans
• Family support (relocation/ childcare facilities)
• Company share purchasing plan
• Mental health and wellbeing programs
• Mobility solutions (Jobrad bike leasing, subvention Jobticket)
• Career opportunities within the entire Allianz Group
• Self‑guided learning & development
• Volunteering time
• … and so much more!

About Allianz Global Investors

Allianz Global Investors is a leading active asset manager, managing assets for individuals, families,and institutions worldwide. Our shared goal to shape a future of opportunity for our clients is foundational to our culture and our values. We look to attract, and actively nurture, individuals with the drive to achieve excellence at every stage of the investment experience.

We believe an inclusive culture is key to a flourishing workforce. Not only do we celebrate the broad range of experiences, backgrounds, and attributes that our workforce brings, but we support our people to live and work in a way that promotes their own wellbeing.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges is what makes us a unique employer.

We are united by a shared commitment: to put our clients first and at the centre of everything we do. Their needs inspire our thinking and guide our actions.

Together, we can build an environment where everyone feels empowered and confident to explore, grow and shape a better future – for our clients and for the world around us. At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. The foundation for this is our inclusive workplace, where people and performance both matter, and where integrity, fairness, inclusion and trust are at the heart of our culture.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

If you feel inspired to promotethe activeasset management experience, this is the place for you. Join our international, technology-enabled, and agile environment. Simply upload your CV in English to apply for this position. If you need support to navigate our websites or at any stage during your application, please send an email with your request to [e-mail].

Great to have you on board.Let'scarefortomorrow.