Senior Data Engineer - AI-Ready Metadata & Lineage

Mindmatch; León; Presencial; EUR 70.000 - 95.000; Jornada completa

Descripción de la vacante

MindMatch is seeking a Senior Data Engineer (Metadata & Lineage Integration) to architect and implement data foundations enabling AI adoption. You will design catalogues, semantic and analytical layers to convert large on‑premise estates into agent‑usable data, with a focus on security, governance and scalable pipelines.

The role blends data engineering, AI, and financial services—offering ownership of foundational platforms and collaboration across teams in a regulated environment.

Formación

• 5+ years building production data systems in Python with strong engineering fundamentals and solid SQL.
• Experience building crawlers, harvesters or connector frameworks to extract inventories, schemas and lineage.
• Event‑driven integration with Kafka or similar, including secure patterns and schema‑managed topics.
• Experience with search/document stores (Elasticsearch/OpenSearch/MongoDB).
• Knowledge of lineage capture/modelling (OpenLineage) and proprietary in‑house event models.
• Experience applying LLMs to metadata work and drafting descriptions.
• Willingness to work within another team's codebase and review processes.
• Fluent English for written and spoken communication with client teams.

Responsabilidades

• Build extraction, enrichment and registration paths that populate domain catalogs from live estates and connect to the central catalogue.
• Extend scraping capabilities from dataset inventories to full metadata including descriptions and dictionaries.
• Load vendor schema metadata at scale into a persistent knowledge base for humans and LLMs.
• Seed inventories from existing metadata tables and ETL sources, with LLMDrafted descriptions.
• Integrate lineage into the client’s lineage backend across batch, streaming and cross‑system chains.
• Implement federation contracts with stable identities, ownership and availability state exposed by local catalogs.

Conocimientos

Python

SQL

Data pipelines

Kafka

OpenLineage

LLMs

Crawlers

Elasticsearch

MongoDB

Airflow

English

Herramientas

Kafka

Elasticsearch

OpenSearch

MongoDB

Airflow

Descripción del empleo

MindMatch is seeking a Senior Data Engineer (Metadata & Lineage Integration) to architect and implement data foundations enabling AI adoption. You will design catalogues, semantic and analytical layers to convert large on‑premise estates into agent‑usable data, with a focus on security, governance and scalable pipelines.

The role blends data engineering, AI, and financial services—offering ownership of foundational platforms and collaboration across teams in a regulated environment.