Senior Data Engineer: Real-Time AI-Ready Pipelines

Marvik; Municipio de Esquel; Vor Ort; ARS 106.110.000 - 166.745.000; Vollzeit

Zusammenfassung

Marvik is seeking a hands-on Data Engineer to turn messy fabrication data into AI-ready datasets. You will build scalable ingestion and ETL pipelines within Microsoft Fabric and cloud lakehouse environments, focusing on real-time streams and layered Medallion architectures.

You’ll ensure data quality through deduplication, schema validation, and lineage tracking, delivering curated datasets for BI, analytics, vector search, and AI/ML workloads with strong observability.

Qualifikationen

• 5+ years of hands-on data engineering experience building and operating production data pipelines at scale.
• Strong proficiency in Python, SQL, and PySpark / Apache Spark with solid software engineering fundamentals (Git, CI/CD).
• Experience with real-time data ingestion and streaming pipelines (not just batch).
• End-to-end data modeling, schema design, and layered lakehouse architectures (Medallion architecture).
• Experience with cloud-native lakehouse platforms; familiarity with Microsoft Fabric is highly preferred.

Aufgaben

• Build and operate scalable ingestion, ELT/ETL, and orchestration pipelines within Microsoft Fabric and cloud lakehouse environments.
• Design and implement low-latency real-time data ingestion flows for live operational analytics.
• Implement layered lakehouse architectures (Bronze/Silver/Gold) using PySpark/SQL with idempotent, backfillable, and incrementally loaded jobs.
• Apply deduplication, normalization, schema validation, and lineage tracking for data quality and governance.
• Deliver feature-ready, curated datasets to support BI, analytics, vector search, and AI/ML workloads.
• Establish testing, monitoring, and pipeline observability with alerting to resolve failures proactively.

Kenntnisse

Python

SQL

PySpark / Apache Spark

Data pipelines

Git / CI/CD

Unit / integration testing

Tools

Microsoft Fabric

Data Factory

Synapse Analytics

MongoDB

Kafka

Event Hubs

Jobbeschreibung

Marvik is seeking a hands-on Data Engineer to turn messy fabrication data into AI-ready datasets. You will build scalable ingestion and ETL pipelines within Microsoft Fabric and cloud lakehouse environments, focusing on real-time streams and layered Medallion architectures.

You’ll ensure data quality through deduplication, schema validation, and lineage tracking, delivering curated datasets for BI, analytics, vector search, and AI/ML workloads with strong observability.