Senior Azure Data Platform Engineer — ETL & Lakehouse

Valiant Technologies LLC; Redmond (WA); Vor Ort; USD 140.000 - 190.000; Vollzeit

Zusammenfassung

VALIANT TECHNOLOGIES LLC seeks a senior data engineer to design ETL pipelines using Microsoft Fabric and Azure Data Factory in a Redmond, WA setting. The role focuses on building a metadata-driven Lakehouse, data lakes, and an orchestration layer to support multi-agent queries for customer inquiries.

You will enable interactive data consumption via Power BI and Azure OpenAI, while upholding data quality, governance, and security standards. Collaboration with product teams is essential.

Qualifikationen

• 7+ years of data engineering experience focused on Microsoft Azure Data stack and Fabric technologies.
• Ability to design metadata-driven architectures and reusable components.
• Experience with agent-based architectures and conversational AI integration is a plus.
• Strong governance, data modeling and security practices.

Aufgaben

• Design and implement ETL pipelines using Microsoft Fabric and Azure Data Factory.
• Build and maintain a metadata-driven Lakehouse architecture with threaded datasets.
• Develop agent-specific data lakes and an orchestration layer for cross-agent querying to answer customer questions.
• Enable interactive data consumption via Power BI, Azure OpenAI and analytics tools.
• Ensure data quality, lineage, and governance across ingestion and transformation processes.
• Collaborate with product teams to understand data needs and deliver scalable solutions.
• Optimize performance and cost across storage and compute layers.

Kenntnisse

Executive comms

Fabric Lakehouse

Azure Data Factory

ADLS Gen2

Power BI

SQL & Python

PySpark/Scala

JSON/XML/Parquet

Metadata driven

Agent architectures

Security best practices

Tools

Microsoft Fabric

Azure Data Factory

Azure SQL

Azure Data Lake Gen2

Power BI

PySpark

Scala

Jobbeschreibung

VALIANT TECHNOLOGIES LLC seeks a senior data engineer to design ETL pipelines using Microsoft Fabric and Azure Data Factory in a Redmond, WA setting. The role focuses on building a metadata-driven Lakehouse, data lakes, and an orchestration layer to support multi-agent queries for customer inquiries.

You will enable interactive data consumption via Power BI and Azure OpenAI, while upholding data quality, governance, and security standards. Collaboration with product teams is essential.