Remote Senior Data Consultant: Microsoft Fabric Lakehouse

Trility Consulting; Clayton (MO); À distance; USD 120 000 - 180 000; Plein temps

Avantages offerts par ce poste

Paid time-off

Résumé du poste

Trility Consulting is seeking an experienced Senior Data Consultant to modernize an Azure Synapse data platform and migrate workloads to Microsoft Fabric. This remote W2 role collaborates with client engineers to design scalable, metadata-driven ingestion and CI/CD pipelines.

You'll implement PySpark-based data transformations, validate migrations, and establish repeatable patterns for maintainability while using Claude Code to review AI-generated code and ensure quality across multiple sources

Qualifications

• 5+ years with Azure Data Services including Fabric, Data Factory, Databricks, or Synapse
• Hands-on Fabric production experience with lakehouses, pipelines, notebooks, and capacity management
• Experience building production ETL/ELT pipelines and reusable data ingestion frameworks
• Designing metadata-driven ingestion frameworks with config-based onboarding
• Spark notebooks and data transformations using PySpark and Spark SQL
• Strong SQL for data transformation and performance tuning
• Experience with Git-based CI/CD for data platform artifacts (GitHub Actions / Azure DevOps)
• Hands-on Claude Code for AI-assisted development and code review
• Data migration validation and data quality assurance across systems
• Knowledge of data governance, lineage, metadata management, and catalog concepts
• Experience migrating data from Azure Synapse to Microsoft Fabric
• Experience with Oracle and DB2 sources
• Incremental ingestion, CDC, and source-specific loading strategies
• Data reconciliation and validation frameworks for large-scale migrations
• Evaluating data catalogs like Microsoft Purview

Responsabilités

• Refine Fabric architecture and ingestion patterns
• Build metadata-driven ingestion framework configurable via settings
• Migrate data sources from Azure Synapse to Fabric
• Develop Fabric pipelines and Spark notebooks (PySpark, Spark SQL)
• Analyze Synapse pipelines to translate into Fabric
• Create data validation and reconciliation for migration accuracy
• Determine full-load, incremental, or CDC loading strategies
• Implement Git-based CI/CD and environment promotions for Fabric artifacts
• Leverage Claude Code to define specs and review AI-generated code
• Evaluate data dictionary, metadata, and lineage requirements
• Apply data engineering best practices for performance, testing, security

Connaissances

ETL/ELT pipelines

SQL skills

Spark notebooks

PySpark

CI/CD concepts

AI-assisted development

Outils

Azure Data Services

Microsoft Fabric

Azure Data Factory

Databricks

Synapse

PySpark

Spark SQL

GitHub Actions

Azure DevOps

Claude Code

Description du poste

Trility Consulting is seeking an experienced Senior Data Consultant to modernize an Azure Synapse data platform and migrate workloads to Microsoft Fabric. This remote W2 role collaborates with client engineers to design scalable, metadata-driven ingestion and CI/CD pipelines.

You'll implement PySpark-based data transformations, validate migrations, and establish repeatable patterns for maintainability while using Claude Code to review AI-generated code and ensure quality across multiple sources