Senior Data Consultant: Microsoft Fabric & Lakehouse

Trility Consulting LLC; Kansas; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Paid time-off, company holidays, and浮动

Job summary

Trility Consulting LLC is seeking an experienced Senior Data Consultant to modernize an Azure Synapse data platform and migrate workloads to Microsoft Fabric. You will partner with client engineers to refine Fabric architecture, build reusable ingestion patterns, and migrate data workloads into a modern lakehouse environment.

You will develop scalable data pipelines, implement validation processes for migration accuracy, and establish automated CI/CD workflows supporting deployment from

Qualifications

• 5+ years of experience with Azure Data Services and Microsoft Fabric in production environments.
• Hands-on Fabric experience in production environments, including lakehouses, pipelines, notebooks, medallion architecture, workspaces, and capacity management.
• Strong track record building and maintaining production ETL/ELT pipelines and reusable data ingestion frameworks.
• Experience designing a metadata-driven ingestion framework with configuration-based onboarding and extensible processing patterns.
• Experience developing Spark notebooks and data transformation workflows using PySpark and Spark SQL.
• Strong SQL skills for data transformation, validation, and performance tuning, including reading and understanding T-SQL logic.
• Experience with Azure Synapse, including pipeline analysis and understanding complex data processing dependencies.
• Experience implementing Git-based CI/CD workflows for data platform artifacts using GitHub Actions, Azure DevOps, or similar tooling.
• Hands-on experience using Claude Code for AI-assisted software development, including directing code generation and reviewing generated code.
• Experience validating data migrations and ensuring data quality and consistency across source and target systems.
• Experience integrating data from REST APIs, relational databases, and structured or semi-structured data sources.
• Familiarity with data governance, lineage, metadata management, and data catalog concepts.
• Experience migrating data platforms or workloads from Azure Synapse to Microsoft Fabric.
• Experience with Oracle and DB2 data sources.
• Experience incremental ingestion, CDC, and source-specific loading strategies.
• Experience developing data reconciliation and validation frameworks for large-scale migrations.
• Experience evaluating Microsoft Purview or other data catalog and lineage solutions.

Responsibilities

• Evaluate and refine the existing Microsoft Fabric architecture, including metadata structures, table-level event triggering, and reusable ingestion patterns.
• Design and implement a metadata-driven ingestion framework that allows new tables and sources to be onboarded through configuration rather than custom development.
• Migrate prioritized data sources and workloads from Azure Synapse into Microsoft Fabric.
• Develop and orchestrate Fabric data pipelines and Spark notebooks using PySpark and Spark SQL.
• Analyze existing Azure Synapse pipelines, including complex and nested workflows, to understand dependencies and translate existing processing logic into Fabric solutions.
• Build data validation and reconciliation processes to verify migration accuracy, including duplicate detection, key alignment, and record-level comparisons between Synapse and Fabric.
• Analyze source-specific loading requirements and recommend appropriate full-load, incremental, or change data capture (CDC) patterns.
• Implement Git-based CI/CD and automated environment promotion for Fabric artifacts, including pipelines and notebooks, using GitHub Actions, Azure DevOps, or similar tooling.
• Leverage Claude Code to define implementation specifications, accelerate development, and review AI-generated code for quality, correctness, and maintainability.
• Evaluate data dictionary, metadata, and lineage requirements, including potential catalog solutions such as Microsoft Purview.
• Apply data engineering best practices related to performance, testing, monitoring, documentation, security, and operational support.

Skills

Azure Data Services

Microsoft Fabric

ETL/ELT pipelines

Metadata-driven ingestion

PySpark

Spark SQL

SQL

GitHub Actions

Azure DevOps

Claude Code

Data migration

Data governance & lineage

REST APIs integration

Tools

Claude Code

GitHub Actions

Azure DevOps

Microsoft Purview

Job description

Trility Consulting LLC is seeking an experienced Senior Data Consultant to modernize an Azure Synapse data platform and migrate workloads to Microsoft Fabric. You will partner with client engineers to refine Fabric architecture, build reusable ingestion patterns, and migrate data workloads into a modern lakehouse environment.

You will develop scalable data pipelines, implement validation processes for migration accuracy, and establish automated CI/CD workflows supporting deployment from