Senior Data Engineer: Microsoft Fabric Lakehouse

Cloud Analytics Technologies, LLC; Hartford (CT); Presencial; USD 140.000 - 190.000; Tempo integral

Resumo da vaga

Cloud Analytics Technologies, LLC is seeking a senior data engineering lead to design and deliver Microsoft Fabric data pipelines and analytics models for a client migrating to a unified lakehouse platform. You will shape the solution, lead hands‑on build, and mentor junior developers onsite in Hartford, CT.

You will architect ingestion pipelines, Delta Lake layouts, and Power BI models while coordinating with business and IT stakeholders to ensure production‑ready, governed assets.

Qualificações

• 8+ years in data engineering with end‑to‑end delivery on Microsoft Fabric: Lakehouse, Data Factory pipelines, notebooks, One Lake, production deployment.
• Expert-level SQL (T-SQL) and PySpark/Python with Delta Lake knowledge (ACID, partitioning, optimize/vacuum, schema evolution, performance).
• Strong dimensional modelling and medallion architecture experience with design trade‑offs.
• Experience with incremental loads, CDC patterns, and data quality frameworks at scale; Git-based CI/CD for data assets.
• Demonstrated leadership or mentoring and strong client/stakeholder communication.

Responsabilidades

• Lead design and build of ingestion and transformation pipelines using Fabric Data Factory and Spark notebooks.
• Define Lakehouse vs Warehouse choices and medallion layering on One Lake with Delta tables; document rationale.
• Design and optimize Power BI semantic models and guide report developers.
• Lead migration of workloads (SSIS, ADF, Synapse, on‑prem SQL Server) to Fabric with assessment, sequencing, and cutover.
• Set up CI/CD using Fabric Git integration and deployment pipelines with branching and promotion standards.
• Define and enforce security and governance: workspace roles, row/column security, sensitivity labels, Purview integration.
• Own performance and capacity management; tune pipelines, Spark jobs, and queries.
• Conduct code/design reviews, set engineering standards, mentor junior/mid-level developers.
• Run requirement workshops; estimate/work breakdown; flag risks and maintain runbooks.

Conhecimentos

Data engineering

SQL (T-SQL)

PySpark / Python

Delta Lake

Dimensional modelling

Mentoring/leadership

Stakeholder communication

Data CI/CD

Ferramentas

Microsoft Fabric

Delta Lake

Power BI

Azure DevOps

GitHub

Spark (PySpark)

Descrição da vaga

Cloud Analytics Technologies, LLC is seeking a senior data engineering lead to design and deliver Microsoft Fabric data pipelines and analytics models for a client migrating to a unified lakehouse platform. You will shape the solution, lead hands‑on build, and mentor junior developers onsite in Hartford, CT.

You will architect ingestion pipelines, Delta Lake layouts, and Power BI models while coordinating with business and IT stakeholders to ensure production‑ready, governed assets.