Senior Data Engineer – Microsoft Fabric Lakehouse & Pipelines

Machine Learning Technologies, LLC; Hartford (CT); On-site; USD 82,000 - 117,000; Full time

Job summary

Machine Learning Technologies, LLC is seeking a Data Engineer with expert-level Microsoft Fabric experience in Hartford, CT to own the data pipelines and analytics models on the Fabric Lakehouse platform.

You will lead design, hands-on build, and mentor junior developers, acting as the day-to-day technical contact for business and IT stakeholders, with minimal supervision and production-ready governance of assets.

Qualifications

• 8+ years in data engineering with end-to-end Fabric delivery.
• Expert SQL (T-SQL) and PySpark with Delta Lake knowledge.
• Strong dimensional modelling and medallion architecture experience.
• Experience with incremental loads, CDC patterns, and data quality frameworks at scale.
• Git-based version control and CI/CD for data assets, with experience setting it up.

Responsibilities

• Lead design and build of ingestion and transformation pipelines using Fabric Data Factory (pipelines, Dataflows Gen2) and Spark notebooks (PySpark / Spark SQL).
• Define Lakehouse vs Warehouse choices and the medallion (bronze/silver/gold) layering on One Lake with Delta tables, and document the rationale.
• Design and optimize Power BI semantic models, including Direct Lake where appropriate, and guide report developers.
• Lead migration of existing workloads (SSIS, ADF, Synapse, on-prem SQL Server) to Fabric, including assessment, sequencing, and cutover.
• Set up and own CI/CD using Fabric Git integration and deployment pipelines (Azure DevOps or GitHub), including branching strategy and promotion standards.
• Define and enforce security and governance: workspace roles, row/column-level security, sensitivity labels, Purview integration.
• Own performance and capacity management (CU consumption, throttling, SKU sizing inputs), and tune pipelines, Spark jobs, and queries.
• Conduct code and design reviews, set engineering standards, and mentor junior and mid-level developers.
• Run requirement workshops and demos, estimate and break down work, flag risks early, and keep designs and runbooks current.

Skills

Microsoft Fabric

SQL (T-SQL)

PySpark / Python

Delta Lake

Data modeling

Leadership/mentoring

Stakeholder communication

CI/CD

Tools

Fabric Data Factory

Spark notebooks

Azure DevOps

GitHub

One Lake

Job description

Machine Learning Technologies, LLC is seeking a Data Engineer with expert-level Microsoft Fabric experience in Hartford, CT to own the data pipelines and analytics models on the Fabric Lakehouse platform.

You will lead design, hands-on build, and mentor junior developers, acting as the day-to-day technical contact for business and IT stakeholders, with minimal supervision and production-ready governance of assets.