Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC; Hartford (CT); On-site; USD 84,000 - 116,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC in Hartford, CT is seeking a seasoned Data Engineer to design and deliver data pipelines and analytics models on Microsoft Fabric for a client moving to a unified lake house platform. You will lead hands-on build, mentor junior developers, and serve as the technical lead for business and IT stakeholders on site.

You will drive ingestion pipelines, medallion architecture decisions, and secure data assets while coordinating with BI teams and operations.

Qualifications

• 8+ years of data engineering with end-to-end delivery on Microsoft Fabric including Lakehouse, Data Factory pipelines, and notebooks.
• Expert-level SQL (T-SQL) and PySpark/Python with Delta Lake knowledge and performance tuning.
• Strong dimensional modelling and medallion architecture experience with sound trade-offs.
• Power BI semantic modelling and Direct Lake considerations for reporting.
• Experience with CDC, incremental loads, data quality, and CI/CD for data assets.

Responsibilities

• Lead design and build of ingestion and transformation pipelines on Fabric Data Factory and Spark notebooks.
• Define lakehouse vs warehouse strategies and medallion layering documentation.
• Design and optimize Power BI models and guide report developers.
• Lead migration of workloads to Fabric, including assessment and cutover planning.
• Set up and own CI/CD with Fabric Git integration and deployment pipelines.
• Define security and governance: workspace roles, data sensitivity, and Purview integration.
• Manage performance, capacity, and tuning of pipelines, Spark jobs, and queries.
• Conduct code reviews, mentor junior developers, and establish engineering standards.
• Run workshops, estimate work, and maintain current runbooks.

Skills

8+ years data engineering

Microsoft Fabric expertise

SQL (T-SQL)

PySpark/Python

Delta Lake knowledge

Power BI modelling

CDC patterns & data quality

Git CI/CD for data assets

Mentoring / code reviews

Tools

Microsoft Fabric

Azure DevOps

GitHub

Job description

Job Overview

Job ID: J53974
Job Title: Data Engineer
Location: Hartford, CT
Duration: 7 Months + Extension
Bill Rate: $72/hour (C2C)
Domain Exposure: Insurance
Client: To Be Discussed Later
Employment Type: W-2, 1099, C2C
Work Authorization: USC, GC, H-1B, OPT-EAD, GC-EAD, H4-EAD

Job Summary

You will own the design and delivery of data pipelines and analytics models on Microsoft Fabric for a client moving to a unified lake house platform. Working onsite, you are the senior technical voice in the room: you shape the solution design, lead hands-on build, mentor junior developers, and be the day-to-day technical contact for business and IT stakeholders. You are expected to work with minimal supervision and hand over production-ready, governed assets.

Responsibilities

• Lead design and build of ingestion and transformation pipelines using Fabric Data Factory (pipelines, Dataflows Gen2) and Spark notebooks (PySpark / Spark SQL).
• Define Lakehouse vs Warehouse choices and the medallion (bronze/silver/gold) layering on One Lake with Delta tables, and document the rationale.
• Design and optimize Power BI semantic models, including Direct Lake where appropriate, and guide report developers.
• Lead migration of existing workloads (SSIS, ADF, Synapse, on-prem SQL Server) to Fabric, including assessment, sequencing, and cutover.
• Set up and own CI/CD using Fabric Git integration and deployment pipelines (Azure DevOps or GitHub), including branching strategy and promotion standards.
• Define and enforce security and governance: workspace roles, row/column-level security, sensitivity labels, Purview integration.
• Own performance and capacity management (CU consumption, throttling, SKU sizing inputs), and tune pipelines, Spark jobs, and queries.
• Conduct code and design reviews, set engineering standards, and mentor junior and mid-level developers.
• Run requirement workshops and demos, estimate and break down work, flag risks early, and keep designs and runbooks current.

Mandatory Skills

• 8+ years in data engineering, with proven end-to-end delivery on Microsoft Fabric: Lakehouse, Warehouse, Data Factory pipelines, notebooks, One Lake shortcuts, production deployment.
• Expert-level SQL (T-SQL) and PySpark/Python, with deep Delta Lake knowledge (ACID, partitioning, optimize/vacuum, schema evolution, performance tuning).
• Strong dimensional modelling and medallion architecture experience, including making and defending design trade-offs.
• Power BI semantic modelling: DAX, Direct Lake vs Import vs Direct Query trade-offs, model performance.
• Experience with incremental loads, CDC patterns, and data quality frameworks at scale. Git-based version control and CI/CD for data assets, with experience setting it up, not just using it.
• Demonstrated experience leading or mentoring a small team and doing code reviews. Strong communication skills for working directly with client stakeholders, including senior business and IT leaders.

Equal Opportunity Employer

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.