Senior Data Engineer – Microsoft Fabric & Healthcare Integrations

Quantum Pain & Sports Medicine; Irving (TX); Presencial; USD 120.000 - 160.000; Jornada completa

Descripción de la vacante

Quantum Pain & Sports Medicine is building a centralized data and analytics platform on Microsoft Fabric to power reliable reporting, operational analytics, and AI use cases.

The Senior Data Engineer will design Fabric architecture, implement ingestion pipelines, and ensure data quality and governance as the platform grows, with a strong emphasis on security and scalable pipelines.

Formación

• 4+ years of professional data engineering or data integration experience.
• Hands-on with Microsoft Fabric or Azure data stack.
• Strong SQL skills.
• Proficient in Python and/or PySpark.
• Experience designing ETL/ELT pipelines and ingestion frameworks.
• Knowledge of Lakehouse architectures and Delta tables.
• Experience with REST APIs and third-party integrations.
• Familiarity with Git and CI/CD practices.
• Ability to implement incremental data loads.

Responsabilidades

• Design and implement the Microsoft Fabric data architecture using OneLake and Lakehouse components.
• Build and maintain ingestion pipelines from eClinicalWorks, HubSpot, APIs, and databases.
• Create analytics-ready datasets to support Power BI reporting and AI use cases.
• Implement incremental loading, change data capture, schema management, and monitoring.
• Establish data quality checks and robust security and governance practices.
• Use Git and CI/CD to keep pipelines version-controlled and maintainable.
• Document architecture, data definitions, and operating procedures.

Conocimientos

SQL

Python

PySpark

ETL/ELT design

Data integration

Dimensional modeling

Herramientas

Microsoft Fabric

OneLake

Lakehouse/Warehouse

Data Factory pipelines

Delta tables

Git

CI/CD

REST APIs

Descripción del empleo

Quantum Pain & Sports Medicine is building a centralized data and analytics platform in Microsoft Fabric, with a focus on dependable healthcare and business integrations. This Senior Data Engineering role is centered on infrastructure and ingestion pipelines in Fabric, so trusted, analytics-ready datasets can power reporting, operational analytics, automation, and future AI use cases.

You will help design the foundation early, implement the ingestion and data layers that make data reliable, and operate the platform so additional systems and new analytics needs can be added without rebuilding everything.

What you will do

• Design and implement the Microsoft Fabric data architecture using OneLake, Lakehouse/Warehouse, Data Factory pipelines, notebooks, and related Fabric services.
• Build and maintain ingestion pipelines from eClinicalWorks, HubSpot, APIs, databases, files, and future enterprise systems.
• Integrate healthcare and operational systems using REST APIs, FHIR APIs, webhooks, batch exports, and other extraction approaches.
• Implement a medallion-style structure with raw, validated, and analytics-ready layers.
• Clean, standardize, deduplicate, and reconcile data across systems, including building patient, referral, appointment, provider, procedure, case, outcome, and other operational data models.
• Create matching methods for records when identifiers are inconsistent or unavailable.
• Deliver reusable curated datasets that support Power BI reporting, operational analytics, automation, and future AI applications.
• Implement incremental loading, change detection, schema management, error handling, logging, alerting, and pipeline monitoring.
• Set up data quality checks to identify incomplete, inconsistent, or unexpected source data.
• Apply healthcare-appropriate security, access controls, and governance practices.
• Use Git and CI/CD so pipelines and transformations remain version-controlled, documented, and maintainable.
• Monitor Fabric usage, performance, capacity, and cost as the platform grows.
• Document architecture, data definitions, pipelines, dependencies, and operating procedures.
• Maintain and improve the platform after initial implementation, rather than treating it as a one-time migration.
• Assess existing systems, data sources, reporting requirements, and the Microsoft environment, then design and configure the initial Fabric architecture.

Required qualifications

• 4+ years of professional experience in data engineering, data platform, or data integration.
• Hands-on experience building production solutions with Microsoft Fabric or a closely related Azure data stack.
• Strong SQL skills.
• Strong Python and/or PySpark skills.
• Experience designing ETL/ELT pipelines and data ingestion frameworks.
• Experience with Microsoft Fabric Lakehouse, OneLake, Data Factory/Pipelines, notebooks, Delta tables, or equivalent technologies.
• Experience integrating third-party systems via REST APIs.
• Experience implementing incremental data loads rather than relying only on full refreshes.
• Strong understanding of dimensional modeling, relational modeling, lakehouse architecture, and analytics-oriented data modeling.
• Experience troubleshooting API authentication, pagination, rate limits, schema changes, malformed records, pipeline failures, and other real-world integration issues.
• Experience with Git, source control, and CI/CD practices.
• Experience implementing monitoring, logging, data-quality validation, and alerting for production pipelines.