Data Engineer – Microsoft Fabric & Cloud Data Platforms

Thinkgrid Labs; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Health Insurance

5-day work week

100% remote setup with flexible work

Job summary

Thinkgrid Labs is expanding its data practice and building a modern data platform for a US health insurer, using Microsoft Fabric as the primary platform. You will design, build, and operate reliable data pipelines, lakehouse and warehouse models, and ensure data quality and governance.

We seek engineers with strong Fabric, Databricks, and cloud data platform experience, who can deliver end-to-end data solutions at scale in a remote work environment.

Qualifications

• Strong SQL and data modeling skills for large, complex databases.
• Production experience with Fabric, Databricks or similar cloud data platforms.
• Python and Spark for processing, validation, and transformation.
• Experience with CDC, incremental loading, and schema evolution.
• Knowledge of CI/CD practices for data solutions.

Responsibilities

• Build and maintain ingestion and ETL/ELT pipelines using Fabric primarily.
• Develop lakehouse/warehouse solutions across raw, refined, and curated layers.
• Transform and model data for analytics-ready datasets with clear documentation.
• Manage incremental data loads, CDC, and schema evolution safely.
• Orchestrate workflows with Fabric Data Factory and notebooks, ensuring reliability.
• Ensure quality, observability, and cost-conscious performance tuning.
• Collaborate with architects, security, analysts, and stakeholders; maintain runbooks.

Skills

SQL & relational databases

Python

Apache Spark

Data modeling

CDC / incremental loading

Pipeline orchestration

CI/CD for data solutions

Education

Bachelor’s or Master’s in Computer Science / Health Informatics / Business

Tools

Microsoft Fabric

Databricks

Fabric Data Factory

OneLake

Fabric Lakehouse/Warehouse

Job description

Thinkgrid Labs designs and builds custom web, mobile, cloud, data, and AI solutions. Our team brings together software engineers, architects, data engineers, and UI/UX designers to deliver reliable systems for clients around the world. We’re expanding our data practice and building a modern data platform for a US health insurer, with Microsoft Fabric as the primary platform. You’ll design, build, and operate reliable data pipelines—from source ingestion and transformation to analytics-ready lakehouse and warehouse models. Hands-on Microsoft Fabric experience is strongly preferred, and we welcome engineers with strong Databricks or comparable cloud data platform experience who are ready to apply their skills in Fabric. Job Title: Data Engineer – Microsoft Fabric & Cloud Data PlatformsLocation: RemoteWorking Hours: 2 PM IST to 11 PM ISTExperience Required: 6–10 years in data engineering, including 2+ years working with cloud data platformsEducation: Bachelor’s or Master’s degree in Computer Science, Health Informatics, or BusinessPreferred Certifications: Relevant Microsoft Fabric/Azure, Databricks, or equivalent data engineering certifications Who are you?

• Strong in SQL and Data Modeling: You work confidently with complex relational databases and design practical data models for integration, reporting, and analytics.
• Experienced with Cloud Data Platforms: You’ve built production solutions on Microsoft Fabric, Databricks, or comparable platforms. Experience with Fabric Mirroring, OneLake, Lakehouse/Warehouse, and Fabric Data Factory is particularly valuable.
• A Reliable Pipeline Builder: You design ingestion and transformation pipelines that handle incremental loads, CDC, schema changes, and recovery from failures.
• Comfortable with Python and Spark: You use code and notebooks to process data, implement reusable workflows, and validate results.
• Focused on Data Quality: You build checks, reconciliation, and clear data contracts into your work so downstream teams can trust the data.
• Operationally Minded: You monitor pipeline health, troubleshoot production issues, document runbooks, and balance performance with cost.

• Security Conscious: You handle sensitive data responsibly, including PII/PHI, and apply appropriate access controls and governance practices.

  What will you be doing?

• Build Data Pipelines: Develop and maintain ingestion and ETL/ELT pipelines across relational databases, files, APIs, and other sources, primarily using Microsoft Fabric.
• Develop Lakehouse and Warehouse Solutions: Organize data across raw, refined, and curated layers, using OneLake and Fabric Lakehouse/Warehouse to support reliable downstream consumption.
• Transform and Model Data: Build reusable transformations and analytics-ready models that translate source data into useful, well‑documented datasets.
• Manage Incremental Data and Change: Implement CDC, replication, or watermark‑based loading as appropriate. Handle deletes, late‑arriving data, backfills, and schema evolution safely, using Fabric Mirroring where suitable.
• Orchestrate Resilient Workflows: Manage dependencies, scheduling, retries, idempotency, and failure recovery using Fabric Data Factory, notebooks, and appropriate orchestration tools.
• Ensure Quality and Observability: Implement data validation, reconciliation, logging, metrics, lineage, and alerts to maintain data accuracy and pipeline reliability.
• Optimize Performance and Cost: Tune queries, Spark workloads, partitioning, file sizes, and compute usage to meet delivery expectations efficiently.
• Collaborate and Document: Work with platform architects, security teams, analysts, and business stakeholders on requirements, governance, and delivery. Maintain pipeline documentation, data definitions, SLAs, and operational runbooks.

Must-have skills

• Strong SQL and experience working with large, complex relational databases
• Production experience with Microsoft Fabric, Databricks, or a comparable cloud data platform
• Python and Apache Spark for data processing, transformation, and validation
• Experience building and operating ingestion and ETL/ELT pipelinesData modeling and lakehouse or data warehouse fundamentals
• CDC/incremental loading, schema evolution, and data quality practices
• Pipeline orchestration, monitoring, troubleshooting, and performance tuning
• Git-based development workflows and familiarity with CI/CD for data solutions

Benefits

• 5‑day work week
• Health Insurance
• 100% remote setup with flexible work culture and international exposure
• Opportunity to work on mission‑critical healthcare projects impacting providers and patients globally