DataBricks Consultant-Freelancing Work-Preferred only US candidates

InOpTra Digital; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

InOpTra Digital is seeking a Databricks Architect / Lead to design and implement scalable healthcare data platforms. The role involves integrating various healthcare data sources, ensuring compliance with standards such as HIPAA, and leading data engineering initiatives. Strong expertise in Databricks, Apache Spark, and healthcare analytics is essential. The position is remote and supports advanced analytics and data modernization in healthcare environments.

Qualifications

• Strong hands-on experience with Databricks Lakehouse Platform.
• Expertise in building data pipelines using Databricks workflows.
• Understanding of healthcare data and compliance standards.

Responsibilities

• Design and implement Databricks Lakehouse architecture for healthcare.
• Integrate healthcare data sources including EHR and claims systems.
• Support AI/ML initiatives using Databricks MLflow.

Skills

Databricks Lakehouse Platform

Apache Spark (PySpark/Scala)

Data pipeline architecture

Healthcare data models

CI/CD for data pipelines

Education

Bachelor’s or Master’s degree in Computer Science or Data Engineering

Tools

Microsoft Azure

AWS

Google Cloud Platform

Delta Lake

Terraform

Job description

Role Title: Databricks Architect / Lead – Healthcare Data Platform
Experience: 15–20 Years
Location: Remote / Client Engagement Model
Domain: Healthcare & Life Sciences

Role Overview

We are seeking an experienced Databricks Architect / Lead with strong expertise in building modern healthcare data platforms on the Databricks Lakehouse architecture. The role will focus on designing scalable, secure, and compliant data solutions that support healthcare analytics, clinical insights, regulatory reporting, and AI/ML initiatives. The candidate will work closely with data engineering teams, healthcare domain experts, and enterprise architects to enable advanced analytics and data interoperability while ensuring compliance with healthcare data standards and privacy regulations.

Key Responsibilities

Data Platform Architecture

• Design and implement Databricks Lakehouse architecture for healthcare data platforms.
• Build scalable pipelines using Apache Spark, Databricks Delta Lake, and Databricks Workflows.
• Architect ingestion frameworks for structured and semi‑structured healthcare data from EMR/EHR systems, claims systems, and clinical repositories.
• Implement data governance, lineage, and security frameworks within the Databricks ecosystem.

Healthcare Data Integration

• Integrate healthcare data sources including:
  • Electronic Health Records (EHR)
  • Claims and billing systems
  • Clinical data repositories
  • Healthcare IoT and medical devices
• Enable interoperability using healthcare standards such as:
  • FHIR
  • HL7
  • DICOM (for imaging datasets where applicable)

Data Engineering & Analytics

• Develop high‑performance ETL/ELT pipelines using Databricks.
• Optimize large‑scale healthcare datasets using Delta Lake optimization techniques.
• Enable analytics for:
  • Population health
  • Clinical outcomes
  • Claims analytics
  • Healthcare operations and financial analytics
• Support AI/ML initiatives using Databricks MLflow and machine learning pipelines.

Security, Compliance & Governance

• Implement secure data architectures aligned with healthcare compliance standards such as:
  • HIPAA
  • HITRUST
  • PHI data protection frameworks
• Implement data masking, encryption, and role‑based access control.
• Enable auditing, monitoring, and regulatory reporting.

Collaboration & Leadership

• Work with clinical data teams, business analysts, and product owners to translate healthcare use cases into scalable data solutions.
• Provide technical leadership to data engineering and analytics teams.
• Define data platform standards, architecture guidelines, and best practices.
• Support client engagements and contribute to solution architecture discussions.

Required Skills & Experience

Databricks & Data Engineering

• Strong hands‑on experience with Databricks Lakehouse Platform
• Expertise in Apache Spark (PySpark/Scala)
• Experience with Delta Lake architecture and optimization
• Building data pipelines using Databricks workflows or orchestration tools

Cloud Platforms

• Microsoft Azure (preferred)
• AWS
• Google Cloud Platform

Data Architecture

• Experience with data lake, data warehouse, and lakehouse architectures
• Knowledge of data modeling (Medallion Architecture, dimensional models)
• Experience working with large‑scale healthcare datasets

Healthcare Domain Knowledge

• Understanding of healthcare data models and regulatory frameworks
• Experience with FHIR, HL7, ICD codes, CPT codes, and claims data models
• Knowledge of clinical analytics and healthcare reporting

DevOps & Automation

• Experience with CI/CD for data pipelines.
• Infrastructure as Code using Terraform / ARM / CloudFormation.
• Monitoring and performance tuning of data pipelines.

Preferred Qualifications

• Databricks certifications (Data Engineer / Architect)
• Experience implementing Real World Evidence (RWE) analytics platforms
• Exposure to GenAI or AI‑driven healthcare analytics
• Experience working with payer, provider, or life sciences organizations

Education

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, ata lakehouse platform
• Enable secure access to PHI‑compliant healthcare data
• Support advanced analytics, AI/ML, and clinical insights
• Drive data modernization initiatives within healthcare organizations