Lead Data Engineer

Verisma; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

Verisma is seeking a Lead Data Engineer to shape our Azure-based data platform, focusing on healthcare interoperability and EHR data exchange standards. You will design scalable data architectures, build robust pipelines, and guide a team of engineers while ensuring HIPAA/HITRUST compliance across PHI data.

Responsibilities include developing ETL/ELT processes, integrating with REST APIs and external data sources, and refining Power BI models for analytics and reporting.

Qualifications

• 8+ years of data engineering experience.
• Expert-level SQL Server/T-SQL proficiency.
• Expert-level Python proficiency.
• Experience building REST APIs and background workers in Python.
• Understanding of event-driven architectures and async processing.
• Strong experience with Azure data services (SQL, Data Factory, Synapse, Blob Storage).
• Hands-on experience building and maintaining data pipelines and ETL/ELT processes.
• Power BI development skills including data modeling and performance optimization.
• Solid database design principles and data modeling.
• Version control and CI/CD practices with Git.
• Strong analytical and problem-solving abilities with communication skills.
• MicrosoftAzure certifications related to data engineering.
• Knowledge of Infrastructure as Code (Terraform, Bicep).

Responsibilities

• Design scalable, secure Azure data architectures for BI, analytics, and operations.
• Build and maintain data pipelines using Azure Data Factory and Synapse.
• Contribute to database designs across Azure SQL, PostgreSQL, CosmosDB.
• Optimize storage strategies across Azure Blob Storage, Data Lake, and databases.
• Design data models and structures for analytics and reporting.
• Develop stored procedures, functions, triggers, and views.
• Implement DB DevOps: schema versioning, deployments, rollback.
• Tune performance, indexing, and query optimization.
• Ensure data integrity, security, and HIPAA/HITRUST compliance.
• Manage database lifecycle across environments with Flyway or similar.
• ETL/ELT processes and data integration from multiple sources.
• Event-driven processing with Azure Service Bus, Event Grid, and workers.
• Integrate with external REST APIs; develop internal APIs (FastAPI/Flask).
• Develop data transformation logic and orchestration workflows.
• Ensure data quality and lineage; leverage Azure AI services as needed.
• Review Power BI semantic models, reports, and dashboards.
• Collaborate with stakeholders to translate requirements into solutions.
• Mentor junior engineers; contribute to best practices and patterns.
• Participate in architecture reviews and DevOps collaboration.

Skills

8+ years data eng

T-SQL & SQL Server

Python (expert)

REST APIs in Python

Event-driven architectures

Azure data services

ETL/ELT pipelines

Power BI data modeling

Database design & modeling

Git & CI/CD

HIPAA/HITRUST familiarity

Education

Bachelor’s degree in CS/IS/Data Eng or related

Tools

Azure Data Factory

Azure Synapse

Azure SQL Database

CosmosDB

PostgreSQL

Terraform

Bicep

ARM templates

OpenAPI/Swagger

Git

CI/CD tooling

Power BI

Job description

Lead Data Engineer

We are seeking an experienced Lead Data Engineer to join our healthcare technology team with a primary focus on healthcare interoperability. This role will be instrumental in designing, building, and maintaining our Azure-based data infrastructure, including databases, data pipelines, analytics platforms, and business intelligence solutions - with an emphasis on integrating with Electronic Health Record (EHR) systems and healthcare data exchange standards (FHIR, C-CDA, HL7). The ideal candidate will have deep expertise in the Microsoft data stack, strong architectural thinking, and experience with healthcare data compliance requirements. This senior-level position will also provide technical leadership, mentorship to the broader engineering team and reports to the Director of Data Engineering.

Duties & Responsibilities:

• Design and implement scalable, secure data architectures on Azure to support business intelligence, analytics, and operational systems
• Build and maintain data pipelines using Azure Data Factory, Azure Synapse and related services
• Contribute to database designs across multiple platforms (Azure SQL Database, SQL Managed Instance, PostgreSQL, CosmosDB)
• Optimize data storage strategies across Azure Blob Storage, Data Lake, and structured databases
• Design data models and structures that enable efficient analytics and reporting

• Design, develop, and maintain complex stored procedures, functions, triggers, and views
• Implement database DevOps processes including schema versioning, deployment automation, and rollback procedures
• Perform database performance tuning, indexing strategies, and query optimization
• Ensure data integrity, security, and compliance with HIPAA/HITRUST requirements for PHI data
• Manage database lifecycle across development, pre-production, and production environments utilizing RedGate Flyway

• Design and implement ETL/ELT processes to integrate data from multiple sources
• Build event-driven data processing solutions including message producers, consumers, and workers using Azure Service Bus, Event Grid, and related messaging services
• Integrate with external REST APIs for data ingestion and develop internal data APIs using Python (FastAPI/Flask) on Azure Container Apps or Functions
• Develop data transformation logic and orchestration workflows
• Ensure data quality, consistency, and lineage across the data platform
• Integrate with Azure AI services and machine learning endpoints as needed
• Review designs of Power BI solutions including semantic models, reports, & dashboards

• Collaborate with stakeholders to translate business requirements into technical solutions
• Optimize data structures for reporting and analytics performance

• Provide technical guidance and mentorship to junior developers and engineers
• Contribute to data engineering best practices, standards, and patterns
• Participate in architecture reviews and technical decision-making
• Collaborate with DevOps and Infrastructure teams on platform improvements
• Drive continuous improvement in data quality, performance, and reliability

• Create and maintain comprehensive technical documentation including data dictionaries, architecture diagrams, and process flows
• Document data lineage, transformation logic, and business rules
• Ensure all data handling practices meet HIPAA/HITRUST compliance requirements
• Maintain audit trails and documentation for compliance purposes

• Troubleshoot and resolve complex data-related issues across the platform
• Provide escalation support for production incidents
• Collaborate with cross-functional teams to support business operations
• Participate in on-call rotation as needed

Preferred:

• Build and maintain integrations with EHR systems using FHIR resources (e.g., Patient, DocumentReference) and C-CDA document parsing/transformation
• Support HL7 v2/MDM/ADT message flows where needed, including message parsing and acknowledgment handling
• Define and maintain OpenAPI specifications and API versioning; write contract tests to prevent breaking changes
• Lead the document templating framework and server-side rendering pipeline for HTML-to-PDF output; ensure accessibility and compliance requirements

Experience & Qualifications:

Required:

• Bachelor’s degree in computer science, Information Systems, Data Engineering, or related field, or equivalent work experience
• 8+ years of experience in data engineering, database development, or related roles
• Expert-level proficiency in T-SQL and Microsoft SQL Server
• Expert-level proficiency of Python
• Experience building REST APIs and background workers in Python (FastAPI/Flask)
• Understanding of event-driven architectures and asynchronous processing patterns
• Strong experience with Azure data services (Azure SQL, Data Factory, Synapse, Blob Storage, etc.)
• Hands-on experience building and maintaining data pipelines and ETL/ELT processes
• Knowledge of Power BI development skills including DAX, data modeling, and performance optimization
• Solid understanding of database design principles, normalization, and data modeling
• Experience with version control systems (Git) and CI/CD practices
• Strong analytical and problem-solving abilities with attention to detail
• Excellent written and verbal communication skills
• Ability to work both independently and collaboratively in a team environment
• Microsoft certification (Azure Data Engineer Associate, Azure Database Administrator, Power BI Data Analyst)
• Knowledge of Infrastructure as Code (Terraform, Bicep, ARM templates)

• Knowledge of additional Azure services (Container Apps, Functions, Logic Apps)
• Experience with other database platforms (PostgreSQL, CosmosDB)
• Understanding of HIPAA/HITRUST compliance requirements for healthcare data
• Experience PowerShell for automation and scripting
• Experience with Azure DevOps or similar platforms for CI/CD
• Experience with data governance, cataloging, and metadata management
• Background in healthcare technology or regulated industries
• Experience with OpenAPI/Swagger specification design and contract testing
• Proficient in advanced usage of AI development tooling (e.g., Claude Code, GitHub Copilot)
• Azure API Management and private networking patterns
• Travel up to 15%.

Preferred:

• Practical experience with FHIR (resources, search parameters, OAuth2/SMART on FHIR) and C-CDA parsing/transformation
• Experience integrating with Epic, Cerner, Athena, or other EHR/EMR systems
• Experience with HL7 v2 (MLLP/MDM/ADT) message handling