Lead Data Cloud Engineer

Talentola; Virginia (MN); Remote; USD 140,000 - 170,000; Full time

Job summary

TalentOla is seeking a senior data engineer to design, build, and optimize scalable data pipelines across GCP, Azure, and on-prem environments. You will lead API development in .NET and work with Kafka-based streaming and healthcare data standards.

The role is remote in the United States, requires 8+ years of experience, and offers mentorship to junior engineers while aligning with healthcare regulatory needs.

Qualifications

• 8+ years of experience in data engineering or a related field.
• Strong track record designing and maintaining scalable data pipelines and ETL processes.
• Experience with healthcare data standards (FHIR/HL7/ADT) and regulatory needs.

Responsibilities

• Design, build, and maintain scalable data pipelines using StreamSets, Azure Data Factory, and related services on Azure and GCP.
• Manage and optimize API development in .NET; some Python exposure.
• Develop and maintain event streaming with Kafka and StreamSets.

Skills

StreamSets

Azure Data Factory

Databricks

Kafka

API development (.NET)

Python

FHIR/HL7/ADT

Postgres

Snowflake

MongoDB

GCP

Azure

Kubernetes

Docker

Apigee

Datapower

Education

BE/ME/BTech/MTech/BSc/MSc

Tools

Databricks

Apigee

Datapower

StreamSets

Azure Data Factory

Kafka

Postgres

Snowflake

MongoDB

Kubernetes

Docker

GCP

Azure

Job description

Responsibilities: -

• Design, build, and maintain scalable data pipelines using StreamSets, Azure Data Factory, and other related services on Azure and GCP.
• Manage and optimize API development within the .NET environment, and to a lesser extent, Python.
• Develop and maintain event streaming services using Kafka and StreamSets.
• Administer and work with databases including Snowflake, MongoDB, and FHIR servers (Firely and Azure FHIR).
• Ensure compliance with healthcare data standards such as UDAP, CDEX, FHIR, HL7, and ADT.
• Collaborate with cross-functional teams to gather requirements and deliver tailored data solutions that meet business and healthcare regulatory needs.
• Implement API Gateways using tools like Apigee and Datapower.
• Work seamlessly across a multi-cloud environment, with 50% of infrastructure on GCP, 25% on Azure, and 25% on-premises.
• Provide technical leadership and mentorship to junior data engineers.
• Monitor, troubleshoot, and optimize the performance and scalability of data solutions.
• Stay updated with the latest trends and technologies in healthcare interoperability and cloud services.

Experience: -

8+ Years

Location: -

Remote

Educational Qualifications: -

• Engineering Degree BE/ME/BTech/MTech/BSc/MSc.

Technical certification in multiple technologies is desirable.

Skills: -

Mandatory skills

• Strong experience in Healthcare Payer Proven Management experience Strong experience in Postgres
• Proven experience with StreamSets, Azure Data Factory, and related ETL tools including Databricks.
• Proficiency in managing API development primarily using .NET and familiarity with Python.
• API Gateway management (e.g., Apigee, Datapower).
• Hands-on experience with streaming technologies like Kafka and StreamSets.
• Working knowledge of healthcare data standards such as FHIR, HL7, and ADT.
• Expertise in databases such as Snowflake, MongoDB, and FHIR servers (Firely, Azure FHIR).
• Expertise in working with multiple cloud infrastructures, including 50% on GCP, 25% on Azure, and 25% on-premises setups.
• Understanding of Firely and Azure FHIR services.
• Strong understanding of container orchestration K8s, Docker

Good to have skills: -

• Familiarity with healthcare payor systems and regulatory standards such as HIPAA etc.
• Knowledge of Programming languages such Dot Net, Python and standards such as, HL7 V2/V3, ADT, FHIR, and CCDA.
• Experience with other cloud services related to GCP and Azure ecosystems.
• Excellent communication and collaboration abilities, especially in a multi-functional, multi-cloud environment.