Senior Data Engineer - Healthcare Pipelines on Databricks

Robotics Technologies; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Robotics Technologies in New York, NY is seeking a Senior Data Engineer to build a modern, scalable enrollment data platform focused on X12 EDI processing. This onsite role demands hands-on data engineering with Databricks and Spark to design production pipelines for healthcare data.

You will work with Delta Lake, PySpark, Python, and Databricks tools to handle enrollment, eligibility, and claims data while maintaining HIPAA compliance and safeguarding PII/PHI.

Qualifications

• 6+ years of experience in Data Engineering.
• Hands-on Databricks experience including Delta Lake, Spark, Notebooks, and Pipelines.
• Strong knowledge of X12 EDI standards: 837, 835, 834.
• Proficiency in Python and PySpark.
• Experience building high-volume, production-grade pipelines.
• Healthcare domain experience with PII/PHI and HIPAA compliance.
• Familiarity with healthcare enrollment and eligibility data workflows.
• Ability to work independently in a fast-paced environment.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build X12 EDI processing capabilities with 837, 835, 834 transactions.
• Develop high-volume, production-grade data solutions.
• Work with Delta Lake, Apache Spark, Databricks Notebooks, and Databricks Pipelines.
• Process and transform healthcare enrollment, eligibility, and claims data.
• Ensure data security and HIPAA compliance (PII/PHI).
• Troubleshoot, optimize, and maintain pipelines.
• Operate independently within a fast-paced program environment.

Skills

Data Engineering

Healthcare data

Python

PySpark

X12 EDI

HIPAA compliance

PII/PHI handling

Independent work

Healthcare enrollment data

Tools

Databricks

Delta Lake

Apache Spark

Databricks Notebooks

Databricks Pipelines

Notebooks

Job description

Robotics Technologies in New York, NY is seeking a Senior Data Engineer to build a modern, scalable enrollment data platform focused on X12 EDI processing. This onsite role demands hands-on data engineering with Databricks and Spark to design production pipelines for healthcare data.

You will work with Delta Lake, PySpark, Python, and Databricks tools to handle enrollment, eligibility, and claims data while maintaining HIPAA compliance and safeguarding PII/PHI.