Principal Data Platform Engineer - Databricks & AI Scale

CODAMETRIX; Boston (MA); Hybrid; USD 175,000 - 200,000; Full time

Benefits offered by this job

Health Insurance

401(k) plan

Paid Time Off

Career development

Job summary

CodaMetrix in Boston, MA is seeking a Principal Data Engineer to lead the Databricks-based data platform for 30+ healthcare customers. You will own architecture, CI/CD, and data governance, enabling ML and analytics teams with scalable pipelines.

The role requires 8+ years in data engineering, expert PySpark/Python, Terraform, Kafka, and HIPAA-compliant design. You will mentor engineers, shape technical standards, and drive cost optimization while delivering reliable data services.

Qualifications

• 8+ years of data engineering experience with progressive responsibility.
• Expert proficiency in PySpark and Python; working knowledge of Scala.
• Expert experience with Terraform for infrastructure-as-code.
• Expert experience with Kafka / AWS MSK for streaming ingestion.
• Huge plus: healthcare data experience (EHR, ICD-10, CPT) and HIPAA familiarity.

Responsibilities

• Own the technical strategy and roadmap for the CodaMetrix data platform.
• Design, build, and maintain scalable streaming and batch data platforms on Databricks.
• Implement data governance and security controls to ensure HIPAA compliance.
• Drive platform cost reduction and ensure operational excellence.
• Mentor engineers and establish data platform standards across the organization.

Skills

PySpark

Python

SQL

Databricks

Terraform

Kafka / MSK

CI/CD

Jenkins

GitHub Actions

AWS

HIPAA

Lakehouse

Architecture leadership

Education

Bachelor's degree in CS

Master's degree

Ph.D.

Tools

Unity Catalog

Delta Lake

Structured Streaming

Spark SQL

Terraform

Jenkins

GitHub Actions

AWS IAM / S3

Job description

CodaMetrix in Boston, MA is seeking a Principal Data Engineer to lead the Databricks-based data platform for 30+ healthcare customers. You will own architecture, CI/CD, and data governance, enabling ML and analytics teams with scalable pipelines.

The role requires 8+ years in data engineering, expert PySpark/Python, Terraform, Kafka, and HIPAA-compliant design. You will mentor engineers, shape technical standards, and drive cost optimization while delivering reliable data services.