Senior Data Platform Engineer, AI-Driven Data Lakehouse

CodaMetrix; Boston (MA); On-site; USD 130,000 - 190,000; Full time

Benefits offered by this job

Health Insurance

401(k) plan

Flexible PTO

Professional development

Boston office in Back Bay

Job summary

CodaMetrix, based in Boston, is seeking a Senior Data Engineer to join the Data Platform team. You will design and scale data pipelines on Databricks, implement governance, and drive cost-efficient, secure data services for ML, BI, and analytics across the organization.

Bring 3–5+ years of data engineering, strong PySpark/Python, Terraform, and AWS experience, plus a deep understanding of lakehouse architectures and HIPAA-compliant data handling.

Qualifications

• Degree in Computer Science or related field or equivalent experience.
• 3-5+ years of data engineering with progressive responsibility.
• 3-5+ years with Databricks platform (Unity Catalog, Delta Lake, Structured Streaming).
• Expert in PySpark and Python; working knowledge of Scala.
• Terraform for infrastructure-as-code and CI/CD pipelines (Jenkins, GitHub Actions).
• Expert SQL skills for large-scale queries and performance tuning.
• Strong AWS experience (S3, IAM, Secrets Manager, VPC) and HIPAA readiness.

Responsibilities

• Contribute to the data platform vision, architecture, and roadmaps.
• Design, build, and maintain scalable streaming and batch data platforms with Databricks.
• Implement least-privilege access, HIPAA compliance, and data governance.
• Drive platform cost reduction and operational excellence.
• Mentor engineers and enable ML/BI/customer success teams with data pipelines.

Skills

PySpark

Python

SQL

Databricks

Terraform

Kafka

AWS MSK

Jenkins

GitHub Actions

Lakehouse

CI/CD

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks platform

Terraform

Jenkins

GitHub Actions

Apache Kafka / AWS MSK

Job description

CodaMetrix, based in Boston, is seeking a Senior Data Engineer to join the Data Platform team. You will design and scale data pipelines on Databricks, implement governance, and drive cost-efficient, secure data services for ML, BI, and analytics across the organization.

Bring 3–5+ years of data engineering, strong PySpark/Python, Terraform, and AWS experience, plus a deep understanding of lakehouse architectures and HIPAA-compliant data handling.