Data Engineering Intern: Azure Databricks & PySpark

Transverse Insurance Group; New York (NY); On-site; USD 110,000 - 160,000; Full time

Job summary

MS Transverse is seeking a data engineering professional to design, develop, and maintain data pipelines on Azure Databricks using PySpark and SQL. You will contribute to Kimball-style dimensional models supporting actuarial, underwriting, claims, and reinsurance teams.

The role requires pursuing a Master’s in Data Science or related field and Databricks Data Engineer certification, with research experience preferred. Collaboration with cross-functional teams will be essential.

Qualifications

• Bachelor's degree in Computer Science or related field.
• Familiarity with Databricks platform and PySpark.
• Knowledge of Kimball-style dimensional modeling.
• Working knowledge of Claude Code and the Claude ecosystem.
• Git and version-controlled development workflows.

Responsibilities

• Support the design, development, and maintenance of Bronze, Silver, and Gold layer pipelines in Azure Databricks using PySpark and SQL
• Contribute to Kimball-style dimensional models (fact and dimension tables) for internal MST teams
• Build data quality checks and validation logic to keep Lakehouse data accurate and timely
• Contribute to internal automation tooling and Claude skills with a human-in-the-loop approach
• Document technical work clearly for technical and non-technical stakeholders
• Utilize Git and GitHub for version control and SCRUM-based collaborative development

Skills

Analytical thinking

Problem solving

Education

Bachelor's degree in Computer Science or related field

Master's degree in Data Science or related field (in progress)

Tools

Python

SQL

PySpark

Databricks

Kimball dimensional modeling

Claude Code

Git/GitHub

Job description

MS Transverse is seeking a data engineering professional to design, develop, and maintain data pipelines on Azure Databricks using PySpark and SQL. You will contribute to Kimball-style dimensional models supporting actuarial, underwriting, claims, and reinsurance teams.

The role requires pursuing a Master’s in Data Science or related field and Databricks Data Engineer certification, with research experience preferred. Collaboration with cross-functional teams will be essential.