Senior Data Engineer — Databricks & Lakehouse at Scale

Mheducation; Ohio; On-site; USD 130,000 - 185,000; Full time

Job summary

McGraw Hill is seeking a Senior Engineer - Data Engineering (Databricks) to own data pipelines in a Databricks lakehouse powering reports for educators and administrators. You will craft scalable batch and incremental pipelines using PySpark, Spark SQL, and Delta Lake while collaborating with product, UI, and data-platform teams.

Responsibilities include designing reporting data models with bronze/silver/gold medallion architecture, optimizing SQL views, and guiding engineers on data engineering

Qualifications

• 5+ years of software or data engineering experience.
• 3+ years building production data pipelines on Databricks or Spark in AWS.
• Advanced SQL skills and data warehousing knowledge.
• Bachelor's degree in Computer Science or related field, or equivalent experience.

Responsibilities

• Translate business requirements into technical designs with product and engineering leaders.
• Build and maintain scalable batch and incremental data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows.
• Design reporting data models with dimensional modeling and bronze/silver/gold medallion architecture.
• Tune Spark and SQL performance and manage compute cost.
• Lead design reviews and mentor engineers on data engineering best practices.

Skills

Databricks

PySpark

SQL

Data modeling

AWS

Go

Education

Bachelor's degree in Computer Science or related field

Tools

Delta Lake

Unity Catalog

Databricks SQL

Terraform

Docker

Kubernetes

GitHub Actions

Job description

McGraw Hill is seeking a Senior Engineer - Data Engineering (Databricks) to own data pipelines in a Databricks lakehouse powering reports for educators and administrators. You will craft scalable batch and incremental pipelines using PySpark, Spark SQL, and Delta Lake while collaborating with product, UI, and data-platform teams.

Responsibilities include designing reporting data models with bronze/silver/gold medallion architecture, optimizing SQL views, and guiding engineers on data engineering