Senior Data Engineer: Cloud Data Platform & Pipelines

Heritage Construction + Materials; Indianapolis (IN); On-site; USD 120,000 - 180,000; Full time

Job summary

Heritage Construction + Materials seeks a Senior Data Engineer to design and implement strategic data initiatives. You will develop data pipelines in a Cloud Data Platform (Databricks or Snowflake), promote governance, and serve as the cloud data platform expert.

Responsibilities include optimizing Databricks clusters, automating CI/CD with Azure DevOps, integrating Databricks with Azure services, and building monitoring dashboards to ensure data reliability and cost efficiency.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.
• 5+ years of experience in data engineering or a related field.
• Experience with programming languages such as Python/PySpark, SQL, or Scala.
• Experience with cloud environments (Azure preferred) and strong cloud data architecture knowledge.
• Hands-on experience with Databricks or Snowflake Cloud Data Platforms.

Responsibilities

• Develop and support data pipelines within a Cloud Data Platform (Databricks or Snowflake).
• Monitor and optimize Databricks clusters for cost-effective scaling and performance.
• Implement and maintain Delta Lake for reliable data storage with versioning.
• Automate CI/CD pipelines for data workflows using Azure DevOps.
• Integrate Databricks with Azure services like Azure Data Factory, Synapse, and ADLS.
• Design monitoring solutions and dashboards for cluster and job performance.
• Collaborate with cross-functional teams to strengthen data governance via Unity Catalog.

Skills

Python/PySpark

SQL

Scala/Java

Azure Cloud

Databricks/Snowflake

CI/CD

Data governance

BI tools

Education

Bachelor’s or Master’s in CS/Engineering/Data Science

Tools

Databricks

Snowflake

Azure Data Factory

Azure Synapse

ADLS

Job description

Heritage Construction + Materials seeks a Senior Data Engineer to design and implement strategic data initiatives. You will develop data pipelines in a Cloud Data Platform (Databricks or Snowflake), promote governance, and serve as the cloud data platform expert.

Responsibilities include optimizing Databricks clusters, automating CI/CD with Azure DevOps, integrating Databricks with Azure services, and building monitoring dashboards to ensure data reliability and cost efficiency.