Senior Data Engineer - Databricks (No Sponsorship Available)

The Heritage Group; Indianapolis (IN); On-site; USD 110,000 - 165,000; Full time

Job summary

Heritage Construction + Materials in Indianapolis seeks a Senior Data Engineer to design and maintain data pipelines on a cloud data platform, with Databricks and Delta Lake. You will optimize cluster performance, ensure data reliability, and support governance through Unity Catalog.

The role requires 5+ years in data engineering, Python/PySpark, SQL, and cloud experience (Azure preferred); knowledge of Databricks or Snowflake is essential.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.
• 5+ years of experience in data engineering or a related field.
• Experience with Python/PySpark/SQL/Scala.
• Hands-on experience with cloud environments (Azure preferred) and Databricks or Snowflake.
• Experience with workflow orchestration and CI/CD pipelines.

Responsibilities

• Develop and support data pipelines within a Cloud Data Platform (Databricks/Snowflake).
• Monitor and optimize Databricks cluster performance for cost-effective scaling.
• Automate CI/CD pipelines for data workflows using Azure DevOps.
• Integrate Databricks with Azure services (Data Factory, Synapse, ADLS).
• Design monitoring solutions and support data governance through Unity Catalog.

Skills

Python/PySpark

SQL

Databricks

Delta Lake

Azure

Azure Data Factory

Azure Synapse

ADLS

Data governance

Education

Bachelor’s or Master’s in CS/Engineering/Data Science

Tools

Databricks

Snowflake

Azure DevOps

Azure Data Factory

Azure Synapse

ADLS

Job description

## Senior Data Engineer - Databricks (No Sponsorship Available)Applylocations: Indianapolis, INtime type: Full timeposted on: Posted 2 Days Agojob requisition id: R-1622# **Build Your Career at Heritage Construction + Materials!**The Senior Data Engineer will help develop and implement strategic data initiatives. They will be responsible for developing and maintaining data pipelines, promoting sustainable data governance, and serving as the organization’s cloud data platform expert. **Essential Functions*** Develop and support data pipelines within a Cloud Data Platform, such as Databricks or Snowflake* Monitor and optimize Databricks cluster performance, ensuring cost-effective scaling and resource utilization* Implement and maintain Delta Lake for optimized data storage, ensuring data reliability, performance, and versioning* Automate CI/CD pipelines for data workflows using Azure DevOps* Integrate Databricks with other Azure services, such as Azure Data Factory, Azure Synapse, and ADLS* Design and implement monitoring solutions, leveraging Databricks monitoring tools, Azure Log Analytics, or custom dashboards for cluster and job performance* Collaborate with cross-functional teams to support data governance through the use of Databricks Unity Catalog* *Additional duties and responsibilities as assigned, including but not limited to continuously growing in alignment with the Company’s core values, competencies, and skills.***Education Qualifications*** Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field**Experience Qualifications*** 5+ years of experience in data engineering or a related field* Expertise with programming languages such as Python/PySpark, SQL, or Scala* Experience working in a cloud environment (Azure preferred) with strong understanding of cloud data architecture* Hands-on experience with Databricks or Snowflake Cloud Data Platforms* Experience with workflow orchestration (e.g., Databricks Jobs, or Azure Data Factory pipelines)**Skills and Abilities*** Strong analytical and problem-solving skills* Experience with programming languages such as Python, Java, R, or SQL* Experience with Databricks or Snowflake Cloud Data Platforms* Knowledge of statistical analysis and data visualization tools, such as PowerBI or Tableau* Understanding of business financials and NPV analysis* Proficient in Microsoft Office* Creative thinker that is open to working collaboratively with a team**About Heritage Construction + Materials**Heritage Construction + Materials (HC+M) is part of The Heritage Group, a privately held, family-owned business headquartered in Indianapolis. HC+M has core capabilities in infrastructure building. Its collection of companies provides innovative road construction and materials services across the Midwest. HC+M companies, including Asphalt Materials, Inc., Evergreen Roadworks, The Hoosier Company, Milestone Contractors and US Aggregates, proudly employ 3,000 people at 68 locations across seven states. Learn more at www.heritagebuilds.com.All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.