Databricks Lead Engineer | Data Lakehouse

Healthcare Triangle, Inc.; Minneapolis (MN); On-site; USD 130,000 - 180,000; Full time

Job summary

Subash US IT Consulting seeks a Databricks Lead in Minneapolis to architect scalable data engineering solutions on Databricks. You will drive PySpark, Spark SQL, and Delta Lake-based pipelines and guide the data team.

You will collaborate with architects, analysts, and business stakeholders, mentoring engineers and delivering robust, cost-efficient data lakehouse architectures in a long-term, on-site role.

Qualifications

• Proficient in Databricks platform and PySpark programming.
• Strong SQL skills and experience with Spark SQL and Delta Lake.
• Experience building ETL/ELT pipelines and data lakehouse architectures.

Responsibilities

• Lead the design and development of scalable data engineering solutions using Databricks
• Develop and optimize PySpark, Spark SQL, and Delta Lake pipelines
• Build and manage ETL/ELT workflows and data lakehouse architecture
• Work with Azure/AWS, Databricks Workflows, Unity Catalog, and cloud storage
• Optimize performance, data quality, and cost of Databricks workloads
• Mentor engineers and provide technical leadership
• Collaborate with architects, analysts, and business teams

Skills

Databricks

PySpark

Spark SQL

Delta Lake

Python

SQL

ETL/ELT

Data Lakehouse

Azure/AWS

Unity Catalog

CI/CD

Git

Job description

Subash US IT Consulting seeks a Databricks Lead in Minneapolis to architect scalable data engineering solutions on Databricks. You will drive PySpark, Spark SQL, and Delta Lake-based pipelines and guide the data team.

You will collaborate with architects, analysts, and business stakeholders, mentoring engineers and delivering robust, cost-efficient data lakehouse architectures in a long-term, on-site role.