Databricks Lakehouse Architect & Platform Lead

Quantum Technologies. LLC; Boston (MA); On-site; USD 180,000 - 260,000; Full time

Job summary

Quantum Technologies. LLC is seeking an experienced Databricks expert to design and deliver scalable, secure data and AI platforms on the Databricks Lakehouse.

You will define the target architecture, set engineering standards, and guide data ingestion, transformation, governance, and operations. You will lead performance tuning, cost optimization, CI/CD, and monitoring while mentoring engineering teams and coordinating with stakeholders, security, and vendors to translate business needs into

Qualifications

• 12-15 years of overall experience in data engineering, integration, or cloud/data architecture.
• Strong hands-on expertise with Databricks and cloud architecture.
• Strong knowledge of Apache Spark, Delta Lake, Unity Catalog, SQL, and Python.
• Experience designing enterprise-scale data and AI platforms.
• Strong understanding of data governance, security, performance tuning, and cost optimization.
• Experience with CI/CD and production monitoring.
• Excellent communication and stakeholder-management skills.

Responsibilities

• Design scalable and secure Databricks Lakehouse architectures.
• Define target architecture, engineering standards, and reusable frameworks.
• Lead data ingestion, transformation, governance, and production operations.
• Conduct architecture reviews and technology evaluations.
• Drive performance and cost optimization across the platform.
• Establish CI/CD, monitoring, and operational best practices.
• Partner with client stakeholders, data engineers, analysts, security teams, and vendors to translate business requirements into technical solutions.
• Provide technical leadership and mentoring to engineering teams.

Skills

Databricks

Apache Spark

Delta Lake

Unity Catalog

SQL

Python

Cloud arch

Data governance

Performance tuning

CI/CD

Production monitoring

Stakeholder mgmt

Job description

Quantum Technologies. LLC is seeking an experienced Databricks expert to design and deliver scalable, secure data and AI platforms on the Databricks Lakehouse.

You will define the target architecture, set engineering standards, and guide data ingestion, transformation, governance, and operations. You will lead performance tuning, cost optimization, CI/CD, and monitoring while mentoring engineering teams and coordinating with stakeholders, security, and vendors to translate business needs into