Databricks Lakehouse Architect & Platform Lead

Machine Learning Technologies, LLC; Boston (MA); On-site; USD 180,000 - 240,000; Full time

Job summary

Machine Learning Technologies, LLC is seeking an experienced Databricks expert to design and deliver scalable data and AI platforms on the Databricks Lakehouse. You will define target architecture, establish engineering standards, and guide ingestion, transformation, governance, performance optimization, and production operations.

With 12-15 years in data engineering or cloud architecture, you will lead architecture reviews, drive cost efficiency, and mentor engineering teams while partnering

Qualifications

• 12-15 years of experience in data engineering, integration, or cloud/data architecture.
• Hands-on Databricks and cloud architecture experience.
• Strong knowledge of Spark, Delta Lake, Unity Catalog, SQL and Python.
• Experience designing enterprise-scale data and AI platforms.
• Understanding of governance, security, performance tuning and cost optimization.
• Experience with CI/CD and production monitoring.
• Excellent communication and stakeholder-management skills.

Responsibilities

• Design scalable and secure Databricks Lakehouse architectures.
• Define target architecture, engineering standards, and reusable frameworks.
• Lead data ingestion, transformation, governance, and production operations.
• Conduct architecture reviews and technology evaluations.
• Drive performance and cost optimization across the platform.
• Establish CI/CD, monitoring, and operational best practices.
• Partner with stakeholders to translate business requirements into technical solutions.
• Provide technical leadership and mentoring to engineering teams.

Skills

Data architecture

Cloud architecture

Data governance

Security

CI/CD

Stakeholder management

Leadership

Tools

Databricks

Apache Spark

Delta Lake

Unity Catalog

SQL

Python

Job description

Machine Learning Technologies, LLC is seeking an experienced Databricks expert to design and deliver scalable data and AI platforms on the Databricks Lakehouse. You will define target architecture, establish engineering standards, and guide ingestion, transformation, governance, performance optimization, and production operations.

With 12-15 years in data engineering or cloud architecture, you will lead architecture reviews, drive cost efficiency, and mentor engineering teams while partnering