Databricks Architect | Canada

Cuesta Partners, LLC; United States; Hybrid; USD 150,000 - 230,000; Full time

Benefits offered by this job

Flexible location

Healthcare benefits

401k

Paid time-off

Job summary

Cuesta Partners is seeking a Databricks Architect to lead production-grade data platforms for enterprise clients. This hands-on leadership role requires working with C-level stakeholders to design scalable Databricks solutions across AWS and Azure.

You will guide end-to-end data pipelines, implement CI/CD, and mentor data engineering teams while ensuring governance and best practices. Local and client-location flexibility is offered.

Qualifications

• 6+ years in Data Engineering or Cloud Architecture.
• 3+ years hands-on Databricks experience.
• Strong PySpark, Spark SQL, Python and SQL skills.
• Deep Delta Lake/Unity Catalog experience and governance.
• Experience delivering Databricks projects in consulting/Professional Services.

Responsibilities

• Design and architect scalable Databricks-based solutions for enterprise clients.
• Lead end-to-end cloud data platform implementations on AWS and/or Azure.
• Build production pipelines using Databricks Workflows and Spark Declarative Pipelines.
• Partner with client stakeholders to translate business needs into technical designs.
• Provide technical leadership to data engineering teams.
• Implement CI/CD and IaC for Databricks resources.
• Drive collaborative development with Git workflows and code reviews.
• Collaborate cross-functionally with analytics, engineering, and business teams.
• Serve as the primary technical point of contact for clients, including C-level executives.

Skills

Databricks

PySpark

SQL

Git

Communication

Tools

Delta Lake

Unity Catalog

Delta Live Tables

Terraform

GitHub Actions

Job description

About Cuesta Partners We are looking for a Databricks Architect to lead the delivery of data engineering solutions in a client-facing environment. This is a hands-on leadership role where you will work directly with clients — including C-level stakeholders — to design, build, and deploy production-grade Databricks platforms.

What You’ll Do

• Design and architect scalable Databricks-based solutions for enterprise clients.

• Lead end-to-end cloud data platform implementations across AWS and/or Azure.

• Design and build production pipelines using Databricks Workflows and Spark Declarative Pipelines.

• Design scalable ETL/ELT pipelines using Spark, PySpark, and Databricks Workflows.

• Partner directly with client stakeholders to understand business needs and translate them into technical solutions.

• Provide technical leadership and guidance to data engineering teams.

• Implement CI/CD pipelines and infrastructure-as-code for Databricks resources.

• Drive collaborative development through Git workflows, pull requests, and code reviews.

• Collaborate cross-functionally with analytics, engineering, and business teams.

• Serve as the primary technical point of contact for clients, including C-level executives.

What We’re Looking For

• 6+ years of experience in Data Engineering, Cloud Architecture, or related roles.

• 3+ years of hands-on experience with Databricks.

• Strong proficiency in PySpark, Spark SQL, Python, and SQL.

• Deep experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs.

• Proven experience delivering Databricks projects in a consulting or professional services environment.

• Strong understanding of data lake concepts and formats (Delta, Iceberg, Parquet, etc.).

• Experience designing and managing Unity Catalog for governance, access control, and lineage.

• Familiarity with medallion architecture (bronze/silver/gold) patterns.

• Hands-on experience with Git version control, pull requests, code reviews, and collaborative development workflows.

• CI/CD and infrastructure-as-code for Databricks resources (Terraform, Asset Bundles, GitHub Actions).

• MLflow experience: experiment tracking, model registry, deployment, and observability.

• Cloud platform experience.

• Experience working in client-facing, consulting, or cross-functional environments.

• Experience leading technical projects or mentoring engineering teams.

Nice to Have

• Databricks certifications (Data Engineer Associate or Professional, Machine Learning Associate or Professional, Data Engineer Associate or Professional or ML Engineer Associate or Professional).

• Experience with agentic AI workflows, including agents, skills, and tools on the Databricks side (Agentbricks), as well as LangChain and Claude agentic workflows.

Benefits: What you’ll gain:

• At Cuesta, we value entrepreneurship, humility, diversity, learning, speed, and balance. We provide our team members with:

• Constant opportunities for exposure & learning

• Flexible working location and enabling personal-life harmony with work

• Agency and influence in the company’s total strategy and direction

• Collaboration with a high-performing team

• Competitive base salary (outlined in this listing) and target bonus of 20-25%

• 401k, healthcare benefits, paid time-off, and more!