Databricks Solution Architect

Clearstead Advisors LLC; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Clearstead Advisors is seeking an experienced Databricks Solution Architect to lead the design, development, and ongoing management of our enterprise data and AI platform. This role combines hands-on data engineering with strategic architecture leadership, collaborating with IT partners and stakeholders to advance our data capabilities.

The candidate will establish standards, build scalable data solutions, and ensure security, reliability, and performance of the Databricks environment while

Qualifications

• Bachelor's degree in a data/IS/computer science related field or equivalent experience.
• Advanced degree preferred in information systems/data analytics.
• CDMP or Databricks certification preferred.
• 10+ years in data engineering/architecture roles.
• 4+ years designing and managing production Databricks environments.
• Expert knowledge of Azure Databricks, Unity Catalog, Delta Lake.

Responsibilities

• Lead the architecture and development of the Azure Databricks platform with Delta Lake, Unity Catalog, and medallion architecture.
• Establish technical standards, security controls, and scalable data integration frameworks.
• Oversee platform performance, monitoring, and cost optimization.

Skills

PySpark

SQL

Data pipelines

Azure Databricks

Security controls

CI/CD

Communication

Education

Bachelor's degree in Data, Information Systems, Computer Science, Business Analytics, or related field

Advanced degree in Information Systems, Data Analytics, or related discipline

Tools

Databricks

Azure

Terraform

CI/CD tools

Job description

All Jobs > Databricks Solution Architect

Clearstead Advisors is seeking an experienced Databricks Solution Architect to lead the design, development, and ongoing management of our enterprise data and AI platform. As a founding member of our AI & Data team, this individual will establish architecture standards, develop scalable data solutions, and ensure the security, reliability, and performance of our Databricks environment.

This is a highly technical, hands-on position combining strategic architecture leadership with active data engineering responsibilities. The successful candidate will collaborate with internal stakeholders, technology partners, and fellow team members to advance Clearstead's data capabilities and AI initiatives.

KEY RESPONSILBILITIES

• Lead the architecture and development of our Azure Databricks platform, including Delta Lake, Unity Catalog, and medallion architecture.
• Establish technical standards, security controls, and scalable data integration frameworks.
• Oversee platform performance, monitoring, operational readiness, and cost optimization.
• Manage the transition of platform ownership from external implementation partners.
• Design, build, and maintain production data pipelines using PySpark, SQL, and Databricks Workflows.
• Integrate information from Salesforce, Orion, Addepar, PARis, and other enterprise systems.
• Develop data transformation, reconciliation, and quality assurance processes.
• Implement automated testing, deployment, and infrastructure management practices.
• Develop data infrastructure supporting AI applications, intelligent agents, and advanced analytics.
• Support document ingestion, vector search, model serving, and retrieval-augmented generation.
• Collaborate with the AI Engineer and Data Management Lead to deliver secure, scalable solutions.

Technical Leadership & Collaboration

• Serve as the technical design authority for the enterprise data platform.
• Evaluate vendor solutions and oversee implementation partner deliverables.
• Partner with business and technology leaders to translate requirements into effective solutions.
• Mentor developing engineers and promote engineering best practices.

Role Scope and Working Relationships

• Direct Reports: None
• Decision-making responsibility: Recommendations with approval
• Key internal partner(s): IT Partners
• Key external relationship(s): Vendors, or other external partners
• Primary systems/tools: Databricks, Azure, Databases, Other Dev tools
• Travel Requirements: Limited to None

Requirements

Required Qualifications

• Bachelor's degree in Data, Information Systems, Computer Science, Business Analytics, or a related field, or equivalent combination of education and relevant professional experience.
• Advanced degree in Information Systems, Data Analytics, or a related discipline preferred.
• Certified Data Management Professional (CDMP), Databricks Data Engineer Associate or Professional, or comparable industry certifications preferred.
• 10+ years of experience in data engineering, data architecture, or related disciplines.
• 4+ years designing and managing production Databricks environments.
• Expert knowledge of Azure Databricks, Unity Catalog, Delta Lake, and data architecture principles.
• Strong hands-on experience with PySpark, SQL, data pipelines, and data integration.
• Experience supporting AI or machine learning workloads within Databricks.
• Knowledge of Azure security, identity management, Terraform, and CI/CD practices.
• Demonstrated experience managing enterprise data platforms and external technology partners.
• Strong communication skills with the ability to translate technical concepts for business stakeholders.

Preferred Qualifications

• Databricks Data Engineer Professional certification and additional platform administration or architecture credentials.
• Experience with financial services, wealth management, or regulated data environments.
• Familiarity with Salesforce Financial Services Cloud, Orion, or Addepar.
• Experience with Power BI, Databricks AI/BI Genie, and Delta Sharing.

Work and Physical Requirements

• Work environment: Home
• Physical requirements: Sitting, keyboarding
• Cognitive / communication requirements: Attention, analysis, prioritization, interaction, and collaboration

Performance Expectations

• Platform Architecture & Reliability: Design, implement, and maintain a secure, scalable, and high-performing Databricks platform that supports enterprise data integration, analytics, and AI initiatives.
• Data Engineering & Operational Excellence: Develop and optimize production data pipelines, ensuring data accuracy, system reliability, effective cost management, and timely resolution of technical issues.
• Technical Leadership & Collaboration: Establish architecture standards, guide implementation partners, and collaborate with cross-functional teams to deliver solutions aligned with business objectives and regulatory requirements.
• May require occasional flexibility in work hours to support system implementations, production incidents, or critical business operations.

General Statement

This job description summarizes the primary duties and requirements of the position. It is not intended to be an exhaustive list of every responsibility or task. Responsibilities may be adjusted based on business needs, consistent with applicable law. Nothing in this job description creates a contract of employment or changes the at‑will employment relationship where applicable.

About Clearstead

Clearstead is a financial advisory firm serving wealthy families and leading institutions. Our mission is to enable wealthy families and leading institutions to meet their financial objectives and achieve their aspirations, building stronger futures for their families, their communities, and themselves.

We fulfill our mission by creating a unified investment framework for our clients and integrating financial planning, tax planning and compliance, trust administration services with investment management to address complex financial situations.