Solution Architect, Delivery (Databricks)

Jobgether; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Remote work within the United States

Competitive compensation

Professional development

Industry exposure

Job summary

Jobgether, on behalf of a partner company, seeks a Solution Architect, Delivery (Databricks) in the United States to lead enterprise lakehouse architecture and modernization initiatives. You will design scalable Databricks Data Intelligence Platform solutions across Bronze-Silver-Gold, implement governance with Unity Catalog, and guide delivery teams through CI/CD practices and cost optimization.

This remote role offers opportunities to integrate MLflow, Feature Store, Vector Search, and RAG

Qualifications

• 8+ years of experience in data architecture, data engineering, or enterprise software development with 3+ years designing Databricks-based solutions.
• Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional certification is preferred.
• Strong cloud architecture skills across AWS/Azure/GCP with secure data governance.

Responsibilities

• Lead architecture and design of scalable Lakehouse solutions on Databricks, including Delta Lake and Medallion Architecture.
• Design and implement governance using Unity Catalog and ABAC, with data masking and Delta Sharing.
• Drive modernization by migrating legacy warehouses to Databricks-based architectures and optimize Spark/Delta performance.

Skills

Python

PySpark

SQL

Terraform

Airflow

Databricks

Tools

Databricks

Power BI

Tableau

dbt

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Solution Architect, Delivery (Databricks) based in the United States.

Accountabilities

• Enterprise Architecture and Solution Design: Lead the architecture and design of scalable, production-ready Lakehouse solutions using the Databricks Data Intelligence Platform, including Medallion Architecture (Bronze, Silver, and Gold layers), Delta Lake, and Databricks SQL.
• Data Governance and Security: Design and implement enterprise-wide governance and security frameworks using Unity Catalog, row- and column-level access controls, data masking, attribute-based access control (ABAC), and Delta Sharing.
• Platform Modernization and Migration: Define and lead technical strategies for migrating legacy enterprise data warehouses and Hadoop environments, including Teradata, Netezza, Snowflake, and Cloudera, to Databricks-based architectures.
• Data Ingestion and Processing: Establish scalable patterns for batch and real-time data processing using Structured Streaming, Delta Live Tables (DLT), Auto Loader, and Apache Spark optimization techniques.
• Technical Leadership and Quality Assurance: Guide engineering teams through complex delivery initiatives, establish reference architectures, conduct architectural reviews, review code contributions, and implement standardized CI/CD practices using GitHub Actions, GitLab, Azure DevOps, Terraform, and Databricks Asset Bundles (DABs).
• Performance Optimization: Identify and resolve complex Spark and Delta Lake performance challenges by optimizing query execution, partitioning, Z-Ordering, caching, and Photon configurations.
• Cloud Cost Management: Establish FinOps practices and compute governance standards, including cluster policies, autoscaling, auto-termination, serverless configurations, and cost allocation models to optimize Databricks consumption.
• AI and Advanced Analytics Integration: Collaborate with machine learning and AI specialists to integrate MLflow, Feature Store, Model Serving, Vector Search, and Retrieval-Augmented Generation (RAG) pipelines into enterprise data platforms.
• Client Advisory and Stakeholder Engagement: Serve as a trusted technical advisor to senior client stakeholders, including Chief Architects, Directors of Engineering, and Chief Data Officers. Lead technical workshops, architecture design sessions, and proof-of-concept evaluations.
• Technology Partner Collaboration: Work closely with Databricks Partner Solutions Architects and field engineering teams to assess emerging capabilities, evaluate preview features, and ensure alignment with platform best practices.
• Architectural Innovation and Knowledge Sharing: Develop reusable architectural blueprints, migration accelerators, reference architectures, and standardized configurations to strengthen internal technical capabilities and delivery practices.

Requirements

• Professional Experience: At least 8 years of experience in data architecture, data engineering, or enterprise software development, including a minimum of 3 years designing and architecting enterprise solutions based on Apache Spark and the Databricks Data Intelligence Platform.
• Databricks Certification: Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional certification.
• Cloud Architecture Expertise: Strong architectural proficiency in at least one major cloud platform (AWS, Azure, or GCP), including VPC/VNet networking, private endpoints, cloud identity and access management, Entra ID integration, and cloud object storage solutions such as S3, ADLS Gen2, or GCS.
• Data Modeling and Governance: Advanced knowledge of modern data modeling methodologies, including dimensional modeling, Data Vault, and Medallion Architecture, with hands-on experience implementing Unity Catalog across multiple workspaces in production environments.
• Programming and Automation: Advanced proficiency in Python, PySpark, and SQL, combined with practical experience using Infrastructure as Code tools such as Terraform and orchestration platforms such as Databricks Workflows or Airflow.
• Technical Delivery Leadership: Demonstrated experience leading technical delivery teams, translating complex business and functional requirements into architectural designs, technical specifications, and user stories, and resolving critical implementation challenges.
• DevOps and Engineering Practices: Strong understanding of CI/CD pipelines, code quality standards, infrastructure automation, and modern software engineering practices.
• Client Communication and Advisory Skills: Excellent communication, presentation, and stakeholder management abilities, with the confidence to facilitate technical discussions and advise senior decision-makers.
• Additional Technical Experience: Familiarity with modern data ecosystem integrations, including Power BI, Tableau, reverse ETL, dbt, and data observability platforms, is highly desirable.
• Security and Compliance: Experience designing and implementing solutions aligned with enterprise regulatory and security frameworks such as HIPAA, HITRUST, SOC 2, or FedRAMP.
• Security Clearance: An active Public Trust or Secret clearance, or eligibility to obtain one, is considered an advantage, particularly for regulated and public-sector engagements.

Benefits

• Competitive Compensation: Estimated annual salary range of $140,000-$210,000, depending on experience, technical expertise, qualifications, certifications, and other relevant factors.
• Remote Work: Fully remote working arrangement within the United States.
• Technical Leadership Opportunities: Opportunity to take ownership of complex enterprise architecture initiatives and influence large-scale data modernization projects.
• Professional Development: Exposure to advanced cloud technologies, the Databricks ecosystem, artificial intelligence, machine learning, and emerging data engineering capabilities.
• Industry Exposure: Opportunity to collaborate with enterprise clients, senior technology leaders, and specialized engineering teams across complex technical environments.
• Innovation and Knowledge Sharing: Contribute to reusable technical solutions, architectural frameworks, and best practices that support continuous improvement and innovation.