Senior Databricks & AI Platform Architect (Remote)

Eliassen Group; Olympia (WA); Remote; USD 138,000 - 145,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401k matching

Life insurance

Job summary

Eliassen Group is seeking a Senior Solution Architect to lead the Databricks, Data & AI modernization effort. This remote, contract role requires hands-on architectural direction across lakehouse platforms, governance, and AI-enabled data products.

You will partner with platform, data, security, business, and engineering teams, shaping target-state architecture, standards, and scalable data solutions while mentoring senior engineers.

Qualifications

• Deep understanding of Databricks Lakeflow connect capabilities and AWS analytics and ingestion services.
• Extensive experience addressing ingestion challenges from Lakeflow connect sources, including CDC handling and reconciliation techniques.
• Expertise with data quality libraries and configuration-based frameworks that plug DQ into ingestion pipelines.
• Extensive experience designing enterprise Databricks lakehouse architectures with Spark, Delta Lake, Lakeflow, and Unity Catalog.
• Strong experience with medallion architecture and Bronze/Silver/Gold data patterns.
• Experience designing enterprise data governance, lineage, security, and access-control frameworks including ABAC and RBAC, data classification, tagging, and least-privilege security.
• Experience designing production-grade data pipelines, reusable ingestion frameworks, observability, CI/CD, deployment automation, and production support.
• Experience designing semantic layers and governed enterprise data products, and architecting AI or GenAI solutions operating against governed enterprise data, including Databricks Genie.
• Experience with MCP and agent-based architectures, GenAI governance, tracing, and evaluation preferred.
• Strong communication skills and ability to work across engineering, security, architecture, and business teams.

Responsibilities

• Lead a Databricks Well‑Architected assessment across platform architecture, observability, pipelines, consumption, data quality, security, governance, and cost.
• Define target-state architecture and technical standards for the Data & AI platform.
• Identify architectural gaps, technical debt, platform risks, and remediation priorities.
• Establish architectural patterns supporting scalable data ingestion, transformation, analytics, AI, and agent-based solutions.
• Guide engineering teams in implementing standards and remediation priorities.
• Establish standards for development, test, production promotion, CI/CD, monitoring, and operational support.
• Design and implement governance standards leveraging Databricks Unity Catalog.
• Define least-privilege access models, attribute-based access control policies, data classification, tagging, and lineage standards.
• Establish monitoring for system health, jobs, pipelines, audit activity, data quality, and platform performance.
• Implement platform approaches for cost monitoring and optimization.
• Define data architecture standards for Bronze, Silver, and Gold medallion layers.
• Establish reusable patterns for source ingestion, transformation, data quality, lineage, and deployment.
• Architect configuration-driven onboarding frameworks using Databricks Lakeflow, Asset Bundles, YAML/control tables, and automated CI/CD.
• Guide development of conformed dimensions, shared business keys, and cross-domain data products.
• Define the architecture for an AI-ready Gold layer with certified, governed, documented data products.
• Design semantic and context layers using governed business definitions, metric views, KPIs, metadata, and business glossary information.
• Architect natural-language analytics using Databricks Genie.
• Define agent-ready interfaces using governed Unity Catalog functions, MCP endpoints, tracing, and evaluation.
• Provide architectural direction and technical mentorship to Data & AI Engineers.
• Partner with business SMEs and data owners to translate requirements into scalable technical solutions.
• Lead architecture and design reviews.
• Establish reusable engineering standards, reference architectures, runbooks, and implementation patterns.
• Participate in milestone and steering reviews.
• Deliver a secure, observable, governed, cost-aware, AI-ready Databricks architecture that can scale beyond a single use case.
• Produce assessment scorecards, remediation plans, platform standards, reusable data architecture patterns, and semantic and AI architecture assets.
• Enable operational readiness with monitoring, runbooks, documentation, and knowledge transfer.

Skills

Databricks Lakeflow

AWS analytics

Data governance

Unity Catalog

GenAI governance

Tools

Databricks

Lakeflow

Unity Catalog

CI/CD

Job description

Eliassen Group is seeking a Senior Solution Architect to lead the Databricks, Data & AI modernization effort. This remote, contract role requires hands-on architectural direction across lakehouse platforms, governance, and AI-enabled data products.

You will partner with platform, data, security, business, and engineering teams, shaping target-state architecture, standards, and scalable data solutions while mentoring senior engineers.