Databricks Solution Architect

International Business Machines Corporation; Boston (MA); Remote; USD 150,000 - 210,000; Full time

Job summary

IBM is seeking a Databricks Solution Architect to define end-to-end data and AI architecture for a large modernization initiative. You will integrate cloud platforms (AWS/Databricks, GCP) and ensure data products, governance, and scalable AI services across hyperscalers.

You will lead migration architecture, modeling standards, and reporting layers while embedding responsible AI, security, and cost controls across the platform. The role can be performed remotely within the United States.

Qualifications

• Expertise in building data and AI architecture across cloud platforms.
• Experience leading data migration and integration patterns.
• Ability to model lakehouse architectures and data products.
• Familiarity with CI/CD, observability, and cost controls on cloud platforms.

Responsibilities

• Define the target data and AI architecture across AWS/Databricks and GCP ML/GenAI services.
• Lead data migration architecture including sequencing, cutover, and risk management.
• Own data modeling standards covering conceptual, logical, and physical models.
• Define data engineering patterns for ingestion, transformation, orchestration, and cost control.
• Architect reporting/analytics with governed metrics and self-service access.
• Design the ML/GenAI lifecycle including MLOps and evaluation.
• Architect secure cross-cloud data movement across hyperscalers.
• Ensure Databricks lakehouse data products are used (SAP data via SAP BTP / BDC Connect).
• Define governance guardrails for AI use cases and privacy.
• Provide architectural oversight on major migrations and roadmap for AI capabilities.
• Lead architecture reviews and communicate decisions to stakeholders.
• Design reference patterns integrating AI services with existing systems.
• Collaborate with stakeholders to align business objectives with scalable designs.
• Drive governance for AI integration with observability and responsible AI principles.

Skills

Databricks

Architecture

Cloud engineering

Data migration

Data modeling

MLOps

GenAI

Governance

Cross-cloud

Stakeholder collaboration

Tools

Fivetran

SAP BTP

BDC Connect

Job description

Databricks Solution Architect to define and lead the end-to-end architecture for a large data modernization program. You will bring deep expertise across cloud-based engineering, Databricks platform, data migration, data modeling, data architecture, data engineering, and reporting, and connect governed data products to ML, GenAI, and advanced AI solutions across hyperscalers.

Responsibilities

• Define the target data and AI architecture across the AWS/Databricks data platform and GCP-based ML/GenAI services, and the integration patterns between them.
• Lead the data migration architecture: re-engineering approach, migration patterns, Integration & Ingestion patterns, sequencing, cutover, reconciliation, and risk management.
• Own data architecture and data modeling standards, covering conceptual, logical, and physical models, lakehouse/medallion layers, and data product design.
• Define data engineering patterns and standards for ingestion (Fivetran), transformation, orchestration, CI/CD, observability, and cost control on cloud-based platforms.
• Architect the reporting and analytics layer, including semantic models, governed metrics, and self-service access on top of data products.
• Design the ML/GenAI lifecycle: feature engineering, model training and serving, RAG and agentic patterns, MLOps/LLMOps, monitoring, and evaluation.
• Architect secure, scalable cross-cloud data movement and integration across hyperscalers (AWS, Microsoft Azure, GCP), selecting services based on fit, cost, and performance.
• Ensure the architecture uses data products from the Databricks lakehouse, including SAP-sourced data via SAP BTP / BDC Connect.
• Define security, data privacy, responsible AI, and governance guardrails for AI use cases.
• Provide architectural oversight on related initiatives, such as the Tier-1 application migration from MySQL to Databricks Lakebase.
• Evaluate and shape the roadmap for advanced AI capabilities (agentic AI, frontier models, and emerging superintelligence-class technologies) as they apply to the client.
• Lead architecture reviews and communicate design decisions to executive and technical stakeholders.
• Design end-to-end architectures that embed AI capabilities into enterprise applications and workflows, defining reference patterns for AI services, models, and orchestration layers that integrate with existing systems, APIs, and data platforms.
• Collaborate with stakeholders to translate business objectives into scalable, resilient, and policy-compliant designs, balancing vendor-managed services with custom development and meeting performance, security, compliance, and cost objectives.
• Drive governance for AI integration: prompts, guardrails, and lifecycle management, with observability, privacy, and responsible AI principles embedded from design through operations.

This job can be performed from anywhere in the US